Hey producers, I need your help.

I have a list of videos that are pending, and many of them are pending for this reason: Their bit rates are too high! I am trying to figure out what to do about it.

It's been around for a little while now, but the store system is still pretty new, and its rules regarding acceptable bit rates have been, well, nonexistent. That has to change. When some really large files have been uploaded occasionally, I would usually just ask the producer if he didn't mind if I re-rendered the scene myself and swapped the file. But it's really a growing trend for these really large 1.5 or 2 gigabyte files to be uploaded. I mean, if there is a lot of motion in the vids, or if it's shot in really clear quality, then of course a higher bit rate is needed to maintain the quality. That, and if it's a really long scene, it can end up being a really huge file. I know producers want to offer their customers top-quality content, and if the source material necessitates it, then of course that's acceptable. But man, I just ran into an 11-minute file that was almost 2 gigs (over 18,000Kbps). I do think that some producers don't do much experimentation with different bit rates to try and squeeze the videos down at all. Perhaps they just select a default really high-bit rate setting in their rendering program and go with that?

I have been pointing to mudmadphil's stuff as a standard. He renders his 1280x720 videos at around 2,000Kbps, and they are crystal clear. I am thinking of making a policy for the maximum acceptable bit rate for each possible frame size. This is where I could use some suggestions from your guys.

For the videos already uploaded, I was intending to let the producer decide whether they want me to re-render the video myself and do the file swap, or they could re-render them themselves. These downloads are your babies and I know that. I just want to work with you to keep things under control. It will help bandwidth, and the customers won't have as many problems downloading them either.

I am going to go ahead and approve the videos for now. Based on your suggestions though, I am going to come up with a new policy and ask if some of the larger stuff can be brought down in size. I'll do the work if you want, so you don't have to render and re-upload.

I'm all ears.

Thanks,
Messmaster

My first word on this (but not my last), is that the quality of the camera seems to affect how low a bit rate I can use. I noticed that the video from my Nikon D7000 seems to look better after compression than the video from my other cameras. I have not re-tested with the Nikon video to see how low I can go, but my recent results with it lead me to believe I can squeeze the files down some more ... from that camera.

Obviously, the encoder quality is also going to factor in, among other things.

So, using Mudmadphil as the standard for people with consumer cameras is probably going to be catastrophic. I would think that 2-5Mbps MIGHT be a good target range.

I'll have more to say about this, later. It's Monday morning and I have a bunch of work to get done, here.

By "standard" I only meant that it's the best that I've seen so far and the example that I want to use. It's the bar that has been set to which I can point other producers and say, "hey, that's probably how much your videos can be compressed and still have good quality." I have not yet decided what actual policy to set regarding max bitrates, but phil's stuff just seems to be a good example to live up to. Starting with that, I'm still coming to you guys to see what your input would be.

Thanks!

A lot of it is going to depend on what kind of sensor is in the camera and whether the source video is stored as AVCHD, HDV, or some other codec. Most consumer cams are going to look substandard at any bitrate. If the source is a Red One or a broadcast-grade cam, you could probably get away with 1mbps and still have a better looking distribution product than even the raw video from a consumer camcorder.

In my experience with DV, the bitrate is only half the battle. Just as much quality can be gained by judicious use of VBR and tweaking the i-frame rate.

Soundguy, have you actually had any experience with AVCHD? The attached images are a couple of frames from video shot on my Nikon D7000. Just FYI, since you seem to enjoy denigrating AVCHD. I think the video looks pretty damn good.

I know that there are a ton of knobs to turn, and infinite combinations of equipment and software that people are using, but I just want producers to at least think about getting better file sizes out of their videos. If they know enough about keyframes and all that jazz to squeeze a big clear video into a tiny file, then great. If all they know is to be aware that there are higher preset compression settings to test out in their video program, then that's okay too. I just want to develop guidelines that will make people aware that a little thought should go into rendering material. I think that starting with a maximum of 2500kbps for a 1280x720 video could be a good starting point.

FYI, the lowest I can go on the iPhone movies is 1Mbps. That's as far as the slider goes on the low end. I'm doing some tests with the 24fps Nikon D7000 footage and the 30fps footage from the JVC. I'm not sure how much I'll be using my Canon for mud videos in the future, so that will be my last test.

I use 960x540 image size because all of my cameras produce pristine results at that size. My JVC shoots 1080i, so 540 is a single field, giving it the same progressive frame look as my other cameras. So, that's the frame size at which I'm doing my testing.

The 24fps/Nikon test looks good at 2Mbps with automatic keyframes. I generally force a long GOP (like 2 seconds), but it looks like automatic performs just as well. The Nikon is still a best case scenario, so I need results from the other cameras.

2Mbps is as low as I can go with a tripod shot at 30fps, 960x540 and keep the quality I want.

Thanks for your input so far guys. It's really helping me out. I really think that this 2000 or 2500 ceiling is right about where I want to set a maximum for the download stores--at least for frames no larger than about 1280 wide. Of course exceptions can and will be made when necessary, but this is a nice guideline I think.

Hmm. I encode at 5000 I believe but have yet to come in at over a gig.

I see the difference when I encode at 2000 when I watch it on a TV.



Maybe put a size limit on? People need to learn how to encode, enforcing just one part of it probably won't be the best tactic.

I can get 45 minutes in at under a gig at a great quality using basic encoding software and consumer level cameras.

Technically (Soundguy may correct me here) I believe Hi Def is not technically Hi Def under 5000kbps bit rate.


I think a lot of producers should tag their videos "Shot in Hi Def, fucked up in encoding."

LOL, I shoot in 1080p or 1080i, then downsize to 540p and squeeze as much as possible. I lament that my customers never get to see the video in it's full quality; especially when I take a huge hit in computer performance, editing at 1920x1080. Everything takes forever, and I'm looking at buying an 8 core machine just for handling the HD video ... and, in the end, nobody sees it.

Ok, having said that, the 540p is really sharp, so it is still considerably better than SD, all else being equal.

I'm pretty sure there is no bit rate requirement for high definition. AVCHD is a high definition standard used for recording and distribution of HD and it maxes out at 2.8Mbps.

See: http://en.wikipedia.org/wiki/AVCHD

It's not about the quality of the video. AVCHD is an unnecessary, proprietary, LICENSED "container" that requires specific software to read and was designed to facilitate add-on features and DRM on blu-ray disk media. If you don't have native handling ability in your media player or editor, you have to strip out the A/V streams into something more useful. It has no business being used for acquisition. It's a distribution format.

Additionally, the codec normally used in the container is mpeg 4 part 10 (h.264) which is a VERY highly compressed interframe codec (i-frames & b-frames). You need ample processing power to process even one stream. If you are doing complex production editing with 2 or more video tracks, the power needed increases considerably. Even a quad-core can struggle with 3 or 4 layered tracks, especially when using extensive FX, color matching, or frame sizing. That means you either need to buy a video card with a hardware decoder or transcode the individual streams into something more "raw", less processor-intensive, and ideally using intraframe encoding (something resembling DV or MJPEG)

Worst of all, it's now difficult and expensive to buy even a semi-pro camera that provides any other storage format options. I don't think it's possible at all in the consumer lines. Very clearly the manufacturers have expended a lot of effort to drive every serious videographer off the consumer gear and into the exponentially-higher-margin semi-pro and pro lines.

I didn't want to specifically limit the size because somebody may want to sell a video that is just really long or big in its dimensions. If they record that at halfway decent bitrate, it can still become a huge file--and it should be a huge file if the length requires it.

On the flip side, somebody can upload a file that is, maybe, only 500 megs, but the video is only like a few minutes long. Now that's just wasteful, and an absolute size limit would not address that problem.

I think we need rules / standards that guide people into creating files that are only a certain number of megabytes per minute, aka max bitrate. And the rules can vary based on the frame size. If they want to sell a video that's 2 gigs in file size, that's fine, if it's a whopping frame size like Doug's original 1920x1080, and pretty long as well.

Yes, all of your points are valid. I can edit the AVCHD native, but performance is a problem. I transcode to Apple ProRes at 1GB/minute when necessary. As long as I work off striped RAID storage, I can limp along. I am seriously looking at an 8 core machine. It is still more cost effective to shoot with an AVCHD camera and buy a $3000 computer than it is to buy a high end camera ... especially if I need 2 or 3 of them.
I resisted HDV when it came out. I fought it and I screamed about the compression. Eventually, I gave up. I'm not going to jump up and down and shout about AVCHD, since nobody listens, anyway. The quality from both of my AVCHD cameras is really very good. I'll deal with the workflow issues. I can archive the original clips, so I only have to keep the 50GB clips around until I'm finished with a project. The one thing I really miss is timecode. When I think of it, I'm going to look into adding a timecode track to the movie files.

In theory, the 1920x1080 video can look very nice at 2.4Mbps (that's what the Nikon D7000 records at), but that implies a very efficient hardware encoder.

Which brings us to the software issue, which we have not yet discussed. I am testing with Apple's Compressor, which is a pretty decent audio/video compression tool. For people using low end software, it may be pretty hard to hit the same bit rates with comparable quality. Maybe we should test and suggest best in class compression software that is free or inexpensive. Also, if we were to find and recommend specific software, we could post screenshots of a range of setups for various source types and target sizes, or offer presets for download.

It's going to be difficult to get compliance from people who just don't understand what you're asking them to do.

I see your point.

Flipside is, I don't want to sell a video on one download store that looks awesome on a 50" screen and one in another store that looks crappy on a 50" screen.

Someone that buys it on the download store with the not so good version then thinks my videos all look crap on a big TV.

I'm SO glad you brought that up. This was in the back of my head and I just neglected to mention it. YES, I'd like to come up with help pages that can guide people on what software to get, and what settings have been tested to produce great results. We have so many producers on here--some more professional than others--and it would be great if we could share what software / settings we use. If we can figure out the free ones and the best settings, I'll put that info up on the Webmaster Help page.

What does everybody here use? What settings?

Well the whole point is to figure out what settings will produce results that are efficient, yet still look good. As a producer myself, I have no intention on forcing you guys to re-render your slaved-over projects until it's nothing but blocky artifacts. If the quality and size of your material deserves it, then hell, you'll require a large file. I should make clear that I'm not trying to stymie everyone with arbitrary size limits just because I'm cheap. I just want to help the amateur producers who so far have paid no attention to compression, as well as professionals to just select the max-bitrate setting thinking that anything lower would greatly reduce quality.

I had standardized on 3.5Mbps for my premium content some time ago, and, thanks to your prodding, I have learned that I can reduce that to 2.0 - 2.5Mbps, which not only reduces file sizes by 30-40%, it saves me hours of upload time. I was also surprised to discover that automatic keyframes works as well as long GOP settings. That will probably improve quality, in that the software can insert keyframes at scene/angle changes, and avoid them when not much is changing. As much as I have screamed about this in the past, I might have to thank you, Derek, for pushing me to revisit my compression settings.

Regarding software, as an Apple user, I am not going to be much help in evaluating MS-Windows software. I assume that MS-Windows has native WMV compression. I would think that would be a good starting point.

I am a Mac user as well, but had a little experience with people using the standard MS stuff and the WMV settings aren't great. Pinnacle doesn't seem to bad, but if they are using that they shouldn't be using WMV.

MS should really withdraw WMV at this point and switch to MP4 which they have been backing.