Compression and codecs: time to deliver2 July 2010
Putting video onto the web, or even mobile devices, isn’t difficult, but doing it well is tough. There are a huge number of variables to worry about, and the best choice for one video may not work well for another. Analysis by David Fox.
Each minute, 24-hours worth of video is uploaded to YouTube. In May, just five years after it first went ‘on air’, the website reported that it serves two billion video streams per day (with viewers watching an average of 15 minutes each). The TV set may still be the dominant way of reaching the public, but internet video is growing fast.
A lot of encoding techniques are "voodoo", requiring you to test and test again — and then make subjective judgements — according to Chris Roberts, editor and Final Cut Studio master trainer. Roberts delivered a training day on Compression and Codecs for the Guild of Television Cameramen recently.
"Why do we need to compress video? Because it’s big. And IT networks are not designed for video," said Roberts.
You will often start with some form of compression in your broadcast video – whether in terms of the frame size (1440×1080 instead of 1920×1080, for example) or frame rates (25 frames per second could almost halve your file size compared to 50fps), or interlacing — "probably the thing that causes us the most grief."
Adding one form of compression on top of a different form of compression will almost always lead to a reduction in quality. "If I am delivering for a computer screen, I need to work progressive, so as not to see comb artefacts," he said.
Amongst the main compression systems, the biggest difference is in whether they deal in individual frames (Intraframe) or in groups of frames (Interframe). Intraframe compression, such as DV, DV 50, DVCPRO HD, or ProRes, gives great image quality and ease of editing, but produces larger file sizes, because they do spatial compression within a frame, so each frame generally takes up the same amount of space.
Interframe codecs (such as MPEG-2, MPEG-4, H.264, HDV and AVCHD) use temporal compression, spreading the information in the video over a group of pictures, so that if little changes between frames, it can considerably reduce the file size. This makes them excellent for distribution, but not so good as production formats.
Intraframe usually uses constant bit rates, while Interframe is usually done at variable bit rates, although Apple’s ProRes is a variable bit rate codec. Some editing systems can deal with Interframe codecs natively (Avid now has realtime support for H.264 and AVCHD), while Adobe Premiere and Grass Valley Edius both have native support for all the main codecs. Final Cut Pro typically requires you to transcode to ProRes if you want realtime editing.
Players, Containers & Codecs
The video can end up being viewed on different Player software (QuickTime, Windows Media, Real, Adobe Flash, VLC, or DivX), using different Containers (the wrapper that stores audio and video data) and different Codecs.
A QuickTime .mov file can contain about 140 different codecs, but sometimes the wrapper and the codec can be the same thing (as with MPEG-4). He also explained the difference between the various versions of Adobe’s Flash format. Importantly, the most recent versions of Flash (9r3 and 10) are able to view H.264, which is now the most widely used video codec for new material on the internet. "The choice of how to encode and what player to aim for is fraught with difficulties, especially if your viewers are going to be in a corporate environment, where the best you might be able to expect is Flash 8," he said.
Unless you have to deliver something else, Roberts recommends an MPEG-4 wrapper (.mp4), H.264 codec, with AAC audio as an efficient format for web delivery that will be able to be viewed by the largest number of people. It is also the best format for YouTube or Vimeo and compatible with Apple devices. It is supported by the latest Flash Player, by Microsoft’s Silverlight 3 and Windows Media Player in Windows 7, and in HTML 5 (by Apple Safari, Google Chrome, and the upcoming Microsoft Internet Explorer 9 web browsers).
HTML5 is a set of new web standards that includes support for dynamic web applications and interfaces (similar to Flash in many ways, but not requiring a separate player as they are built in to the web browser). They are supported by most modern browsers, and will mean that web pages will just require a <video> tag to place embedded video. The video then plays within the browser. HTML 5 is already implemented within YouTube, Vimeo and Facebook, and will support other video codecs (notably Ogg Theora and VP8).
"There is no format that rules them all. It depends on what you are dealing with," he said. Common video codecs include: MPEG-4, Simple Proﬁle; MPEG-4 Part 10, Advanced Video Codec, H.264; Windows Media 9; VC-1; Sorenson Video 3; Sorenson Spark; MPEG-1 / MPEG-2; RealVideo; and DivX (a variant of MPEG-4). About 75% of video on the web is viewed using Flash, but H.264 is currently the most common codec for web video, with some 66% of uploads, according to Encoding.com, which claims to be the world’s largest encoding service (it encoded five million videos over the past year for the likes of MTV, Brightcove, Nokia and MySpace).
H.264 use rose from 31% of all its videos in Q2 2009 to 66% in Q1 2010. Over the same period, Flash-encoded video dropped from a share of 69% to 26% (for both Flash VP6 and FLV). Of course, a lot of the H.264 video ended up wrapped in a Flash wrapper, but it is fairly simple to offer H.264 both within Flash and by itself. YouTube, which represents about 40% of videos on the web, offers its video in both versions.
Also part of HTML 5 is Ogg Theora – which leapt from nothing to 4% between Q4 2009 and Q1 2010 (overtaking the aging and fast disappearing FLV format, which uses the old H.263 codec). Theora is a currently free, open source format (although that might be challenged by patent holders), and is being adopted by some web browsers for use with HTML 5 video (Chrome, Firefox and Opera). However, Theora currently isn’t as advanced as H.264 (it is based on the older VP3 codec). There is also the new VP8 codec, which has been developed by On2 Technologies. On2 is now owned by Google, which recently announced that VP8 would be available free. It is currently supported in beta versions of Chrome, Firefox and Opera. VP8 could offer similar performance to H.264, but "advanced" is not always a good thing.
"One problem with H.264 is that there are so many interdependencies within the Interframe GoP that the processor has to do a lot of work, which is why it doesn’t work with an older computer," said Roberts.