H.265/HEVC Ratification and 4K Video Streaming

OK, so maybe it was a shorter break from blogging than I expected. As it turns out the world does not stop when I change jobs. ;)

The media world today is abuzz with news of H.265/HEVC approval by the ITU. In case you’ve been hiding from NAB/IBC/SM events for the past two years – or if you’re a WebM hermit – I will have you know that H.265 is the successor standard to H.264, aka MPEG-4 AVC. As was the case with its predecessor it is the product of years of collaboration between the ISO/IEC Moving Picture Experts Group (MPEG) and the International Telecommunications Union (ITU) Video Coding Experts Group (VCEG). The new video coding standard is important because it promises bandwidth savings of about 40-45% for the same quality as H.264. In a world where video is increasingly being delivered over-the-top and bandwidth is not free – that kind of savings is a big deal.

What most media reports seem to have focused on is the potential effect that H.265 will have on bringing us closer to 4K video resolution in OTT delivery. Most reports speculate that H.265 will allow 4K video to be delivered over the Internet at bit rates between 20 and 30 Mbps. In comparison, my friend Bob Cowherd recently theorized on his blog that 4K delivery using the current H.264 video standard would require about 45 Mbps to deliver 4K video OTT.

While I think the relative difference between those two estimates is in the ballpark of the 40% bandwidth savings that H.265 promises, I actually think that both estimates are somewhat pessimistic. Given the current state of video streaming technology, I think we’ll actually be able to deliver 4K video at lower bit rates when the time comes for 4K streaming.

A common mistake that most people dealing with lossy video compression seem to make is to assume that the ratio between bit rate (bps) and picture size (pixels/sec) remains proportional and fixed as the values of both axis change. I don’t think that’s the case. I believe that the relationship between bit rate and picture size is not linear, but closer to a power function that looks like this:

H.264 Bits/Pixel Graph

In other words, I believe that as the pixel count gets higher a DCT-based video codec requires fewer bits to maintain the same level of visual quality. Here’s why:

  1. The size of a 16×16 macroblock, which is the smallest unit of DCT-based compression used in contemporary codecs such as H.264 and VC-1, grows smaller relative to the total size of the video image as the image resolution grows higher. For example,  in a 320×180 video the 16×16 macroblock represents 0.444% of the total image size, whereas in a 1920×1080 video the 16×16 macroblock represents only 0.0123% of the total image. A badly compressed macroblock in a 320×180 frame would therefore be more objectionable than a badly compressed macroblock in a 1920×1080 frame.
  2. As many studies have shown, the law of diminishing returns applies to video/image resolution too. If you sit at a fixed distance from your video display device eventually you will no longer be able to distinguish the difference between 720p, 1080p and 4K resolutions due to your eye’s inability to resolve tiny pixels from a certain distance. Ipso facto, as the video resolution goes up your eyes become less likely to distinguish compression artifacts too – which means the video compression can afford to get sloppier.
  3. Historically the bit rates used for OTT video delivery and streaming have been much lower than those used in broadcasting, consumer electronics and physical media. For example, digital broadcast HDTV typically averages ~19 Mbps for video (in CBR mode), while most Blu-ray 1080p videos average ~15-20 Mbps (in 2-pass VBR mode). Those kinds of bit rates are possible because those delivery channels have the luxury of either dedicated bandwidth or high-capacity physical media. However, in the OTT and streaming world video bit rate has always been shortchanged in comparison. Most 720p30 video streaming today, whether live or on-demand, is encoded at average 2.5-3.5 Mbps (depending on complexity and frame rate). 1080p30 video, when available, is usually streamed at 5-6 Mbps. Whereas Blu-ray tries to give us movies at a quality level approaching visual transparency, streaming/OTT is completely driven by the economics of bandwidth and consequently only gives us video at the minimum bit rate required to make the video look generally acceptable (and worthy of its HD moniker). To put it bluntly, streaming video is not yet a videophile’s medium.

So taking those factors into consideration, what kind of bandwidth should we expect for 4K video OTT delivery? If 1080p video is currently being widely streamed online using H.264 compression at 6 Mbps, then 4K (4096×2304) video could probably be delivered at bit rates around 18-20 Mbps using the same codec at similar quality levels. Again, remember, we’re not comparing Blu-ray quality levels here – we’re comparing 2013 OTT quality levels which are “good enough” but not ideal. If we switch from H.264 to H.265 compression we could probably expect OTT delivery of 4K video at bit rates closer to 12-15 Mbps (assuming H.265′s 40% efficiency improvements do indeed come true). I should note that those estimates are only applicable to 24-30 fps video. If the dream of 4K OTT video also carries an implication of high frame rates - e.g. 48 to 120 fps – then the bandwidth requirements would certainly go up accordingly too. But if the goal is simply to stream a 4K version of “Lawrence of Arabia” into your home at 24 fps, that dream might be closer to reality than you think.

 

One last thing: In his report about H.265 Ryan Lawler writes that “nearly every video publisher has standardized [H.264] after the release of the iPad and several other connected devices. It seems crazy now, but once upon a time, Apple’s adoption of H.264 and insistence on HTML5-based video players was controversial – especially since most video before the iPad was encoded in VP6 to play through Adobe’s proprietary Flash player.” Not so fast, Ryan. While Apple does deserve credit for backing H.264 against alternatives, they were hardly the pioneers of H.264 web streaming. H.264 was already a mandatory part of the HD-DVD and Blu-ray specifications when those formats launched in 2006 as symbols of the new HD video movement. Adobe added H.264 support to Flash 9 (“Moviestar”) in December 2007. Microsoft added H.264 support to Silverlight 3 and Windows 7 in July 2009. The Apple iPad did not launch until April 2010, which was also the same month Steve Jobs posted his infamous “Thoughts on Flash” blog post. So while Apple certainly did contribute to H.264′s success, they were hardly the controversial H.264 advocate Ryan makes them out to be. H.264 was already widely accepted at that point and its success was simply a matter of time.

About Alex Zambelli

Alex is a Principal Product Manager at iStreamPlanet Co. in Redmond, Washington. Prior to his current job he was a Technical Evangelist for Microsoft Media Platform at Microsoft Corporation. He specializes in video streaming, adaptive HTTP streaming, VC-1 and H.264 video, and video processing best practices.
This entry was posted in H.264, H.265. Bookmark the permalink.

9 Responses to H.265/HEVC Ratification and 4K Video Streaming

  1. Bob Cowherd says:

    Awesome detailed explanation, Alex! I’ve updated my post to reflect your research.

  2. Kiran Adimatyam says:

    Very nice explanation Alex. Thanks for sharing. Never knew you moved to iStreamPlanet. :)

  3. Pingback: What Is Streaming? » A history of media streaming and the future of connected TV

  4. shachar says:

    Great post. thanks

  5. Stuart Espey says:

    Hi Alex, we met at NAB,

    Absolutely agree with you.

    As someone who’s had experience developing video codecs since the early days of MPEG-4, its an easily observed phenomenon that bitrate requirements for a certain level of subjective quality reduce as the spatial and temporal complexity reduce.

    This is why when you double the frame rate (30 to 60fps) the data rate does not have to be doubled for the same quality, and also why if you double the resolution, bitrate does not have to be quadrupled. Basically, increasing the spatial resolution does not increase the spatial complexity at the same rate. But, it is also fair to say that doubling horizontal and vertical resolution (a quadrupling in the pixel rate), would probably require at least a doubling in bitrate.

    Also, if a new video standard provides an average 40% reduction in bitrate, this can often have an even more appreciable affect depending on where the savings are made on the complexity curves

    In short. You’re right

    Regards

  6. Mike says:

    Three things:

    1. It’s my understanding that blu-ray 1080p is maxed at 40Mbps and that a lot of movies are encoded in the 20′s or even 30′s. The AVC averge seems to be in teh mid 20′s, not 15-20 as you say. At least based on these lists here:
    http://forum.blu-ray.com/showthread.php?t=3338

    2. Apple was in fact one of the first to push H.264. All the way back on January 11th, 2005 with the introcution of Quicktime 7. Steve Jobs made a big deal about it in his keynote. I’ve linked to that section of the keynote video for you:
    http://www.youtube.com/watch?v=_6S_IRec4HQ&t=17m45s

    I remember watching this back then, it was the first time I heard of H.264. So they had a software player on the market before Blu-ray or HDDVD hit the market.

    3. So what would you deem a good bitrate for 4K video up to 120fps? Assuming this video was going to a disc format that cinephiles/collectors would consider perfectly valid and not complain about?

    • Hi Mike,
      1. Yes, Blu-ray bitrate averages are actually a little higher than what I posted, and it’s not uncommon these days to find Blu-ray titles encoded at 25-35 Mbps. The 15-20 Mbps I was thinking of is more like a lowest acceptable average bitrate for achieving VQ transparency to the 1080p master. In the early days of HD optical disc encoding many studios tried to save money (and hedge their bets) by encoding movie titles only once and then re-using the same encode for both HD-DVD and Blu-ray. Since HD-DVD discs had less storage capacity than Blu-ray discs, the early encodes ended up being more constrained. Although many people were skeptical early on (I still remember the numerous skeptical AVSForum posts prior to the HD-DVD launch), it turned out that most post-production houses were able to (with some tweaking and re-encoding) deliver nearly transparent VC-1 and H.264 encodes at 15-20 mbps. These days, with everyone encoding for Blu-ray only, such constraints no longer apply so many studios are encoding at 25-35 Mbps simply because they can. Now, the question is: are they getting significantly better VQ at 30 Mbps than at 20 Mbps? Many would argue that the extra bit rate is nice icing on the cake, but not strictly necessary to deliver the desired VQ.

      2. Duly noted. Still, I don’t think the iPad deserves that much credit for the advancement of H.264 as Ryan gives it, especially since H.264 adoption was already in full swing by 2010 when it launched.

      3. For 120 fps? It’s really hard to say without any real test data available, but I would guess at least 50 Mbps for H.264, and 30 Mbps for H.265. That would, of course, only apply to newly produced 120p 4K content. 4K scans of existing films would still be only 24 fps so such high bit rates wouldn’t be necessary.

      • Mike says:

        Thank you for your response. I understand your perspective better now.

        I think it’s interesting you say 50Mbps for 4K 120fps. If that is in fact true, that’s pretty amazing and we really aren’t too far off from 4K home distribution. At least from a physical media perspective. If there will be physical media to support it.

  7. Wow great post Alex, thank you very much for the information and explanation! Cheers