An inside look at NBC Olympics video player

It’s the second week of the Beijing 2008 Olympics and though the press coverage of the NBC Olympics website has been more than thorough, one thing that hasn’t been fully explained is – what exactly are you watching when exploring the different parts of the NBCO video player – and what kind of quality should you expect anyway?

Let’s begin by explaining the 2 video player user interfaces and the plugins that power each.

User Interface

The NBC Olympics video player is available in 2 flavors: Standard and Enhanced. The Standard player UI is what you get when you first launch the video player. The Enhanced player UI is what you get when you click on the “Enhanced” button in the lower right corner of the Standard player.

The Standard player has a video rectangle of size 592×336 (roughly a 16:9 aspect ratio) and can be experienced with either WMP or Silverlight plugins. As explained in earlier posts, if you are running Windows OS + Internet Explorer or Firefox browser + WMP9 or better (ideally WMP11), you can choose to use the WMP plugin instead of the Silverlight plugin to view video by choosing “Watch without Plugin” when prompted to install Silverlight. The video streams available in the Standard player are identical regardless of whether you’re using WMP or Silverlight. The bitrate of those streams never exceeds 650 kbps in the Standard player.

The Enhanced player is only available to those who have installed the Silverlight plugin. It provides a more interactive experience and features a larger video window, as well as higher resolution and higher bitrate video streams (for some content). The video rectangle is 848×480 (also roughly 16:9 aspect ratio).

Video and Audio Codecs

All video on the NBC Olympics website is encoded as VC-1 Advanced Profile in CBR mode at various bitrates (described below).

All audio is encoded as WMA 10 Professional audio at 48 kbps, 44.1 kHz, stereo. The special Low Bitrate (LBR) mode of the WMA Professional codec offers improved fidelity over the more commonly used WMA Standard codec and is comparable with HE-AAC quality.

Content Categories

The content is generally divided into 2 categories: Live/Rewind and Highlights/Encore.

Live video (and its archived counterpart Rewind) is encoded on site in Beijing, then beamed back to New York and distributed to homes via CDNs. It comes in 2 bitrates and sizes:

  • 592×336 at 600 kbps
  • 320×176 at 300 kbps

The reason why higher bitrates aren’t offered for Live streams is because NBC’s link from Beijing to New York has a fixed bandwidth and needs to be able to sustain many simultaneous live streams (1 Mbps per event, and there can be as many as 30 events happening at the same time). In addition, delivering more than 1 Mbps of video around the world without losing packets all over the place or running into last-mile bottlenecks – is still incredibly difficult even in 2008.

Highlights/Encore video is content produced and encoded by NBC in New York. It typically features highlights, previews, recaps, interviews – so generally anything that’s not a full rewind of an event. It comes in 4 bitrates:

  • 320×176 at 350 kbps
  • 424×240 at 600 kbps
  • 592×336 at 1050 kbps
  • 848×480 at 1450 kbps

As mentioned above, the higher bitrates are only available in the Silverlight-exclusive Enhanced player interface. The Standard player is only able to consume the first 2 lower-bitrate streams.

In addition to all the bitrates and resolutions mentioned above, all content is available for thumbnail-sized Picture-In-Picture viewing. PiP video is always encoded as 128×96 at 50 kbps and half the source framerate.

This means that the minimum bandwidth needed to view the highest quality video + PiP is 1550 kbps (1450 video + 48 audio + 50 PiP) in perfect conditions. In reality, you probably need at least 100 kbps overhead on top of that in order to compensate for Internet unreliability.

Much of the press coverage of the NBC Olympics website has referred to the video content as being “HD quality.” The definition of “HD” for television has always been pretty clear: you need at least 1280×720 to call something “HD.” Unfortunately, the definition of HD video on the web has been far more ambiguous. It’s the YouTube effect. Once you get used to watching 320×240 poorly compressed video for so long, anything above that suddenly starts looking like Digital Cinema. :) Whether or not you choose to think of 848×480 video as HD is up to you. I personally wouldn’t, but then again – it’s my job to be nitpicky about video quality.

Streaming Methods

Finally, there’s the actual delivery of the content. Two basic methods of streaming are used on the NBC Olympics website.

All Live video – regardless of which plugin is consuming it – is streamed via WMS HTTP streaming protocol from Windows Server 2008 servers running Windows Media Services. The same streaming method is also used for all delivery to the WMP plugin. If you’re using the WMP plugin, you always get the WMS stream, regardless of content type.

As mentioned before, the Silverlight-powered Enhanced player has several features that make it a superior experience to the Standard player and WMP plugin. One of them is its ability to seamlessly switch between streams of different bitrates and resolutions during playback to dynamically match the user’s bandwidth and CPU power. This feature, often referred to generically as Adaptive Streaming, is something that Microsoft developed for NBC based on Silverlight 2′s MediaStreamSource interface. NBC’s website does not utilize the Move Networks adaptive streaming technology, as has been widely rumored. Silverlight 2 supports hooks to multiple adaptive streaming approaches, including Move’s - but in this particular case Microsoft provided the solution.

The easiest way to recognize that you’re watching an adaptively streamed video while in the Enhanced player is by seeking to another point in the video. If the player is using adaptive streaming, you will see the video start up very quickly without a buffering notification and the resolution will briefly drop. After a few seconds the blurry video will get sharper, and then sharper again… and then sharper again, your bandwidth allowing, of course.

Summary – Setting expectations

Here’s the best quality you can expect for NBC Olympics video, as well as minimum requirements:

  • Live/Rewind content:
    • Either plugin: 592×336 at 650 kbps
  • Highlights/Encore content:
    • WMP plugin:  424×240 at 650 kbps
    • Silverlight plugin and Enhanced player:  848×480 at 1500 kbps

About Alex Zambelli

Alex is a Principal Product Manager at iStreamPlanet Co. in Redmond, Washington. Prior to his current job he was a Technical Evangelist for Microsoft Media Platform at Microsoft Corporation. He specializes in video streaming, adaptive HTTP streaming, VC-1 and H.264 video, and video processing best practices.
This entry was posted in Olympics, Silverlight. Bookmark the permalink.

14 Responses to An inside look at NBC Olympics video player

  1. Informative. It also exposes clearly the occaisional difficulty in getting open-source developers to stop screaming bloody murder and start contributing — to “make it happen”. In this case, it is clear that Microsoft has been very positive and helpful in getting the Linux crowd to come on board. I can’t help, but I’d really like to see some folks step up to the plate.

  2. James Seng says:

    Did you guys look at H.264 extension scalable video coding instead of doing adaptive streaming?

  3. Not as a viable option for this project, no. However, it’s certainly a technology that’s been discussed at MS. I personally feel that is has potential to be a truly standard method of doing adaptive streaming, however, it still remains to be proven in the real world.

  4. Tom says:

    I am looking for any code example of Silverligt Adaptive Streaming implementation. I was checking MSDN (http://msdn.microsoft.com/en-us/library/system.windows.media.mediastreamsource(VS.95).aspx) but I could not find any examples how to use MediaStreamSource. Can anyone help me with that?

  5. Hi Tom,

    I am working with some folks in the Silverlight product team to see if we could get more detailed descriptions or sample code of MediaStreamSource online soon. Stay tuned.

  6. Cagdas Gerede says:

    Thanks for the article.

    One question I have is why adaptive streaming only works with Highlights/Encore content. What is the challenge with Live/Rewind content?

    Also, you mentioned that Live video is streamed via WMS HTTP streaming protocol. And you also mentioned “If you’re using the WMP plugin, you always get the WMS stream, regardless of content type.”

    How is this different for Silverlight? When the content is not live, does it delivered via Progressive Download? if it does, why is this the case?

    I appreciate your answer,
    thanks

  7. Restricting adaptive streaming to only Highlights/Encore content was mostly a design decision. Rewind videos could’ve been created as dual-bitrate adaptive streaming assets, but the thought during the design phase was that doing so would require additional testing. In hindsight, that was probably an overly cautious decision.

    As for Live streams, we simply didn’t have the encoder tools available that could easily create adaptive streams and upload them to CDNs in realtime. On the other hand, Windows Media-ready live encoders (such as the Digital Rapids Stream encoders used by NBC) were widely available and of great quality.

    As for your last question… Adaptive streaming to Silverlight was done entirely over HTTP using progressive download. This was made possible by Silverlight’s low-level MediaStreamSource API which allows you to download data using .NET methods, parse it and hand it off to Silverlight to decode and render. Engineering the same functionality in WMP would’ve required an additional ActiveX plugin, at the very least.

  8. Cagdas Gerede says:

    Thank you very much for your response.

    If I may, I have some follow up questions.

    I have some experience with MediaStreamSource and I am confused to hear “Adaptive Streams” and “dual-bitrate adaptive streaming”.

    When I thought about how Adaptive Streaming works, I thought there are multiple separate streams (there are, for instance, in NBC case four streams with different bitrates and resolutions), and after we hook-in to a MediaElement with our custom MediaStreamSource, in the MediaStreamSource’s method preparing the next sample to play in MediaElement, we decide from which stream we should generate the sample.

    Does “Adaptive Stream” or “dual-bitrate adaptive streaming” refer to a single stream with some special features? For example, would there be 4 separate adaptive streams in NBC case for encore content. If that is the case, would we need different tools to create such streams than the tools to create regular media streams?

    Also, you pointed out that it was a challenge to do adaptive streaming in realtime. Do you think Adaptive Streaming is a feature suitable for recorded media, but not for live media? If it is, why is this the case?
    Is it because, it takes time to create these kinds of “special” streams, and therefore the real-time requirement would be unmet.

    Lastly, when you mentioned “Adaptive streaming to Silverlight was done entirely over HTTP using progressive download”, I conclude that the backend server is a web server, not a streaming server. Am I correct? If that is the case, then, I assume the complete logic of adaptation is on the Silverlight site so I can use my choice of server technologies.

    I appreciate your answers,
    thanks

  9. Pingback: Venkatarangan's Blog [???????????? ??????????] - Notes on PDC2008 Sessions - Inside the NBC Olympics Player

  10. Cagdas,

    There’s no broad standard on how to implement adaptive streaming in Silverlight.

    MediaStreamSource API merely allows A/V samples to be independently downloaded and then handed over to Silverlight to decode and render. How the samples are actually encoded, contained and transported – can vary.

    For example, you could chunk up the media stream into thousands of pieces at encoding time and then have the client request tiny files. Or, you could generate a contiguous stream but have the client request a byte offset corresponding to a particular time sequence in the video. Either way works.

    The most important thing in preparing content for adaptive streaming is to make sure each video sample has a valid sequence header in order to allow seamless switching in the decoder. This is why encoding for adaptive streaming typically requires video GOPs of fixed length (i.e. 2 seconds) and entry point headers at the start of each GOP. This allows the decoder to treat every GOP as a standalone unit, making it possible to switch between streams even when the resolution or bitstream configs change.

    It’s definitely possible to do adaptive streaming from live sources. It requires an encoder that’s able to encode, index and upload the content to a CDN – all in realtime. Move Networks already provides this type of service today.

    Correct, NBC Olympics used purely web servers for adaptive streaming.

    Related to all this: Microsoft announced a productized version of our adaptive streaming technology this week – IIS 7.0 Smooth Streaming. The technology is based on IIS 7.0 and Silverlight, and is currently available through Akamai to end customers. You can visit http://www.smoothhd.com for demo and info.

  11. Pingback: Futile » links for 2008-11-27

  12. abrardLak says:

    Much the interesting has found here

  13. Pingback: Alex Zambelli’s Silverlight Media Blog » Blog Archive » The Birth of Smooth Streaming

  14. Pingback: Hiroshi Okunushi's Blog ?? : ?IIS7? IIS7 ??????????????????????2?