Anyway, here’s the big news:

We will be adding support for H.264 and AAC-LC decoding to the next version of Silverlight (post v2). This is in response to the loud and clear customer demand for H.264/AAC that we’ve been hearing since Silverlight 1.0, the general convergence of the video industry around H.264, and a continuation of Microsoft’s own investment into the MPEG-4 standard.

Here’s the official Microsoft press release, framed as Q&A with Silverlight’s Scott Guthrie:

http://www.microsoft.com/presspass/features/2008/sep08/09-09silverlight.mspx

The Q&A does a very good job of answering some of the questions regarding the seemingly complex relationship between H.264, VC-1 and Windows Media, but one answer in particular bears repeating:

Addition of H.264 support in Silverlight does not mark a departure from VC-1 or Windows Media, but instead serves to enhance and expand the existing video/audio format ecosystem. The idea is to give users more choice and allow Silverlight to adapt to their existing workflows, rather than forcing it the other way around. I’ve seen too many people over the past few years get bogged down in “VC-1 vs H.264 codec wars” and I always found such obsessions to be very counterproductive. Both codecs are efficient enough to deliver excellent video quality at similar bitrates, so the question of which one to use should really be answered with “whichever one best fits your workflow and project constraints.” By adding H.264 support to Silverlight, we hope to get to a point where Silverlight is codec agnostic and customers can spend their valuable time focusing on end-to-end media delivery.

We will be showing a technology preview of H.264/AAC playback at IBC in Amsterdam (September 12-16), for which we partnered with Inlet Technologies to produce the demo content. The exact techhnical details of “what” and “how” are still being worked out, but one thing I can tell for sure is that MP4 file progressive download will definitely be supported.

A few links to blogosphere coverage of this announcement:

• http://blog.streamingmedia.com/the_business_of_online_vi/2008/09/microsoft-silve.html
• http://timheuer.com/blog/archive/2008/09/09/silverlight-to-support-h264-and-aac.aspx
• http://realworldvideocompression.com/2008/09/h264-support-for-silverlight/

Check it out at:

http://gallery1.demconvention.com/

The DNC website is providing live streaming HD video coverage of the convention using Silverlight 2 Beta 2 and Move Networks plugins. This is a great example of Move providing their own adaptive streaming technology and integrating it with Silverlight. I don’t have the exact encoding specs available, but it looks like video is being streamed in true HD - at least 720p in full screen, as far as I can tell. It’s really amazing video quality - and it’s live!

Let’s begin by explaining the 2 video player user interfaces and the plugins that power each.

User Interface

The NBC Olympics video player is available in 2 flavors: Standard and Enhanced. The Standard player UI is what you get when you first launch the video player. The Enhanced player UI is what you get when you click on the “Enhanced” button in the lower right corner of the Standard player.

The Standard player has a video rectangle of size 592×336 (roughly a 16:9 aspect ratio) and can be experienced with either WMP or Silverlight plugins. As explained in earlier posts, if you are running Windows OS + Internet Explorer or Firefox browser + WMP9 or better (ideally WMP11), you can choose to use the WMP plugin instead of the Silverlight plugin to view video by choosing “Watch without Plugin” when prompted to install Silverlight. The video streams available in the Standard player are identical regardless of whether you’re using WMP or Silverlight. The bitrate of those streams never exceeds 650 kbps in the Standard player.

The Enhanced player is only available to those who have installed the Silverlight plugin. It provides a more interactive experience and features a larger video window, as well as higher resolution and higher bitrate video streams (for some content). The video rectangle is 848×480 (also roughly 16:9 aspect ratio).

Video and Audio Codecs

All video on the NBC Olympics website is encoded as VC-1 Advanced Profile in CBR mode at various bitrates (described below).

All audio is encoded as WMA 10 Professional audio at 48 kbps, 44.1 kHz, stereo. The special Low Bitrate (LBR) mode of the WMA Professional codec offers improved fidelity over the more commonly used WMA Standard codec and is comparable with HE-AAC quality.

Content Categories

The content is generally divided into 2 categories: Live/Rewind and Highlights/Encore.

Live video (and its archived counterpart Rewind) is encoded on site in Beijing, then beamed back to New York and distributed to homes via CDNs. It comes in 2 bitrates and sizes:

• 592×336 at 600 kbps
• 320×176 at 300 kbps

The reason why higher bitrates aren’t offered for Live streams is because NBC’s link from Beijing to New York has a fixed bandwidth and needs to be able to sustain many simultaneous live streams (1 Mbps per event, and there can be as many as 30 events happening at the same time). In addition, delivering more than 1 Mbps of video around the world without losing packets all over the place or running into last-mile bottlenecks - is still incredibly difficult even in 2008.

Highlights/Encore video is content produced and encoded by NBC in New York. It typically features highlights, previews, recaps, interviews - so generally anything that’s not a full rewind of an event. It comes in 4 bitrates:

• 320×176 at 350 kbps
• 424×240 at 600 kbps
• 592×336 at 1050 kbps
• 848×480 at 1450 kbps

As mentioned above, the higher bitrates are only available in the Silverlight-exclusive Enhanced player interface. The Standard player is only able to consume the first 2 lower-bitrate streams.

In addition to all the bitrates and resolutions mentioned above, all content is available for thumbnail-sized Picture-In-Picture viewing. PiP video is always encoded as 128×96 at 50 kbps and half the source framerate.

This means that the minimum bandwidth needed to view the highest quality video + PiP is 1550 kbps (1450 video + 48 audio + 50 PiP) in perfect conditions. In reality, you probably need at least 100 kbps overhead on top of that in order to compensate for Internet unreliability.

Much of the press coverage of the NBC Olympics website has referred to the video content as being ”HD quality.” The definition of “HD” for television has always been pretty clear: you need at least 1280×720 to call something “HD.” Unfortunately, the definition of HD video on the web has been far more ambiguous. It’s the YouTube effect. Once you get used to watching 320×240 poorly compressed video for so long, anything above that suddenly starts looking like Digital Cinema. Whether or not you choose to think of 848×480 video as HD is up to you. I personally wouldn’t, but then again - it’s my job to be nitpicky about video quality.

Streaming Methods

Finally, there’s the actual delivery of the content. Two basic methods of streaming are used on the NBC Olympics website.

All Live video - regardless of which plugin is consuming it - is streamed via WMS HTTP streaming protocol from Windows Server 2008 servers running Windows Media Services. The same streaming method is also used for all delivery to the WMP plugin. If you’re using the WMP plugin, you always get the WMS stream, regardless of content type.

As mentioned before, the Silverlight-powered Enhanced player has several features that make it a superior experience to the Standard player and WMP plugin. One of them is its ability to seamlessly switch between streams of different bitrates and resolutions during playback to dynamically match the user’s bandwidth and CPU power. This feature, often referred to generically as Adaptive Streaming, is something that Microsoft developed for NBC based on Silverlight 2’s MediaStreamSource interface. NBC’s website does not utilize the Move Networks adaptive streaming technology, as has been widely rumored. Silverlight 2 supports hooks to multiple adaptive streaming approaches, including Move’s - but in this particular case Microsoft provided the solution.

The easiest way to recognize that you’re watching an adaptively streamed video while in the Enhanced player is by seeking to another point in the video. If the player is using adaptive streaming, you will see the video start up very quickly without a buffering notification and the resolution will briefly drop. After a few seconds the blurry video will get sharper, and then sharper again… and then sharper again, your bandwidth allowing, of course.

Summary - Setting expectations

Here’s the best quality you can expect for NBC Olympics video, as well as minimum requirements:

• Live/Rewind content:
  • Either plugin: 592×336 at 650 kbps
• Highlights/Encore content:
  • WMP plugin:  424×240 at 650 kbps
  • Silverlight plugin and Enhanced player:  848×480 at 1500 kbps

We need Silverlight on Linux - and we need your help to make it happen.

Jason writes:

Yeah, it would have been nice to be able to watch the Olympics event playbacks and live feeds on Linux using Moonlight.  But right now, Moonlight only supports Silverlight 1.0 apps, and NBCOlympics.com is implemented using 2.0. As Novell’s chief Mono/Moonlight developer, Miguel de Icaza told me several weeks ago before the NBCOlympics content launch, “Work on this has started, but it will take a lot of work. And sadly, there are very few people willing to contribute to make this happen on time.”

That’s incredibly disappointing to hear, because here’s just a sample of the type of feedback I’ve been seeing from Linux users regarding NBC’s use of Silverlight:

“It’s infuriating to be summarily left out just because I choose to use a superior OS, Linux, instead of the crap M$ puts out. Oh well, I guess NBC doesn’t care how many viewers–and, yes, we ARE viewers as well, not just people online–they’re alienating by their idiotic decision to go with a Micro$oft only application.”  - from a comment on an LA Times blog

“I triple boot. XP, Vista and Ubuntu. I refuse to boot into XP or Vista to watch this online. If they don’t care about Linux users then I will return the favor and find alternatives.” - from a comment on Digg

“Nevertheless, NBC’s official stance is to support Internet Explorer and Firefox for Windows and the Mac, but there is no Linux support. This seems absolutely foolish. How hard is it cater to users of Firefox on Linux?” - from a blog post on OStatic

So on one hand we have Linux users who are infuriated because they feel left out, and on the other hand we have a statement from Moonlight project leader indicating there is not enough interest and support in the OSS community to deliver a solution. What’s going on here? Some might say it’s just a case of Linux users growing too complacent, or use this as an example of a counterproductive anti-Microsoft bias in the OSS community - but an article on the not-very-subtly named Boycott Novell site hints there might be more to it than meets the eye. As it turns out, the OSS community doesn’t seem to like Novell very much either.

My own opinion is: if you want it, help make it happen. Miguel is waiting for your e-mail. 

I’d love to hear everyone’s opinion on this, especially from Linux users and active OSS contributors.

http://silverlight.net/forums/p/22318/80644.aspx

Microsoft’s Tom Taylor has provided some context and explanation for the controversial design decision.

The NBCO website specifically lists the plug-ins required for experiencing all sections of the website: http://www.nbcolympics.com/pluginsneeded.html

As you can see, only Flash is a requirement for access to the main (non-video) site. To access the video content on the website, Silverlight 2 Beta 2 plug-in or Windows Media Player are required. The various Silverlight supported platforms were outlined in my previous post, but what about WMP support?

WMP “fallback” mode was a key part of the NBC Olympics player design from the very start. Because the Silverlight 1 and 2 media pipeline is  built on top of the Windows Media format (ASF) and codecs (VC-1, WMV8, WMV7, WMA9 Standard & Pro, MP3), any media content produced for Silverlight is also backwards compatible with WMP - so it only made sense to re-use the same streams for WMP as a “fallback” option in case certain users didn’t wish to install Silverlight 2 (after all, it is a beta) or simply couldn’t install it due to non-admin restrictions or due to being on an unsupported Windows OS.

Now, I know some may immediately ask, “Well, if you wanted to provide a fallback option, why not just use Flash as an alternative?” All business politics aside, it’s important to understand that creating all the content in duplicate (and trust me, there’s A LOT of content being produced for these Olympic games) would’ve been extremely inefficient with regards to both time and cost. Not only would all content need to be encoded twice, but the bandwidth of NBC’s direct link from Beijing to New York would need to be doubled, and NBC would need to deploy twice the number of encoders and servers, etc, etc, etc. Supporting Flash would’ve also doubled the engineering cost of designing, implementing and testing the video player application. Speaking from a purely engineering perspective, I can say that getting this project off the ground and to this final stage was an incredibly ambitious undertaking with just Windows Media alone. Supporting a whole additional set of formats, codecs and RIA technology would’ve been nothing short of impossible.

But back to WMP support…

At the moment WMP support is limited to Windows OS only, and Internet Explorer and Firefox browsers only. The good news is that any Windows OS running WMP9 or later ought to work - and that includes even ancient Windows 98SE, Millennium and 2000 systems. However, if using the WMP fallback mode, it is definitely recommended that you use WMP11 (XP, Vista and WS2008) for optimal video and audio playback quality. If you are using WMP9 and WMP10, upon visiting the NBC Olympics video player page for the first time you may be prompted with an ActiveX security dialog asking you to install “wvc1dmo.cab” or “Windows Media Audio Codec.” It is safe to install these updates - and actually required to make the NBCO video player work with WMP9/WMP10. If you don’t get any security dialog prompt, but can’t see any video either - your browser’s security settings might be blocking the ActiveX install prompts. In that case you can install the necessary video codec update manually from http://support.microsoft.com/kb/942423.

It has also been suggested that Mac PPC users might be able to get the WMP fallback solution working for them by installing Flip4Mac WMV. Unfortunately, at the time of this writing that solution is not working, partially due to an apparent incompatibility between the NBCO player Javascript code and Flip4Mac/Safari/Firefox. If this changes at any point, I’ll make sure to post about it immediately.

Finally, for those who do have a choice of installing Silverlight instead of using WMP, what advantage does Silverlight bring to the table? For starters, all WMP-targeted video streams are limited to 650 kbps, whereas the Silverlight plug-in can take advantage of higher-bitrate and higher-resolution video streams, all the way up to 1500 kbps. Furthermore, all WMP playback is single-bitrate only with no dynamic/adaptive stream-switching capability. The Silverlight-based player, on the other hand, can use adaptive streaming (dynamic bitrate switching) for most NBCO content that’s not Live or Rewind. So the short answer to the question of what advantage Silverlight has over WMP is: better video quality and more reliable streaming methods.

Anyone living in the U.S. can watch live and archived streaming video of nearly every Olympic event for free by visiting:

http://www.nbcolympics.com/video/index.html

The reason that only United States residents have access to NBC’s video streams is because NBC owns the rights to broadcasts of the 2008 Summer Olympics only for the United States. The International Olympic Committee sells broadcasts rights to only one broadcaster per country, so if you’re in Canada - you have to watch CBC; if you’re in the UK - you have to watch the BBC, and so on. I’m sure NBC wouldn’t have minded selling ads globally if the IOC had let them.

The NBC Olympics website doesn’t require (as some news reports and blogs have stated) Microsoft Silverlight to run, but is optimized for it. This is even stated on the NBC Olympics website. The Silverlight 2 Beta 2 plugin needed for the optimal (and intended) rich web experience is a minimal download and a very light-weight install. And despite being dubbed a “Beta 2,” it’s actually easily one of the most stable Microsoft products I’ve seen in years. Here’s the list of Silverlight 2 supported operating systems and browsers:

• Windows Vista: Internet Explorer 7 or better, Firefox 1.5 or better
• Windows XP SP2 or SP3: Internet Explorer 6 or better, Firefox 1.5 or better
• Mac OS X 10.4.8+ (Intel only): Firefox 1.5 or better, Safari 2 or better

Microsoft’s Silverlight website also mentions the following platforms as being supported by Silverlight 2, but I guess NBC isn’t explicitly supporting them, so proceed at your own risk: Windows 2000 w/ IE6, Windows Server 2003 w/ IE6+ or Firefox 1.5+. And if Silverlight 2 works on Vista, one would also assume it works on Windows Server 2008 too.

A list of Frequently Asked Questions is also available on the NBC Olympics Video website.

Finally, here’s a few articles that describe the project and showcase its highlights:

http://news.cnet.com/8301-13860_3-10002909-56.html
http://seattlepi.nwsource.com/business/373888_msftoly07.html?source=mypi

MSDN’s Channel 9 also has a 20-minute interview with fellow Silverlight evangelist Eric Schmidt who talks about the various features of the NBCO video player and what it took to pull off this massive effort:

http://channel9.msdn.com/shows/Continuum/Building-NBCOlympicscom-with-Silverlight/