There’s no broad standard on how to implement adaptive streaming in Silverlight.

MediaStreamSource API merely allows A/V samples to be independently downloaded and then handed over to Silverlight to decode and render. How the samples are actually encoded, contained and transported – can vary.

For example, you could chunk up the media stream into thousands of pieces at encoding time and then have the client request tiny files. Or, you could generate a contiguous stream but have the client request a byte offset corresponding to a particular time sequence in the video. Either way works.

The most important thing in preparing content for adaptive streaming is to make sure each video sample has a valid sequence header in order to allow seamless switching in the decoder. This is why encoding for adaptive streaming typically requires video GOPs of fixed length (i.e. 2 seconds) and entry point headers at the start of each GOP. This allows the decoder to treat every GOP as a standalone unit, making it possible to switch between streams even when the resolution or bitstream configs change.

It’s definitely possible to do adaptive streaming from live sources. It requires an encoder that’s able to encode, index and upload the content to a CDN – all in realtime. Move Networks already provides this type of service today.

Correct, NBC Olympics used purely web servers for adaptive streaming.

Related to all this: Microsoft announced a productized version of our adaptive streaming technology this week – IIS 7.0 Smooth Streaming. The technology is based on IIS 7.0 and Silverlight, and is currently available through Akamai to end customers. You can visit http://www.smoothhd.com for demo and info.

If I may, I have some follow up questions.

I have some experience with MediaStreamSource and I am confused to hear “Adaptive Streams” and “dual-bitrate adaptive streaming”.

When I thought about how Adaptive Streaming works, I thought there are multiple separate streams (there are, for instance, in NBC case four streams with different bitrates and resolutions), and after we hook-in to a MediaElement with our custom MediaStreamSource, in the MediaStreamSource’s method preparing the next sample to play in MediaElement, we decide from which stream we should generate the sample.

Does “Adaptive Stream” or “dual-bitrate adaptive streaming” refer to a single stream with some special features? For example, would there be 4 separate adaptive streams in NBC case for encore content. If that is the case, would we need different tools to create such streams than the tools to create regular media streams?

Also, you pointed out that it was a challenge to do adaptive streaming in realtime. Do you think Adaptive Streaming is a feature suitable for recorded media, but not for live media? If it is, why is this the case?
Is it because, it takes time to create these kinds of “special” streams, and therefore the real-time requirement would be unmet.

Lastly, when you mentioned “Adaptive streaming to Silverlight was done entirely over HTTP using progressive download”, I conclude that the backend server is a web server, not a streaming server. Am I correct? If that is the case, then, I assume the complete logic of adaptation is on the Silverlight site so I can use my choice of server technologies.

I appreciate your answers,
thanks

As for Live streams, we simply didn’t have the encoder tools available that could easily create adaptive streams and upload them to CDNs in realtime. On the other hand, Windows Media-ready live encoders (such as the Digital Rapids Stream encoders used by NBC) were widely available and of great quality.

As for your last question… Adaptive streaming to Silverlight was done entirely over HTTP using progressive download. This was made possible by Silverlight’s low-level MediaStreamSource API which allows you to download data using .NET methods, parse it and hand it off to Silverlight to decode and render. Engineering the same functionality in WMP would’ve required an additional ActiveX plugin, at the very least.

One question I have is why adaptive streaming only works with Highlights/Encore content. What is the challenge with Live/Rewind content?

Also, you mentioned that Live video is streamed via WMS HTTP streaming protocol. And you also mentioned “If you’re using the WMP plugin, you always get the WMS stream, regardless of content type.”

How is this different for Silverlight? When the content is not live, does it delivered via Progressive Download? if it does, why is this the case?

I appreciate your answer,
thanks

I am working with some folks in the Silverlight product team to see if we could get more detailed descriptions or sample code of MediaStreamSource online soon. Stay tuned.