A Brief History of Multi-Bitrate Streaming

It’s been a while since I last posted. Probably the most significant news from Microsoft regarding Silverlight media since IBC (when we announced H.264/AAC support in Silverlight 3) has been on the IIS and Expression fronts. The Expression Encoder team released Service Pack 1 for Encoder 2 in October, which really should’ve been called a 2.5 release considering all the new feature additions: Silverlight 2 templates; simple H.264/AAC encoding for devices; publishing to servers via WebDAV; and most importantly, on-demand encoding to the new Smooth Streaming format. Read more about the SP1 release on Expression Encoder Team’s and James Clarke’s blogs.

The IIS Media team, meanwhile, released IIS Media Pack 1.0, a free download for IIS7 (Windows Server 2008) which adds media features to IIS. The first media pack release features bitrate throttling and web playlists. But the big news came on October 28th at Digital Hollywood when we announced that Media Services 2.0 for IIS7 will feature a new HTTP-based adaptive streaming technology named Smooth Streaming. In order to promote the new technology we launched a showcase website in partnership with Akamai Technologies – SmoothHD.com. Akamai will also offer the first media delivery service based on Smooth Streaming, named Akamai AdaptiveEdge Streaming for Microsoft Silverlight. The better the technology, the longer the product name. 🙂

For more details on IIS media extensions, visit Chris Knowlton’s, Vishal Sood’s, and John Bocharov’s excellent blogs.

So with all this talk of adaptive streaming going on lately, it’s highly likely that you find yourself a little lost and confused. What is adaptive streaming? What is Smooth Streaming? Is it related to MBR streaming? Is it backwards compatible with Windows Media? Does it work with Windows Media Services? Is it supported by Silverlight? Where does Move Networks fit into all this?

So before I dive into all the technical details of the new technology, let’s take a step back first and take a look at Microsoft’s history of multi-bitrate streaming and how we got to Smooth Streaming.

The first effort to adapt streams to client conditions was called “stream thinning”, which Microsoft introduced as part of NetShow Services 3.0 and Windows Media Player 6.1 in 1998. Stream thinning automatically detected deteriorating network conditions and decreased the video frame rate in response. In dire network conditions the client could even suspend video playback entirely and stream only audio.

The earliest form of multiple bit rate streaming from Microsoft was introduced in 1999, as part of Windows Media Technologies 4.0 (in Windows NT 4.0) together with Windows Media Player 6.4. The ASF file format allows storage of multiple video and audio tracks inside a single file, and the Windows Media streaming protocols support switching streams during broadcast. This technology is most commonly referred to as Multiple Bit Rate ASF, or simply MBR.

In 2002 Microsoft released the Windows Media 9 Series products. Windows Media Services 9 (in Windows Server 2003) and Windows Media Player 9 introduced an improved MBR technology dubbed Intelligent Streaming. Intelligent Streaming combined bandwidth detection, stream thinning, MBR ASF, and better image handling in Windows Media Player. Intelligent Streaming, of course, still required the media to be encoded as MBR ASF files with a tool such as Windows Media Encoder 9.

While the technology itself was well designed, its implementations suffered from numerous shortcomings. It was limited to streaming only (no progressive download), and only from Windows Media servers. The encoders never required the multiple video streams to be temporally aligned, let alone keyframe aligned, which made switching between streams difficult to do in a seamless fashion. Because the media was streamed, and streaming protocols function at constant rates, it was almost impossible to accurately predict overall client bandwidth – particularly in a timely fashion. By the time poor network conditions were detected, it was usually already too late – the player often went through several iterations of re-buffering before finally downgrading the bitrate. WMP’s heuristics code never fully lived up to its potential. MBR streaming worked great for some people, and less than stellar for others.

Meanwhile… One of the trends that emerged in the streaming media industry over the past several years has been a steady shift away from classic streaming protocols (RTSP, MMS, RTMP, etc.) and back towards plain HTTP download. Need proof? Just visit YouTube.

Why is that? There are several strong reasons for this industry trend:

• Web download services are typically cheaper than media streaming services offered by CDNs and hosting providers.
• Media protocols often run into trouble getting around firewalls and routers because they are commonly based on UDP sockets over unusual port numbers. HTTP-based media delivery has no such problems because every firewall and router knows to let through HTTP downloads over port 80.
• HTTP delivery doesn’t require special proxies or caches. Any web cache will do.
• It’s much easier and cheaper to move HTTP data to the edge of the network, closer to the end users.

Even though streaming protocols were designed with media delivery in mind, the fact of the matter is that the Internet was built on HTTP and optimized for HTTP delivery. So instead of trying to adapt the entire Internet to streaming protocols – why not just adapt media delivery to the Internet instead?

Move Networks, a strategic Microsoft Silverlight partner, seized the opportunity early on by providing exceptional end-to-end HTTP-based media delivery services, and pioneering a new hybrid form of media streaming called adaptive streaming. They proved again and again in 2008 that HTTP-based media delivery can be done successfully on a large scale – both on-demand (ABC, ESPN, Discovery, etc.) and live (Democratic National Convention). Microsoft demonstrated this again during the NBC Olympics, when it prototyped its own HTTP-based adaptive streaming solution.

So what exactly is adaptive streaming and how is it different than classic streaming? Stay tuned for my next blog post. 🙂