Smooth Streaming FAQ

Editor’s note: This Smooth Streaming FAQ was last updated in 2012, and most of the information in it reflects the state of technology around 2008-2012. Though its content is likely no longer relevant I have decided to keep the page alive for archival purposes.

Q: What is Smooth Streaming?
A: Smooth Streaming is Microsoft’s implementation of HTTP-based adaptive streaming.

Q: What is adaptive streaming?
A: It is a hybrid content delivery method which acts like traditional streaming but is in fact based on HTTP progressive download. In a nutshell, it involves the content being delivered from the server to the client as a series of short HTTP downloads, rather than one long download or a constant-bitrate stream. Because the content is typically encoded at multiple bitrates and delivered as a series of small “chunks”, the client can also dynamically switch between chunks of different sizes (quality levels) depending on network bandwidth and CPU capability, allowing users to always receive the best level of video quality they possibly can.

Q: How is Microsoft enabling this new technology? Do I need to install a new plug-in or something?
A:
This depends on what you’re trying to do with Smooth Streaming:

  • If you are an end user – you don’t need to do anything except have Silverlight installed on your PC or Mac. As of Moonlight 2.0 beta, Smooth Streaming is also supported on the Linux platform.
  • If you are a content creator, you will need to upgrade your encoding software to one that supports encoding to the new Smooth Streaming file format (based on MP4). This currently includes the entry-level Expression Encoder 4, as well as enterprise-level encoding products from Cisco (formerly Inlet Technologies), Envivio, ElementalDigital RapidsTelestream (formerly Anystream) and Harmonic (formerly Rhozet).
  • If you are a hosting provider or CDN, you will need to install IIS Media Services on your Windows Server 2008 or R2 origin servers. This component allows IIS7 to serve Smooth Streaming media assets to clients. Note that you will only need to install this on your origin servers – your edge cache/proxy infrastructure can remain unchanged, though you may need to configure it to correctly pass through Smooth Streaming RESTful URLs.
  • If you are a Silverlight application developer, you will need to add Smooth Streaming playback support to your Silverlight application. The easiest way to accomplish this is by using the MMP Player Framework with Smooth Streaming Client, or by using Silverlight player templates included with Expression Encoder.

Q: Where can I see Smooth Streaming in action?
A:
Visit the IIS.net demo which features HD video at quality up to 720p and lets you simulate variable network bandwidth during playback. Media service provider iStreamPlanet has a great demo of multi-camera playback (with the ability to dynamically change camera angles) and a presentation player (slides + video). For a demo of live Smooth Streaming, check out Home Shopping Network‘s 24-hr live shopping channel, also offering up to 720p in full-screen mode.

Q: Is Smooth Streaming an exclusive feature of Silverlight?
A: No. Though Silverlight is considered the preferred premium client for Smooth Streaming, the streaming technology itself is client-agnostic and can be implemented on any client platform that supports VC-1/WMA/H.264/AAC decoding and HTTP transport.

Q: How can you do live streaming from a Web server? I thought live streaming could only work with streaming protocols such as RTSP?
A:
No, live streaming using plain HTTP is absolutely possible. In fact, from an architectural standpoint, live and on-demand Smooth Streaming are surprisingly similar. Encoders upload video/audio fragments to the IIS7 ingest server in real-time as a long HTTP post. The IIS7 server, upon receiving all the expected fragments for a given timestamp, appends those fragments to the Fragmented MP4 (*.ismv) files on disk and updates the manifest with the most recent timestamps. The client downloads the most recent manifest from the server when seeking or connecting for the first time, and maintains an internal incremental manifest the rest of the time.

Q: Is Smooth Streaming backwards compatible with Windows Media Services or Windows Media Player?
A:
No, not at this time. Smooth Streaming serving requires at least IIS7 and IIS Media Services, whereas Smooth Streaming playback is officially supported in Silverlight applications and native applications based on the Smooth Streaming Client Porting Kit. Other 3rd party client implementations of Smooth Streaming are also possible.

Q: How is this streaming technology different from Windows Media MBR (multiple bit rate) streaming?
A:
Windows Media Services uses traditional streaming protocols like RTSP and WMS HTTP to deliver content to clients. Content streamed with such protocols is always streamed at the rate at which it’s encoded at, which also makes it very difficult for the clients to detect overall network bandwidth and adapt to changing network conditions in a timely fashion. This means that deteriorating network conditions can’t be detected until it’s too late and the bit rate cannot be safely upgraded if the network conditions improve.

Smooth Streaming, on the other hand, is based entirely on HTTP download. This allows the clients to constantly evaluate download speeds and adapt the downloaded bit rate to the changing network conditions. Furthermore, because the Internet was predominantly built for HTTP communication, Smooth Streaming doesn’t require specialized caches and proxies and can take advantage of existing Web caches/proxies, such as the ones already built out by CDNs and ISPs. This improves scalability, accessability (no firewall issues) and reach.

Q: Does Smooth Streaming support H.264/AAC?
A:
Yes. However, delivery of H.264/AAC-encoded Smooth Streaming content to Silverlight clients requires Silverlight 3 or better, since H.264/AAC support is a new playback feature of Silverlight 3.

Q: Does Smooth Streaming support DRM content protection?
A:
Yes. PlayReady Server and PC SDKs are available to all encoder ISVs and DRM ASPs wishing to implement DRM protection for Smooth Streaming. For end users, PlayReady DRM service providers offer both on-the-fly live PlayReady DRM encryption of Smooth Streaming streams as well as offline encryption of on-demand Smooth Streaming content.

Q: Does Smooth Streaming really split up video files into thousands of chunks?
A:
No, not on disk. Encoders that support the Smooth Streaming file format produce contiguous MP4 files (1 file per encoded bitrate) with file extensions *.ismv or *.isma, and manifest XML files with file extensions *.ism and *.ismc. The minimum number of files in a Smooth Streaming asset is therefore 3 files. Typically encoded assets usually include 6-8 files.
When Smooth Streaming clients request time-based slices from the IIS7 server, the contiguous MP4 files are virtually split up into fragments (chunks) by the server and sent over the wire as individual files. This allows the client requests to be cached, enabling greater scalability and efficiency.

Q: Will Smooth Streaming ensure I can deliver HD video to all my customers?
A:
No. Smooth Streaming can’t perform any miracles; it’s simply an enhanced HTTP delivery method. Delivery of HD video is still bound by the efficiency of the codecs used: VC-1, H.264, etc. For example, most video experts would agree that 720p/24 video can’t be delivered at a satisfactory quality level with less than 2 Mbps encoded bitrate. Well, 2 Mbps of encoded video still requires 2 Mbps of network bandwidth in order to be delivered in real-time – and there’s nothing IIS7 or Silverlight can do to change that. After all, if there existed a way of delivering 2 Mbps of data in real-time at only 1 Mbps – wouldn’t we all be using that technology? 🙂

What Smooth Streaming does allow, however, is for content providers to provide HD video to those customers who are able to receive it (sufficient network bandwidth and CPU power) without sacrificing the overall quality of the video or catering to the lowest common quality denominator. In other words, with Smooth Streaming content providers no longer have to guess the capabilities of their audience – instead they can simply encode to a wide range of bit rates and let the customers automatically adjust to the ones that suit them best.

Q: Didn’t Microsoft already implement adaptive streaming technology once before?
A:
Yes, Microsoft developed a prototype implementation of adaptive streaming for the NBC Olympics 2008 video site. Smooth Streaming represents Microsoft’s productization of that work.

Q: Will I need to re-encode all my content for Smooth Streaming?
A:
Unfortunately, yes. Even though Smooth Streaming uses standard codecs, it requires exact temporal alignment between streams of different bit rates, as well as other encoding constraints such as closed GOPs, entry point headers, etc. We therefore recommend that you re-encode your content for Smooth Streaming from the original sources.

Q: Is Smooth Streaming based on Microsoft’s ASF file format? Or does it use some new proprietary file format?
A:
No, the Smooth Streaming File Format is based on the ISO-standardized Base Media File Format, better known as the MP4 file format. It follows the ISO/IEC 14496-12 specification.

Q: Is Smooth Streaming based on new codecs? Is it restricted only to VC-1?
A:
No. Smooth Streaming supports industry-standard VC-1 and H.264 (MPEG-4 AVC) video codecs, as well as WMA and AAC audio codecs. Note that delivery of H.264/AAC-encoded Smooth Streaming content to Silverlight clients requires Silverlight 3, since H.264/AAC support is a new playback feature of Silverlight 3.

Q: Why can’t I play Smooth Streaming files in my MP4 player even though you claim it’s based on the MP4 file format?
A:
The Smooth Streaming file and wire formats heavily utilize Movie Fragments, a particular MP4 box structure that was originally designed with live encoding in mind. The ISO Base Media File Format specification (better known as the MP4 file format specification) covers a lot of format options and allows for a large degree of implementation flexibility. Consequently, not all developers and manufactures implement the full MP4 specification, so it’s possible that your particular MP4 reader doesn’t support MP4 Movie Fragments (“moof” and “mdat” boxes) and Smooth Streaming’s specific MP4 box structure.

Furthermore, even though SMPTE VC-1 has been officially approved for storage in MP4 containers by the MP4 Registration Authority, the MP4 file format is largely associated with H.264 video and AAC audio codecs and therefore many MP4 players only accept those codecs as valid. Finally, WMA Standard and Professional audio codecs have not been officially approved for storage in MP4 files by any standards organization at this moment so it’s possible that they may be unsupported by your particular platform.