It’s been too long since my last post, so I thought I’d get warmed up and back into blogging form by posting something about recent releases on the Microsoft media platform.
After PDC 2010 in November which focused on new web technologies (e.g. IE9, HTML5, WP7), we hosted a special event in December called Silverlight Firestarter which focused exclusively on Silverlight. Scott Guthrie announced some of the new features going into Silverlight 5. Among the many features, the ones most interesting to media developers were:
- Support for GPU-accelerated H.264 decoding, intended to make H.264 video perform better on low powered computers and increase battery life on laptops and netbooks.
- Variable speed playback, also known as “trick play”, which will allow you to fast forward and rewind videos as well as play them back at up to 2x speed while maintaining the audio pitch. This is a feature particularly useful for e-learning, as well as suffering through boring corporate training videos. Note that some aspects of trick play, such as rew/ffwd, have already been implemented in the Silverlight Media Framework for Smooth Streaming videos.
- OS power scheme awareness. This will allow you to watch long videos in Silverlight without worrying about your computer going to sleep during the season finale of “Dexter”.
- Remote control support for media scenarios. I presume this will include support for Media Center remote controls.
- Advanced DRM management. I don’t know much about this, but I’m guessing it has to do with supporting seamless key rotations.
Other noteworthy SL5 features include 64-bit support and improved GPU-accelerated graphics APIs.
Around the same time we shipped Windows Phone 7 which uses Silverlight as one of its two app development platforms (the other one being XNA). WP7 support ended up being one of the main features of the Smooth Streaming Client 1.5 (aka SSME) too. Vishal Sood wrote a great post about encoding video for Windows Phone 7. The biggest difference between encoding for WP7 versus the desktop is WP7 video driver’s current lack of support for mid-stream resolution changes. What this effectively means is that all streams in a Smooth Streaming asset delivered to WP7 must be encoded at the same resolution. We hope to fix this in future WP7 updates, but for now we have to live with it.
2010 also saw the release of IIS Media Services 4.0. The most exciting new feature of this release is support for dynamic repackaging and delivery of Smooth Streaming assets (VOD and live) to Apple iOS devices such as the iPhone and iPad. Apple iOS devices support HTTP Live Streaming, an HTTP adaptive streaming technology developed by Apple and based on the MPEG-2 Transport Stream file format. Because both Microsoft and Apple support H.264/AAC codecs, it is possible to repackage (remux) Smooth Streaming H.264/AAC assets into Apple HTTP Live assets by simply changing the container formats, without a need for re-encoding the video and audio streams. The IIS Media Services 4.0 release also supports live DVR features, AES encryption, archiving and B-frame encoding (H.264 Main/High Profile) for iPhone4 and iPad.
Apple has documented some of their own encoding best practices for iOS devices, though frankly, they seem a little on the conservative side. I’ve seen many online reports of iPhone4/iPad being able to handle H.264 High Profile at 1280×720, so it seems strange that Apple would limit their recommendations to just Main Profile at 640×360 (especially considering iPad’s display resolution is 1024×768). If you have experience with streaming to iOS devices and would like to share your encoding settings and feedback, please leave a comment. Even better – if you work for Apple and your focus is HTTP Live Streaming, send me an e-mail.
IIS Media Services 4.0 also includes support for low-latency streaming which can bring down the typical Live Smooth Streaming end-to-end latency to under 2 seconds. This requires the latest Smooth Streaming Client version and a low-latency compatible realtime video encoder, such as the ones made by Allegro DVT, Digital Rapids, Elemental, Envivio and Inlet. (Did I forget your company? Let me know.)
Though not officially a part of the IIS Media Services package, the IIS Media team also recently released IIS Transform Manager 1.0 alpha, another IIS extension which allows you to automate your transcoding/transmuxing workflow. The most recent public release is just an alpha, but shows a lot of promise. It features integration with Microsoft Expression Encoder (for transcoding to WMV, MP4 and Smooth) and offline transmuxing (repackaging) of Smooth VOD assets into Apple HTTP adaptive assets. The final release, coming soon, will also feature support for distributed transcoding, DRM packaging tasks and – my favorite – MP4-to-Smooth tasks. The last one is particularly exciting because it will allow you to encode to Smooth Streaming using any H.264 encoder regardless of its lack of Smooth Streaming file support. For example, you’ll be able to encode videos with the awesome open-source x264 encoder and automatically convert its MP4 output files into Smooth Streaming assets as a background task.
Completing the slew of recent media releases are:
- Silverlight Media Framework 2.3
- New SMF features include support for WP7 and SSME 1.5.
- Silverlight Rough Cut Editor 1.0 SP1
- Native Extensions 1.0 for Microsoft Silverlight
- While not necessarily directly related to media, NESL provides features in out-of-browser Silverlight apps that aren’t available in the browser, such as integration with Windows sensors, access to portable devices, webcam capture/encoding/archiving, speech recognition and Windows 7 taskbar integration.
Last, but certainly not the least, stay tuned for the upcoming Microsoft Expression Encoder 4 service pack which will enable GPU-accelerated H.264 encoding using NVIDIA CUDA technology. Expression Encoder Pro’s H.264 compression engine is powered by MainConcept, and the new acceleration feature will be powered by MainConcept’s CUDA H.264 Encoder SDK. SP1 is scheduled for release in Q1 2011.