MediaStreamSource Takes On a New Life

When the Silverlight team originally designed the MediaStreamSource API, its main purpose was to allow asynchronous reading of compressed video/audio samples from formats other than ASF. We took full advantage of this API to implement Smooth Streaming support in Silverlight 2. In Silverlight 3, the team decided to extend the API to also allow reading of uncompressed samples – YV12, RGBA and PCM. The primary goal behind this was to allow developers to build their own codecs. If you could parse a format and decode it in .NET – you could now play it back in Silverlight.

But one of the other potential uses for MediaStreamSource that emerged during SL3 development was video and audio synthesis. After all, why limit A/V creation to just decode from existing content? You can create a sound waveform or a raster bitmap using .NET math functions and then present it to the Silverlight runtime to render like any other audio or video.

Well, I was extremely happy to find out today that developers are catching on to this fantastic new feature. Namely, Pete Brown, a Washington DC-based .NET developer and evangelist has been using MediaStreamSource to synthesize video, audio and – my favorite – emulate a Commodore 64 computer!

Check out:

Creating Sound using MediaStreamSource in Silverlight 3 Beta

Silverlight 3 – Creating Video from Raw Bits using a MediaStreamSource

My MIX09 Silverlight 3 ShowOff Video – Commodore 64 Emulator

All awesome stuff. Way to go, Pete!

But what about custom codec development? If you’re a codec developer, I invite you to take a look at MediaStreamSource and consider writing a C# decoder for Silverlight. There are plenty of open-source codecs and formats out there that would make for fantastic Silverlight demos. Just to list a few:

Containers

• Matroska
• Ogg
• Ogg Media

Video

• Dirac
• Theora
• HuffYUV
• Lagarith

Audio

• Vorbis
• FLAC
• Monkey’s Audio
• Shorten

These are just some of the codecs and formats out there with easily accessible source code that could be ported to C# or another .NET language. But of course, why stop there? There are also formats such as MPEG-2 TS, FLV, AVI, and codecs such as H.263, MPEG-4 ASP, MJPEG, MPEG Audio Layer II, and others that would be incredibly useful to have supported in Silverlight too.

Will you be the first to develop those?