MediaStreamSource Takes On a New Life

When the Silverlight team originally designed the MediaStreamSource API, its main purpose was to allow asynchronous reading of compressed video/audio samples from formats other than ASF. We took full advantage of this API to implement Smooth Streaming support in Silverlight 2. In Silverlight 3, the team decided to extend the API to also allow reading of uncompressed samples – YV12, RGBA and PCM. The primary goal behind this was to allow developers to build their own codecs. If you could parse a format and decode it in .NET – you could now play it back in Silverlight.

But one of the other potential uses for MediaStreamSource that emerged during SL3 development was video and audio synthesis. After all, why limit A/V creation to just decode from existing content? You can create a sound waveform or a raster bitmap using .NET math functions and then present it to the Silverlight runtime to render like any other audio or video.

Well, I was extremely happy to find out today that developers are catching on to this fantastic new feature. Namely, Pete Brown, a Washington DC-based .NET developer and evangelist has been using MediaStreamSource to synthesize video, audio and – my favorite – emulate a Commodore 64 computer!

Check out:

Creating Sound using MediaStreamSource in Silverlight 3 Beta

Silverlight 3 – Creating Video from Raw Bits using a MediaStreamSource

My MIX09 Silverlight 3 ShowOff Video – Commodore 64 Emulator

All awesome stuff. Way to go, Pete!

But what about custom codec development? If you’re a codec developer, I invite you to take a look at MediaStreamSource and consider writing a C# decoder for Silverlight. There are plenty of open-source codecs and formats out there that would make for fantastic Silverlight demos. Just to list a few:

Containers

  • Matroska
  • Ogg
  • Ogg Media

Video

  • Dirac
  • Theora
  • HuffYUV
  • Lagarith

Audio

  • Vorbis
  • FLAC
  • Monkey’s Audio
  • Shorten

These are just some of the codecs and formats out there with easily accessible source code that could be ported to C# or another .NET language. But of course, why stop there? There are also formats such as MPEG-2 TS, FLV, AVI, and codecs such as H.263, MPEG-4 ASP, MJPEG, MPEG Audio Layer II, and others that would be incredibly useful to have supported in Silverlight too.

Will you be the first to develop those?

About Alex Zambelli

Alex is a Principal Product Manager at iStreamPlanet Co. in Redmond, Washington. Prior to his current job he was a Technical Evangelist for Microsoft Media Platform at Microsoft Corporation. He specializes in video streaming, adaptive HTTP streaming, VC-1 and H.264 video, and video processing best practices.
This entry was posted in Silverlight and tagged , , . Bookmark the permalink.

13 Responses to MediaStreamSource Takes On a New Life

  1. Pete says:

    Thanks Alex!

    Pete

  2. Curtis says:

    Any idea if Smooth Streaming output works with the player in Windows Media Center?

    • No, not yet. Silverlight is currently the only supported client platform, but that’s likely to change in the future. Because the format spec is so new there was no time to get it into Windows 7.

  3. Richie says:

    I am currently in the process of porting OGG Decoder into Silverlight MediaStreamSource. I already have it done in C#, just need to now port it into SL.

    :-)

  4. I try using MediaStreamSource for VC-1 Playback speed.

    Playback speed is very important for korean students. (I’m korean)

    Korean Students study very very hard. they have no time for normal speed, online lecture. usually they play vod lecture 1.5 or 2x speed.

    but I found out yesterday that I have no chance for using Silverlight Decoder output.

    If I don’t use raw a/v, silverlight decoder, I must make(or port) a decoder. but performance issue.

    May I have any method, access raw a/v ?

  5. buchi says:

    youve got a nice blog here buddy, bookmark it already. ^_^

  6. sam says:

    I have custom ASF stream coming from Windows Media Server. The custom stream contains mpeg4 part2, mjpeg or proprietary format.

    How to play custom ASF stream with proprietary format data and other formats data which is not directly supported by silverlight?

    Does MediaStreamSource help?

    • Yes, MediaStreamSource could be of help, but you’d need to write an ASF streaming receiver/parser in .NET, plus managed code decoders for any codecs not natively supported in Silverlight (in other words, anything that’s not H.264, VC-1 or WMV). The extensibility framework is there – somebody just needs to take the plunge and write the necessary code.

  7. i have written a custom mediastreamsource, that can play media from growing source files (mpeg transport streams).
    Once it reaches the end of its mediastream, it reads the new duration from the mediafile and continues to deliver samples. The MediaElement plays continously.

    Unfortunately i haven´t found a way to update the MediaElement.NaturalDuration property. Hence i cannot seek into the “reloaded” area, because ME doesn´t know about it and sets my position change to its NaturalDuration value.

    I tried to call ReportOpenMediaCompleted after getting the new stream length. Then Naturalduration get´s updated, but i cannot play anymore.

    Is there any other way to deal with it ?

    Many thanks

    Tilo

  8. Sometime life can be so easy :-)

    I solved it giving MediaElement a “fantasy” duration value when initializing my MediaStreamSource :

    protected override void OpenMediaAsync()
    {

    mediaSourceAttributes[MediaSourceAttributesKeys.Duration] = TimeSpan.FromHours(10).Ticks.ToString(CultureInfo.InvariantCulture);
    this.ReportOpenMediaCompleted(mediaSourceAttributes, mediaStreamDescriptions);
    }

    The only thing left to do was to update my slider control with the “real” duration. Now it works like a charm …

  9. Let me check this. I´ll keep you informed …

  10. Just wanted to add to this discussion to say that I’ve developed something similar to Tilo – a MediaStreamSource that handles HTTP Live Streaming (segmented Mpeg-1 Transport Streams, often used to stream to iPhones/iPads/etc.)

    I’ve not seen it done anywhere else, so quite excited about it! Means that you can now stream seekable video to Silverlight using just a normal web server.

    If you want to check it out, I’ve posted a live demo and downloadable sample – it’s called ‘SilverLive’ and is at my website. (click on my name above, I assume!)

    Carl