Silverlight Wireshark Analysis

Silverlight smooth streaming

Wireshark analysis

Yossi Cohen

www.dsp-ip.comFast Forward Your Development

Yossi Cohen

DSP-IP

SILVERLIGHT INTRODUCTION


Smooth Streaming

• Microsoft’s implementation of HTTP-based adaptive streaming

• A hybrid media delivery method that acts like streaming but is in fact a series of short progressive downloads


• Leverages existing HTTP caches• Client can seamlessly switch video quality and bit

rate based on perceived network bandwidth and CPU resources

Streaming or Progressive Download?

Traditional Streaming

• Responsive User Experience

• Bandwidth Use• User Tracking

Progressive Download

• Works from a Web Server

• World-wide scale w/HTTP


Challenges

• No cache-ability• Separate,

smaller streaming networks

Challenges

• Limited User Experience

• User tracking• Bandwidth Use

(20% watched)

Smooth Streaming Design

• Smooth Streaming File Format based on MP4 (ISO Base Media File Format)

• Video is encoded and stored on disk as one contiguous MP4 file▫ Separate file for each bit rate

• Each video Group of Pictures (GOP) is stored in a


• Each video Group of Pictures (GOP) is stored in a Movie Fragment box▫ This allows easy fragmentation at key frames

• Contiguous file is virtually split up into chunks when responding to a client request

Content Provider Benefits

• Cheaper to deploy

▫ Can utilize any generic HTTP caches/proxies

▫ Doesn’t require specialized servers at every node

• Better scalability and reach

▫ Reduces “last mile” issues because it can


▫ Reduces “last mile” issues because it can dynamically adapt to inferior network conditions

• Audience can adapt to the content, rather than requiring the content providers to guess which bit rates are most likely to be accessible to their audience

End User Benefits• Fast start-up and seek times

▫ Start-up/seeking can be initiated on the lowest bit rate before moving up to a higher bit rate

• No buffering, no disconnects, no playback stutter

▫ As long as the user meets the minimum


▫ As long as the user meets the minimum bit rate requirement

• Seamless bit rate switching based on network conditions and CPU capabilities.

• A generally consistent, smoothplayback experience

Evolution

• Previous versions of MS streaming divide the file into many chunkc 0001.vid 0002.vid etc

• Problematic in caching, CDNs, CMS etc

• Today all fragments of a file are contained in a single bitstream container. Typically 1 fragment = 1 video GOP.


= 1 video GOP.

SILVERLIGHT FILES


Containers & Configuration files

Format options

• ASF/WMV – native Microsoft Format

• MPEG4 File-Format

• AVI

• OGG


MP4 over ASF file format• MP4 is a lightweight container format with less

overhead than ASF

• MP4 is easier to parse in managed (.NET) code

• MP4 is based on a widely used standard, making 3rd party adoption and support easier

• MP4 has native H.264 video support


• MP4 has native H.264 video support

• MP4 was designed to natively support payload fragmentation within the file

MP4 File format

• MP4 has two format types

▫ Disk Format - for file storage

▫ Wire format - for transport

• Wire format enables easy CDN support and integration


Smooth Streaming File Format


Smooth Streaming Wire Format


File extensions

• Media Files

▫ *.ismv - Audio & Video

▫ *.isma – Audio only

• Manifest Files

▫ *.ism – Server manifest. Describes to the server Relation between tracks, bitrates & files on disk.


Relation between tracks, bitrates & files on disk. Based on SMIL 2.0 XML format specification

▫ *.ismc – Describes to the client the available streams, CODECS used, bitrates encoded, video resolutions, markers, captions. First file delivered to client. It’s the first file delivered to client (“SDP” like).

Directory Structure

Manifest Files

Media file in different bitrates


Manifest files• VC-1, WMA, H.264 and AAC codecs

• Text streams

• Multi-language audio tracks

• Alternate video & audio tracks (i.e. multiple camera angles, director’s commentary, etc.)

• Multiple hardware profiles (i.e. same bitrates


• Multiple hardware profiles (i.e. same bitrates targeted at different playback devices)

• Script commands, markers/chapters, captions

• Client manifest Gzip compression

• URL obfuscation

• Live encoding and streaming

ISM file sample<?xml version="1.0" encoding="utf-16" ?>

- 

- <smil xmlns="http://www.w3.org/2001/SMIL20/Language">

- <head>

<meta name="clientManifestRelativePath" content="NBA.ismc" />

</head>

- <body>

- <switch>

- <video src="NBA_3000000.ismv" systemBitrate="3000000">


<param name="trackID" value="2" valuetype="data" />

</video>



</video>



</video>

ISM file sample- <video src="NBA_1300000.ismv" systemBitrate="1300000">


</video>



</video>



</video>


- <audio src="NBA_3000000.ismv" systemBitrate="64000">


</audio>

</switch>

</body>

</smil>

*.ISMC sample<?xml version="1.0" encoding="utf-16" ?>

- 

- <SmoothStreamingMedia MajorVersion="1" MinorVersion="0" Duration="4084405506">

- <StreamIndex Type="video" Subtype="WVC1" Chunks="208" Url="QualityLevels({bitrate})/Fragments(video={start time})">

<QualityLevel Bitrate="3000000" FourCC="WVC1" Width="1280" Height="720" CodecPrivateData="250000010FD3FE27F1678A27F859E80C9082DB8D44A9C00000010E5A67F840" />

<QualityLevel Bitrate="2400000" FourCC="WVC1" Width="1056" Height="592"


<QualityLevel Bitrate="2400000" FourCC="WVC1" Width="1056" Height="592" CodecPrivateData="250000010FD3FE20F1278A20F849E80C9082493DEDDCC00000010E5A67F840" />

<QualityLevel Bitrate="1800000" FourCC="WVC1" Width="848" Height="480" CodecPrivateData="250000010FCBF81A70EF8A1A783BE80C908236EE5265400000010E5A67F840" />

<QualityLevel Bitrate="1300000" FourCC="WVC1" Width="640" Height="352" CodecPrivateData="250000010FCBE813F0AF8A13F82BE80C9081A7ABF704400000010E5A67F840" />

SILVERLIGHT SESSION


Initiation and Flow

Smooth Streaming Protocol

• Smooth Streaming Protocol uses HTTP [RFC2616] as its underlying transport .

• The Server role in the protocol is stateless

▫ Enabling (potentially) different instance of the server to handle client requests


server to handle client requests

▫ Request can utilize any generic HTTP caches/proxies - > Lowering CDN costs

Messages

• Smooth Streaming Protocol uses 4 different messages:

▫ Manifest Request

▫ Manifest Response

▫ Fragment Request

▫ Fragment Response


▫ Fragment Response

• All messages follow the HTTP/1.1 specification

Messages Flow

Server Client

Manifest Request

Manifest Response

Fragment Request


Fragment Response

Fragment Request(s)

Messages

• Manifest Request and Fragment Request message MUST use the HTTP "GET" method, generated by the client.

• Manifest Request and Fragment Request message use the HTTP Response messages.


message use the HTTP Response messages.

Status-Code SHOULD be 200.

Smooth Streaming Transport

Protocol Session Details

Manifest Request

Manifest ResponseVideo Fragment Request

Fragment ResponseAudio Fragment Request


Session Details - Manifest Request

• In order to initiate a presentation the Client MUST send the server a Manifest Request using the HTTP GET method.


the HTTP GET method.

Session Details - Manifest Response

• The Response is a ISMC Manifest file describing the session. - <SmoothStreamingMedia MajorVersion="1" MinorVersion="0" Duration="5965419999">


<QualityLevel Bitrate="2750000" FourCC="WVC1" Width="1280" Height="720" CodecPrivateData="250000010FD3BE27F1678A27F859E804508253EBE8E6C00000010E5AE7F840" />

…..

<c n="0" d="20000000" />

<c n="1" d="20000000" />


<c n="1" d="20000000" />

.....

<c n="297" d="20000000" />

<c n="298" d="5000001" />

</StreamIndex>

- <StreamIndex Type="audio" Subtype="WmaPro" Chunks="299" Url="QualityLevels({bitrate})/Fragments(audio={start time})">

<QualityLevel Bitrate="64000" WaveFormatEx="6201020044AC0000451F0000CF05100012001000030000000000000000000000E00042C0" />

<c n="0" d="20433560" />

....

<c n="297" d="20433560" />

<c n="298" d="4393197" />

</StreamIndex>

</SmoothStreamingMedia>

Manifest Response reviewed• We can see in the ISMC file that the server can support 8 different levels

of quality (bitrate) for the client can chose from between 2.75Mbit to 0.35 Mbit.

- <SmoothStreamingMedia MajorVersion="1" MinorVersion="0" Duration="5965419999">


<QualityLevel Bitrate="2750000" FourCC="WVC1" Width="1280" Height="720" CodecPrivateData="250000010FD3BE27F1678A27F859E804508253EBE8E6C00000010E5AE7F840" />

<QualityLevel Bitrate="2040000" FourCC="WVC1" Width="1056" Height="592" CodecPrivateData="250000010FD3BE20F1278A20F849E80450823E414DD1400000010E5AE7F840" />

<QualityLevel Bitrate="1520000" FourCC="WVC1" Width="848" Height="480" CodecPrivateData="250000010FCBAE1A70EF8A1A783BE8045081AE62F3F7400000010E5AE7F840" />


CodecPrivateData="250000010FCBAE1A70EF8A1A783BE8045081AE62F3F7400000010E5AE7F840" />

<QualityLevel Bitrate="1130000" FourCC="WVC1" Width="704" Height="400" CodecPrivateData="250000010FCBA215F0C78A15F831E8045081A27BD635C00000010E5AE7F840" />

<QualityLevel Bitrate="845000" FourCC="WVC1" Width="576" Height="320" CodecPrivateData="250000010FCB9A11F09F8A11F827E804508199C94077400000010E5AE7F840" />

<QualityLevel Bitrate="630000" FourCC="WVC1" Width="448" Height="256" CodecPrivateData="250000010FCB920DF07F8A0DF81FE804508113396020C00000010E5AE7F840" />

<QualityLevel Bitrate="470000" FourCC="WVC1" Width="368" Height="208" CodecPrivateData="250000010FC38E0B70678A0B7819E80450810E5747B6C00000010E5AE7F840" />

<QualityLevel Bitrate="350000" FourCC="WVC1" Width="320" Height="176" CodecPrivateData="250000010FC38A09F0578A09F815E80450808AADEACF400000010E5AE7F840" />

Manifest Response – reviewed• The client also receives the number of chunks for audio and video tracks

and the duration of each chunk so it can request the chunk which fits the desired position in the file

<c n="0" d="20000000" />

<c n="1" d="20000000" />

<c n="2" d="20000000" />

<c n="3" d="20000000" />

....

<c n="297" d="20000000" />

<c n="298" d="5000001" />

</StreamIndex>


</StreamIndex>

- <StreamIndex Type="audio" Subtype="WmaPro" Chunks="299" Url="QualityLevels({bitrate})/Fragments(audio={start time})">

<QualityLevel Bitrate="64000" WaveFormatEx="6201020044AC0000451F0000CF05100012001000030000000000000000000000E00042C0" />

<c n="0" d="20433560" />

<c n="1" d="19969161" />

<c n="2" d="19969161" />

<c n="3" d="20433560" />

<c n="4" d="20433560" />

<c n="297" d="20433560" />

<c n="298" d="4393197" />

</StreamIndex>

</SmoothStreamingMedia>

Session Details – Fragment Request

• Client-Server requests are based on RESTFull URLs:GET /mediadl/iisnet/smoothmedia/Experience/BigBuckBunny_720p.ism/QualityLevels(350000)/Fragments(video=0)

• The URL includes reference to:


• The URL includes reference to:

▫ Bitrate as QualityLevels which maps to a media file

▫ Fragment number

Session Details – Fragment Response

• The Server:

▫ checks “BigBuckBunny_720p.ism” server manifest file to find the media file associated with the quality level(350000)

▫ Opens and parses the associated media file to get the chunk with requested time offset (0).


requested time offset (0).

▫ Sends the requested media fragment to the client as HTTP response with status code set to 200

DSP-IP Contact information

For Course materials licensing & lecture request contact: For Course materials licensing & lecture request contact:

For projects development services contact: For projects development services contact:

Adi YakovAdi YakovTraining and Courses ManagerTraining and Courses Manager

adi@[email protected]

++972972--99--86519338651933


wwwwww..dspdsp--ipip..comcomMailMail :: info@[email protected]:: ++972972--99--88509568850956,,FaxFax :: ++972972--5050-- 89629108962910

Alona AshkenaziAlona Ashkenazi

Technology Management Technology Management Services Services : :

alona@[email protected]

+972+972--99--88509568850956

Refrences

• Most valuable refrence: http://alexzambelli.com/blog/2009/02/10/smooth-streaming-architecture/


DSP-IP Contact information

For Course materials licensing & lecture request contact: For Course materials licensing & lecture request contact:

For projects development services contact: For projects development services contact:

Adi YakovAdi YakovTraining and Courses ManagerTraining and Courses Manager

adi@[email protected]

+972+972--99--86519338651933


wwwwww..dspdsp--ipip..comcomMailMail :: info@[email protected]:: ++972972--99--88509568850956,,FaxFax :: ++972972--5050-- 89629108962910

Alona AshkenaziAlona Ashkenazi

Technology Management Technology Management Services Services : :

alona@[email protected]

+972+972--99--88509568850956