Upload
jeff-tapper
View
11.945
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Slides from Streaming Media West Presentation, "How to Create A DASH-264 player for Adobe Flash"
Citation preview
Building a Dash-264 Player
Jeff Tapper / Michael LabriolaDigital Primates
[email protected]@digitalprimates.netTwitter: @jtapper / @mlabriola
Jeff Tapper
• Senior Consultant at Digital Primates– Building next generation client applications
• Built video applications for clients including…
• Developing Internet applications for 17 years• Author of 12 books on Internet technologies
Who am I?
• Michael Labriola• Senior Consultant at Digital Primates– Building next generation client applications
• Built video applications for clients including…well lots…
• Developing Internet applications for 17 years• Author of 5 books on Internet technologies
Agenda
• Video and the Internet today• Understanding HTTP Streaming• What is DASH• Flash as a video platform• The inner workings of DASH• The challenge • Getting video bytes info flash• Introducing the Digital Primates DASH-264 Player• Questions
Video is dominating the Internet• Desktop: Video makes up 50% of traffic at peak periods – notably 30% from Netflix and 11% from YouTube
• Mobile: Video traffic is growing exponentially
Fixed Internet Mobile Internet
HTTP Adaptive Streaming
Media Capture & Encoding
Media Origin Servers HTTP Cache Servers Client Devices
001010100001010
010101010001110
01110100011010101
001010100001010
010101010001110
01110100011010101
001010100001010
010101010001110
01110100011010101
001010100001010
010101010001110
01110100011010101
Encode each segment at multiple bitrates
2 Split the video into small segments
1
Make each segment addressable via a HTTP-URL3
Client makes decision on which segment to download
4
Client splices together and plays back5
HTTP Streaming Landscape
• Apple’s HTTP Live Streaming (HLS)• Microsoft’s Smooth Streaming • Adobe’s HTTP Dynamic Streaming (HDS)• And many more…
The challenge
• Most agree that HTTP Streaming is the most efficient choice
• Different devices support different streaming protocols
• No one standard is currently supported ubiquitously
• Results in media being served in several different formats to support the broadest range of devices
What is MPEG-DASH
DASH – Dynamic Adaptive Streaming via HTTP International open standard, developed and
published by ISO Addresses both simple and advanced use cases Enables highest-quality multiscreen distribution
and efficient dynamic adaptive switching Enables reuse of existing content, devices and
infrastructure Attempts to unify to a single standard for HTTP
Streaming
DASH and codecs
• The DASH specification is codec agnostic• Any existing or future codec can work with
DASH• DASH manifest describes which codec is used– Different codecs store the actual video data
differently
DASH264
• H.264 is dominant format today• Many vendors and service providers are
committed to supporting/enabling DASH264• Provides support for today’s requirements
such as DRM• H.264 is backed by rigorous testing and
conformance
DASH Industry Forum
• Addressing the dramatic growth of broadband video by recommending a universal delivery format that provides end users with the best possible media experience by dynamically adapting to changing network conditions.
DASH Industry Forum
• Objectives: – promote and catalyze market adoption of MPEG-DASH – publish interoperability and deployment guidelines– facilitate interoperability tests – collaborate with standard bodies and industry consortia
in aligning ongoing DASH standards development and the use of common profiles across industry organizations
• Currently 46 members• Visit http://dashif.org for more information
Members
Building a Desktop DASH player
• There are many technologies available for playing video in modern web browsers– Flash– SilverLight– HTML5– Etc.
• We focused on Flash for our first DASH player– Largest percentage of users can support it
Building Video Players in Flash
• Flash Player includes a native video player– flash.media.Video
• Video accepts a source from a Camera or a NetStream– flash.media.Camera– flash.net.NetStream
• Netstream supports:– Streaming FLV file– Manually appending FLV Bytes into its buffer
Flash Video (continued)
• NetStream and Video classes can be manipulated manually
• Open Source frameworks are available for easing the process– Adobe’s Open Source Media Framework (OSMF)– Open Video Player (OVP)
• We have chosen to build on OSMF
OSMF
• Extensible framework with plugin architecture• Native support for DRM• Contains classes to work with HTTP Streaming
technologies– Only HDS is supported natively– Extensible architecture makes it possible to
support new technologies
How to DASH in Flash
• Extract Video Bytes from DASH segments• Wrap bytes in format understood by flash
(FLV)• Append FLV Bytes into Netstream
Understanding DASH structure
• Three types of files– Manifest (.mpd)• XML file describing the segments
– Initialization file• Contains headers needed to decode bytes in segments
– Segment Files• Contains playable media• Includes:
– 0…many video tracks– 0…many audio tracks
DASH Manifest
• Manifest contains:– Program Info (including title)– 1 or more periods
• Periods contain 1 adaptation set per video stream and • Periods contain 1 adaptation set per audio stream• Adaptation Sets contain:
– Content Composition nodes (for each video or audio track)– 1 or more Representation node
» Each representation describes a single bitrate» Representations contain data on finding the actual segments» Different ways a representation can describe segments
Describing Representations
• SegmentBase– Describes a stream with only a single Segment per bitrate– Can be used for Byte Range Requests
• SegmentList– A SegmentList will contain a specific list of each SegmentURL
(individual HTTP packet with media data)– Can be used for Byte Range Requests
• SegmentTemplate– Defines a known url for the fragment with wildcards resolved
at runtime to request a segments (see bbb.mpd)– Alternatively, can specify a list of segments based on duration
SegmentList<Representation id="h264bl_hd" mimeType="video/mp4" codecs="avc1.42c01f" width="1280" height="720" startWithSAP="1" bandwidth="514864"> <SegmentList timescale="1000" duration="10000"> <Initialization sourceURL="mp4-main-multi-h264bl_hd-.mp4"/> <SegmentURL media="mp4-main-multi-h264bl_hd-1.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-2.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-3.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-4.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-5.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-6.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-7.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-8.m4s"/>
SegmentTemplate fixed segment duration
<AdaptationSet> <ContentComponent id="1" contentType="video"/> <SegmentTemplate initialization="BigBuckBunny_720p_1800kbps_44khz_track1_dash.mp4"/> <Representation id="1" mimeType="video/mp4“ codecs="avc1.64001f" width="1280" height="720“ startWithSAP="1" bandwidth="1809954"> <SegmentTemplate timescale="1000" duration="13809" media="bbb_seg_BigBuckBunny_720p_1800kbps_44khz_track1$Number$.m4s" startNumber="1"/> </Representation></AdaptationSet>
SegmentTemplate variable segment duration
<AdaptationSet group="2" mimeType="video/mp4" par="16:9“ minBandwidth="475000“ maxBandwidth="6589000" minWidth="176" maxWidth="1680" minHeight="99" maxHeight="944“ segmentAlignment="true“ startWithSAP="1"> <SegmentTemplate timescale="1000"
initialization="dash/ateam-video=$Bandwidth$.dash"media="dash/ateam-video=$Bandwidth$-$Time$.dash">
<SegmentTimeline> <S t="0" d="4171" /> <S d="2503" /> <S d="2961" /> <S d="2461" /> <S d="2127" r="2" />…
Understanding MP4 – H.264 structure
• Each segment is binary, with a series of boxes contained within
• First 8 bytes of every box defines– Box type– Number of bytes for the box.
Initialization Segment
• FTYP – file type and compatibility• FREE – free space• MOOV – container of all metadata– MVHD – movie header– 1.* TRAK – container for individual stream data• TKHD – track header• MDIA – container for all media info
– Lots of child boxes which contain more detailed stream info
Content Segments
• STYP - segment type• SIDX - segment index• MOOF -movie fragment– MFHD - movie fragment header– TRAF - track fragment
• TFHD - track fragment header• TFDT - track fragment decode time• TRUN - track fragment run
• MDAT - media data container (actual media data)– Contains one or more samples (smallest described piece of
content)
Pulling it all together
• Using OSMF, we wrote a DASH specific IndexHandler and FileHandler
• IndexHandler responsible for deciding url of next http request
• FileHandler responsible for parsing data from segments and returning FLV Bytes
DashIndexHandler
• When OSMF determines a new fragment needs to be loaded, it calls the DashIndexHandler.
• When playing, getNextFile is called• When seeking, getFileForTime is called• Both return an HTTPStreamRequest, which
OSMF uses to request the proper segment
DashFileHandler
• When the requested file is returned, it is handed to the DashFileHandler for processing– initializeProcessing – gets decoder– processFileSegment – asks decoder to process data
• We are using a Decoder called MP4Decoder• MP4Decoder returns FLVBytes for the data
within the MDAT• processFileSegment returns ByteArray (FLVBytes)
to OSMF, for adding to the NetStream buffer
MP4Decoder
• processData() – iterates through all boxes in segment– Calls getNextBox()
• getNextBox knows how to process each specific box type
• When processing MDAT, individual samples are wrapped with FLVBytes
• FLVBytes wrap specific binary header around data, as required by NetStream.appendBytes()
Digital Primates DASH Player
• Currently our DASH player is not commercially available
• We use it as the basis for DASH players we build for our clients
• Initially built as a learning exercise to better understand DASH
Current DP Dash Player
• We currently support SegmentLists, SegmentTemplates and SegmentBase in manifest– Byte Range Requests only currently supported via
Akamai• We support Dynamic Adaptive Streaming for
VOD Streams• We have tested with streams up to 1080p• We have tested with streams from several
different encoders
DP Dash Player plans
• 4Q12 – Basic Seek Fixes– Improved multi-channel
audio support– Support for Live
Streams– Continue to test and fix
streams from more encoding partners
• 1Q13 – Initial JavaScript Player– Support for DRM– Support for Enhanced
seeking• 2Q13 – Investigate HEVC (h.265)– Initial Android Player
• 3Q13 – Initial iOS Player
Resources
• GPAC – http://gpac.wp.mines-telecom.fr– Provides baseline test streams– Provides baseline player
• MP4Parser– http://code.google.com/p/mp4parser/ – Open Source java project – Allows for display of contents within boxes
• DASH Industry Forum– http://www.dashif.org
Questions?
?