MPEG-DASHOverview, State-of-the-Art, and Future Roadmap

Priv.-Doz. Dr. Christian Timmerer[Ack: Ali C. Begen, Networked Media & Ozyegin University | Thomas Stockhammer, Qualcomm Inc.]

Alpen-Adria-Universität Klagenfurt (AAU) Faculty of Technical Sciences (TEWI) Department of Information Technology (ITEC) Multimedia Communication (MMC) Sensory Experience Lab (SELab)

http://blog.timmerer.com http://selab.itec.aau.at/ http://dash.itec.aau.at christian.timmerer@itec.aau.atChief Innovation Officer (CIO) at bitmovin GmbH

http://www.bitmovin.com christian.timmerer@bitmovin.com

Presentation at EBU/ASBU Week of Technology, Tunis, Oct 19, 2016

http://www.slideshare.net/christian.timmerer

Importance of Multimedia Delivery

• Multimedia is predominant on the Internet• Real-time entertainment: Streaming video and audio; >70% of Internet traffic at peak periods• Popular services: YouTube (17.53%), Netflix (35.15%), Amazon Video (4.26%), Hulu (2.68%); all

delivered over-the-top (OTT); huge potential for mobile access!

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 2

Global Internet Phenomena Report: June 2016

Open Digital Media Value Chain

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 3

Create Content

AggregateMonetize

Distribute Content

Consume Content

Any Content Any Storefront Any Network Any Device

CDNsMedia Protocols

Internet Transport

DRMEncoding

EncapsulationDynamic

Ads

Clients

Happy User

Example Platform/Infrastructure

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 4

https://bitmovin.com/

Common Annoyances in Streaming• Wrong format• Wrong protocol• Plugin requirements• DRM issues• Long start-up delay• Poor quality• Frequent stalls• Quality oscillations• No seeking features

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 5

Over-The-Top – Adaptive Media Streaming• In a nutshell…

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 6

Adaptation logic is within the client, not

normatively specified by the standard, subject to

research and development

Multi-Bitrate Encoding and Representation Switching

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 7

Contents on the Web Server

Request Movie A (200 Kbps) for t=0

Movie A – 200 Kbps

Movie A – 400 Kbps

Movie A – 1.2 Mbps

Movie A – 2.2 Mbps

. . .

. . .

Request Movie A (400 Kbps) for t=16

Request Movie A (800 Kbps) for t=28

Request Manifest for Movie A

Movie K – 200 Kbps

Movie K – 500 Kbps

Movie K – 1.1 Mbps

Movie K – 1.8 Mbps

. . .

. . .

Time (s)

Start quickly

Keep requestingImprove quality

Loss/congestion detection

Revamp quality

. . .

. . .Segments

Manifest

Request Movie A (400 Kbps) for t=2

Request Movie A (800 Kbps) for t=4

Adaptive Streaming over HTTP

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 8

…………

HTTP GETs

ClientBuffer

MediaPlayer

HTTPServer

Formats and Standards• Adobe

– HTTP Dynamic Streaming (HDS)– Switched to DASH

• Apple– HTTP Live Streaming (HLS)– Required for iOS

• Microsoft– Smooth Streaming– Switched to DASH, almost..

• MPEG Dynamic Adaptive Streaming over HTTP (DASH)– Supported by Netflix, YouTube, Bitmovin, etc.

• MPEG Common Media Application Format (MPEG-A Part 19)– The new kid on the block – support for “fragmented mp4 in HLS”– DASH/HLS convergence at segment level – some open issues with encryption format

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 9

Source: http://xkcd.com/927/

Scope of DASH: what is specified?

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 10

Media Presentation on HTTP Server

DASH-enabled ClientMedia Presentation Description

.

.

.

Segment

…

.

.

.Segment

…

.

.

.

Segment

…

.

.

.Segment

…

…

Segments located by HTTP-

URLs

DASH Control Engine

HTTP/1.1 HTTP Client

MPD Parse

rMedia Engine

On-time HTTP requests to segments

Scope of DASH: what is specified?

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 11

Media Presentation on HTTP Server

DASH-enabled ClientMedia Presentation Description

.

.

.

Segment

…

.

.

.Segment

…

.

.

.

Segment

…

.

.

.Segment

…

…

Segments located by HTTP-

URLs

DASH Control Engine

HTTP/1.1 HTTP Client

MPD Parse

rMedia Engine

On-time HTTP requests to segments

DASH Data Model

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 12

MPD

Period id = 1start = 0 s

Period id = 3start = 300 s

Period id = 4start = 850 s

Period id = 2start = 100 s

Adaptation Set 0subtitle turkish

Adaptation Set 2audio english

Adaptation Set 1BaseURL=http://abr.rocks.com/

Representation 2Rate = 1 Mbps

Representation 4Rate = 3 Mbps

Representation 1Rate = 500 Kbps

Representation 3Rate = 2 Mbps

Resolution = 720p

Segment InfoDuration = 10 s

Template:3/$Number$.mp4

Segment Access

Initialization Segmenthttp://abr.rocks.com/3/0.mp4

Media Segment 1start = 0 s

http://abr.rocks.com/3/1.mp4

Media Segment 2start = 10 s

http://abr.rocks.com/3/2.mp4

Adaptation Set 3audio german

Adaptation Set 1video

Period id = 2start = 100 s

Representation 3Rate = 2 Mbps

Selection of components/tracks

Well-defined media format

Selection of representations

Splicing of arbitrary content like ads

Chunks with addresses and timing

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 13

type=static typically,for on demand content

Base URL of the segments

Subtitles

Audio adaptation set with different representations (bw)

Video adaptation set with different representations (bw)

Different codecs (profiles)

Segment URL constructed with template and base URL

https://bitmovin.com/demo/

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 14

ISO/IEC 23009-1 Timeline

Fastest time ever that a standard was developed in MPEG to address the demand of the market

• Other Relevant Specifications– 14496-12: ISO Base Media File Format– 14496-15: Carriage of NAL unit structured video in the ISO Base Media File Format– 23001-7: Common encryption format for ISO base media file format – 23001-8: Coding-Independent Code Points– 23001-10: Carriage of Timed Metadata Metrics of Media in ISO Base Media File Format

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 15

CfP Issued April 2010

18 Responses and Working Draft (WD)July 2010

Committee Draft (CD)Oct. 2010

Draft International

Standard (DIS)Jan. 2011

Final Draft International

StandardNov. 2011

Published as International

Standard April 2012

See also here https://multimediacommunication.blogspot.co.at/2010/05/http-streaming-of-mpeg-media.html

ISO/IEC 23009 Parts• 23009-1: Media Presentation Description and Segment Formats

– 2nd edition has been published– 1st amendment (high profile and availability time synchronization)– 2nd amendment (spatial relationship description, generalized URL parameters, etc.)– 3rd amendment (authentication, MPD linking, callback event, period continuity, etc.)– 4th amendment (segment independent SAP signalling (SISSI), MPD chaining, MPD reset,

etc.)– 3rd edition draft available

• 23009-2: Conformance and Reference Software– 1st edition has been published– WD for 2nd edition is in progress, incl. support for ISOBMFF, M2TS and Web-based

conformance checking provided by DASH-IF (http://testassets.dashif.org/)

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 16

ISO/IEC 23009 Parts• 23009-3: Implementation Guidelines

– 1st edition is done, will be published soon– 2nd edition is in progress

• 23009-4: Segment Encryption and Authentication– Published by ISO in 2013

• 23009-5: Server and Network Assisted DASH (SAND) – FDIS – to be published soon

• 23009-6: DASH with Server Push and WebSockets– DIS available is in progress

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 17

…as of this week!

Ongoing Work in MPEG DASH (as of MPEG 116)

• Currently Running Core Experiments– High Quality VR delivery with DASH (DASH-VR)

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 18

https://bitmovin.com/mpeg-vr-one-ring-rule/https://bitmovin.com/lighter-faster-interactive-ads-cross-platform-vr-bitmovins-latest-major-player-version-6-0/

New work item proposalCoded Representation of Immersive Media

MPEG CMAF: Threat or Opportunity?+ CMAF Media Object Model compatible with DASH Data Model

+ Segment formats based on ISOBMFF

- Different manifest formats (MPD vs. m3u8)

- CENC: AES-128 CBC (HLS) vs. AES-128 CTR (all others) mode

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 19

https://bitmovin.com/what-is-cmaf-threat-opportunity/

Conclusions• MPEG-DASH defines formats only

– Media Presentation Description (MPD)– Segment format: mp4, ts

• MPEG-DASH is not– System, protocol, presentation, codec, interactivity, DRM, client specification

• Other standards required for a complete ecosystem– DASH-IF, WAVE, HMTL5, MSE, EME, 3GPP, DVB, etc.

• Do we need MPEG-DASH? (for adaptive media streaming)– Not necessarily: e.g., WebM + VPx + manifest & control end-to-end– Required to address heterogeneous environments to solve interop. problem

• Role of standards sometimes overrated but often underestimated• CMAF and VR addressing new challenges for adaptive media streamingOct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 20

Deployment Thoughts• Proprietary ecosystems will disappear (Silverlight, Flash)• No more plugins – HTML5!

– MSE/EME available on all major browser platforms– Support for both DASH/HLS (+CMAF) and CENC

• Rich feature set: codecs, ads, DRM, multi-language/-audio, subtitles, VR/360, UHD, HFR, HDR, live, on-demand, analytics, …

• Common implementation issues: start-up, buffering, high-quality, seamless switching, platform support, cost-effective, …

• Solutions available for adaptive streaming, advertising, VR/360, live streaming, and DRM– Details available at https://bitmovin.com/

Oct 19, 2016 EBU Week at ASBU, Dr. Timmerer [Bitmovin] 21

Christian Timmerer / @timse7Bitmovin, Inc. 530 Lytton Avenue | Palo Alto | CA 94301 | USASchleppe Platz 7 | 9020 Klagenfurt | Austria | Europe

https://bitmovin.com@bitmovin | facebook.com/bitmovin | github.com/bitmovin

MPEG-DASH: OVERVIEW & ROADMAP