Adaptive Media Streaming: The Role of Standards

AdaptiveMediaStreaming:TheRoleofStandardsDynamicAdaptiveStreamingoverHTTP

Priv.-Doz.Dr.ChristianTimmerer[Ack:AliC.Begen,MediaMelon,Inc.,OzyeginUniversity]

Alpen-Adria-UniversitätKlagenfurt(AAU)w FacultyofTechnicalSciences (TEWI) w DepartmentofInformationTechnology(ITEC)wMultimediaCommunication(MMC)w SensoryExperience Lab(SELab)

http://blog.timmerer.com w http://selab.itec.aau.at/ w http://dash.itec.aau.at w christian.timmerer@itec.aau.atChiefInnovationOfficer(CIO)atbitmovinGmbH

http://www.bitmovin.comw christian.timmerer@bitmovin.com

Tutorial@QoMEX 2016,June2016

http://www.slideshare.net/christian.timmerer

ImportanceofMultimediaDelivery• Multimediaispredominanton

theInternet• Real-timeentertainment

– Streamingvideoandaudio– Morethan70%ofInternettraffic

atpeakperiods• Popularservices

– YouTube(17.85%),Netflix(37.05%),AmazonVideo(3.11%),Hulu (2.58%)

– Alldeliveredover-the-top (OTT)

June2016 QoMEX2016Tutorial 2GlobalInternetPhenomenaReport:Dec2015

OpenDigitalMediaValueChain

June2016 QoMEX2016Tutorial 3

CreateContent

AggregateMonetize

DistributeContent

ConsumeContent

AnyContent AnyStorefront AnyNetwork AnyDevice

CDNsMediaProtocols

InternetTransport

DRMEncoding

EncapsulationDynamicAds

Clients

HappyUser

WhatisStreaming?

June2016 QoMEX2016Tutorial 4

Streamingistransmissionofacontinuouscontentfromaservertoaclientanditssimultaneousconsumptionbytheclient

TwoMainCharacteristics1. Clientconsumption ratemaybelimitedbyreal-timeconstraintsasopposed tojust

bandwidth availability2. Servertransmissionrate(looselyortightly)matchestoclientconsumptionrate

CommonAnnoyancesinStreaming• Wrongformat• Wrongprotocol• Pluginrequirements• DRMissues• Longstart-updelay• Poorquality• Frequentstalls• Qualityoscillations• Noseekingfeatures

June2016 QoMEX2016Tutorial 5

Over-The-Top– AdaptiveMediaStreaming• Inanutshell…

June2016 QoMEX2016Tutorial 6

Adaptation logic is within the client, not normatively

specified by the standard, subject to research and

development

Multi-BitrateEncodingandRepresentationSwitching

June2016 QoMEX2016Tutorial 7

Contents ontheWebServer

MovieA– 200Kbps

MovieA– 400Kbps

MovieA– 1.2Mbps

MovieA– 2.2Mbps

...

...

MovieK– 200Kbps

MovieK– 500Kbps

MovieK– 1.1Mbps

MovieK– 1.8Mbps

.. .

.. .

Time(s)

Startquickly

Keep requestingImprove quality

Loss/congestiondetection

Revampquality

...

...Segments

AdaptiveStreamingoverHTTP

June2016 QoMEX2016Tutorial 8

…………

HTTP GETs

ClientBuffer

MediaPlayer

HTTPServer

FormatsandStandards• Adobe

– HTTPDynamicStreaming (HDS)– Almostdead

• Apple– HTTPLiveStreaming (HLS)– RequiredforiOS

• Microsoft– SmoothStreaming– Almostdead, too!Ornot?

• MPEGDynamicAdaptiveStreamingoverHTTP(DASH)– Supported byNetflix,YouTube,Bitmovin,etc.

June2016 QoMEX2016Tutorial 9

Source: http://xkcd.com/927/

ScopeofDASH:whatisspecified?

June2016 QoMEX2016Tutorial 10

Media Presentation on HTTP Server

DASH-enabled ClientMedia Presentation Description

.

.

.

Segment

…

.

.

.Segment

…

.

.

.

Segment

…

.

.

.Segment

…

…

Segments located by HTTP-URLs

DASH Control Engine

HTTP/1.1 HTTP Client

MPD Parser

Media Engine

On-time HTTP requests to segments

ScopeofDASH:whatisspecified?

June2016 QoMEX2016Tutorial 11

Media Presentation on HTTP Server

DASH-enabled ClientMedia Presentation Description

.

.

.

Segment

…

.

.

.Segment

…

.

.

.

Segment

…

.

.

.Segment

…

…

Segments located by HTTP-URLs

DASH Control Engine

HTTP/1.1 HTTP Client

MPD Parser

Media Engine

On-time HTTP requests to segments

DASHDataModel

June2016 QoMEX2016Tutorial 12

MPD

Period id=1start=0s

Period id=3start=300s

Period id=4start=850s

Period id=2start=100s

AdaptationSet0subtitleturkish

AdaptationSet2audioenglish

Adaptation Set1BaseURL=http://abr.rocks.com/

Representation2Rate=1Mbps

Representation4Rate=3Mbps

Representation1Rate=500Kbps

Representation 3Rate=2Mbps

Resolution =720p

SegmentInfoDuration=10s

Template:3/$Number$.mp4

Segment Access

InitializationSegmenthttp://abr.rocks.com/3/0.mp4

MediaSegment1start=0s

http://abr.rocks.com/3/1.mp4

MediaSegment2start=10s

http://abr.rocks.com/3/2.mp4

AdaptationSet3audiogerman

AdaptationSet1video

Period id=2start=100s

Representation3Rate=2Mbps

Selectionofcomponents/tracks

Well-definedmediaformat

Selectionofrepresentations

Splicingofarbitrarycontentlikeads

Chunkswithaddressesandtiming

June2016 QoMEX2016Tutorial 13

type=static typically,for on demand content

Base URL of the segments

Subtitles

Audio adaptation set with different representations (bw)

Video adaptation set with different representations (bw)

Different codecs (profiles)

Segment URL constructed with template and base URL

http://www.dash-player.com/demo/

June2016 QoMEX2016Tutorial 14

OStandard,WhereArtThou?WhatisaStandard?http://en.wikipedia.org/wiki/Standards

Atechnicalstandardisanestablishednormorrequirement.Itisusuallyaformaldocumentthatestablishes uniform engineeringortechnicalcriteria,methods,processesandpractices.

è I N T E R O P E R A B I L I T Y

TypesofStandard• Openstandards

– ITU-T,W3C,IETF,MPEG,Linux,…• Closedstandards

– MSMediaPlayer,…

• Dejurestandards– ANSI,ISO,DIN,…

• Defactostandards– MSWindows,OSX,Unix,Linux,…

• Mandatorystandards– Health,radiation,…

• Voluntarystandards– JPEG,…

June2016 QoMEX2016Tutorial 15

AdaptiveStreamingContentWorkflow

June2016 QoMEX2016Tutorial 16

Source Transcoding Encapsulation Encryption OriginServer

HelperDistribution

Client

Linear:MulticastVoD:FTP,RTMP,HTTP,etc.

UnicastHTTP(PUSH),FTP,etc.

HTTPGETsmallobjects

Singlehighest-bitratestream Multiplestreamsat

targetbitrates

Multiplestreamsattargetencapsulationformats

Largevideo/virtualfilesandmanifests

AdaptiveStreamingContentWorkflowSimplified

June2016 QoMEX2016Tutorial 17

StandardDeliveryInfrastructure(CDN)Source Transcoding Encapsulation Encryption

Multiplestreams:video[bitrate(32000-20000000),profile(baseline,main,high),preset(standard,professional,premium),height(128-7680),width(96-4320),framerate(1-120),codec(h264,hevc)],audio:[bitrate(8000-256000),samplerate(0,8000,11025,12000,16000,22050,24000,32000,44100,48000,64000,88200,96000)]

Singlehighest-bitratestream:HTTP,FTP,RTMP;mp4,ts;AVC,AAC,Subtitles

Multiplestreamsattargetencapsulationformats:DASH(MPD+mp4),HLS(m3u8,ts)

MultiplestreamswithmultipleDRMformats:MPEG-CENC,Widewine,PlayReady,PrimeTime,Fairplay

Player

HeterogeneousClientse.g.BitmovinHTML5AdaptivePlayerDASH,HLS,HTML5,MSE,EME

QoEforDASHServices• DifferentapplicationdomainshavedifferentQoE

requirements– NeedtoprovidespecializationsofthegeneralQoE

definition– Takeintoaccountrequirements formulatedbymeansof

influence factorsandfeaturesofQoE• QoEinfluencefactorsforDASH

– Initial/start-updelay(low)– Bufferunderruns, stalls,freezes(zero)– Qualityswitches(low)– Mediathroughput (high)– …

June2016 QoMEX2016Tutorial 18

!

http://www.bitcodin.com/

http://www.dash-player.com/

QoEEvaluationforDASH-basedServices• Testsequence

– Manydatasetsavailable– AdoptedBigBuckBunny&DASHed itwithbitcodin

• Players– bitdash– Proprietarysolutions(smooth,HLS,HDS)– YouTube,dash.js,DASH-JS– …andcompareitwithtendifferentadaptationalgorithms

• Objectiveevaluation– Commontestsetupusingnetworkemulation&bandwidthshaping– Predefinedbandwidthtrajectory(orrealnetworktraces)

• Subjectiveevaluation– Lab[ITU-TB.500/P.910]vs.

crowdsourcingwithspecialplatformsorsocialnetworks

June2016 QoMEX 2016Tutorial 19

CrowdsourcedQoEEvaluation• QualityofExperience …

– MeanOpinionScore[0..100]– [otherobjectivemetrics:

start-uptime,throughput,stalls]• …Web-basedAdaptiveHTTPStreamingClients…

– HTML5+MSE: DASH-JS (dash.itec.aau.at), dash.js (DASH-IF,v1.1.2),YouTube• …Real-WorldEnvironments…

– DASH-JS,dash.js hostedatITEC/AAU (~10Gbit/s)– YouTubehostedatGoogledatacenters– Content:TearsofSteel @144p(250kbit/s),240p(380kbit/s),360p(740kbit/s),480p(1308kbit/s),and

720p(2300kbit/s);segment size:2s– Usersaccess contentovertheopenInternet (i.e.,real-worldenvironment)

• …Crowdsourcing– CampaignatMicroworker platform(othersalsopossible:Mechanical Turk,socialnetworks) limitedto

Europe,USA/Canada,India– ScreeningTechniques: Browserfingerprinting,stimuluspresentationtime, QoEratingsandpre-

questionnaire

June2016 QoMEX2016Tutorial 20B. Rainer, C. Timmerer, “Quality of Experience of Web-based Adaptive HTTP Streaming Clients in Real-World Environments using Crowdsourcing”, Proceedings of International Workshop on VideoNext: Design, Quality and Deployment of Adaptive Video Streaming, Sydney, Australia, Dec. 2014.

MOS andAverageBitrate

• 288microworkers,33screened(Fingerprinting: 20,presentationtime:6,QoEratingsandpre- questionnaire: 7),175maleand80female,majority(80%)isagedbetween18and37

June2016 QoMEX2016Tutorial 21

StartupTimeandNumberofStalls

June2016 QoMEX2016Tutorial 22

ResultsSummary• DASH-JS

– Highstart-uptime– Lownumberofstalls– Goodthroughput,QoE

• dash.js– Lowstart-uptime– High#stalls– Lowthroughput– LowQoE

• YouTube– Lowstart-uptime– Lownumberofstalls– Bestthroughput,QoE

June2016 QoMEX2016Tutorial 23

Now,10differentadaptationlogics…

June2016 QoMEX2016Tutorial 24

Adaptationlogicswell-known inresearchliterature

Predefinedbandwidthtrajectoryandtestsetup

Differentsegmentsizes,RTTs,HTTP/2,etc.

C. Timmerer, M. Maeiro, B. Rainer, “Which Adaption Logic? An Objective and Subjective Performance Evaluation of HTTP-based Adaptive Media Streaming Systems”, arXiv cs.MM, June 2016, http://arxiv.org/abs/1606.00341.

June2016 QoMEX2016Tutorial 25

June2016 QoMEX2016Tutorial 26

DASH-JSvs.bitdash

June2016 QoMEX2016Tutorial 27

C. Timmerer, D. Weinberger, C. Mueller, and S. Lederer, “Ultra-High-Definition-Quality of Experience with MPEG-DASH”, Proceedings of the Broadcast Engineering Conference (BEC), NAB2015, Las Vegas, NV, USA, April 2015.

ObjectiveEvaluations

June2016 QoMEX2016Tutorial 28

Stalls(lowerisbetter)AverageBitrate(higherisbetter)

Stallsarereallybad…

June2016 QoMEX2016Tutorial 29

Conviva:ViewerExperienceReport.2014

Conclusions(1)• MPEG-DASHdefinesformatsonly

– MediaPresentationDescription (MPD)– Segment format:mp4, ts

• MPEG-DASHisnot– System,protocol,presentation,codec,interactivity,DRM,clientspecification– Otherstandardsrequired foracompleteecosystem:e.g.,DASH-IF,WAVE,

HMTL5,MSE,EME• DoweneedMPEG-DASH?(foradaptivemediastreaming)

– Notnecessarily:e.g.,WebM +VPx +manifest&controlend-to-end– Requiredtoaddressheterogeneous environments

• Roleofstandardssometimesoverrated butoftenunderestimated

June2016 QoMEX2016Tutorial 30

Conclusions(2)• QoEforDASH-basedservices(aruleofthumb)

– Startupdelay(low[butlivevs.on-demand&shortvs.long-tailcontent])

– Bufferunderrun /stalls(zero)– Qualityswitches(low)andmediathroughput(high)– Energy- andcost-awareness(dataplan)

• NogeneralapplicableQoEmodelforDASH– (Too)manyfactorsinfluencing/featuresofQoEforDASH-based

services– Methodologyforreproducibleresearchisinplaceandwellestablished– Ampleresearchopportunities

June2016 QoMEX2016Tutorial 31

MainQoEfactorsforDASH

➡ ComeupwithourownQoEfactoranddesign, conduct,analyzeasmall-scaleexperiment