Impact of New Protocols: What Happens When Everybody DASHes? Begen... · 2.80 2.99 2.89 2.86 3.05 2.99 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 WiFi LTE 3G) TCP QUIC QUIC provided a higher

Workshop on Media Streaming Meets 5G – Dec. 2019

Impact of New Protocols: What Happens When Everybody DASHes?

Ali C. Begen, PhDhttp://ali.begen.net

Tradeoffs in Adaptive Streaming

Media Streaming Meets 5G – Dec. 2019 2

I DASH, You DASH, Everybody DASHes

User experienceOverall quality

Quality stability

StallsZapping/seeking timeLive latency

A Single Microsoft Smooth Streaming Client under a Controlled EnvironmentDemystifying a Streaming Client

0

1

2

3

4

5

0 50 100 150 200 250 300 350 400 450 500

Bitra

te (M

bps)

Time (s)

Available Bandwidth Requests Chunk Tput Average TputReading: “An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP,” ACM MMSys 2011

Buffer-filling StateBack-to-back requests

Steady StatePeriodic requests


• Leverage tried-and-true Web– Video is just ordinary Web content

• Leverage tried-and-true TCP– Congestion avoidance– Reliable transport– No need for special QoS for video

Doe

s it

just

wor

k?

Mostly yes, when streaming clients compete with other types of traffic

Not really, when streaming clients compete with each other

Streaming clients form an “accidental” distributed control-feedback system, causing all kinds of issues in• House with multiple screens• Crowded public hotspots• ISP access/aggregation links

Streaming over HTTP – The Promise


10 Streaming Clients Sharing a 10 Mbps LinkSelfishness Hurts Everyone

0

400

800

1200

1600

0 100 200 300 400 500

Req

uest

ed B

itrat

e (K

bps)

Time (s)

Alice Bob Lisa John Jane


Two Competing ClientsUnderstanding the Root Cause

• Depending on the timing of the ON periods:– Unfairness, underutilization and/or instability may occur– Clients may grossly overestimate their fair share of the available bandwidth

Clients cannot figure out how much bandwidth to use until they use too much(Just like TCP)

Reading: “What happens when HTTP adaptive streaming players compete for bandwidth?,” ACM NOSSDAV 2012Media Streaming Meets 5G – Dec. 2019 6

How to Solve the Issues?

• Use a better adaptation algorithm like PANDA or BOLA• Use machine learning or deep learning like Pensieve• Improve the HTTP/TCP/QUIC stack, try out the new developments (HTTP/2/3)• Adopt ideas from game/consensus theory (GTA)

Fix the clients and/or the transport

• QoS in the core/edge• SDN

Get support from the network

• Assist the clients and network elements through metrics and analytics

Enable a control plane


Review of over 100 Adaptation Schemes


Looking for a Permanent head Damage (Degree)?

Bitrate Adaptation Schemes

Client-based

Adaptation

Bandwidth-based

Buffer-based

Mixed adaptation

Proprietary solutions MDP-based

Server-based

Adaptation

Network-assisted

AdaptationHybrid

Adaptation

SDN-basedServer and network-assisted

Reading: “A survey on bitrate adaptation schemes for streaming media over HTTP,” IEEE Commun. Surveys Tuts., 2019

Conflicting Goals in the Media Delivery Chain

• I want to make sure that my content is protected and looks awesome on any device

• I want to make sure that my ads are viewed, trackable and measurable

• I want to make sure that my servers are properly used and latency is low

• I want to control the QoE of my customers, make $ from and differentiate my own video services

• I want to make sure that my device/app provides the best possible video quality

• I want the best quality for minimal $

ConsumerContent Provider ISPCDN

ProviderAdvertiser Device/App


A Control Plane Approach


Origin (HTTP)Server

Encoder/Transcoder Packager

CDN

RG

Clients

MediaParameters for enhancing reception (PER)

Metrics and status messagesParameters for enhancing delivery (PED)

AnalyticsServer

•CTA-2066: Streaming Quality of

Experience Metrics (almost done)

•New proposal at CTA: Common

Media Client Data

Service Provider:

“Your video or CDN provider must be slow”

Video/CDN Provider:

“Your homenetwork must

suck”

Consumer:“The device or the app is

slow”

Device/App Vendor:

“It must be the OS”

OS Vendor:“Your Internet

connection must be bad”


• Use a common language (e.g., CTA-2066) across players

• Beware infobesity

“Overabundance of information implies a scarcity of user attention”

Yes, at the Source, Encoder, Packager, Origin, Cache, GW, Player, etc.Do We Need Analytics?

What about LatencyPredicting the Bandwidth Accurately in Low-Latency Apps

Reading: “Bandwidth prediction in low-latency chunked streaming,” ACM NOSSDAV 2019

Live Twitch Data* (Nov. 2018)Bandwidth Measurement is Tricky

0

1

2

3

4

5

0 100 200 300 400 500 600

Bitra

te (M

bps)

Time (s)

tc Bandwdith Measured Bandwidth Selected Bitrate

* Encoded at {0.18, 0.73, 1.83, 2.5, 3.1, 8.8} Mbps with three resolutions of {540p, 720p, 1080p}, and packaged with CMAFMedia Streaming Meets 5G – Dec. 2019 13

No upshifting despite the available bandwidth

6HUYHU

1RQ�ORZ�ODWHQF\�

FOLHQW

6WRUH�DQGVHUYH�LQ�VHJPHQWV�

��'HWHUPLQH�WKH�OLYH�HGJH�VHJPHQW�EDVHG�RQ�WKH�PDQLIHVW��5HTXHVW�LW�DQG�UHFHLYH�LQ�FKXQNV

5HT��VHJ��

��['�['�['

5HT��VHJ��

/RZ�ODWHQF\�

FOLHQW

��'HWHUPLQH�WKH�PRVW�UHFHQW�IXOO�VHJPHQW�EDVHG�RQ�WKH�PDQLIHVW��5HTXHVW�LW

6HUYHU

6WRUH�LQ�VHJPHQWV��VHUYH�LQ�FKXQNV�

6HVVLRQ�

VWDUW�WLPH

5HT��

VHJ�� 5HT

��

VHJ��

6HJ��GRZQORDG�WLPH��ᶦ��

,GOH�

0HGLD�WLPHOLQH

6HJ��GRZQORDG�WLPH��ᶦ��

6HJ��GRZQORDG�WLPH��ᶦ� �a�'��

6HJ��GRZQORDG�WLPH��ᶦ� �a�'��

ABR for Chunked Transfer Encoding (ACTE)

BandwidthMeasurement

Sliding window

based moving

average method

BandwidthPrediction

Online linear

adaptive filter

based RLS

algorithm

ABR Controller

Throughput-

based bitrate

selection logic


Identifying the “Good” Chunks


Bandwidth Measurement

• Compute the download rate for the chunks where the transmission is network limited– If there is a negligible idle period after a chunk download, use that chunk, otherwise disregard it

Media timeline

Chu

nk 4

Size

: Q4

Chu

nk 5

Size

: Q5

Chu

nk 6

Size

: Q6

Chu

nk 7

Size

: Q7

b4e4e5 b5e6 b6e7 b7

……

Chunk download beginning time (unknown)

Chunk download end time

Chunk download end time is known from HTTP Fetch API

Chunk size is determined from the received data

Identifying the “Good” Chunks without the bn Values


Bandwidth Measurement

• Reasonable assumption: Idle periods cannot happen within a chunk, happen only between the chunks– Since the server pushes the chunks at full network speed

• For each chunk, compute its download rate, which equals its size divided by this chunk’s end time minus previous chunk’s end time– If this download rate is close (+/- 20%) to the average segment download rate, there must be

significant idle time between these two chunks• Transmission is source limited• Disregard the current chunk

– Else, the idle time is negligible • Transmission is network limited • The current chunk’s download rate is a good approximation of the available bandwidth

• Use a sliding window based moving average method over the last three successful chunk downloads

Online Linear Adaptive Filter Using Recursive Least Squares (RLS)


(Future) Bandwidth Prediction

Bandwidth prediction for the next step

Measured bw

Measured bw

Measured bw

Schemes Implemented


Performance Evaluation

BandwidthMeasurement

ABR Schemes

Throughput-based Buffer-based Hybrid

SLBW THsl - -

EWMA THew - -

SWMA THsw BOLAsw Dynamicsw

WSSL THwss - -

SLBW: Segment-based last bandwidth

EWMA: Chunk-based exponentially weighted moving average

SWMA: Chunk-based sliding window moving average

WSSL: Will’s simple slide-load

28.6% Improvement by ACTE over Other Schemes


Average Selected Bitrate

0

0.5

1

1.5

2

2.5

ACTE THsl THew THsw THwss BOLAsw Dynamic-sw

Aver

age

Bitra

te (M

bps)

Prediction

accuracy > 96%



Average Live Latency

0

1

2

3

4

5

6


Aver

age

Late

ncy

(s)



Average Normalized QoE

0

0.2

0.4

0.6

0.8

1


Aver

age

Nor

mal

ized

QoE

Yin Model P.1203

What about QualityEncoding and Streaming in a Quality-Aware Fashion

Reading: “Spending quality time with the Web video,” IEEE Internet Comput., 2016, and “Quality-aware HTTP adaptive streaming,” IBC 2015

Picking the Bitrate Ladder Based on the ContentContent-Based (Content-Aware) Encoding

Content-aware encoding gives us fairness in quality as opposed to fairness in bitrate


Guidelines Limited Bitrate Variability to (Mostly) 10% So FarAdaptation Feature Delivers Inconsistent Quality

Eas

y

Mod

erat

e

Eas

y

Eas

y

Eas

y

Diff

icul

t

Diff

icul

t

Diff

icul

t

Mod

erat

e

Mod

erat

e

Mod

erat

e

Mod

erat

e

…S

Time (s)

Segment Size

0 2 4 6 8 10 12 14 16 18 20 22 24

Segment Quality

QCBR

Small variation in encoding bitrate

Large variation in quality

If there is something worse than having to watch a video at a lousy quality, it is to watch that video with varying quality


What If We Encode in a More Subtle Fashion? E

asy

Mod

erat

e

Easy Eas

y

Eas

y

Diff

icul

t

Diff

icul

t

Diff

icul

t

Mod

erat

e

Mod

erat

e

Mod

erat

e

Mod

erat

e

…

Time (s)

Segment Size

0 2 4 6 8 10 12 14 16 18 20 22 24

Segment Quality

QVBR

While we spend the same total amount of bits, we not only increase average quality but also reduce quality variation

Large variation in encoding bitrate

Low variation in quality

S

HLS authoring spec for ATV allows 2x capping rate for VoD. For linear content, variability is limited to 10-25% range.Media Streaming Meets 5G – Dec. 2019 26

Content-awareEncoding

Content-awareStreaming

Generating VBR-encoded segments is easy, but streaming them is not!


We can Still Save Bandwidth and/or Improve QualityWhat If the Content is Already CBR Encoded

Reading: “Streaming video over HTTP with consistent quality,” ACM MMSys 2014

2.8 Mbps Network

HTTP Server

…k+1 k+24M

bps

k+3k

k+1 k+2 k+3k …3M

bps

…2M

bps

…1M

bps

Representations (4 bitrate levels)

Smart Clients

Naive (Most) Clients

A Bit Smarter (Few) Clients

…

…

…


We can Still Save BandwidthWhat If There is No Smartness in the Client

3.2 Mbps Network

RegularHTTP Server

…k+1 k+24M

bps

k+3k

k+1 k+2 k+3k …3M

bps

…2M

bps

…1M

bps

Representations (4 bitrate levels)

Naive Clients

3.2 Mbps Network

Quality-awareHTTP Server (or Packager)

Naive Clients

…

…

Reading: “More juice less bits: content aware streaming,” ACM MMSys 2016

Bandwidth

Savings

The server/packager replacessome of the 3 Mbps segmentswith the 2 Mbps ones sincedelta quality is insignificant


What If the Content is VBR Encoded

2.8 Mbps Network

HTTP Server

k+1 k+2 k+3k …

QL_

3

…

QL_

2

…

QL_

1

Representations (3 quality levels)

Smart Clients

…

The client streams the highestconsistent-quality video withoutdraining its buffer while respectingthe available bandwidth(Solved using dynamic programming)

The resolution stays the same butthe encoding rate varies in a givenrepresentation (per quality level)


• Most common scenario– CBR encoders and naive streaming clients– Capped VBR encoders (small caps) and

naive clients

• Emerging scenario– Capped VBR encoders (small cap) and a

bit smarter clients

• Best possible scenario – Capped VBR encoders (large cap) and

smart clients

Deployment ScenariosWhat If We Have Both CBR and VBR Encoded Content

If you do not control both

ends, follow Postel’s Law

(RFC 1122)

Be liberal in what you

accept, conservative in

what you send

If there are unknown clients:

- Send CBR encoded segments

- Set minBufferTime sufficiently

large

Otherwise, the clients might

get confused, break, stall or

even crash


What about QUICAddressing Slow Startups, High Latency and Frequent Connection Changes

Reading: “Quickly starting media streams using QUIC,” Packet Video Wksp. 2018


Frame-Seek Scenario Results

2.802.99 2.862.89

3.05 2.99

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

WiFi LTE 3G

Aver

age

Play

back

Bitr

ate

(Mbp

s)

TCP QUIC

QUIC provided a higher (or at least an equal) average playback bitrate in all

cases and for all algorithms

Max Bitrate Available: 3.9 Mbps



2.09 2.162.38

1.28 1.291.42

0.0

0.5

1.0

1.5

2.0

2.5

3.0

WiFi LTE 3G

Aver

age

Wai

t Tim

e af

ter S

eeki

ng (s

)

TCP QUIC

QUIC reduced wait times to less than 1.5 seconds



3.67 3.67

5.67

1.001.33

2.00

0

1

2

3

4

5

6

WiFi LTE 3G

Reb

uffe

rRat

e (%

)

TCP QUIC

QUIC’s benefits are bigger when delays are higher

WiFi - LTE Switches


Connection-Switch Results

3.00

4.00

5.00

2.00

3.00

4.00

0

1

2

3

4

5

6

BASIC SARA BBA-2

Reb

uffe

rRat

e (%

)

TCP QUIC

QUIC provided a higher (or at least an equal) average playback bitrate for all algorithms

WiFi - 3G Switches


Connection-Switch Results

13.00

2.00

5.006.00

1.00

4.00

0

2

4

6

8

10

12

14

BASIC SARA BBA-2

Reb

uffe

rRat

e (%

)

TCP QUIC

QUIC provided a higher (or at least an equal) average playback bitrate for all algorithms

• Two grand challenges– Improving open-source HEVC encoding– Low-latency live streaming

• Focus areas in 2020– Machine learning and statistical modeling for video streaming– Volumetric media: from capture to consumption– Fake media and tools for preventing illegal broadcasts

• A challenge dedicated to high-school students

• Three confirmed keynotes from Google, MIT and Tsinghua

• Expecting reduced registration fees thanks to strong support

Important Dates Submit by

Research Track Jan. 10 (firm)

Demo Track Feb. 29

Open Source/Dataset Feb. 29

Workshops Mar. 27

Conference June 8-11

NEW

Visit http://acmmmsys.org today!

NEW

Visit

http://ali.begen.net

for tutorials and papers

Also AMAAAS

[email protected]

DON’T DRINK

AND DRIVE,

JUST STREAM

AND FLY!

http://ali.begen.net/

Documents

Impact of New Protocols: What Happens When Everybody DASHes? Begen... · 2.80 2.99 2.89 2.86 3.05 2.99 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 WiFi LTE 3G) TCP QUIC QUIC provided a higher