Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO

Improving the user experience of mul4media streaming services in highly dynamic environments

Frank Eliassen ([email protected]) Verdikt conference, 26th April 2012

4/26/2012 ROMUS project @ VERDIKT 1

Many problems still hampering Internet live video streaming

•  Increased heterogeneity of networks and terminals

•  Variability in resource availability such as bandwidth, CPU, and joining and leaving of devices

•  Challenge: How to provide each consumer with the best possible viewing experience when considering heterogeneity and variability, while maintaining efficiency and scalability

Car computer

Home PC

GSM /UMTS

The Internet

BT/ WLAN


ROMUS project: main objec4ve •  To inves4gate and provide solu4ons for mul4media (video) streaming services to provide each consumer best possible experience in a highly dynamic environment – best possible quality (image quality, con4nuous playback)

–  least possible visual distor4on


ROMUS – three main areas of results •  Adapta4on and Robustness in Live P2P Streaming

–  Chameleon: a novel adap4ve P2P streaming protocol targe4ng live video streaming (video “broadcasts”)

–  S*r: A social network based P2P streaming solu4on to be[er handle peer dynamics in live video streaming

•  Video quality assessment –  Randomized Pair Comparison (R/PC), a novel test method for

subjec4ve video quality assessment –  A set of guidelines to reduce visual distor4on in scalable video

streaming •  Mul4core Processing to handle mul4media workloads

–  Techniques for exploi4ng mul4core processing and graphical processing units on individual peer nodes to improve video quality

–  P2G: a framework for distributed processing on computer nodes in a cluster suppor4ng mul4media workloads with so_ deadlines.


ROMUS team •  Professor Frank Eliassen, University of Oslo (project leader) •  Professor Carsten Griwodz, Simula Research Laboratory •  Professor Pål Halvorsen, Simula Research Laboratory •  Dr. Viktor S. Wold Eide, University of Oslo (post doc for 18

months) •  Dr. Eli Gjørven, University of Oslo (post doc for 6 months) •  Anh Tuan Nguyen, University of Oslo (PhD scholar) •  Pengpeng Ni, Simula Research Laboratory (PhD scholar) •  Håkon Stensland, Simula Research Laboratory (PhD scholar)


Adapta4on and Robustness in Live Peer-‐to-‐Peer Streaming

ROMUS project


Mo4va4on •  Limita4ons of tradi4onal live P2P streaming systems – No differen4ated QoS: users must receive the same stream regardless of their bandwidth (high capacity users perceive the same low quality as average users)

– No con4nuous playback/black block images: with the current best-‐effort Internet and the peer dynamics, the streaming quality at each peer is easily impaired (when the available bandwidth at a peer drops below the streaming rate, it may suffer playback skips)


Main hypotheses

•  Adaptable coding techniques (such as SVC) can bring significant benefits in terms of differen4ated QoS and con4nuous playback to live P2P streaming

•  Network coding and social networking can improve the robustness of the P2P system with respect to network fluctua4ons and peer dynamics

•  Quality-‐aware overlays can ensure high capacity peers will receive high quality video


Scalable Video Coding (H.264 AVC/SVC)

•  A video coding technique: encodes a video into layers of quality

•  Standardized in July 2007 by ITU-‐T (H.264)

•  ~10% bitrate overhead and an indis4nguishable visual quality compared to H.264 AVC


Scalable Video Coding (source: h[p://www.hhi.fraunhofer.de)

The three scalability dimensions “Any” sub-stream can be extracted


Network Coding (Linear network coding)

•  Instead of simply forwarding data, intermediate nodes may recombine several input packets into one or several output packets

•  Perfect collabora4on –  Poten4al throughput improvements –  A high degree of robustness


Chameleon: a pull-‐based P2Pstreaming protocol

Chameleon’s architecture with key components


Evalua4on: Baseline •  FABALAM: Y. Liu, W. Dou, and Z. Liu, “Layer Alloca4on

Algorithms in Layered Peer-‐to-‐Peer Streaming,” in Proc. of IFIP interna*onal conference on network and parallel compu*ng (NPC), Oct. 2004, pp. 167–174

•  Commons –  Pull-‐based P2P streaming protocol –  Adaptability

•  Differences

Chameleon FABALAM

H.264/SVC Synthe4c layered data

Network coding -‐ A layer is delivered from mul4ple senders

Approxima4on algorithm -‐ A layer is delivered from one sender


Evalua4on: on scalability

Chameleon is scalable and offers much lower skip rates and higher quality satisfaction

Chameleon vs. FABALAM: Skip rates Chameleon vs. FABALAM: Quality satisfaction


Evalua4on: on peer dynamics

Using weibull(k,2) for generation of different levels of peer dynamic Skip rates

Quality satisfaction

Chameleon can adapt well to peer dynamics to maintain low skip rates and high quality satisfaction


Video Quality Assessment

Flicker effects in Adap4ve Video Streaming


ROMUS project

Quality adapta4on mechanism in Chameleon at work

What are acceptable limits of quality fluctuations for the user? Can we provide guidelines for how to adapt to reduce visual distortion?


Media Performance Group

Visual perception of dynamically adaptive video (1)!Understanding and using limits of user perception and perceived quality"

Signal-‐to-‐noise ra4o (SNR) scaling Noise flicker

Resolu4on scaling Blur flicker

Frame rate scaling Judder

Three main types of visual artifacts"


Visual Perception of dynamically adaptive video (2)

Encoding Layers

Frames over Time

High Frequency

Low Frequency

Two main fluctuation factors"

Amplitude" Frequency"

Field study"•  mobile devices, free seating, resolution 480x320@30fps, no sunlight,

lounge chairs"Experiment design"•  repeated measures, single-stimulus, randomized block design"•  blocking by flicker type and amplitude level"•  baselines for highest and lowest quality without quality fluctuations"


Visual Perception of dynamically adaptive video (3)!

Three influential factors"

Amplitude"Most dominant effect"Flicker is almost undetectable at amplitudes < 8QP and almost always detectable for larger amplitudes"

Frequency"Major effect"Acceptance thresholds compared to constant low quality video:"worse when above 1 Hz,"often better when below 0.5 Hz""

Content"Minor effect"But: content can influence"flicker perception;"low interaction for noise flicker and stronger for blur flicker""

P2G: Parallel Processing Graphs

A Framework for Distributed Real-‐Time Processing of Mul*media Data


ROMUS project

Mo4va4on •  The poten4al to use mul4core processing and new heterogeneous technology (e.g., GPUs) to handle mul4media workloads on individual peer nodes to meet new demands

•  Challenge: difficult to program because of heterogeneous architectures (e.g., data parallelism vs. thread parallelism)

•  Exis4ng frameworks have all some short-‐comings; do not meet all requirements of mul4media data processing


P2G main features •  The P2G framework allows developers to

express continuous multimedia workloads with fine grained parallelism in a single language

•  The runtime decides itself how a program should be partitioned, and which execution node should execute them

•  Open source at http://p2gproject.org 4/26/2012 ROMUS project @ VERDIKT 23

Mo4on JPEG Experiment

•  Continuous workload, CIF resolution •  DCT consumes most of the encoding time.


Mo4on JPEG Results

21

11

8 7 7

6 6 5

0

2

4

6

8

10

12

14

16

18

20

22

1 2 3 4 5 6 7 8

4-‐way Core i7 30

15

10

8

6 5 5 5

0

2

4

6

8

10

12

14

16

18

20

22

24

26

28

30

1 2 3 4 5 6 7 8

8-‐way Opteron

Threads Threads

Tim

e (s

)

Tim

e (s

)


Conclusions (1) •  The development of a simulation model to simulate and

evaluate novel adaptive and robust live P2P video streaming solutions is essentially important for long-term research in the field

•  It is essential to find solutions to important limitations of live P2P streaming technologies. Our basic solutions and findings could be inherited by other initiatives to build a more practical protocol taking other network metrics into account.

•  Tools and guidelines for how to design real time video streaming infrastructure adopting SVC techniques is crucial for being able to provide the best possible user experience with minimal visual distortion.


Conclusions (2) •  Our results indicate that quality adaptation can

outperform constant low quality, but frequency and amplitude, as well as switching patterns are relevant

•  Multicore processor scheduling to handle multimedia workloads on individual peer nodes will be important in the future and may improve the multimedia experience of the user even further.

•  Our results shows the feasibility of providing a programming framework for automatic parallel, real-time processing of multimedia workloads exploiting heterogeneous multicore processors.


Thank you!

Q&A


Education

Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO