Video Quality Research @ IBCN

2

Video Quality Research @ IBCN

Real-time Video Quality Monitoring/Testing Monitor Probe Virtual Wall

Video Quality Metrics Full Length Movie Quality Assessment Visual Quality Impairment Detector

VQEG involvement Video Streaming

xStreamer

3

Real-time Video Quality Monitoring/Testing

Monitor probe Real-time monitoring of H.264/AVC streams at several

demarcation points Gather both network statistics and video statistics

Packet loss, delay, jitter Macroblock & motion vector information

N. Vercammen, N. Staelens, B. Vermeulen and P. Demeester, “Distributed Video Quality Monitoring”, to appear in Proceedings of 2nd IEEE International Workshop on Internet and Distributed Computing Systems (IDCS'09), December 10-12, Korea

Monitor probe

4

5

Real-time Video Quality Monitoring/Testing

Virtual Wall & Video testbed Video testbed

Automate video quality tests Virtual wall

Enables creation of multiple video testbeds N. Vercammen, N. Staelens, B. Vermeulen and P.

Demeester, “Extensive video quality evaluation: A scalable video testing platform”, Proceedings of 1st IEEE International Workshop on Internet and Distributed Computing Systems

Scalable Video Testing Platform

6

7

Video Quality Metrics

Full Length Movie Quality Assessment Existing video quality assessment methodologies:

Evaluate short video sequences (~ 15s) Users are actively evaluating visual quality Overall test duration limited to 30 minutes

Watching television At home, living room, with family (social viewing) Longer content: movies, television programs Lean backward TV viewing experience

=> how is quality perceived while watching full length movies, when users are not focused on quality evaluation

8

Full Length Movie Quality Assessment

Error Visibility Frame freeze in movie <-> frame freeze in short

sequences: 42% <-> 91% Blockiness in movie and short sequences: both 98%

Error Annoyance Standard test: freezes rated higher quality than

blockiness Movie: frame freezes are more annoying

Conclusion Focus is important Flow experience

9

Full Length Movie Quality Assessment

N. Staelens, B. Vermeulen, S. Moens, J.-F. Macq, P. Lambert, R. Van de Walle and P. Demeester, “Assessing the influence of packet loss and frame freezes on the perceptual quality of full length movies”, Proceedings of Fourth International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM-09)

N. Staelens, S. Moens, W. Van den Broeck, I. Mariën, B. Vermeulen, P. Lambert, R. Van de Walle and P. Demeester, “Assessing the perceptual influence of H.264/SVC signal-to-noise ratio and temporal scalability on full length movies”, Proceedings of First International Workshop on Quality of Multimedia Experience (QoMEX 2009)

10

Video Quality Metrics

Visual Quality Impairment Detector Current research focus Real-time visual impairment detection

Network level => detect losses Video level => determine severity and visibility

Target No-Reference metric or Reduced-Reference Will be implemented in the monitor probe

11

VQEG involvement

Ghent University – IBBT involved in VQEG Meeting hosted, September 22 – 26, 2008

Contributes to MM testplan (already finalized)

ITU-T J.246 & ITU-T J.247 Hybrid testplan:

co-editors HDTV testplan

Independent Lab Group (ILG) Joint Effort Group (JEG)

Toolchain for creating impaired sequences H.264/AVC parser, based on JM reference software

xStreamerModular Multimedia Streaming

xStreamer

In-house developed modular multimedia streamer Alexis Rombaut (alexis.rombaut@intec.ugent.be)

Written in C++ Uses libraries:

libavformat/libavcodec (parsing/encoding/decoding)

live555 (RTSP)

jrtplib (RTP)

Released under General Public License (GPL) Freely available at http://xstreamer.atlantis.ugent.be/

13

14

Modular Multimedia Streaming

Inspired by Click Modular Router & DirectShow

Offers different components Performs basic functions Readers, packetizers, multiplexers, schedulers,

transmitters, receivers, writers, classifiers, analyzers

Streamer is directed graph of components

15

Modular Multimedia Streaming

Supports audio and video

Using RTP packetization: MPEG-1/2/4 Video & Audio

Using MPEG-2 Transport Streams: MPEG-1/2/4 Video & Audio H.264 AVC/SVC

16

Modular Multimedia Streaming

Multitude of supported protocols

RTP/UDP

RTSP/RTP/UDP

UDP

TCP

What can xStreamer do?

Advanced streaming server Own MPEG-2 TS multiplexer SVC streaming Differentiated streaming using classifiers

Proxy/client Proxy: convert differentiated stream into

a single stream Client: save captured stream to file

Video tool No ‘real’ streaming involved Simulate packet loss Collect tracefiles during streaming

17

Create xStreamer configuration

Configuration saved in XML-based file Describes directed graph of components and

connections between components

Graphical User Interface Visualize directed graph Drag components and draw connections Configure components

18

Example: Differentiated SVC streaming

19

Read raw H.264 video

stream

Packetize frames into packets as

defined in RFC3984

Avoid bursts by smoothing packets

over time

Classify NAL units depending

on SVC layer

Stream different layers over different

connections

Example: Proxy/Client

20

Proxy Client

xStreamer as video tool

Offline simulator No ‘real’ streaming Simulate packet loss using Classifier component

Random, Gilbert-Elliott Write resulting packet stream back to file

Tracefile generation Packetizer: video trace Transmitter: sender trace Receiver: receiver trace Classifier: sender & receiver trace

21

Publication

‘xStreamer: Modular Multimedia Streamer’ accepted for publication on ACM Multimedia 2009 - Open Source Software Competition, Beijing, ChinaOctober 19-24, 2009

Distributed transcoding with xStreamer

Current: Architecture

Transcoder

1

Transcoder

1

ProxyProxyTranscoder

…

Transcoder

…

Transcoder

N

Transcoder

N

ServerServer

Input

File

Input

File

Output

File

Output

File

Current: Server

DifferentiatorDifferentiatorffmpeg-

reader

ffmpeg-

reader

RTP

transmitter 1

RTP

transmitter 1

RTP

transmitter …

RTP

transmitter …

RTP

transmitter N

RTP

transmitter N

Send each new GOP to the next transcoder

= RR (round robin) distribution

note: the schemes omit some components for clarity

Current: Transcoder

xstreamer

unpacketizer

xstreamer

unpacketizerRTP

receiver

RTP

receiver

transcodertranscoder

xstreamer

packetizer

xstreamer

packetizerRTP

transmitter

RTP

transmitter

Current: Proxy

ffmpeg-

writer

ffmpeg-

writermultiplexermultiplexer

RTP

receiver 1

RTP

receiver 1

RTP

receiver …

RTP

receiver …

RTP

receiver N

RTP

receiver N

First experimental results

The following slides show the results from the first experiments on the virtual wall.

Experiment parameters: All sequences are 90 minutes long, encoded with H.264 at 25

frames per second.

Future work will experiment with different bit rates for each resolution.

Sequence Resolution Bitrate (kbps)

QCIF 176x144 100

CIF 352x288 400

4CIF 704x576 1600

720p 1280x720 3600

1080p 1920x1080 8100

Scalability

The figure shows the how much faster than real-time (factor) we can transcode in function of the number of nodes.

For example, using 20 nodes the system can transcode from 1080p to QCIF 100 times faster than real-time, transcoding 90 minutes of video in less than 1 minute.

Some curves flatten because the server cannot stream more than 1 Gbps to feed the transcoding notes, future work will alleviate this by using several network interfaces.

Codec comparison

Provides the same information as the previous figure but organized in function of the source resolution.

As target resolutions become larger the influence of the source resolution decreases (encoding, determined by target resolution, consumes more resources than decoding, determined by source resolution).

For example transcoding 1080p to 4CIF is not much slower than transcoding 720p to 4CIF (blue bars).

Transcoding duration

The figure shows the considerable processing power by transcoding 90 minutes of video in mere 10 seconds for the smallest resolutions.

For the highest resolutions, the system can transcode 90 minutes of video from 1080p to 720p in less than 4 minutes.

The times to transcode 1080p to QCIF and 1080p to CIF are same because the server could not feed the former adequately.

Conclusion

The first experiments show promising results by transcoding between 25 and 500 faster than real-time depending on the resolutions using 32 nodes.

Some combinations did not fully scale up to 32 nodes because the server bit rate would exceed 1 Gbps. However, future work, using multiple interfaces or multiple servers will avoid this.

Future experiments will increase the number of nodes up to 100.