CS 294-9 :: Fall 2003

Introduction

Topics in Multimedia Computing and Networking

Ketan Mayer-Patel

CS 294-9 :: Fall 2003

The medium is the message.• Marshall McLuhan (1967)• Themes of the course:

– In multimedia communication, how the message arrives is often as important as what the message is.

– The two are entangled.• How the message arrives: Networking• What the message is: Multimedia Encoding

CS 294-9 :: Fall 2003

Characterizing media data• Continuous media is a bit more accurate.

– Periodic– Robust– Often large– Correlated– Endless?– Video, audio, stock ticker?, depth?

• Compare with web pages.– Web pages are definitely multimedia, but don’t fit

the characteristics of continuous media.

CS 294-9 :: Fall 2003

Video TaxonomyCompressed Uncompressed

AnalogDigitalTape Streaming

Digital Betacam

DVDVCAM

DVCPro (D-7)

Digital-8D-9

DVCPro50

MPEG-1MPEG-2MPEG-4M-JPEG

H.261H.263+

RealSorenson

IndeoCinepak

Video for Windows

D-1 (CCIR 601)D-2D-5

VHSS-Video

Betacam

Video-8Hi-8

Betamax

CS 294-9 :: Fall 2003

Audio TaxonomyCompressed Uncompressed

-lawA-law

Linear PCM

Speech-based General

LPC-10ECELPGSM

ADPCMDolby AC-3MPEG-1 L3

MPEG-2 AACDTS

SDDSTHX

MiniDisc (ATRAC)

CS 294-9 :: Fall 2003

Network Basics

Source Dest

Throughput = Amount of data arriving per unit time.

Jitter = Variance of interpacket arrival times

Interpacket Arrival Time

Delay

Loss

CS 294-9 :: Fall 2003

Problems of Delay• Interactive applications

– Video conferencing• Human communication starts to break down at

around 500 msec delays.– Feedback loops

• Haptic interfaces

• Quality of Service– Best effort and end-to-end mechanisms can do

little to control delay.

CS 294-9 :: Fall 2003

Problems of Jitter• Buffer management

– Buffers are used to smooth jitter, but how big should the buffer be?

– Unexpected jitter can lead to starvation.– Over-provisioning buffers creates delay.

• Jitter is hard to bound even with QoS.• Media jitter vs. packet jitter

– Variable bitrate media (where number of packets for “unit” of media varies) can make this relationship hard to model.

CS 294-9 :: Fall 2003

Problems of Throughput• Estimating throughput is problematic.

– Media coding may be tunable, but generally changes in quality are perceptually annoying.

– Throughput will often vary faster than feedback delay which makes estimation hard.

• Multicast makes this worse.• Scalable encoding is one solution path.• QoS mechanisms are another.

CS 294-9 :: Fall 2003

Problems of Loss• Effect of loss is very media and encoding

specific.– Loss of one packet in a video stream can result in

the loss of multiple frames.– Other packets received can be rendered useless.

• Goodput vs. Throughput

• Retransmission often not feasible.• Encoding may be designed to alleviate packet

loss effects.• Forward Error Correction

CS 294-9 :: Fall 2003

Administrivia• Who am I?

– kmp@cs.unc.edu– On campus Mon. and Wed. (629 Soda)

• Who are you?• Class web page

– http://inst.eecs.berkeley.edu/~cs294-9– Schedule of topics and readings posted here.

• Need to sign up for presenting papers.– More on this next week.

• Reading for next week.– Jim Blinn’s article on NTSC

CS 294-9 :: Fall 2003

TCP/UDP Review• Transport protocols on top of IP• TCP: ordered, byte-stream, reliable• UDP: datagram, unreliable, unordered• Programming API: sockets

– Originally from UNIX, but available in Windows as WinSock and Java

• Network byte-order is big endian .

CS 294-9 :: Fall 2003

UDP Packet Format

Src Port # Dest Port #

ChecksumLength

Data

32 bits

CS 294-9 :: Fall 2003

UDP Review• Connectionless

– No communication necessary other than the datagram itself.

• Unreliable– No guarantee or feedback whether data got ther

or not.• Unordered

– Packets may arrive in any order– Limit on delay: 1 minute (comes from IP)

CS 294-9 :: Fall 2003

TCP Packet Format

Src Port # Dest Port #

Sequence #

Acknowledgement #Hdr Len.

Un-used Flags Window Size

Checksum Urgent Ptr

Options

Data

32 bits

CS 294-9 :: Fall 2003

TCP Lifecycle• 3-way handshake

– Establishes connection.• Data transfer

– Congestion controlled– Reliable– In order.

• FIN– Connection teardown.

CS 294-9 :: Fall 2003

3-way handshake• One side must be “listening” for a connection.

– Whoever is listening is “the server.”• Client sends a SYN packet.

– SYN flag is set.– Seq. # is randomly chosen starting byte number.

• Server sends back SYN-ACK packet.– SYN flag is set, ACK flag is set.– Seq. # is set and Ack # reflects client’s Seq. #

• Client sends back ACK

CS 294-9 :: Fall 2003

Data Transfer• Full duplex connection.

– Both sides can send data simultaneously.• Seq. # set to byte number of first byte.• Ack # set to next byte expected.• Receiver generates an ACK even if it doesn’t

have data to send back.– Some implementations delay sending ACK to

optimize piggyback case.– In general, implementations expected to generate

one ACK for each data packet received.

CS 294-9 :: Fall 2003

Reliability• Sender maintains data until acknowledged.

– This is done at the OS. The application is free from this responsibility.

• Sender estimates RTT from ACK’s.• Timeout values dynamically calculated from RTT and

RTT variance.• If timeout occurs, data retransmitted.• Holes in the byte stream are possible.

– Receiver still generates ACK, but ACK # reflects next byte needed in seq.– Multiple ACK’s used to retransmit quickly.

CS 294-9 :: Fall 2003

Transfer ExampleSeq. 0

Seq. 10Seq. 20Seq. 30

Ack 10

Ack 10Ack 10

TimeoutSeq. 10

Ack 40

CS 294-9 :: Fall 2003

Flow Control

Application

OSTCP

Send BufferFirst un-ACK’d byte

Last un-ACK’d byteFree Space

Application

OSTCP

Recv Buffer

Last ACK’d byte

Recv. Window

Recv. Window is advertised to the sender in ACK.

Sender Receiver

CS 294-9 :: Fall 2003

Congestion Control• Basic question: how fast to send?

– Too slow: underutilize network– Too fast: no one gets anything useful through– Not responsive: congestion collapse

• TCP congestion control is window-based.– TCP maintains a “window” of data in the network.– Successful transmissions allow window to increase.– Packet drops cause window to shrink.– Reliability feedback and congestion control coupled

together (may not have been the best idea).

CS 294-9 :: Fall 2003

Slow Start• Slow Start

– When starting, don’t know what the window should be, so probe for capacity.

– Initial window: 1 segment size• A “segment” is a set number of bytes (i.e., packet size).

– For every sucessful ACK, increase window by 1 segement size.

– Doubles window size every RTT.

CS 294-9 :: Fall 2003

Slow Start IllustratedSeq. 0

Seq. 10, 20

Seq. 30, 40, 50, 60Ack 20, 30

Ack 10

Ack 40, 50, 60, 70

CS 294-9 :: Fall 2003

Congestion Avoidance• Eventually slow-start results in a packet

drop. This establishes “ssthresh”.• Ssthresh set at 1/2 of current window.• Current window shutdown to 1 segment.• Data retransmitted.• Slow-start begins again.• When window gets back to ssthresh, every

ACK results in window += 1/window.• Additive increase, multiplicative decrease.

CS 294-9 :: Fall 2003

AIMD IllustratedW

indo

w S

ize

Time

ssthreshssthresh

ssthresh

CS 294-9 :: Fall 2003

Misc. TCP Oddities• Window is actually minimum of receiver

advertised window and congestion window.• Duplicate ACK’s used to detect congestion

before actual timeout. (Fast retransmission)• Special-case calculation of congestion

window after fast retransmission to allow new data to be sent. (Fast recovery)

• Lots of tweaks and options (SACK, T/TCP)

CS 294-9 :: Fall 2003

Multimedia and TCP/UDP• Streaming multimedia rarely uses TCP. Why?

– Retransmission is pointless• By the time data is recovered, too late.

– AIMD congestion makes jitter and throughput changes unbearable.

• So, why did we just review TCP?– Everything else in the world is TCP.– How non-TCP traffic affects TCP traffic is an important

question.

• In general, streaming media protocols built on top of UDP.