Computer Network Architectures and Multimedia …leduc/cours/SRM/SRM-ch4.pdf1 ©From Computer Networking, by Kurose&Ross 4: Multimedia App. & Transp . 4-1 Computer Network Architectures

1

4: Multimedia App. & Transp. 4-1 ©From Computer Networking, by Kurose&Ross

Computer Network Architectures and Multimedia

Guy Leduc

Chapter 4 Multimedia Applications

& Transport

Sections 9.1 to 9.4 from Computer Networking: A Top

Down Approach, 7th edition.

Jim Kurose, Keith Ross Addison-Wesley, April 2016.

Also 7.4.2 and 7.4.7 from Computer Networks - 4th edition

Andrew S. Tanenbaum Prentice-Hall International, 2003


Multimedia networking: outline

4.1 multimedia networking applications 4.2 streaming stored video 4.3 voice-over-IP 4.4 protocols for real-time conversational

applications

2


Multimedia: audio !  PCM (Pulse Code Modulation):

analog audio signal sampled at constant rate "  telephone: 8,000

samples/sec "  CD music: 44,100

samples/sec !  each sample quantized, i.e.,

rounded "  each quantized value

represented by bits, "  e.g., rounded to one of

28=256 values "  8 bits/sample

!  receiver converts bits back to analog signal: "  some quality reduction

time au

dio

sign

al a

mpl

itude

analog signal

quantized value of analog value

quantization error

sampling rate (N sample/sec)


Multimedia: audio Examples: !  Telephony:

!  8,000 samples/sec, 8 bits/sample: 64 kbps

!  CD music: !  44,100 samples/sec,

16 bits/sample: 705.6 kbps

!  Stereo: 1.411 Mbps Other example rates !  MP3: 96, 128, 160 kbps !  Internet telephony: 5.3

kbps and up

time

audi

o si

gnal

am

plitu

de

analog signal

quantized value of analog value

quantization error

sampling rate (N sample/sec)

3


More on Audio Compression

MP3 (MPEG 1 audio layer 3) takes masking effects into account and does not encode masked signals. Can compress stereo CD down to 96-128 kbps.

From Computer Networks, by Tanenbaum © Prentice Hall


❒  video: sequence of images displayed at constant rate ❍  e.g. 25 images/sec

❒  digital image: array of pixels ❍  each pixel represented

by bits ❒  coding: use redundancy

within and between images to decrease # bits used to encode image ❍  spatial (within image) ❍  temporal (from one image

to next)

Multimedia: video

……………………...…

spatial coding example: instead of sending N values of same color (all purple), send only two values: color value (purple) and number of repeated values (N)

……………………...…

frame i

frame i+1

temporal coding example: instead of sending complete frame at i+1, send only differences from frame i

4


Multimedia: video

……………………...…

spatial coding example: instead of sending N values of same color (all purple), send only two values: color value (purple) and number of repeated values (N)

……………………...…

frame i

frame i+1

temporal coding example: instead of sending complete frame at i+1, send only differences from frame i

!  CBR: (constant bit rate): video encoding rate fixed

!  VBR: (variable bit rate): video encoding rate changes as amount of spatial, temporal coding changes

!  examples: "  MPEG 1 (CD-ROM) 1.5

Mbps "  MPEG2 (DVD) 3-6 Mbps "  MPEG4 (often used in

Internet, < 1 Mbps)


Video - Digital Systems

❒  Consider a rectangular 4:3 grid of pixels, such as ❍  VGA: 640 x 480 ❍  XGA: 1024 x 768

❒  Pixel = 8 bits for each of the RGB colours ❒  25 frames per sec ❒  With XGA :

❍  24 bits/pixel x 1024 x 768 x 25 frames/sec = 472 Mbps! ❒  Needs compression!


5


Data compression

❒  Encoding/decoding schemes ❒  Video on Demand (VoD)

❍  Encoding can be slow (done once) ❍  Decoding must be fast (done many times) ❍  Asymmetrical schemes

❒  Real-time multimedia (e.g. videoconference) ❍  Symmetrical schemes

❒  Lossy compression ❍  Encode/decode is not neutral ❍  When acceptable, leads to better compression ratios

❒  Two main compression schemes: ❍  Entropy encoding (lossless) ❍  Source encoding (lossy)



Entropy encoding

❒  Lossless ❒  Three typical examples:

❍  Run-length encoding •  repeated symbols are encoded as “Special symbol + number

of occurrences” ❍  Statistical encoding

•  short codes for frequent symbols (Huffman encoding) ❍  Look up table

•  e.g. CLUT (Colour Look Up Table) •  define the table of the colours actually used •  send table index instead of a 24-bit colour value


6


Source encoding ❒  Lossy ❒  Three main examples:

❍  Differential encoding •  sequence of values are encoded by representing the differences

from the previous values •  makes sense if differences are encoded with less bits •  lossy when there are large jumps between two values and a fixed

number of bits per difference •  lossless if variable-length encoding is used

❍  Transformation •  e.g. Fourier or DCT Transform •  lossy since only the first amplitudes are sent

❍  Variant of Look Up Table with approximations to closest value



JPEG ❒  Joint Photographic Experts Group ❒  ISO/IEC and ITU standard for compressing still pictures ❒  Compression ratio 20:1 is typical ❒  Roughly symmetrical scheme (decoding as long as encoding) ❒  Lossy sequential mode:

❍  6 steps

Blockpreparation

DiscreteCosine

transformQuantization

Differentialquantization

Run-lengthencoding

StatisticalOutput

encoding


7


JPEG - Step 1

❒  Step 1: block preparation ❍  Translate RGB into luminance (Y) and 2 chrominance (I,Q) values

•  gives better compression •  we get 3 matrices of pixels

❍  Average square blocks of 4 pixels for I and Q •  lossy but unnoticeable

❍  Subtract 128 from each element (0 is middle) ❍  Divide up frame into 8x8 blocks

Blockpreparation

DiscreteCosine



Run-lengthencoding

StatisticalOutput

encoding



JPEG - Step 2

❒  Step 2: DCT (Discrete Cosine Transformation) to each block ❍  Sort of 2 dimensional Discrete Fourier Transform, but more compact

•  Advantage: most of the spectral power in the first few terms ❍  Output: block of 8x8 elements (coefficient of DCT) ❍  Slightly lossy in practice (round-off errors)

Blockpreparation

DiscreteCosine



Run-lengthencoding

StatisticalOutput

encoding


8


JPEG - Step 3

❒  Step 3: Quantization ❍  Apply sort of low pass filter to coefficients (lossy)

Blockpreparation

DiscreteCosine



Run-lengthencoding

StatisticalOutput

encoding



JPEG - Steps 4, 5 and 6 ❒  Step 4: Differential quantization

❍  Replace upper-left (DC) coefficient by its difference with corresponding element of previous block

❒  Step 5: Run-length encoding ❍  Applied to a zig-zag scanning pattern

❒  Step 6: Statistical output encoding (Huffman) From Computer Networks, by Tanenbaum © Prentice Hall

9


MPEG

❒  Motion Picture Experts Group - ISO standard ❒  Audio and video ❒  MPEG-1

❍  Video-recorder quality (CD-ROM) ❍  1.2 Mbps output

❒  MPEG-2 ❍  Broadcast quality ❍  4-6 Mbps output is typical but higher for HDTV

❒  MPEG-4 ❍  Medium-resolution videoconferencing with low frame rate

•  10 frames/sec



MPEG-1

❒  Audio and video encoders work independently ❒  Timestamps included in both flows for

synchronization at receiver ❒  Audio compression (MP3)

❍  Also, exploitation of redundancy in the 2 channels of a stereo stream

Audioencoder

Videoencoder

Clock

Audio signal

Video signal

Systemmultiplexer

MPEG-1 output


10


MPEG-1 - Video compression

❒  Exploit spatial and temporal redundancies ❒  Spatial redundancy: like JPEG ❒  But adds temporal redundancy

❍  Many common parts in the following three consecutive frames!



MPEG-1 - Video compression (2)

❒  Temporal redundancy: four kinds of frames: I, P, B, D ❍  I frames (Intracoded)

•  self-contained JPEG-encoded still pictures •  should appear periodically in the output (initial synch, resynch on

error, fast forward or rewind) ❍  P frames (Predictive)

•  block-by-block difference with the previous frame •  search for a macroblock (Y,I,Q) in previous frame which is equal or

slightly different •  encode the offset in position and difference

❍  B frames (Bidirectional): same as P but search also in next I or P frame

❍  D frames (DC-coded): block averages for fast forward (low resolution)

❒  Example of part of an MPEG sequence ❍  I B B P B B I


11


MPEG-2

❒  Similar to MPEG-1 ❒  Better quality (10 x 10 DCT coefficients instead

of 8 x 8) ❒  Several resolution levels (lowest one is comparable

to MPEG-1) ❒  Several profiles (e.g. no B frames to simplify

encoding) ❒  Usually 3-4 Mbps, but can go up to 100 Mbps

(HDTV)



Multimedia networking: 3 application types

❒  streaming, stored audio, video ❍  streaming: can begin playout before downloading entire

file ❍  stored (at server): can transmit faster than audio/video

will be rendered (implies storing/buffering at client) ❍  e.g., YouTube, Netflix, Hulu

❒  conversational voice/video over IP ❍  interactive nature of human-to-human conversation limits

delay tolerance ❍  e.g., Skype

❒  streaming live audio, video ❍  e.g., live sporting event

12


MM Networking Applications

Fundamental characteristics: ❒  typically delay sensitive

❍  end-to-end delay ❍  delay jitter

❒  loss tolerant: infrequent losses cause minor glitches

❒  antithesis of data, which are loss intolerant but delay tolerant

Jitter is the variability of packet delays within the same packet stream

QoS (Quality of Service) refers to performance metrics such as delay, bandwidth, jitter and loss


Multimedia Over Today’s Internet

TCP/UDP/IP: “best-effort service” ❒  no guarantees on delay, bandwidth, jitter, loss (if UDP)

Today’s Internet multimedia applications use application-level techniques to mitigate

(as best possible) effects of delay, loss

But you said multimedia apps require QoS and level of performance to be

effective!

? ? ? ? ? ?

? ? ?

?

?

13


Chapter 4: outline


applications


Internet multimedia: simplest approach

❒  audio or video stored in file ❒  files transferred as HTTP object

❍  received in entirety at client ❍  then passed to player

Media player ❒  jitter removal ❒  decompression ❒  error concealment ❒  graphical user interface

with controls for interactivity

audio, video not streamed in this scenario: ❒  no, “pipelining,” long delays until playout!

14


Internet multimedia: streaming approach

❒  browser GETs metafile ❒  browser launches player, passing metafile ❒  player contacts server ❒  server streams audio/video to player


Streaming from a streaming server

❒  allows for non-HTTP protocol between server and media player ❒  UDP or TCP for step (3), more shortly

15


Streaming Multimedia: client rate(s)

Q: how to handle different client receive rate capabilities?

A: server stores, transmits multiple copies of video, encoded at different rates (info found in meta-file)

1.5 Mbps encoding

28.8 Kbps encoding


Streaming stored video:

1.  video Recorded

(e.g., 30 frames/sec)

2. video sent C

umul

ativ

e da

ta

streaming: at this time, client playing out early part of video, while server still sending later part of video

network delay (fixed in this

example) time

3. video received, played out at client (30 frames/sec)

16


Streaming stored video: challenges

!  continuous playout constraint: once client playout begins, playback must match original timing "  … but network delays are variable (jitter),

so will need client-side buffer to match playout requirements

!  other challenges: "  client interactivity: pause, fast-forward,

rewind, jump through video "  video packets may be lost, retransmitted


constant bit rate video transmission

Cum

ulat

ive

data

time

variable network delay

client video reception

constant bit rate video playout at client

client playout delay

buffe

red

vide

o

❒  client-side buffering and playout delay: compensate for network-added delay, delay jitter

Streaming stored video: revisited

17


Client-side buffering, playout

variable fill rate, x(t)

client application buffer, size B

playout rate, e.g., CBR r

buffer fill level, Q(t)

video server

client

CBR = Constant Bit Rate VBR = Variable Bit Rate







video server

client

1. initial fill of buffer until playout begins at tp 2. playout begins at tp, 3. buffer fill level varies over time as fill rate x(t) varies and playout rate r is constant

18


x < r: buffer may empty, causing freezing of video playout until buffer again fills

initial playout delay tradeoff: buffer starvation less likely with larger delay, but larger delay until user begins watching





video server



Streaming multimedia: UDP

❒  server sends at rate appropriate for client ❍  often: send rate = encoding rate = constant rate ❍  transmission rate can be unaware of network congestion level

❒  short playout delay (2-5 seconds) to remove network jitter ❒  error recovery: application-level, time permitting ❒  encapsulation of audio/video chunks in RTP (Real-Time Transport

Protocol, RFC 3550) and then in UDP ❍  see later for details

❒  needs a control connection in parallel to pause, resume reposition, etc: ❍  Real-Time Streaming Protocol (RTSP, RFC 2326)

❒  issue: UDP may not go through firewalls

19


User Control of Streaming Media: RTSP

HTTP ❒  does not target multimedia

content ❒  no commands for fast

forward, etc. RTSP ❒  Real-Time Streaming

Protocol ❒  client-server application

layer protocol ❒  user control: rewind, fast

forward, pause, resume, repositioning, etc.

What it doesn’t do: ❒  doesn’t define how

audio/video is encapsulated for streaming over network

❒  doesn’t restrict how streamed media is transported (UDP or TCP possible)

❒  doesn’t specify how media player buffers audio/video


RTSP: out-of-band control FTP uses an “out-of-

band” control channel: ❒  file transferred over

one TCP connection ❒  control info (directory

changes, file deletion, rename) sent over separate TCP connection

❒  “out-of-band”, “in-band” channels use different port numbers

RTSP messages also sent out-of-band:

❒  RTSP control messages use different port numbers than media stream: out-of-band ❍  port 554

❒  media stream is considered “in-band”

20


RTSP example Scenario: ❒  metafile communicated to web browser (1) ❒  browser launches player (2) ❒  player sets up an RTSP control connection, data connection to

streaming server (3)


Metafile Example

<title>Twister</title> <session> <group language=en lipsync> <switch> <track type=audio e="PCMU/8000/1" src = "rtsp://audio.example.com/twister/audio.en/lofi"> <track type=audio e="DVI4/16000/2" pt="90 DVI4/8000/1" src="rtsp://audio.example.com/twister/audio.en/hifi"> </switch> <track type="video/jpeg" src="rtsp://video.example.com/twister/video"> </group> </session>

21


RTSP Operation


RTSP exchange example

C: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0 Transport: rtp/udp; compression; port=3056; mode=PLAY

S: RTSP/1.0 200 1 OK Session 4231

C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=0-

C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=37

C: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231

S: 200 3 OK

22


Streaming multimedia: TCP

❒  multimedia file retrieved via HTTP GET ❒  send at maximum possible rate under TCP

❒  issue: fill rate fluctuates much due to TCP congestion control and TCP retransmissions (in-order delivery)

❒  larger playout delay: smooth TCP delivery rate ❒  HTTP/TCP passes more easily through firewalls

variable rate, x(t)

TCP send buffer

video file

TCP receive buffer

application playout buffer

server client

Application Layer 2-44

Video Streaming and CDNs: context

•  Netflix, YouTube: 37%, 16% of downstream residential ISP traffic

•  ~1B YouTube users, ~75M Netflix users "  challenge: scale - how to reach ~1B

users? •  single mega-video server won’t work (why?)

"  challenge: heterogeneity "  different users have different capabilities (e.g.,

wired versus mobile; bandwidth rich versus bandwidth poor)

"  solution: distributed, application-level infrastructure

"  video traffic: major consumer of Internet bandwidth

23


Streaming multimedia: DASH

❒  DASH: Dynamic, Adaptive Streaming over HTTP ❒  server:

❍  divides video file into multiple chunks ❍  each chunk stored, encoded at different rates ❍  manifest file: provides URLs for different chunks

❒  client: ❍  periodically measures server-to-client bandwidth ❍  consulting manifest, requests one chunk at a time

•  chooses maximum coding rate sustainable given current bandwidth

•  can choose different coding rates at different points in time (depending on available bandwidth at time)


Streaming multimedia: DASH

❒ DASH: Dynamic, Adaptive Streaming over HTTP

❒  “intelligence” at client: client determines ❍ when to request chunk (so that buffer

starvation, or overflow does not occur) ❍ what encoding rate to request (higher quality

when more bandwidth available) ❍ where to request chunk (can request from URL

server that is “close” to client or has high available bandwidth)

24


Content distribution networks

❒  challenge: how to stream content (selected from millions of videos) to hundreds of thousands of simultaneous users?

❒  option 1: single, large “mega-server” ❍  single point of failure ❍  point of network congestion ❍  long path to distant clients ❍  multiple copies of video sent over outgoing link

… quite simply: this solution doesn’t scale


Content distribution networks

❒  challenge: how to stream content (selected from millions of videos) to hundreds of thousands of simultaneous users?

❒  option 2: store/serve multiple copies of videos at multiple geographically distributed sites (CDN) ❍  enter deep: push CDN servers deep into many access networks

•  close to users •  used by Akamai, 1700 locations

❍  bring home: smaller number (10’s) of larger clusters in POPs near (but not within) access networks

•  used by Limelight ❍  Google uses both, in addition to its “mega data centers”

responsible for serving dynamic content

25

Content Distribution Networks (CDNs)

…

… …

…

…

"  subscriber requests content from CDN

" CDN: stores copies of content at CDN nodes •  e.g. Netflix stores copies of MadMen

where’s Madmen? manifest file

•  directed to nearby copy, retrieves content •  may choose different copy if network path congested

Application Layer 2-49

Content Distribution Networks (CDNs)

…

… … …

… Internet host-host communication as a service

OTT challenges: coping with a congested Internet ❍  from which CDN node to retrieve content? ❍  viewer behavior in presence of congestion? ❍  what content to place in which CDN node?

“over the top”

26


CDN: “simple” content access scenario Bob (client) requests video http://video.netcinema.com/6Y7B23V actually stored in a KingCDN content distribution server

netcinema.com

KingCDN content distribution server

1

1. Bob gets URL for video http://video.netcinema.com/6Y7B23V from netcinema.com web page 2

2. resolve video.netcinema.com via Bob’s local DNS that relays to netcinema’s authoritative DNS server

netcinema authoritative DNS

3

3. netcinema’s DNS returns a1105.kingcdn.com 4

4&5. Resolve a1105.kingcdn.com via KingCDN’s authoritative DNS, which returns IP address of KingCDN distribution server with video

5 6. request video from KingCDN server, streamed via HTTP

KingCDN authoritative DNS


CDN cluster selection strategy ❒  challenge: how does CDN DNS select “good” CDN node to stream to

client ❍  CDN learns the IP address of the client’s local DNS via the client’s DNS

lookup ❍  CDN can then implement a selection strategy to dynamically direct clients

to a “suitable” server cluster or data center ❒  Possible strategies:

❍  1. pick IP address of CDN node geographically closest to client ❍  2. pick IP address of CDN node with shortest delay (or min # hops) to

client (CDN nodes periodically ping access ISPs, reporting results to CDN DNS)

❍  3. always pick the same IP address, but make sure this IP address is an IP anycast address associated with all CDN nodes

•  see next slide ❒  alternative: let client decide!

❍  give client a list of several CDN servers ❍  client pings servers, picks “best” ❍  Netflix approach

27


IP anycast ❒  An IP anycast address is an IP unicast address, but this address is assigned to

multiple devices that are geographically distributed ❍  Like a private address, but an anycast address is routable, i.e. reachable from

anywhere in the Internet ❒  When a source sends a packet to an IP anycast address, the packet reaches

one of the devices associated with this address ❍  The choice is made “by the network”, not by the source, because it results from a

decision taken at the interdomain routing protocol (BGP) level ❍  In other words, BGP decides by applying its usual routing rules based on

preferences, AS-Path length, Hot Potato criterion, etc. ❍  So, don’t confuse anycast and multicast! ❍  An anycast address is a unicast address ❍  And nothing can tell you if a unicast address is actually anycast or not!

A

B

C

Source 2

CDN cluster (same IP anycast)

CDN cluster (IP anycast)

D

Source 1


IP anycast in a CDN – Role of BGP ❒  The CDN assigns the same IP address range to each of its clusters

❍  Therefore, it’s an anycast address ❒  The CDN relies on standard BGP to advertise this IP address range

from each of the different cluster locations ❒  When a BGP router receives multiple route advertisements for this

same IP address range, it treats them as providing several paths to the same physical location and picks the “best”

A

B

C

1. BGP: AS-PATH “A” to CDN prefix 2. BGP: AS-PATH “BA” to CDN prefix 3. BGP: AS-PATH “C” to CDN prefix

Client

CDN cluster (same IP prefix)

1 2

3

CDN cluster (IP prefix)

D

1

In D, BGP selects the “best” path between “BA” and “C” to reach the CDN prefix

A, B, C, D are Autonomous Systems (AS)

28


Case study: Netflix

❒  30% downstream US traffic in 2011 ❒  owns very little infrastructure, uses 3rd party

services: ❍  own registration, payment servers ❍ Amazon (3rd party) cloud services:

•  Netflix uploads studio master to Amazon cloud •  create multiple version of movie (different encodings) in

cloud •  upload versions from cloud to CDNs •  Cloud hosts Netflix web pages for user browsing

❍  three 3rd party CDNs host/stream Netflix content: Akamai, Limelight, Level-3


Case study: Netflix

1

1. Bob manages Netflix account

Netflix registration, accounting servers

Amazon cloud Akamai CDN

Limelight CDN

Level-3 CDN

2 2. Bob browses Netflix video

3

3. Manifest file returned for requested video

4. DASH streaming

upload copies of multiple versions of video to CDNs

29


Chapter 4: outline


applications


Voice-over-IP (VoIP)

❒  VoIP end-end-delay requirement: needed to maintain “conversational” aspect ❍  higher delays noticeable, impair interactivity ❍  < 150 msec: good ❍  > 400 msec: bad ❍  includes application-level (packetization,playout), network

delays ❒  session initialization: how does callee advertise IP

address, port number, encoding algorithms? ❒  value-added services: call forwarding, screening,

recording ❒  emergency services: 112 (Europe), 911 (North America)

30


VoIP characteristics

❒  speaker’s audio: alternating talk spurts, silent periods. ❍  64 kbps during talk spurt ❍  pkts generated only during talk spurts ❍  20 msec chunks at 8 Kbytes/sec: 160 bytes of data ❍  so, 20 msec of packetization delay

❒  application-layer header added to each chunk ❒  chunk+header encapsulated into UDP (or TCP)

segment ❒  application sends segment into socket every 20 msec

during talkspurt


VoIP: packet loss, delay

❒  network loss: IP datagram lost due to network congestion (router buffer overflow)

❒  delay loss: IP datagram arrives too late for playout at receiver ❍  delays: processing, queueing in network;

end-system (sender, receiver) delays ❍  typical maximum tolerable delay: 400 ms

❒  loss tolerance: depending on voice encoding, loss concealment, packet loss rates between 1% and 10% can be tolerated

31


constant bit rate transmission

Cum

ulat

ive

data

time

variable network delay (jitter)

client reception

constant bit rate playout at client

client playout delay

buffe

red

data

Delay jitter

❒  end-to-end delays of two consecutive packets: difference can be more or less than 20 msec (transmission time difference)


VoIP: fixed playout delay

❒  receiver attempts to playout each chunk exactly q msecs after chunk was generated ❍  chunk has timestamp t: play out chunk at t+q ❍  chunk arrives after t+q: data arrives too late

for playout, data “lost” ❒  tradeoff in choosing q:

❍  large q: less packet loss ❍  small q: better interactive experience

32


VoIP: fixed playout delay

p a c k e t s

t i m e

p a c k e t s g e n e r a t e d

p a c k e t s r e c e i v e d

l o s s

r p p '

•  sender generates packets every 20 msec during talk spurt •  first packet received at time r •  first playout schedule: begins at p •  second playout schedule: begins at p’

playout schedule p - r

playout schedule p’ - r


Adaptive playout delay (1)

❒  goal: low playout delay, low late loss rate ❒  approach: adaptive playout delay adjustment:

❍  estimate network delay, adjust playout delay at beginning of each talk spurt

❍  silent periods compressed and elongated ❍  chunks still played out every 20 msec during talk spurt

❒  adaptively estimate packet delay: (EWMA - exponentially weighted moving average, recall TCP RTT estimate):

di = (1-α)di-1 + α (ri – ti)

delay estimate after ith packet

small constant, e.g. 0.1

time received - time sent (timestamp)

measured delay of ith packet

33


also useful to estimate average deviation of delay, vi :

❒  estimates di, vi calculated for every received packet, but used only at start of talk spurt

❒  for first packet in talk spurt, playout time is:

remaining packets in talkspurt are played out periodically

❒  Q: does it require clock synchronization?

vi = (1-β)vi-1 + β |ri – ti – di|

playout-timei = ti + di + Kvi




Q: How does receiver determine whether packet is first in a talk spurt?

❒  if no loss, receiver looks at successive timestamps ❍  difference of successive stamps > 20 msec -> talk spurt

begins ❒  with loss possible, receiver must look at both time

stamps and sequence numbers ❍  difference of successive stamps > 20 msec and sequence

numbers without gaps -> talk spurt begins

34


VoIP: recovery from packet loss (1)

Challenge: recover from packet loss given small tolerable delay between original transmission and playout

❒  each ACK/NAK takes ~ one RTT ❒  alternative: Forward Error Correction (FEC)

❍  send enough bits to allow recovery without retransmission (recall two-dimensional parity)

simple FEC ❒  for every group of n chunks, create redundant chunk by

exclusive OR-ing n original chunks ❒  send n+1 chunks, increasing throughput by factor 1/n ❒  can reconstruct original n chunks if at most one lost chunk from

n+1 chunks, with playout delay ❒  called “erasure” code



❒  increasing throughput: ❍  by factor 1/n

❒  increasing playout delay: ❍  need enough time to receive all n+1 packets

❒  tradeoff: ❍  increase n, less bandwidth waste ❍  increase n, longer playout delay ❍  increase n, higher probability that 2 or more chunks

will be lost

35


VoIP: recovery from packet loss (3) FEC: Reed-Solomon (RS) scheme ❒  RS is a more sophisticated

error correcting code, which can be used as erasure code

❒  An (n,k) RS code encodes k source packets into n > k packets

❒  Systematic code: the n transmitted packets contain verbatim copies of the k source packets ❍  + n-k new packets ❍  no decoding if no source packet

loss! ❒  Optimal code: Original k

packets can be recovered provided that any k packets among n are received

❒  Linear code: coding/decoding represented by matrix operations: ❍  x is the vector of k source

packets ❍  G is a n x k matrix ❍  y is the vector of n transmitted

packets ❍  y = G x

❒  Decoding: ❍  y’ vector of any k received

packets ❍  G’ is the k x k submatrix of G

with rows corresponding to these packets

❍  x = G’-1 y’



2nd FEC scheme #  “piggyback lower quality stream” #  send lower resolution audio stream as redundant information #  e.g., nominal stream PCM at 64 kbps and redundant stream GSM at 13 kbps.

#  non-consecutive loss, receiver can conceal the loss #  generalization: can also append (n-1)st and (n-2)nd low-bit rate chunks

36



Interleaving to conceal loss ❒  audio chunks divided into smaller

units ❒  for example, four 5 msec units

per 20 ms audio chunk ❒  packet contains small units from

different chunks

❒  if packet lost, still have most of every chunk

❒  no redundancy overhead, but increases playout delay


supernode overlay network

Voice-over-IP: Skype

❒  proprietary application-layer protocol (inferred via reverse engineering) ❍  encrypted msgs

❒  P2P components:

Skype clients (SC)

"  clients: skype peers connect directly to each other for VoIP call

"  super nodes (SN): skype peers with special functions

"  overlay network: among SNs to locate SCs

"  login server

Skype login server supernode (SN)

37


P2P voice-over-IP: skype skype client operation:

1. joins skype network by contacting SN (IP address cached) using TCP

2. logs-in (username, password) to centralized skype login server

3. obtains IP address for callee from SN, SN overlay " or client buddy list

4. initiate call directly to callee

Skype login server


Problem: both Alice, Bob are behind “NATs” ❍  NAT prevents outside peer

from initiating connection to insider peer

❍  inside peer can initiate connection to outside

relay solution: Alice, Bob maintain open connection

to their SNs "  Alice signals her SN to

connect to Bob "  Alice’s SN connects to

Bob’s SN "  Bob’s SN connects to Bob

over open connection Bob initially initiated to his SN

Skype: peers as relays

38


Chapter 4: outline


applications: RTP/RTCP, SIP


Real-Time Protocol (RTP)

❒  RTP specifies packet structure for packets carrying audio, video data

❒  RFC 3550 ❒  RTP packet provides

❍  payload type identification

❍  packet sequence numbering

❍  time stamping

❒  RTP runs in end systems ❒  RTP packets

encapsulated in UDP segments

❒  interoperability: if two Internet phone applications run RTP, then they may be able to work together

39


RTP runs on top of UDP

RTP libraries provide transport-layer interface that extends UDP:

•  port numbers, IP addresses •  payload type identification •  packet sequence numbering •  time-stamping


RTP Example ❒  consider sending 64

kbps PCM-encoded voice over RTP

❒  application collects encoded data in chunks, e.g., every 20 msec = 160 bytes in a chunk

❒  audio chunk + RTP header form RTP packet, which is encapsulated in UDP segment

❒  RTP header indicates type of audio encoding in each packet ❍  sender can change

encoding during conference

❒  RTP header also contains sequence numbers, timestamps

40


RTP and QoS

❒  RTP does not provide any mechanism to ensure timely data delivery or other QoS guarantees

❒  RTP encapsulation is only seen at end systems (not by intermediate routers) ❍  routers provide best-effort service, making no

special effort to ensure that RTP packets arrive at destination in timely manner


RTP entities

❒  End system: application that actually generates/consumes the content carried in RTP packets ❍  SSRC: Synchronisation Source identifier

❒  Translator: intermediate system that changes the encoding scheme without altering the timing. It may also convert multicast into multiple unicast streams

❒  Mixer: intermediate system that receives multiple streams and combines them in some manner. The new stream has its own timing (new SSRC) ❍  CSRC: Contributing Source identifier

End SystemSSRC = 53

End SystemSSRC = 77

TranslatorMixer

SSRC = 19SSRC = 19

CSRC = 53 77

Different encodings Multiple streams Single stream

41


RTP Header

Payload Type (7 bits): Indicates type of encoding currently being used. If sender changes encoding in middle of conference, sender informs receiver via payload type field

• Payload type 0: PCM µ-law, 64 kbps • Payload type 3: GSM, 13 kbps • Payload type 7: LPC, 2.4 kbps • Payload type 26: Motion JPEG • Payload type 31: H.261 • Payload type 33: MPEG2 video

Sequence Number (16 bits): incremented by one for each RTP packet sent, detects packet loss and restores packet sequence

payload type

sequence number type time stamp Synchronization

Source ID Miscellaneous

fields


RTP Header (2)

❒  Timestamp field (32 bytes): sampling instant of first byte in this RTP data packet ❍  for audio, timestamp clock typically increments by one for

each sampling period (for example, each 125 µsecs for 8 KHz sampling clock)

❍  if application generates chunks of 160 encoded samples, then timestamp increases by 160 for each RTP packet when source is active. Timestamp clock continues to increase at constant rate when source is inactive

❒  SSRC field (32 bits): identifies source of RTP stream. Each stream in RTP session should have distinct SSRC.

payload type

sequence number type time stamp Synchronization

Source ID Miscellaneous

fields

42


Real-Time Control Protocol (RTCP)

❒ works in conjunction with RTP

❒  each participant in RTP session periodically transmits RTCP control packets to all other participants

❒  each RTCP packet contains sender and/or receiver reports ❍  report statistics useful

to application: # packets sent, # packets lost, interarrival jitter, etc.

❒  feedback can be used to control performance ❍  sender may modify its

transmissions based on feedback


RTCP: multiple multicast senders

! each RTP session: typically a single multicast address; all RTP / RTCP packets belonging to session use multicast address

! RTP, RTCP packets distinguished from each other via distinct port numbers

! to limit traffic, each participant reduces RTCP traffic as number of conference participants increases

RTCP RTP

RTCP RTCP

sender

receivers

43


RTCP: packet types

Receiver Report (RR) packets:

❒  fraction of packets lost, last sequence number, average interarrival jitter

Sender Report (SR) packets:

❒  SSRC of RTP stream, current time, number of packets sent, number of bytes sent

Source Description (SDES) packets:

❒  e-mail address of sender, sender's name, SSRC of associated RTP stream

❒  provide mapping between the SSRC and the user/host name


RTCP: stream synchronization

❒  RTCP can synchronize different media streams within a RTP session

❒  e.g., videoconferencing app: each sender generates one RTP stream for video, one for audio

❒  timestamps in RTP packets tied to the video, audio sampling clocks ❍  not tied to wall-clock

time

❒  each RTCP sender-report packet contains (for most recently generated packet in associated RTP stream): ❍  timestamp of RTP packet ❍  wall-clock time for when

packet was created ❒  receivers use association to

synchronize playout of audio, video

44


RTCP: bandwidth scaling

❒  RTCP attempts to limit its traffic to 5% of session bandwidth

Example ❒  one sender, sending video

at 2 Mbps ❒  RTCP attempts to limit its

traffic to 100 kbps ❒  RTCP gives 75% of rate to

receivers; remaining 25% to sender

❒  75 kbps is equally shared among receivers: ❍  with R receivers, each

receiver gets to send RTCP traffic at 75/R kbps

❒  sender gets to send RTCP traffic at 25 kbps

❒  participant determines RTCP packet transmission period by calculating average RTCP packet size (across entire session) and dividing by allocated rate


SIP: Session Initiation Protocol [RFC 3261]

SIP long-term vision:

❒  all telephone calls, video conference calls take place over Internet

❒  people are identified by names or e-mail addresses, rather than by phone numbers

❒  you can reach callee (if callee so desires), no matter where callee roams, no matter what IP device callee is currently using

45


SIP Services

❒ SIP provides mechanisms for call setup: ❍ for caller to let

callee know she wants to establish a call

❍ so caller and callee can agree on media type, encoding

❍ to end call

❒  determine current IP address of callee: ❍ maps mnemonic

identifier to current IP address

❒  call management: ❍  add new media streams

during call ❍  change encoding during

call ❍  invite others ❍  transfer, hold calls


Example: setting up a call to known IP address #  Alice’s SIP invite message indicates her port number, IP address, encoding she prefers to receive (PCM µlaw)

#  Bob’s 200 OK message indicates his port number, IP address, preferred encoding (GSM)

#  SIP messages can be sent over TCP or UDP; here sent over RTP/UDP

#  default SIP port number is 5060

µ

time time

Bob'sterminal rings

Alice

167.180.112.24

Bob

193.64.210.89

port 5060

port 38060

µLaw audio

GSMport 48753

INVITE [email protected]=IN IP4 167.180.112.24m=audio 38060 RTP/AVP 0port 5060

200 OKc=IN IP4 193.64.210.89

m=audio 48753 RTP/AVP 3

ACKport 5060

46


Setting up a call (more) ❒  codec negotiation:

❍  suppose Bob doesn’t have PCM µlaw encoder

❍ Bob will instead reply with 606 Not Acceptable Reply, listing his encoders

❍ Alice can then send new INVITE message, advertising different encoder

❒  rejecting a call ❍ Bob can reject with

replies “busy,” “gone,” “payment required,” “forbidden”

❒  media can be sent over RTP or some other protocol


Example of SIP message

INVITE sip:[email protected] SIP/2.0 Via: SIP/2.0/UDP 167.180.112.24 From: sip:[email protected] To: sip:[email protected] Call-ID: [email protected] Content-Type: application/sdp Content-Length: 885

c=IN IP4 167.180.112.24 m=audio 38060 RTP/AVP 0

Notes: ❒  HTTP message syntax ❒  sdp = session description protocol ❒  Call-ID is unique for every call

#  Here we don’t know Bob’s IP address. -> Intermediate SIP servers needed

#  Alice sends, receives SIP messages using SIP default port 5060

#  Alice specifies in “Via:” header that SIP client sends, receives SIP messages over UDP

47


Name translation and user location

❒  caller wants to call callee, but only has callee’s name or e-mail address

❒  need to get IP address of callee’s current host: ❍  user moves around ❍  DHCP protocol ❍  user has different IP

devices (PC, smartphone, car device)

❒  result can be based on: ❍  time of day (work, home) ❍  caller (don’t want boss to

call you at home) ❍  status of callee (calls sent

to voicemail when callee is already talking to someone)

Service provided by SIP servers


SIP Registrar

REGISTER sip:domain.com SIP/2.0

Via: SIP/2.0/UDP 193.64.210.89

From: sip:[email protected]

To: sip:[email protected] Expires: 3600

❒  one function of SIP server: registrar ❒  when Bob starts SIP client, client sends SIP

REGISTER message to Bob’s registrar server

Register Message:

48


SIP Proxy

❒  another function of SIP server: proxy ❒  Alice sends invite message to her proxy server

❍  contains address sip:[email protected] ❒  proxy responsible for routing SIP messages to

callee Bob ❍  possibly through multiple proxies

❒  Bob sends response back through the same set of proxies

❒  proxy returns Bob’s SIP response message to Alice ❍  contains Bob’s IP address

❒  SIP proxy analogous to local DNS server plus TCP setup


SIP example: [email protected] calls [email protected]

1

1. Jim sends INVITE message to UMass SIP proxy.

2. UMass proxy forwards request to Poly registrar server

2 3. Poly server returns redirect response, indicating that it should try [email protected]

3

5. eurecom registrar forwards INVITE to 197.87.54.21, which is running Keith’s SIP client

5

4

4. Umass proxy forwards request to Eurecom registrar server

8 6

7 6-8. SIP response returned to Jim

9 9. Data flows between clients

UMass SIP proxy

Poly SIP registrar

Eurecom SIP registrar

197.87.54.21 128.119.40.186

Note: also a SIP ack message from Jim, which is not shown

49


Comparison with former H.323

❒  H.323 is another signaling protocol for real-time, interactive applications

❒  H.323 is a complete, vertically integrated suite of protocols for multimedia conferencing: signaling, registration, admission control, transport, codecs

❒  SIP is a single component. Works with RTP, but does not mandate it. Can be combined with other protocols, services

❒  H.323 comes from the ITU (telephony)

❒  SIP comes from IETF: borrows much of its concepts from HTTP ❍  SIP has Web flavor,

whereas H.323 has telephony flavor

❒  SIP uses the KISS principle: Keep It Simple Stupid


Chapter 4: Summary Principles ❒  audio and video coding ❒  multimedia applications types over IP

❍  streaming stored audio video, real-time conversational voice/video ❒  UDP versus TCP streaming ❒  making the best of best effort service

❍  DASH: Dynamic, Adaptive Streaming over HTTP ❍  CDN: Content Distribution Networks ❍  adaptive playout delay ❍  loss recovery (FEC, retransmissions) and concealment

Protocols ❒  RTSP ❒  RTP/RTCP ❒  SIP

50


How should the Internet evolve to better support multimedia?

Laissez-faire ❒  just put more capacity where

needed ❒  no major changes in network,

let apps handle it ❒  content distribution networks,

application-layer multicast

Differentiated services philosophy:

❒  fewer changes to Internet infrastructure, yet provide 1st and 2nd class service

What’s your opinion?

Integrated services philosophy: ❒  fundamental changes in

Internet so that apps can reserve end-to-end bandwidth

❒  requires new, complex software in hosts & routers

Documents

Computer Network Architectures and Multimedia …leduc/cours/SRM/SRM-ch4.pdf1 ©From Computer Networking, by Kurose&Ross 4: Multimedia App. & Transp . 4-1 Computer Network Architectures