CSci4211: Multimedia Networking 1
A Quick Primer onMultimedia Networking
• Multimedia vs. (conventional) Data Applications – analog “continuous” media: encoding, decoding & playback– service requirements
• Classifying multimedia applications– Streaming (stored) multimedia– Live multimedia broadcasting– Interactive multimedia applications
• Making the best of best effort service: streaming Stored Multimedia over “Best-Effort “Internet– client buffering, rate adaption, etc.
• Large-scale video delivery over the Internet: YouTube and Netflix case studies (optional)
Required Readings: csci4221 Textbook, sections 7.1-7.2
2
Multimedia and Quality of Service
Multimedia applications: network audio and video(“continuous media”)
network provides application with level of performance needed for application to function.
QoS
CSci4211: Multimedia Networking
3
Digital Audio• Sampling the analog signal
– Sample at some fixed rate – Each sample is an arbitrary real number
• Quantizing each sample– Round each sample to one of a finite number of values– Represent each sample in a fixed number of bits
4 bit representation(values 0-15)
CSci4211: Multimedia Networking
4
Audio Examples• Speech
– Sampling rate: 8000 samples/second– Sample size: 8 bits per sample– Rate: 64 kbps
• Compact Disc (CD)– Sampling rate: 44,100 samples/second– Sample size: 16 bits per sample– Rate: 705.6 kbps for mono,
1.411 Mbps for stereo
CSci4211: Multimedia Networking
5
Why Audio Compression• Audio data requires too much bandwidth
– Speech: 64 kbps is too high for a dial-up modem user– Stereo music: 1.411 Mbps exceeds most access rates
• Compression to reduce the size– Remove redundancy– Remove details that human tend not to perceive
• Example audio formats– Speech: GSM (13 kbps), G.729 (8 kbps), and G.723.3
(6.4 and 5.3 kbps)– Stereo music: MPEG 1 layer 3 (MP3) at 96 kbps, 128
kbps, and 160 kbps
CSci4211: Multimedia Networking
6
A few words about audio compression• Analog signal sampled at
constant rate– telephone: 8,000
samples/sec– CD music: 44,100
samples/sec• Each sample quantized,
i.e., rounded– e.g., 28=256 possible
quantized values• Each quantized value
represented by bits– 8 bits for 256 values
• Example: 8,000 samples/sec, 256 quantized values --> 64,000 bps
• Receiver converts it back to analog signal:– some quality reduction
Example rates• CD: 1.411 Mbps• MP3: 96, 128, 160 kbps• Internet telephony: 5.3 -
13 kbps
7
Digital Video• Sampling the analog signal
– Sample at some fixed rate (e.g., 24 or 30 times per sec)
– Each sample is an image
• Quantizing each sample– Representing an image as an array of picture
elements– Each pixel is a mixture of colors (red, green, and
blue)– E.g., 24 bits, with 8 bits per color
9
A few words about video compression
• Video is sequence of images displayed at constant rate– e.g. 24 images/sec
• Digital image is array of pixels
• Each pixel represented by bits
• Redundancy– spatial– temporal
Examples:• MPEG 1 (CD-ROM) 1.5
Mbps• MPEG2 (DVD) 3-6 Mbps• MPEG4 (often used in
Internet, < 1 Mbps)Research:• Layered (scalable)
video– adapt layers to available
bandwidth
CSci4211: Multimedia Networking
10
Video Compression: Within an Image
• Image compression– Exploit spatial redundancy (e.g., regions of same
color)– Exploit aspects humans tend not to notice
• Common image compression formats– Joint Pictures Expert Group (JPEG)– Graphical Interchange Format (GIF)
Uncompressed: 167 KB Good quality: 46 KB Poor quality: 9 KB
CSci4211: Multimedia Networking
11
Video Compression: Across Images
• Compression across images– Exploit temporal redundancy across images
• Common video compression formats– MPEG 1: CD-ROM quality video (1.5 Mbps)– MPEG 2: high-quality DVD video (3-6 Mbps)– Proprietary protocols like QuickTime and RealNetworks
CSci4211: Multimedia Networking
12
MM Networking Applications Fundamental
characteristics:• Typically delay
sensitive– end-to-end delay– delay jitter
• But loss tolerant: infrequent losses cause minor glitches
• Antithesis of data, which are loss intolerant but delay tolerant.
Classes of MM applications:
1) Streaming stored audio and video
2) Streaming live audio and video
3) Real-time interactive audio and video
Jitter is the variability of packet delays within the same packet stream
CSci4211: Multimedia Networking
13
Application Classes
• Streaming– Clients request audio/video files from
servers and pipeline reception over the network and display
– Interactive: user can control operation (similar to VCR: pause, resume, fast forward, rewind, etc.)
– Delay: from client request until display start can be 1 to 10 seconds
CSci4211: Multimedia Networking
14
Application Classes (more)• Unidirectional Real-Time:
– E.g., real-time video broadcasting of a sport event– similar to existing TV and radio stations, but delivery
on the network– Non-interactive, just listen/view
• Interactive Real-Time :– Phone conversation or video conference– E.g., skype, Google handout, VoIP & SIP, …– More stringent delay requirement than Streaming
and Unidirectional because of real-time nature– Video: < 150 msec acceptable– Audio: < 150 msec good, <400 msec acceptable
CSci4211: Multimedia Networking
15
Multimedia Over Today’s InternetTCP/UDP/IP: “best-effort service”• no guarantees on delay, loss
Today’s Internet multimedia applications use application-level techniques to mitigate
(as best possible) effects of delay, loss
But you said multimedia apps requiresQoS and level of performance to be
effective!
?? ???
?
? ??
?
?
CSci4211: Multimedia Networking
16
Challenges• TCP/UDP/IP suite provides best-effort, no
guarantees on expectation or variance of packet delay
• Streaming applications delay of 5 to 10 seconds is typical and has been acceptable, but performance deteriorate if links are congested (transoceanic)
• Real-Time Interactive requirements on delay and its jitter have been satisfied by over-provisioning (providing plenty of bandwidth), what will happen when the load increases?...
CSci4211: Multimedia Networking
17
Internet (Stored) Multimedia: Simplest Approach
audio, video not streamed:• no, “pipelining,” long delays until
playout!
• audio or video stored in file• files transferred as HTTP object
– received in entirety at client– then passed to player
CSci4211: Multimedia Networking
18
Streaming Stored Multimedia
Streaming: • media stored at source• transmitted to client• streaming: client playout
begins before all data has arrived
• timing constraint for still-to-be transmitted data: in time for playout
CSci4211: Multimedia Networking
19
Streaming Stored Multimedia: What is it?
1. Videopre-recorded
2. videosent
3. video received,played out at client
Cum
ula
tive
data
streaming: at this time, client playing out early part of video, while server still sending laterpart of video
networkdelay
time
CSci4211: Multimedia Networking
20
Internet multimedia: Streaming Approach
• browser GETs metafile• browser launches player, passing metafile• player contacts server
• server streams audio/video to player CSci4211: Multimedia Networking
21
Streaming from a streaming server
• This architecture allows for non-HTTP protocol between server and media player
• Can also use UDP instead of TCP.
CSci4211: Multimedia Networking
22
Streaming Stored Multimedia
Application-level streaming techniques for making the best out of best effort service:– client side buffering– use of UDP versus TCP– multiple encodings of
multimedia
• jitter removal• decompression• error concealment• graphical user interface
w/ controls for interactivity
Media Player
23
Streaming Multimedia: UDP or TCP?
UDP • server sends at rate appropriate for client (oblivious to network congestion !)
– often send rate = encoding rate = constant rate– then, fill rate = constant rate - packet loss
• short playout delay (2-5 seconds) to compensate for network delay jitter• error recover: time permitting
TCP• send at maximum possible rate under TCP• fill rate fluctuates due to TCP congestion control• larger playout delay: smooth TCP delivery rate• HTTP/TCP passes more easily through firewalls
24
video sent at Certain bit rates
Cum
ula
tive
data
time
variablenetwork
delay
client videoreception
constant bit rate video playout at client
client playoutdelay
bu
ffere
dvid
eo
Streaming Multimedia: Client Buffering
• Client-side buffering, playout delay compensate for network-added delay, delay jitter
25
Streaming Multimedia: Client Buffering
• Client-side buffering, playout delay compensate for network-added delay, delay jitter
bufferedvideo
variable fill
rate, x(t)
“constant”
drain rate,
d
CSci4211: Multimedia Networking
26
Streaming Stored Multimedia: Interactivity
• VCR-like functionality: client can pause, rewind, FF, push slider bar– 10 sec initial delay OK– 1-2 sec until command effect
OK
• timing constraint for still-to-be transmitted data: in time for playout
CSci4211: Multimedia Networking
27
Streaming Live Multimedia
Examples:• Internet radio talk show• Live sporting eventStreaming• playback buffer• playback can lag tens of seconds after transmission• still have timing constraintInteractivity• fast forward impossible• rewind, pause possible!
CSci4211: Multimedia Networking
28
Interactive, Real-Time Multimedia
• end-end delay requirements:– audio: < 150 msec good, < 400 msec OK
• includes application-level (packetization) and network delays• higher delays noticeable, impair interactivity
• session initialization– how does callee advertise its IP address, port number, encoding
algorithms?
• applications: IP telephony, video conference, distributed interactive worlds
CSci4211: Multimedia Networking
Large-scale Internet Video Delivery:
YouTube & Netflix Case Studies Based on two active measurement studies we have conducted
•Reverse-engineering YouTube Delivery Cloud– Google’s New YouTube Architectural Design
•Unreeling Netflix Video Streaming Service– Cloud-sourcing: Amazon Cloud Services & CDNs
CSci4211: Multimedia Networking 29
YouTube Video Delivery Basics
User
Front end web-servers
Video-servers(front end)
1. HTTP GET request for video URL
2. HTTP reply containing html to construct the web page and a link to stream the FLV file
3. HTTP GET requestfor FLV stream
4. HTTP replyFLV stream
Internet
CSci4211: Multimedia Networking 30
Google’s New YouTube Video Delivery Architecture
Three components
• Videos and video id space
• Physical cache hierarchy three tiers: primary, secondary, & tertiary primary caches: “Google locations” vs. “ISP locations”
• Layered organization of namespaces representing “logical” video servers five “anycast” namespaces two “unicast” namespaces
Implications: a)YouTube videos are not replicated at all locations!b)only replicated at (5) tertiary cache locationsc) Google likely utilizes some form of location-aware load-balancing (among primary cache locations)
CSci4211: Multimedia Networking 33
YouTube Video Id Space
• Each YouTube video is assigned a unique id e.g., http://www.youtube.com/watch?v=tObjCw_WgKs
• Each video id is 11 char string• first 10 chars can be any alpha-numeric values [0-9, a-
z, A-Z] plus “-” and “_”• last char can be one of the 16 chars {0, 4, 8, ..., A,
E, ...}
Video id space size: 6411
Video id’s are randomly distributed in the id space
CSci4211: Multimedia Networking 34
Physical Cache Hierarchy & Locations
~ 50 cache locations• ~40 primary locations
• including ~10 non-Google ISP locations
• 8 secondary locations• 5 tertiary locations
P: primary P: primary S: secondaryT: Tertiary
Geo-locations using • city codes in unicast
hostnames, e.g., r1.sjc01g01.c.youtube.com• low latency from PLnodes (<
3ms)• clustering of IP addresses
using latency matrix
CSci4211: Multimedia Networking 35
Layered Namespace Organization
Two types of namespaces– Five “anycast” namespaces
• lscache: “visible” primary ns• each ns representing fixed #
of “logical” servers• logical servers mapped to physical servers via DNS
– 2 “unicast” namespaces• rhost: google locations• rhostisp: ISP locations• mapped to a single server
CSci4211: Multimedia Networking 36
YouTube Video Delivery Dynamics: Summary
• Locality-aware DNS resolution• Handling load balancing & hotspots
– DNS change– Dynamic HTTP redirection– local vs. higher cache tier
• Handling cache misses– Background fetch– Dynamic HTTP redirection
CSci4211: Multimedia Networking 37
What Makes Netflix Interesting?• Commercial, feature-length movies and TV shows
and not free; subscription-based
• Nonetheless, Netflix is huge! 25 million subscribers and ~20,000 titles (and growing) consumes 30% of peak-time downstream bandwidth in North
America• A prime example of cloud-sourced architecture
Maintains only a small “in-house” facility for key functions e.g., subscriber management (account creation, payment,
…) Majority of functions are sourced to Amazon cloud (EC2/S3)
user authentication, video search, video storage, … DNS service is sourced to UltraDNS Leverage multiple CDNs (content-distribution networks) for
video delivery Akamai, Level 3 and Limelight
CSci4211: Multimedia Networking
Netflix Architecture• Netflix has its own “data center” for certain crucial
operations (e.g., user registration, billing, …)• Most web-based user-video interaction,
computation/storage operations are cloud-sourced to Amazon AWS
• Video delivery is out/cloud-sourced to 3 CDNs • Users need to use MS Silverlight player for video
streaming
CSci4211: Multimedia Networking
Netflix Videos and Video Chunks
• Netflix uses a numeric ID to identify each movie– IDs are variable length (6-8 digits): 213530,
1001192, 70221086– video IDs do not seem to be evenly distributed in
the ID space– these video IDs are not used in playback operations
• Each movie is encoded in multiple quality levels, each is identified by a numeric ID (9 digits)– various numeric IDs associated with the same movie appear
to have no obvious relations
CSci4211: Multimedia Networking 40
Netflix Videos and Video Chunks • Videos are divided in “chunks” (of roughly 4 secs),
specified using (byte) “range/xxx-xxx?” in the URL path: Limelight:
http://netflix-094.vo.llnwd.net/s/stor3/384/534975384.ismv/range/0-57689?p=58&e=1311456547&h=2caca6fb4cc2c522e657006cf69d4ace
Akamai:
http://netflix094.as.nflximg.com.edgesuite.net/sa53/384/534975384.ismv/range/0-57689?token=1311456547_411862e41a33dc93ee71e2e3b3fd8534
Level3:
http://nflx.i.ad483241.x.lcdn.nflximg.com/384/534975384.ismv/range/0-57689?etime=20110723212907&movieHash=094&encoded=06847414df0656e697cbd
• Netflix uses a version of (MPEG-)DASH for video streaming
CSci4211: Multimedia Networking 41
CSci4211: Multimedia Networking
DASH: dynamic adaptive streaming over
HTTP • Not really a protocol; it provides formats to enable efficient and high-quality delivery of streaming services over the Internet– Enable HTTP-CDNs; reuse of existing technology (codec, DRM,…)– Move “intelligence” to client: device capability, bandwidth adaptation, …
• In particular, it specifies Media Presentation Description (MPD)
Ack & ©: Thomas Stockhammer
42
CSci4211: Multimedia Networking
DASH Data Model and Manifest Files • DASH MPD: Segment Info
Initialization Segment http://www.e.com/ahs-5.3gp
Initialization Segment http://www.e.com/ahs-5.3gp
Media PresentationPeriod, start=0s
…
Period, start=100s
…
Period, start=295s
…
…
Period, •start=100•baseURL=http://www.e.com/
Representation 1500kbit/s
…
Representation 2100kbit/s
…Representation 1•bandwidth=500kbit/s•width 640, height 480
Segment Infoduration=10s
Template:./ahs-5-$Index$.3gs
…
Media Segment 1start=0shttp://www.e.com/ahs-5-1.3gs
Media Segment 1start=0shttp://www.e.com/ahs-5-1.3gs
Media Segment 2start=10shttp://www.e.com/ahs-5-2.3gs
Media Segment 2start=10shttp://www.e.com/ahs-5-2.3gs
Media Segment 3start=20shttp://www.e.com/ahs-5-3.3gh
Media Segment 3start=20shttp://www.e.com/ahs-5-3.3gh
Media Segment 20start=190shttp://www.e.com/ahs-5-20.3gs
Media Segment 20start=190shttp://www.e.com/ahs-5-20.3gs
• Segment Indexing: MPD only; MPD+segment; segment only
Segment Index in MPD only
<MPD> ... <URL sourceURL="seg1.mp4"/> <URL sourceURL="seg2.mp4"/></MPD>
<MPD> ... <URL sourceURL="seg1.mp4"/> <URL sourceURL="seg2.mp4"/></MPD>
seg1.mp4seg1.mp4
seg2.mp4seg2.mp4
......
<MPD> ... <URL sourceURL="seg.mp4" range="0-499"/> <URL sourceURL="seg.mp4" range="500-999"/></MPD>
<MPD> ... <URL sourceURL="seg.mp4" range="0-499"/> <URL sourceURL="seg.mp4" range="500-999"/></MPD>
seg.mp4seg.mp4
Ack & ©: Thomas Stockhammer43
Netflix Manifest Files• A manifest file contains metadata• Netflix manifest files contain a lot of information
o Available bitrates for audio, video and trickplayo MPD and URLs pointing to CDNso CDNs and their "rankings"
<nccp:cdn> <nccp:name>level3</nccp:name> <nccp:cdnid>6</nccp:cdnid> <nccp:rank>1</nccp:rank> <nccp:weight>140</nccp:weight> </nccp:cdn> <nccp:cdn> <nccp:name>limelight</nccp:name> <nccp:cdnid>4</nccp:cdnid> <nccp:rank>2</nccp:rank> <nccp:weight>120</nccp:weight> </nccp:cdn> <nccp:cdn> <nccp:name>akamai</nccp:name> <nccp:cdnid>9</nccp:cdnid> <nccp:rank>3</nccp:rank> <nccp:weight>100</nccp:weight> </nccp:cdn>
CSci4211: Multimedia Networking 44
Netflix Manifest Files …A section of the manifest containing the base URLs, pointing to CDNs<nccp:downloadurls> <nccp:downloadurl>
<nccp:expiration>1311456547</nccp:expiration> <nccp:cdnid>9</nccp:cdnid>
<nccp:url>http://netflix094.as.nflximg.com.edgesuite.net/sa73/531/943233531.ismv?token=1311456547_e329d4271a7ff72019a550dec8ce3840</nccp:url> </nccp:downloadurl> <nccp:downloadurl>
<nccp:expiration>1311456547</nccp:expiration> <nccp:cdnid>4</nccp:cdnid>
<nccp:url>http://netflix-094.vo.llnwd.net/s/stor3/531/943233531.ismv?p=58&e=1311456547&h=8adaa52cd06db9219790bbdb323fc6b8</nccp:url> </nccp:downloadurl> <nccp:downloadurl>
<nccp:expiration>1311456547</nccp:expiration> <nccp:cdnid>6</nccp:cdnid>
<nccp:url>http://nflx.i.ad483241.x.lcdn.nflximg.com/531/943233531.ismv?etime=20110723212907&movieHash=094&encoded=0473c433ff6dc2f7f2f4a</nccp:url> </nccp:downloadurl> </nccp:downloadurls>
CSci4211: Multimedia Networking 45
Netflix: Adapting to Bandwidth Changes
• Two possible approaches Increase/decrease quality level using DASH Switch CDNs
• Experiments Play a movie and systematically throttle available
bandwidth Observe server addresses and video quality
• Bandwidth throttling using the “dummynet” tool Throttling done on the client side by limiting how fast it
can download from any given CDN server First throttle the most preferred CDN server, keep
throttling other servers as they get selected
46CSci4211: Multimedia Networking
Adapting to Bandwidth Changes
• Lower quality levels in response to lower bandwidth• Switch CDN only when minimum quality level cannot
be supported• Netflix seems to use multiple CDNs only for failover
purposes!
47CSci4211: Multimedia Networking
CDN Bandwidth Measurement• Use both local residential hosts and PlanetLab nodes
13 residential hosts and 100s PlanetLab nodes are used Each host downloads small chunks of Netflix videos from all three CDN
servers by replaying URLs from the manifest files• Experiments are done for several hours every day for about 3
weeks total experiment duration was divided into 16 second intervals the clients downloaded chunks from CDN 1, 2 and 3 at the beginning
of seconds 0, 4 and 8. at the beginning of the 12th second, the clients tried to download the
chunks from all three CDNs simultaneously• Measure bandwidth to 3 CDNs separately as well as the combined
bandwidth• Perform analysis at three time-scales
average over the entire period daily averages instantaneous bandwidth
48CSci4211: Multimedia Networking