Bernd Girod. Joint Source-Network Coding for Real-time Media 1
Joint Source-Network Coding for Real-time Media
Bernd Girod
Information Systems LaboratoryStanford University
Bernd Girod. Joint Source-Network Coding for Real-time Media 2
Message from the President of the IEEE Information Theory Society
“Dear Source Coders,
Don’t compress too much and consequently requireerror-free transmission. It makes the task for us channel coders very difficult. We mighthave to add more redundancy for error protection than you removed!”
Professor Joachim Hagenauer, 1996
Bernd Girod. Joint Source-Network Coding for Real-time Media 3
Outline
Joint source-channel coding– Why it’s good idea– Why it’s bad idea
Joint source-network coding– FEC and Smart Prefetching– Why joint source-channel coding is a good idea after all– Adaptive media playout– Packet Path Diversity
Bernd Girod. Joint Source-Network Coding for Real-time Media 4
Optimal Trade-Off Source Coding vs. Channel Coding
0 10 20 30 40 5010
20
30
40
50
60
70
80
90
100
25 % 25 %
25 %
25 %
25 %
9 %
9 % 9 %
9 %
9 %
6 %
6 % 6 %
6 %
6 %
2.5 %
2.5 %2.5 %
2.5 %
2.5 %
0.7 %
0.7 %0.7 %
0.7 %
0.7 %
Intra Rate [%]
Co
de
Ra
te [%
]
Burst Length:Memoryless
2 4
8
16
32
<< less more >>Source coding redundancy
<<
mo
re
les
s >
>C
ha
nn
el c
od
ing
re
du
nd
an
cy
K. Stuhlmüller, N. Färber, M. Link, and B. Girod, "Analysis of Video Transmission over Lossy Channels," IEEE Journal on Selected Areas in Communications, vol. 18, no. 6, pp. 1012-1032, June 2000.
Bernd Girod. Joint Source-Network Coding for Real-time Media 5
Shannon and the Internet
Appl. #1
TransportLayer
InternetworkingLayer
Appl. #2
Appl. #n • • •
Internet Open Architecture: 2000+
NetworkAccess
#1
NetworkAccess
#2
NetworkAccess
#m• • •Channel
coderChannel
coder
ChanneldecoderChanneldecoder
Channel
SourcecoderSourcecoder
SourcedecoderSourcedecoder
Separation PrincipleShannon, 1948
Joint Source-Channel CodingContradicts Open Architecture? Joint Source-Channel Coding
Contradicts Open Architecture?
How to Build a System that Works, When Shannon’s Separation Principle Doesn’t? I hope I have shown you that it is beneficial to consider source and channel coding jointly, and pay particular attention to source decoding also.
How to Build a System that Works, When Shannon’s Separation Principle Doesn’t? I hope I have shown you that it is beneficial to consider source and channel coding jointly, and pay particular attention to source decoding also.
Bernd Girod. Joint Source-Network Coding for Real-time Media 6
“Joint Source-Network Coding”“Joint Source-Network Coding”
From Joint Source-Channel Coding to Joint Source-Network Coding
Internet– Many different channels with link-layer error control– Application (= source coder) shielded from and unaware of physical
layer– Application “sees” late or lost packets, not bit-errors
What can be done nevertheless– Careful network adaptation layer design (Avoid “chop and ship”)– Application specific error control (for packets erasures, not bit
errors)– Jointly design and optimize
– Source coding/decoding, transcoding & error concealment– Forward error correction, packet scheduling and retransmission,
flow control, cacheing, routing
Bernd Girod. Joint Source-Network Coding for Real-time Media 7
FEC across Packets
information symbols
redundancy symbols
3packet
bitstream
code
wor
d K
N-K
1 01 110 010 ...
block of packets
network
Reed-Solomon Codes RS(N,K) with erasure decoding Any K of N packets can used to reconstruct the information packets.
U. Horn, K. Stuhlmüller, M. Link, and B. Girod, "Robust Internet Video Transmission Based on Scalable Coding and Unequal Error Protection," Image Communication, vol. 15, no. 1-2, pp. 77-94, Sept. 1999.
Bernd Girod. Joint Source-Network Coding for Real-time Media 8
Internet Media Streaming
Workstation
PC
DSL
56K modem
Media Server
Internet
GPRS
Receiver buffer & ARQ
Today: >10 sec delay
Bernd Girod. Joint Source-Network Coding for Real-time Media 9
Media Streaming with Smart Prefetching
Basic idea:
Retransmission protocol employing prefetching of more important information
Server ClientInternetInternet
Request stream
Request stream
Rate-distortionpreamble
Rate-distortionpreamblePrefetch
times
Prefetchtimes
Media dataMedia data
Repeatrequest
RepeatrequestRepeatrequest
RepeatrequestRepeatrequest
Repeatrequest
Bernd Girod. Joint Source-Network Coding for Real-time Media 10
Smart Prefetching 2-Layer Video Results
Single Layer Scalable: Optimal Scalable: Equal Prefetch
0 0.2 0.4 0.6 0.8 10
100
200
300
400
500
600
700
Packet loss rate
MS
E d
isto
rtio
n at
dec
oder
Singlelayer Two
layers,equal
prefetchTwo
layers,optimal
prefetch
Bernd Girod. Joint Source-Network Coding for Real-time Media 11
Adaptive Media Playout
State-of-the-Art Improved scheme
Idea: reduce latency and packet loss simultaneously by continuously adapting playout deadline to network conditions
Idea: reduce latency and packet loss simultaneously by continuously adapting playout deadline to network conditions
5% packet loss2 sec average receiver buffer
5% packet loss2 sec average receiver buffer
E. Steinbach, N. Färber and B. Girod, "Adaptive Playout for Low-Latency Video Streaming," Proc. International Conference on Image Processing, ICIP-2001, Thessaloniki, Greece, October 2001.
Bernd Girod. Joint Source-Network Coding for Real-time Media 12
Adaptive Media Playout
Initialdelay
Initialdelay
Start of playoutStart of playout
Throughput
Receiver buffer level
t
t
Target level
Target rate
Reduced playout speedReduced playout speed
buffer underflowavoided
buffer underflowavoided
Bernd Girod. Joint Source-Network Coding for Real-time Media 13
Reduced Pre-roll Time for Stored Streams
Probabilityof buffer
underflow< 1%
Probabilityof buffer
underflow< 1%
M. Kalman, E. Steinbach, and B. Girod, "Adaptive Media Playout for Low Delay Video Streaming over Error-Prone Channels," IEEE Transactions on Circuits and Systems for Video Technology, Special Issue on Wireless Video, submitted August 2000.
Bernd Girod. Joint Source-Network Coding for Real-time Media 14
Packet Path Diversity
Idea: set up multiple connections along different paths
Improved congestion resiliency– lower combined latency– better loss characteristics
Realization in current Internet, e.g., through relay server
D
S
RelayRelay1 2
crosstraffic
crosstrafficMedia traffic
Bernd Girod. Joint Source-Network Coding for Real-time Media 15
Multipath Voice Streaming Experiment
Exodus Comm.
Exodus Comm.
BBN PlanetBBN Planet
Netergy Networks
192.84.16.176
MIT18.184.0.50
(5 ms) (45 ms)(5 ms)
Source
Destination
QwestQwest
Harvard140.247.62.110
(40 ms) (5 ms)
Relay
Bernd Girod. Joint Source-Network Coding for Real-time Media 16
Adaptive playout with two streams
Yi J. Liang, E. Steinbach and B. Girod, "Multi-stream Voice Transmission over the Internet Using Path Diversity," Proc. ACM Multimedia 2001, Ottawa, Canada, Sept./Oct. 2001.
Bernd Girod. Joint Source-Network Coding for Real-time Media 17
Multiple Description Coding for VoIP
EStream 1
Stream 2
O E O E O
O E O E O E
E O
packet i i+1 i+2 i+3
Even samples: 8 bit Odd samples: 4 bit
Odd samples: 8 bitEven samples: 4 bit
O E
Bernd Girod. Joint Source-Network Coding for Real-time Media 18
Packet Path Diversity: Loss vs. Delay
>45%
Bernd Girod. Joint Source-Network Coding for Real-time Media 19
Conclusions
Joint source-channel coding– Improved performance, if Separation Principle does not hold– Non-obvious trade-off between source coding and channel coding
Joint source-network coding– Jointly optimize source coding/decoding and packet transport– Adaptive media playout: “real-time” more flexible than we thought– FEC across packets and smart prefetching can provide virtual priority
mechanisms– Packet Path Diversity circumvents congestion, reduces latency
Many good & relevant research problems!Many good & relevant research problems!