View
220
Download
0
Embed Size (px)
Citation preview
Reinventing Compression:Reinventing Compression:The New Paradigm ofThe New Paradigm of
Distributed Video CodingDistributed Video Coding
Reinventing Compression:Reinventing Compression:The New Paradigm ofThe New Paradigm of
Distributed Video CodingDistributed Video Coding
BerndBernd Girod Girod
Information Systems LaboratoryInformation Systems LaboratoryStanford UniversityStanford University
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 2
OutlineOutline
Lossless and lossy compression with receiver side information
Shifting the complexity of video encoding to the decoder
Error-resilient video transmission Image authentication
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 3
OutlineOutline
Lossless and lossy compression with receiver side information
Shifting the complexity of video encoding to the decoder
Error-resilient video transmission Image authentication
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 4
Encoder Encoder Decoder Decoder XX
YY
Lossless Compression Lossless Compression with Side Informationwith Side Information
R ≥ H(X|Y)
Statistically dependent
Encoder Encoder Decoder Decoder XX
Y
R ≥ ?
Statistically dependent
Side Information
Side Information
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 5
Encoder Encoder Decoder Decoder XX
YY
Lossless Compression Lossless Compression with Side Informationwith Side Information
R ≥ H(X|Y)
Statistically dependent
Encoder Encoder Decoder Decoder XX
Y
R ≥ H(X|Y)
Statistically dependent
[Slepian, Wolf, 1973]
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 6
Towards Practical Slepian-Wolf CodingTowards Practical Slepian-Wolf Coding
• Convolution coding for data compression [Blizard, 1969]• Convolutional source coding [Hellman, 1975]• Syndrome source coding [Ancheta, 1976]
• Coset codes [Pradhan and Ramchandran, 1999]• Trellis codes [Wang and Orchard, 2001]• Turbo codes
[García-Frías and Zhao, 2001] [Bajcsy and Mitran, 2001] [Aaron and Girod, 2002]
• LDPC codes [Liveris, Xiong, and Georghiades, 2002]• . . .
• . . .
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.70
0.1
0.2
0.3
0.4
0.5
0.6
0.7
H(X|Y)
Rat
e
Slepian-Wolf boundRate = H(X|Y)
Rate-adaptive turbo codes
Rate-Adaptive Slepian-Wolf CodingRate-Adaptive Slepian-Wolf Coding
X X Turbo Decoder
Turbo Encoder
Parity bitsEncoder Buffer
Request bits
Y
L = 8192 bitsTotal simulated bits = 226
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 8
Encoder Encoder Decoder Decoder X X
YY
Encoder Encoder Decoder Decoder X X
Y
Lossy Compression with Side InformationLossy Compression with Side Information
[Wyner, Ziv, 1976] For MSE distortion and Gaussian statistics, rate-distortion functions of the two systems are the same.
[Zamir, 1996] The rate loss R*(d) – RX|Y (d) is bounded.
RX|Y (d)
R*(d)
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 9
Practical Wyner-Ziv CodingPractical Wyner-Ziv Coding
Wyner-Ziv Decoder
QuantizerSlepian-
Wolf Encoder
Wyner-Ziv Encoder
Slepian-
WolfDecode
r
Minimum Distortion
Reconstruction
Y Y
X 'XQ Q
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 10
Non-Connected Quantization RegionsNon-Connected Quantization Regions
Example: Non-connected intervals for scalar quantization
Decoder: Minimum mean-squared error reconstruction with side information
x
1q 2q 3q
x
| |X Yf x y 2
ˆ
conditional centroid
ˆ ˆ | ,arg min x
x E X x y q
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 11
OutlineOutline
Lossless and lossy compression with receiver side information
Shifting the complexity of video encoding to the decoder
Error-resilient video transmission Image authentication
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 12
Interframe Video CodingInterframe Video Coding
PredictiveInterframe Decoder
PredictiveInterframe Decoder
PredictiveInterframe Encoder
PredictiveInterframe Encoder
Side Information
Y YX X’
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 13
Wyner-ZivInterframe Decoder
Wyner-ZivInterframe Decoder
Wyner-ZivIntraframe Encoder
Wyner-ZivIntraframe Encoder
Y
[Witsenhausen, Wyner, 1980] [Puri, Ramchandran, Allerton 2002][Aaron, Zhang, Girod, Asilomar 2002]
Low Complexity EncoderLow Complexity Encoder
X’
Side Information
X
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 15
Pixel-Domain Wyner-Ziv Video CodecPixel-Domain Wyner-Ziv Video Codec
Interframe Decoder
Scalar Quantizer
Turbo Encoder
Buffer
WZ frames
X
Intraframe Encoder
Turbo Decoder
Request bits
Slepian-Wolf Codec
Interpolation/ Extrapolation
Reconstruction
Y
Key frames
I Conventional Intraframe coding
Conventional Intraframe decoding
X’
I’
Side information
[Aaron, Zhang, Girod, Asilomar 2002]
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 16
Decoder side informationgenerated by motion-
compensated interpolationPSNR 30.3 dB
After Wyner-Ziv Decoding16-level quantization – 1.375 bpp
11 pixels in errorPSNR 36.7 dB
Pixel-Domain Wyner-Ziv Video CodecPixel-Domain Wyner-Ziv Video Codec
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 17
Pixel-Domain Wyner-Ziv Video CodecPixel-Domain Wyner-Ziv Video Codec
Decoder side informationgenerated by motion-
compensated interpolationPSNR 24.8 dB
After Wyner-Ziv Decoding16-level quantization – 2.0 bpp
0 pixels in errorPSNR 36.5 dB
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 18
Yk
IDCT
DCT-Domain Wyner-Ziv Video CodecDCT-Domain Wyner-Ziv Video Codec
Request bits
Interpolation/ Extrapolation
Recon
I Conventional Intraframe coding
Conventional Intraframe decoding
DCT
For each transform band k
I’
W’
Y
Scalar Quantizer
DCTTurbo
EncoderBuffer
Turbo Decoder
Side information
WZ frames
W
Key frames
Xk Xk’
Interframe Decoder
Intraframe Encoder
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 19
Rate-Distortion Performance - SalesmanRate-Distortion Performance - Salesman
Every 8th frame is a key frame
Salesman QCIF sequence at 10fps 100 frames
6 dB
3 dB
B frame ~ 100%
WZ DCT ~ 7%
WZ Pixel ~ 6%
I frame ~ 18%
Interframe 100%
Encoder Runtime Pentium 1.73 GHz machine
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 20
Rate-Distortion Performance – Hall MonitorRate-Distortion Performance – Hall Monitor
8 dB
3 dB
Every 8th frame is a key frame
Hall Monitor QCIF sequence at 10fps 100 frames
DCT-based Intracoding 149 kbps
PSNRY=30.0 dB
Wyner-Ziv DCT codec 152 kbps
PSNRY=35.6 dB GOP=8
Salesman at 10 fps
DCT-based Intracoding 156 kbps
PSNRY=30.2 dB
Wyner-Ziv DCT codec 155 kbps
PSNRY=37.1 dB GOP=8
Hall Monitor at 10 fps
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 23
OutlineOutline
Lossless and lossy compression with receiver side information
Shifting the complexity of video encoding to the decoder
Error-resilient video transmission Image authentication
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 24
Systematic Lossy Source/Channel CodingSystematic Lossy Source/Channel Coding
Information theoretic optimality conditions [Shamai, Verdú, Zamir, 1998] Enhancing analog image transmission using digital side information
[Pradhan, Ramchandran, 2001] Lossy source-channel coding of video waveforms
[Rane, Aaron, Girod, 2004,’05,’06]
Encoder Digital Channel
Decoder
Analog Channel
Wyner-ZivEncoder
Sideinfo
Wyner-ZivDecoder
Digital Channel
Wyner-ZivEncoder
Sideinfo
Wyner-ZivDecoder
Digital Channel
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 25
Systematic Lossy Error Protection (SLEP)Systematic Lossy Error Protection (SLEP)
VideoEncoder
Video Decoder With Error Concealment
InputVideo
VideoWith Errors
Cha
nnel
Wyner-ZivEncoder
Wyner-ZivDecoder
Side Information
OutputVideo
“Analog Channel”
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 26
SLEP using H.264/AVC Redundant SlicesSLEP using H.264/AVC Redundant Slices
Encode Redundant Pic(Requantize)
Entropy Decoding
WYNER-ZIV ENCODER WYNER-ZIV DECODER
Err
or-p
rone
Cha
nnel
Decode Redundant Slice
Motion Vecs +Coding Modes
ErasureDecoding
Side info
Motion Vecs +Coding Modes
QP
Recovered motion vectors for erroneously received primary slices
EncodePrimary Pic
Q-1 T-1
H.264/AVC DECODER
+Entropy Decoding
OutputVideo
H.264/AVC ENCODER
InputVideo
Encode Redundant Pic(Requantize)
RSEncoder
Parity SlicesQP
MC
Foreman @ 408 kbps, error resilience bit rate = 40 kbpsSymbol error probability = 5 x 10-4
QP = 28QP = 2835.7 dB35.7 dB
Error concealment onlyError concealment only40 kbps FEC40 kbps FECSLEP with redundant QP = 36SLEP with redundant QP = 36SLEP with redundant QP = 40SLEP with redundant QP = 40SLEP with redundant QP = 48SLEP with redundant QP = 4820.9 dB20.9 dB25.5 dB25.5 dB30.9 dB30.9 dB34.2 dB34.2 dB32.9 dB32.9 dB
Error-free After error propagation
100 kbps FEC
PSNR: 32.5 dB
Recovered 53.7 % of lost macroblocks
100 kbps Wyner-Ziv bit stream
PSNR: 38.0 dB
Recovered 96.6 % of lost macroblocks
Foreman @ 1 MbpsSymbol error probability = 2 x 10-4
Rally, 1 Mbps, 3% packet loss
80 kbps Wyner-Ziv bit stream38.1 dB
80 kbps FEC33.4 dB
Recovered 67.5 % of lost macroblocks Recovered 97.1 % of lost macroblocks
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 30
OutlineOutline
Lossless and lossy compression with receiver side information
Shifting the complexity of video encoding to the decoder
Error-resilient video transmission Image authentication
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 31
Media Authentication ProblemMedia Authentication Problem
Legimate degradation(e.g., compression)
Illegimate degradation(e.g., compression + tampering)
How to distinguish legimate and illegimate signal degradationswithout access to the original?
How to distinguish legimate and illegimate signal degradationswithout access to the original?
Original
Received
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 32
Image Authentication by Distributed CodingImage Authentication by Distributed CodingOriginal Received
Coarse approximationor (random) projection
Slepian-Wolfcoder
Slepian-Wolfdecoder
?
Side information
[Lin, Varodayan, Girod, ICIP 2007][Lin, Varodayan, Girod, MMSP 2007]
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 33
Image Authentication by Distributed CodingImage Authentication by Distributed CodingOriginal Received
Coarse approximationor (random) projection
Slepian-Wolfcoder
Slepian-Wolfdecoder
?
Side information
[Lin, Varodayan, Girod, ICIP 2007][Lin, Varodayan, Girod, MMSP 2007]
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 34
Image Authentication by Distributed CodingImage Authentication by Distributed Coding
Two-state Channel
Original Imagex
Image-to-be-authenticatedy
Two-state Channel
Original Imagex
Image-to-be-authenticatedy
Slepian-WolfEncoder
Slepian-Wolf Decoder
Slepian-Wolf Bitstream S(X)
Image Projection
X
RandomProjection
Side Information YRandom
ProjectionReconstructed Image
Projection X’
Random Seed Ks
Quantization
Random Seed Ks
Two-state Channel
Original Imagex
Image-to-be-authenticatedy
Slepian-WolfEncoder
Slepian-Wolf Decoder
Cryptographic Hash Function
Cryptographic Hash Function
DigitalSignatureD(X,Ks)
ImageDigest
Comparison
Slepian-Wolf Bitstream S(X)
Image Projection
X
RandomProjection
Side Information YRandom
ProjectionReconstructed Image
Projection X’
Asymmetric Encryption
Asymmetric Decryption
ImageDigest
Private Key Public Key
Random Seed Ks
Quantization
Random Seed Ks
[Lin, Varodayan, Girod, ICIP 2007]
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 35
Minimum Rate for Successful DecodingMinimum Rate for Successful Decoding
28 30 32 34 36 38 40 420
0.001
0.002
0.003
0.004
0.005
0.006
0.007
0.008
0.009
0.01
Reconstruction PSNR (dB)
Rat
e (b
its p
er p
ixel
of o
rigin
al im
age)
Minimum Rate for Illegitimate JPEG2000 State
Minimum Rate for Illegitimate JPEG State
Minimum Rate for Legitimate JPEG2000 StateMinimum Rate for Legitimate JPEG State
Selected Slepian-Wolf Bit Rate
Experiment:JPEG or JPEG2000 compression+ illegimate text banner
[Lin, Varodayan, Girod, ICIP 2007]
B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 37
Distributed Image/Video Coding:Distributed Image/Video Coding:Why Do We Care?Why Do We Care?
New Paradigm: Chance to Reinvent Compression from Scratch– Entropy coding– Quantization– Signal transforms– Adaptive coding– Rate control– . . .
Powerful New Tool in the Compression Tool-Box– Very low complexity encoders– Compression for networks of cameras– Error-resilient transmission of signal waveforms– Digitally enhanced analog transmission– Unequal error protection without layered coding– Image authentication – Random access– Compression of encrypted signals– . . .
Further interest:
B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero, "Distributed Video Coding," B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero, "Distributed Video Coding," Proceedings of the IEEE,Proceedings of the IEEE, Special Issue on Video Coding and Delivery. Special Issue on Video Coding and Delivery. January 2005.January 2005.
http://www.stanford.edu/~bgirod/pdfs/DistributedVideoCoding-IEEEProc.pdf http://www.stanford.edu/~bgirod/pdfs/DistributedVideoCoding-IEEEProc.pdf