Seminar On

ERROR CONTROL TECHNIQUES FOR VIDEO COMMUNICATIONS

Prepared By: SHUBHI SINGH

CONTENT Video Communication System Application Challenges Video Compression Compression Techniques Types Of errors Error detection Error detection approaches Forward error correction Error Resilient Error Concealment

Background & Motivation

3

Video becoming more popular

Advances in bandwidth, capacity enhancements

Requirements: ◦ data transmission rate◦ Real-time delivery of multimedia data with least errors

Limitation: ◦ QoS available is not sufficient to guarantee error-free delivery for all receivers

Motivation:◦ Provide means of dealing with various transmission impairments

Video Communication Systems

4

End-to-End Video Transmission

Video Communication Applications• Video storage, e.g. DVD or Video CD• Videophone over PSTN• Videoconferencing over ISDN• Digital TV• Video streaming over the Internet• Wireless video

◦ Videophone over cellular◦ Video over 3G and 4G networks: Interactive games, etc.

Contd….. The video Quality depends on variety of applications:

Minimal undesired attributes.

The compressed video may be afflicted by compression artifacts.

Display device, environmental viewing condition and viewers itself.

Different types of video content influence the video quality.

The quality of audio is paramount for the entire experience.

http://www.spirent.com/Products/ProLab/ProLab_H323 https://vsee.com/blog/tag/hd-video-calling/

Challenges In Video Communication

1. Network and bandwidth constraints 2. Quality Of service 3. Technical Challenges

4. Acquisition Quality 5. Compression 6. Video and audio content

8

Video CompressionWhy video compression technique is important ?

One movie video without compression◦ 720 x 480 pixels per frame◦ 30 frames per second◦ Total 90 minutes◦ Full color

The total quantity of data = 167.96 G Bytes !!

Alternative description of data requiring less storage and bandwidth.

Uncompressed 1 Mbyte

Compressed (JPEG) 50 Kbyte (20:1)

9

Compression / Decompression

Spatial (intra-frame) compression:Compresses each frame in isolation, treating it as

a bitmapped image.Based on quantization of DCT coefficients.

Temporal (inter-frame) compression:Compresses sequences of frames by only storing

differences between them.Record displacement of object plus changed pixels in area

exposed by its movement.Based on Motion Compensation (MC).

Video Compression

Types of Errors

Contd… Single bit errors are the least likely type of errors in serial data transmission because the noise must have a very short duration which is very rare. However this kind of errors can happen in parallel transmission.

The term burst error means that two or more bits in the data unit have changed from 1 to 0 or from 0 to 1.Burst errors does not necessarily mean that the errors occur in consecutive bits, the length of the burst is measured from the first corrupted bit to the last corrupted bit. Some bits in between may not have been corrupted.

Burst error is most likely to happen in serial transmission since the duration of noise is normally longer than the duration of a bit . The number of bits affected depends on the data rate and duration of noise.

13

An example of effect of transmission errors on a compressed video stream.

Coded,No loss

5%

3%

Example of reconstructed video frames from a H.263 coded sequence, subject to packet losses .

10%

http://www.slideshare.net/iamaproudindian/error-resilient-video

Error Propagation

http://research.kustar.ac.ae/pgs/sites/husameldin/about-my-project/

Error Detection Error detection means to decide whether the received data is correct or not without having a copy of the original message. Error detection uses the concept of redundancy, which means adding extra bits for detecting errors at the destination. Some of the important error detection methods are as follows:

Detection methods

Parity check CRC Checksum Repetition

Codes

Different Detection Approaches

Detection By Syntax Analysis: A first error analysis consists of subdividing detectable errors in different sets depending on their characteristics: Illegal Code word, Out of Range Code word and Contextual Error.

The decoding of a slice containing an error at the position b can be described by three intervals illustrated in the below Figure:

Interval [a,b): The slice is correctly decoded from its begin up to the error at the position b. Interval [b,c): The error is undetected until the position c b. This part is decoded incorrectly. Interval [c,d]: Starting from the position c until the end of the slice d concealment is used.

Some other detection approaches are:

Exploitation of header information: Error can be handled by adding the header information at the encoder side i.e. adding the redundancy.

Pattern introduction at encoder: In this method the error handling is described through the introduction of the framing pattern at the encoder.

For PCM and DPCM: Comparing the pel values with the threshold

Detection through DCT coefficients: In order to detect the damage to a single DCT coefficient by examining the difference between the boundary pixels in a block and its four neighbor blocks.

Data hiding technique: The basic idea of error detection scheme is using data hiding techniques to embed useful information at encoder side and extract them at decoder side for video error detection

Spatial and temporal characteristic: For spatial characteristic, AIDB is calculated and for temporal characteristic, ADF is calculated.

Forward Error Correction A FEC code is a system of adding redundant data, or parity data, to a message, such that it can be recovered by a receiver even when a number of errors were introduced, either during the process of transmission, or on storage. Since the receiver does not have to ask the sender for retransmission of the data, a back-channel is not required in this, and it is therefore suitable for simplex communication such as broadcasting. It is frequently used in lower layer communication.

Forward error correction

Some Correction codes are:BCH Codes

Hamming Codes

Walsh-Hadamard Codes

Reed-Solomon Codes

Low Density Parity Check Codes(LDPC)

Turbo Codes

Turbo code application

Reed-Solomon code application

http://phys.org/news/2013-01-nasa-mona-lisa-lunar-reconnaissance.html

Error Resilience Error resilience techniques enable the compressed bit-stream to resist channel errors so that the

impact on the reconstructed image quality is minimal.

Error Resiliency

1

Compression

Because, generally, the error resiliency schemes introduce some redundancy in the data. On the other hand, compression schemes aim to remove various redundancies from the data

Error resilience Error Resilience Employed w.r.t ProfilesBaseline profile includes some enhanced error resilience tools Flexible Macroblock Ordering (FMO), Arbitrary Slice Ordering (ASO), and Redundant Slices (RS)

Extended profile adds further error resilience support in the form of data partitioning (DP).Main profile does not include enhanced error resilience tools like FMO, ASO, RS, DP, SP or SI Slices.

Error Resiliency Tools

Flexible Macroblock Ordering (FMO)

Arbitrary Slice Ordering (ASO)

Data Partitioning (DP)

Redundant Slices (RS)

SP/SI frame for bit stream switching

Reference Frame Selection

Intra-block refreshing by R-D control.

Random Macroblock Intra Refresh

Header error resilience

Parameter Sets SEI messages

Entropy Coding

Slice resync. markers RVLC

Bidirectional bit stream reversal

(a) Resync. Marker with zero motion(b) Intra fresh with zero motion(c) Resync marker with hybrid concealment(d) Intra fresh with hybrid concealment

http://link.springer.com/chapter/10.1007%2F978-3-642-13681-8_40#page-1

Frame level resilience

ASO Data Partitioning

FMO Redundant Slices

SP/SI Multiple Reference

Periodic I-frame

Intra-refresh MB (cyclic)

Intra-refresh MB (random)

Intra-refresh MB (adaptive)

FMO ASO SP/SI

Contd……

Recovery or estimation of lost information due to transmission errors.

Packet losses typically lead to the loss of an isolated segment of a frame.

The lost region can be “recovered” based on the received regions by spatial/temporal interpolation.

24

ERROR CONCEALMENT

Fig.5 Illustration of Decoder Error Concealment

Error Concealment AlgorithmSpatial Concealment – weighted averaging:

Estimate missing pixels by smoothly extrapolating surrounding pixelsCorrectly recovering missing pixels is extremely difficult, however correctly estimating the DC (average) value is

very helpful

Temporal Concealment – copy algorithm: Copy the pixels at the same spatial location in the previous frame Effective when there is no motion, potential problems when there is motion

Motion compensated temporal Concealment–motion vector interpolation:Estimate missing block as motion-compensated block from previous frameCan use coded motion vector, neighboring motion vector, or compute new motion vector

Hybrid error concealment:Combination of intra and inter frame domainSpatio-temporal based approach

Spatial Concealment – weighted averaging (contd.)

Recovery of the damaged macroblock in Foreman and Akiyo video sequence (a) distorted image lying within a smooth area; b) macroblock based weighted averaging applied on a white smooth area; c) block based weighted averaging applied on a white smooth area.

Temporal Concealment – Frame Copy

Frames# 5, 6 and 7 are the output of H.264 encoded frames after it is transmitted in the error prone wireless medium

Frame# 5 is the decoded frame. Here Frame# 6 successfully copied lost information from Frame 5 by copy algorithm; Frame #7 is degraded (Because Frame#7 is reconstructed by collecting the information from previous reference frames)

Motion Vector Interpolation (contd.)

Recovery of the damaged macroblock in Foreman video sequence (a) original sequence b) Distorted Sequence c) Concealed Output using Motion Estimation.

ConclusionReal-time video communication doesn’t require lossless delivery rather signal-reconstruction and error-concealment techniques are more effective. Though, Burstiness of error in transmission has a significant impact on the choice of algorithms for concealment or resiliency.

Various error detection schemes are highlighted where syntax analysis is the old approach on the other side, the exploitation of picture characteristic is most widely used technique.

The error resilient methods are used to avoid the channel error or error propagation. It is applied at the encoder side and various methods are used according to the requirement .The frame level techniques are most commonly used which provide better efficiency than other.

And at the concealment level, the motion compensation techniques are widely used and provide better shield from the errors at the decoder end. Thus, motion vector play an important role while applying concealment techniques.

ReferencesJohn G. Apostolopoulos and Amy R. Reibman, “The Challenge of Estimating Video Quality in Video Communication Applications”, pp.155-158, IEEE signal processing magazine, March 2012.

Iain E. Richardson. " The H.264 Advanced Video Compression Standard”, Second Edition book

Yih-Farn Robin Chen, “Mobile Videos Where Are We Headed?” IEEE Computer Society, pp.86 89, January/February 2015.

Susan O'Donnell, Sonja Perley, Deanne Simms, “Challenges for Video Communications in Remote and Rural Communities,” IEEE, 2008.

Jerry D. Gibson, Al Bovik “Handbook of Image and Video Processing”, Academic Press, 2000.

S. Dogan, A. H. Sadka and A. M. Kondoz “Error-Resilient Techniques For Video Transmission Over Wireless Channels”, Centre for Communication Systems Research (CCSR), pp 1-25.

Yao Wang and Qin-Fan Zhu, “Error Control and Concealment for Video Communication: A Review,” IEEE, vol. 86, no. 5, May 1998.

Rajeshwar Dass, Lalit Singh, Sandeep Kaushik G. “Video Compression Technique ,” International Journal of Scientific & Technology Research Volume 1, Issue 10, pp 114-119, November 2012.

Rubal Chaudhary, Vrinda Gupta., “Error Control Techniques and Their Applications,” International Journal of Computer Applications in Engineering Science,Vol 1, Issue II , pp 187-191, June 2011.

Muhammad Usman, Xiangjian He, Min Xu, Kin Man Lam, “Survey of Error Concealment Techniques: Research Directions and Open Issues,” IEEE, pp. 233-238, 2015.

Contd….

Martin Fleury*, Sandro Moiron, and Mohammed Ghanbari, “Innovations in Video Error Resilience and Concealment,” School of Computer Sci. and Electronic Eng., University of Essex, pp. 1-15.

A. Garzelli, A. Andreadis, G. Benelli and S. Susini, “Error Resilience Coding”, in International Conference on Image Processing, Santa Barbara, CA, pp. 1-23, Oct. 1997.

Yao Wan, Stepbun WenJev, Jianjtao Wen, and Aaelos IC Igtsaaelos, “Video Coding Techniques,” IEEE signal processing Magazine, pp.61 -82, JULY 2000.

Khalid Mohamed Alajel, Wei Xiang, and John Leis “Error Resilience Performance Evaluation of H.264 I-frame and JPWL for Wireless Image Transmission”, IEEE, 2010

Online: “Still Image Compression Standards”, Version 2 ECE IIT, Kharagpur.

Eva Rodriguez Rodriguez, “Robust Error Detection Methods for H.264/AVC Videos”, Universitat Politecnica de Catalunya – EPSC Technical University of Vienna – Institute of Communications and Radio-Frequency Engineering, pp 1-63,2008.

Luca Superiori, Olivia Nemethova and Markus Rupp, “Performance of a H.264/AVC Error Detection Algorithm Based on Syntax Analysis”, Int. Conf. on Advances in Mobile Computing and Multimedia (MoMM), Yogyakarta, Indonesia, Dec. 2006

“Multiplexing protocol for low bit rate multimedia Communication” ITU-T H.223 Telecommunication Standardization Sector of ITU (07/2001).

“Video Codec For Audiovisual Services at p*64 kbits” International Telecommunication Union ITU-T H.261 Telecommunication Standardization Sector of ITU (03/93).

K. N. Ngan and R. Steele “Enhancement of PCM and DPCM Images Corrupted by Transmission Errors." Selected Areas in Communications, IEEE Transactions on Communication, Vol.-30, and Issue. 1, pp-257-265, January 1982.

Contd…

0. Robert Mitchell, and Ali J. Tabatabai, “Channel Error Recovery for Transform Image Coding”, IEEE Transactions on Communication, Vol.29, Issue. 12, pp.1754-1762, December 1981.

Zhi- Wei Gao and Wen-Nung Lie, “MPEG-4 Video Error Detection by using Data Hiding Techniques,” IEEE, Issue-III, pp. 397-400, 2002.

Guan-Lin Wu and Shao-Yi Chien. “Spatial-Temporal Error Detection Scheme for Video Transmission over Noisy Channels" Ninth IEEE International Symposium on Multimedia, pp. 78-85, 2007.

Md. Mehedi Hasan and Oksam Chae, “Faster Detection of Independent Lossy Compressed Block Errors in Images and Videos,” International Journal of Signal Processing, Image Processing and Pattern Recognition Vol. 5, No. 1, March, 2012.

Yu Chen, Keman Yu, Jiang Li and Shipeng Li“An Error Concealment Algorithm For Entire Frame Loss in Video Transmission, “Department of Electronic Engineering, Tsinghua University Microsoft Research Asia.

Yuan Zhou, Wei Xiang, and Gengkun Wang,” Frame Loss Concealment for Multiview Video Transmission Over Wireless Multimedia Sensor Networks”, IEEE Sensors Journal, Vol. 15, Issue. 3, pp. 1892-1901, March 2015

DI Olivia Nemethova, Prof. Dr. Markus Rupp, Dr. Michel Kieffer “Error Resilient Transmission of Video Streaming over Wireless Mobile Networks”. eingereicht an der Technischen Universit¨at Wien Fakult¨at f¨ur Elektrotechnik und Informationstechnik, pp. 8-160, 2007

Future Work

In the next presentation, detailed discussion on error concealment techniques with the practical applications will be done. The comparison of some of the error concealment techniques will be studied with their results.