Upload
ashley-mcdowell
View
240
Download
0
Embed Size (px)
DESCRIPTION
Definition error concealment is a technique in which an error in a transmitted or encoded signal is replaced by synthetic content, often interpolated from other parts of the signal, in an attempt to produce a more pleasant experience for the viewer or listener, at the cost of concealing the presence of the error in the original content. Error concealment by post-processing at the decoder is one of the important strategies for combating transmission errors in video communication 3
Citation preview
Error Concealment
Multimedia Systems and StandardsS2 IF ITTelkom
2
BackgroundEvery communication system has to deal with the problems that may arise during transmission, such as adulteration (bit insertion, deletion or inversion) or loss of the transmitted signalTo solve this problem, a set of techniques were developed, whose purpose was to minimize the influence of the transmission errors at the decoder, taking in consideration the characteristics of the video signal.These techniques are called Error Concealment Techniques
3
Definitionerror concealment is a technique in which an error in a transmitted or encoded signal is replaced by synthetic content, often interpolated from other parts of the signal, in an attempt to produce a more pleasant experience for the viewer or listener, at the cost of concealing the presence of the error in the original content.Error concealment by post-processing at the decoder is one of the important strategies for combating transmission errors in video communication
4
Error Concealment Techniquesthe video decoder attempts to benefit from previously received error-free video information the approximate recovery of lost or erroneous data without relying on additional information from the encoder
5
TechniquesTo estimate and sometimes predict missing video information :
Spatial and temporal interpolation, filtering and smoothingMotion vectors, transform coefficients and administrative bits
6
Techniques (cont’d)take advantage of the human eyes tolerance (high-frequency components than low-frequency components)multi-layer video coding: base & enhancement layersmake use of the spatial and/or temporal correlations between damaged MBs and their neighbouring MBs
7
The goalTo reduce the visual artefacts in segments of a video stream that lie between two error-free synch words. ?: where to insert synch words
Error ConcealmentBasic Idea
Decoder should generate a representation for lost areaMatch as close as possible to the lost infoWithin manageable complexity
TechniquesSpatial Error ConcealmentTemporal Error ConcealmentHybrid ConcealmentOther Techniques 8
9
Spatial Error ConcealmentEmploy the correctly received data of the current frame to conceal the erroneous dataSpatial error concealment methods can be categorized in four main types: interpolation based, stochastic-based, tensor voting-based and exemplar-based.
10
Spatial Error Concealment (Interpolation Based)
conceal each corrupted pixel by interpolation; using the correctly received neighboring pixelsfor instance, directional interpolation methods are proposed that utilize edge directions for interpolationThe weakness of interpolation-based methods is that although they perform very fast, texture preservation is not their intrinsic characteristic and should be imposed separatelyExample : Multi Directional Interpolation
11
Multi Directional InterpolationMDI is one method used to repair the damaged image or video by covering the damaged block using spatial interpolationThe process of MDI are :a. Block Classification and
Directional Decisionb. Block Concealment
12
Multi Directional Interpolation Algorithm
The missing 16×16 MB is recovered 4×4 block by 4×4 block in the proposed error concealment approach.The order of 16 blocks recovered in an MB is from the blocks with dependable available neighbours to the blocks without dependable available neighboursThere are mainly two steps to recover each 4×4 block.In the first step, the block is determined if it is a flat block or a block with one of the eight directional edges according to the estimation of surrounding available pixelsIn the second step, the flat block is recovered using weighted pixel averaging and the block with edge is recovered using Intra_4×4 prediction
13
Multi Directional Interpolation Algorithm
14
Block Classification and Directional Decision
The absence or presence of edges and the most likely direction of the edge, if present, of each 4×4 block in the missing MB are obtained by the use of gradient measures to three layers of pixels of the available adjacent MB as shown in Fig. 1.The local edge gradient components of the pixel x(i,j) in the second layer of the available adjacent MB is computed by:
15
Block Classification and Directional Decision Cont
Which is equivalent to applying 3x3 sobel operator
The angular of gradient at coordinate (i,j) :
And the angle of θ is rounded to the nearest 22.5 degree.
16
Block Classification and Directional Decision Cont
In order to determine if a 4x4 block is lat or with some direction of edge, eight counters Ck, k = 1,....7 are used. The initial value of Ck is set zero.For every pixel x(i,j) and all pixel (y(u,v) to represent any one of these pixels) in the block to be recovered,if y(u,v) satisfy :
Then Ck plus 1/D, where D is given by :
17
Block Classification and Directional Decision Cont
If the largest counter value is below a certain threshold, there is no discernible edge and the block is classified to a flat block
18
Block ConcealmentEach 4×4 block with edge in the missing MB is recovered by Intra_4×4 prediction with edge according to directional decision in the first step.Each flat block is recovered by using the method of weighted pixel averaging.The number of each block in the missing MB is shown
19
Block Concealment ContSince the order of 16 blocks recovered in a MB is from blocks with dependable available neighbours to those without dependable available neighbours, the order of 16 blocks to be recovered will be changed according to the scenario of available neighbour MBsFor examples, if a missing MB has four available edge neighbour MBs, the order of 16 blocks to be recovered is 0, 5, 1, 4, 10, 15, 11, 14, 2, 7, 3, 6, 8, 13, 9, 12;if a missing MB only has up and right available edge neighbour MBs, the order of 16 blocks to be recovered is 5, 4, 1, 0, 7, 6, 3, 2, 13, 12, 9, 8, 15, 14, 11, 10.Furthermore, for each block from 0 to 15, the 16 pixels a, b, …, n, o, p should also be recovered by Intra_4×4 prediction from more reliable pixels X, A, B, C, …, G, H shown at fig 4.
20
Spatial Error Concealment (Stochastic Based)
consider each pixel of the frame as a random variableemploy the Bayes rule to approximate the best pixel values of the corrupted blockThe main disadvantage of these methods is their stationary assumption for input images.
21
Spatial Error Concealment (Tensor Voting Based)
a tensor vector (containing the intensity and gradient information of each pixel) is calculated for each correctly decoded pixelUsing a voting scheme, tensor vectors of the corrupted pixel are approximated and translated to the intensity valuesThe main drawback of tensor voting approaches is their high computational complexity that is not affordable in real-time applications
22
Spatial Error Concealment (Exemplar Based)performed by cloning the best patches from the correctly received region of the image to the corrupted block.In this method, the similarity is measured by the sum-squared difference of the outer boundariesIn this approach, the subblocks near the edge are concealed first.
23
Temporal Error Concealment
Exploits temporal redudancy in a sequence to recover the impaired MacroBlocks (MBs) by utilizing reference frames.Example : Motion Conpensated Temporal Prediction
Temporal Error Concealment
Rely on the continuity of a video sequence in timeUse temporally neighboring areas to conceal lost regionsPrevious Frame Concealment (PFC)
Use previous corresponding data to copy to current frameOnly good when little motionWidely used due to simplicity
24
Hybrid ConcealmentWhen only apply spatial concealment
Concealed regions are significantly blurred
When only use temporal error concealment
Significantly discontinuities in the concealed regions
Hybrid temporal-spatial technique applied
MB mode info of reliable and concealed neighbors decide which concealment method to use
25
Hybrid (cont.)For intra-coded images
Only use spatial concealmentFor inter-coded images
Use temporal concealment when more than half of the available neighbor MBs are inter-codedOtherwise, use spatial concealment
Referred to as Adaptive temporal and spatial Error Concealment (AEC)
26
27
Error DetectionTo activate error concealment techniqueErrors:
loss of synchronization due to error-corrupted VLC parametersthe number of AC coefficients of any 8x8 block of pixels is found to have exceeded 63 The decoded MV component or quantization parameter is outside the acceptable rangetransmission errors
28
Recovery of lost MVs and MB coding modes = INTER
Simplest: replace the erroneous MB by the spatially coinciding MB in the previous frameAlternative: replaced by the motion-compensated MB (what if MV also corrupted?)MV recovery
29
Example: H.263; BER=0.01 %
(a) no concealment, (b) zero-MV technique, (c) MV of spatially corresponding MB in previous frame, (d) MV of MB in previous frame that best moves in the direction of the lost MV
30
Recovery of lost coefficientscan be interpolated from spatially corresponding coefficients: four neighbours
some coefficients damaged use coefficients in the same blockall coefficients are lost using:• four one-pixel wide boundaries• the nearest two one-pixel wide
boundariesmore suitable for INTRA coded blocks
31
Four-one pixel wide boundaries
Based on assumption of continuity of natural scene content in spaceUse pixel values of surrounding available MBs Estimate of lost pixel:
32
αβγ are weighing factorsDetermine relative impact of vertical, Horizontal, upper, lower…
DisadvantageBlurred reconstruction
Four-one pixel wide boundaries
33
the nearest two one-pixel wide boundaries
Selected ResultsPerformance of different error concealment strategies
34