Video Coding with Linear Compensation (VCLC) Arif Mahmood, Zartash Afzal Uzmi, Sohaib A Khan {arifm,zartash,sohaib}@lums.edu.pk Department of Computer Science, Lahore University of Management and Sciences, Lahore, PAKISTAN Consecutive frames from movies and music videos: Blade II (2002), Batman Begins (2005), Pink Floyd: Coming Back to Life (1994). 1 MOTIVATION: Intensity changes are observed frequently in commercial videos 2 Exploitation of Temporal Redundancy Results in Storage and Communication Efficiency Temporal Redundancy is exploited through: Motion Estimation: The process of finding a matching block in the temporally adjacent frame Compensation: The process of computing the signal for communication or storage 3 CURRENTLY: Sum of Absolute Differences (SAD) is used for Motion Estimation ESTIMATION of location of matching block is through SAD COMPENSATION is done through computing the difference at the match location n x n y y x k b y x k b SAD 1 1 ) ' , ' , ' ( ) , , ( ) ' , ' , ' ( ) , , ( y x k b y x k b RESIDUE 4 SAD Fails as an Match Measure if Video undergoes light changes SAD is optimal from coding point of view in the absence of light changes, since it minimizes the residue However, SAD is NON-OPTIMAL if brightness or contrast changes are present in video Example: If all pixels in a block become brighter by Δ, the location where content matches will become brighter by nΔ, and may no longer be the location with least residue 5 OUR PROPOSED SCHEME: Video Coding Using Linear Compensation Gives consideration to brightness and contrast changes in a sequence of frames ESTIMATION of location of matching block is done through correlation coefficient COMPENSATION is done through computing the difference with the linear estimate of block intensity α and β are selected by minimizing the Mean Squared Error and are given by: ) ' , ' , ' ( ) , , ( 1 1 ' ) ' , ' , ' ( ) ' , ' , ' ( ) , , ( ) , , ( y x k b y x k b n x n y bb y x k b y x k b y x k b y x k b ) ' , ' , ' ( ) , , ( ˆ y x k b y x k b ' ' ' ' ' ; b b b bb b b b bb 6 Why Linear Model? Brightness and Contrast Changes are well modeled by first order linear model Going to a higher order model is a case of diminishing returns: Variance is not further decreased by much, and the cost, in terms of number of parameters, becomes larger 7 KEY THEORETICAL RESULTS RESULT 1: For the same motion estimator, i.e. the variance of the difference signal after linear compensation, is upper bounded by (the variance of the simple difference) This means that from compression point of view, the difference signal after linear compensation should always be better, or equal, to the simple difference scheme employed by current codecs RESULT 2: When linear compensation is used, the optimal criterion for motion estimation is correlation coefficient, rather than SAD We have proved that no other motion estimator can give match at a location that results in lesser first order linear compensated difference 2 p 2 d 8 EXPERIMENTAL RESULTS Used a dataset of clips taken from commercial videos Implemented a generic encoder, and measured the PSNR of original and decoded signal at receiver end MSE PSNR 2 10 255 log 10 Variation of PSNR with the variation of bits per pixel for the VCLC scheme and the traditional generic encoder. Red curves being higher indicate more optimal compression The average and standard deviation of Mean Squared Error of different estimation filters. More than 400,000 8x8 blocks were taken to compute these statistics

Video Coding with Linear Compensation (VCLC) Arif Mahmood, Zartash Afzal Uzmi, Sohaib A Khan {arifm,zartash,sohaib}@lums.edu.pk Department of Computer

Download PPT Report

View
224
Download
0

Embed Size (px)

Citation preview

Video Coding with Linear Compensation (VCLC)Arif Mahmood, Zartash Afzal Uzmi, Sohaib A Khan

{arifm,zartash,sohaib}@lums.edu.pkDepartment of Computer Science, Lahore University of Management and Sciences, Lahore, PAKISTAN

Consecutive frames from movies and music videos: Blade II (2002), Batman Begins (2005), Pink Floyd: Coming Back to Life (1994).

1MOTIVATION: Intensity changes are observed frequently in commercial videos

2Exploitation of Temporal Redundancy Results in Storage and Communication EfficiencyTemporal Redundancy is exploited through:

Motion Estimation: The process of finding a matching block in the temporally adjacent frame

Compensation: The process of computing the signal for communication or storage

3CURRENTLY: Sum of Absolute Differences (SAD) is used for Motion EstimationESTIMATION of location of matching block is through SAD

COMPENSATION is done through computing the difference at the match location

yxkbyxkbSAD1 1

)',','(),,(

)',','(),,( yxkbyxkbRESIDUE

4SAD Fails as an Match Measure if Video undergoes light changes

SAD is optimal from coding point of view in the absence of light changes, since it minimizes the residue

However, SAD is NON-OPTIMAL if brightness or contrast changes are present in video

Example: If all pixels in a block become brighter by Δ, the location where content matches will become brighter by nΔ, and may no longer be the location with least residue

5OUR PROPOSED SCHEME: Video Coding Using Linear Compensation

Gives consideration to brightness and contrast changes in a sequence of frames

ESTIMATION of location of matching block is done through correlation coefficient

COMPENSATION is done through computing the difference with the linear estimate of block intensity

α and β are selected by minimizing the Mean Squared Error and are given by:

)',','(),,(

1 1'

)',','()',','(),,(),,(

yxkbyxkb

ybb

yxkbyxkbyxkbyxkb

)',','(),,(ˆ yxkbyxkb

' ; bb

bbbb

bbb

6Why Linear Model?Brightness and Contrast Changes are well modeled by first order linear model

Going to a higher order model is a case of diminishing returns: Variance is not further decreased by much, and the cost, in terms of number of parameters, becomes larger

7KEY THEORETICAL RESULTS

RESULT 1: For the same motion estimator, i.e. the variance of the difference signal after linear compensation, is upper bounded by (the variance of the simple difference)

This means that from compression point of view, the difference signal after linear compensation should always be better, or equal, to the simple difference scheme employed by current codecs

RESULT 2: When linear compensation is used, the optimal criterion for motion estimation is correlation coefficient, rather than SAD

We have proved that no other motion estimator can give match at a location that results in lesser first order linear compensated difference

8EXPERIMENTAL RESULTS

Used a dataset of clips taken from commercial videos

Implemented a generic encoder, and measured the PSNR of original and decoded signal at receiver end

MSEPSNR

255log10

Variation of PSNR with the variation of bits per pixel for the VCLC scheme and the traditional generic encoder.

Red curves being higher indicate more optimal compression

The average and standard deviation of Mean Squared Error

of different estimation filters. More than 400,000 8x8 blocks were taken to compute these

statistics