Image and Multidimensional Signal Processinginside.mines.edu/~whoff/courses/EENG510/lectures/26-CompressionLossy.pdfImage Compression - Lossy . Colorado School of Mines Image and Multidimensional

Image and Multidimensional Signal Processing Colorado School of Mines

Colorado School of Mines

Image and Multidimensional Signal Processing

Professor William Hoff

Dept of Electrical Engineering &Computer Science

http://inside.mines.edu/~whoff/


Image Compression - Lossy


Lossy Compression

• Reconstructed image is different from original

• Hopefully differences are unnoticeable, or minor

• We will look at: – Block transform coding methods, using the discrete cosine transform

(such as the JPEG standard)

– Predictive coding

3


Block Transform Coding

• Divides the image into subimages, or blocks

• Apply a transform (e.g., Fourier) to each block

• Quantize and encode the coefficients

4

Compression


Transform Coding

• General forward transform of image g, size nxn

• Inverse transform

• Example: Fourier transform

1 1

0 0

( , ) ( , ) ( , , , )n n

x y

T u v g x y r x y u v

1 1

0 0

( , ) ( , ) ( , , , )n n

u v

g x y T u v s x y u v

2 ( )/ 2 ( )/

2

1,j ux vy n j ux vy nr e s e

n

r,s are the forward and inverse transformation kernels (also called basis functions)

5


Example: Walsh-Hadamard Transform (WHT)

• Kernels:

• where – nxn is the size of the kernel, and n = 2^m – bk is the kth bit – Summation is done in modulo 2 arithmetic – The p’s are

p0(u) = bm-1(u) p1(u) = bm-1(u) + bm-2(u) p2(u) = bm-2(u) + bm-3(u) : pm-1(u) = b1(u) + b0(u)

1

0

( ) ( ) ( ) ( )

( , , , ) ( , , , )

11

m

i i i i

i

b x p u b y p v

r x y u v s x y u v

n

6


Example WHT Basis Functions

7

+1 -1


Discrete Cosine Transform (DCT)

• Kernels:

• DCT used in JPEG (wavelets are used in JPEG2000)

1,...,2,1for2

0for1

)(

where

2

)12(cos

2

)12(cos)()(

),,,(),,,(

NuN

uNu

N

vy

N

uxvu

vuyxhvuyxg

8

% Show DCT kernels

N = 32;

x=0:N-1;

y=0:N-1;

u = 1;

v = 4;

au = sqrt(2/N);

av = sqrt(2/N);

if u==1

au = sqrt(1/N);

end

if v==1

av = sqrt(1/N);

end

gx = au*cos((2*x+1)*u*pi/(2*N));

gy = av*cos((2*y+1)*v*pi/(2*N));

figure, plot(x,gx);

figure, plot(y,gy);

g = gx'*gy;

figure, surf(g);


DCT 4x4 Basis Functions

9


Approximation Errors

Apply a transform to each 8x8 subimage block

Keep highest 50% of coefficients in each block

Then reconstruct image (by taking the inverse transform) using the remaining coefficients

Reconstructed Error Image

10

RMS error: 2.32 RMS error: 1.78 RMS error: 1.13


Effect of Subimage Size

Image: lena

Truncate smallest 75% coefficients in each subimage

Figure 8.26

11


12


Quantizing Transform Coefficients

• Methods: – Threshold coding: Within each 8x8 subimage, keep the top N% of the

coefficients; or those with magnitude greater than a threshold

• Matlab exercise with “blkproc”

– Zonal coding: Keep coefficients with maximum variance across all subimages

• Matlab’s “dctdemo”

• Then quantize to a fixed or variable number of bits

13


Threshold coding – Matlab example

• dct2

– The function dct2 performs 2D discrete cosine transform on a matrix

B = dct2(A)

– The function idct2 performs the reverse transformation A2 = idct2(B)

• blkproc

– Use “blkproc” to apply dct2 to each 8x8 block J = blkproc(I,[8 8],@dct2);

– You could also apply your own function (eg., threshold) to each block J = blkproc(I,[8 8],@mythresh);

14

Image and Multidimensional Signal Processing Colorado School of Mines 15

• DCT is performed on each 8x8 subimage (block) >> I = imread('cameraman.tif');

>> J = blkproc(I,[8 8],@dct2);

DCT coefficients


A note on functions in Matlab

• You can write a function and call it from your program

• Syntax: function B = myfunc(A)

% This function computes something from A and

returns B

:

B = …

• Store this in a file called myfunc.m

• Put in current working directory

16


Truncating coefficients

• Write a function called “mytrunc” that truncates the smallest 75% coefficients in an image

• The function should – Take the absolute value of each pixel in the image

– Sort the values in ascending order

– Find the value that is 75% down the list

– Threshold the image using that value

17

function B = mytrunc(A)

% Truncate the lowest 75% of the magnitudes within A

Aabs = abs(A);

vals = sort(Aabs(:)); % Sort the values from low to high

:

B = A .* (Aabs > thresh);


18

clear all

close all

I = double(imread('lena.tif'));

wsize = 8;

J = blkproc(I,[wsize wsize],@dct2);

imshow(J,[]), title('J');

% Truncate 75% of the values within each block

Jtrunc = blkproc(J,[wsize wsize],@mytrunc);

figure, imshow(Jtrunc,[]), title('Jtrunc');

pct = sum(sum(Jtrunc == 0))/(size(I,1)*size(I,2));

fprintf('Percentage of zero coeffs: %f\n',100*pct);

K=blkproc(Jtrunc,[wsize wsize],@idct2);

figure, imshow(K,[]), title('K');

R = I - K;

disp('RMS error:');

sqrt(mean2(R .^ 2))

• Try on different block sizes

• Do you get this result?


Quantizing Transform Coefficients

• Methods: – Threshold coding: Within each 8x8 subimage, keep the top N% of the

coefficients; or those with magnitude greater than a threshold

• Matlab exercise with “blkproc”

– Zonal coding: Keep coefficients with maximum variance across all subimages

• Matlab’s “dctdemo”

• Then quantize to a fixed or variable number of bits

19


Zonal coding – Matlab “dctdemo”

• Apply DCT to each 8x8 block

• Discard coefficients with the smallest variance

20

DCT coefficients

Original Saturn Image

Reconstructed Image Error Image


Ordering sequence

• Convert a nxn matrix to a one-dimensional vector

Typical threshold mask (we keep the coefficients in the shaded positions)

Ordering sequence, to convert 8x8 array to a 64x1 vector

Resulting vector will have long runs of 0s

Different mask for each subimage

21


Quantizing Coefficient Magnitudes

• Once we decide which coefficients to keep, we now quantize the remaining non-zero coefficients

• We divide each coefficient by a number (depending on its location) and round to integer – Smaller magnitudes can be represented by fewer bits

– Division by a large number will tend to give a zero result

22

Z(u,v)


Variable quantization

• You can achieve more or less compression by scaling the normalization matrix Z (i.e., dividing by larger values)

• Resulting compression:

(a) 12:1 (b) 19:1 (c) 30:1 (d) 49:1 (e) 85:1 (f) 182:1

23


JPEG Algorithm

• Divide into 8x8 subimages

• Discrete cosine transform on each

• Quantize the coefficients

– Uses threshold coding

– Order coefficients in zig-zag pattern

• Encode the 1D sequence using run-length encoding and Huffman encoding

24

Ordering of coefficients

Threshold quantization array


Input 8x8 subimage

Subtract 128 from each value

25


Do forward DCT

Quantize and truncate values

26

Example: round(-415/16) = -26


JPEG Algorithm (continued)

• Re-order in zig-zag pattern

• Use variable length code words to encode non-zero values

• Use run length encoding to encode # zeros

• Results – bit count = 92

– Compression: 512/92 = 5.6:1

27


Details of coding the coefficients

• We use a pre-computed Huffman code (Appendices A.4-A.5)

• It assumes that values are clustered around zero

• The code word consists of a “base” code (which represents the most significant bits), followed by a coding of the least significant bits

• The base code is determined by the magnitude of the coefficient

First find what range the coefficient value lies in, and the corresponding category K

Then look up the base code for that category

28


Predictive Coding

• Takes advantage of interpixel redundancy

• Predict next pixel from previous pixel, encode only the difference from the actual and the predicted

29


30

A simple predictor: fpred(x,y) = f(x,y-1)


Lossy Predictive Coding

• Error values are quantized

• Predictions by encoder and decoder must be same to prevent error buildup

31


Optimal quantization of Error Values

• Lloyd-Max interval quantizer: staircase function with L values

32


Choosing Intervals

• Assume a Laplacian pdf

• The optimal 2-bit quantizer is

e

e eep

2

2

1)(

Quantized output

Error (e) 0 1.102 -1.102

-0.395 0.395 1.81 -1.81

33


Lloyd-Max Quantization

Quantized output

Error 0 1.102 -1.102

-0.395 0.395 1.81 -1.81

2 bit (4 level):

34


Compression of Image Sequences

• To compress a video we take advantage of the redundancy between successive frames

• See NASA Shuttle Movie

– 1829 color frames (~1 minute)

– Compression using Quicktime (H.264)

• Reduction from 5 GB to 45 MB (100:1)

• Predict the value of each pixel, transmit the residual error • Simplest prediction method: Prediction is the value of the pixel in the

previous image (forward prediction) • Periodically insert I-frames (“independent” frames)

– These are compressed as single images (like DCT) – Needed for initialization – Or to handle cases where there are too many changes between successive

images

• Can also base the prediction on the next frame (backward prediction)

35

nasa_shuttle-r_m720p.mov


Motion Compensation

• Predict motion of small blocks (e.g, 16x16)

Encoder has to estimate motion of each macroblock … usually finds dx,dy to minimize “mean absolute distortion”, which is the average of the absolute values of the differences

36


Example

std dev = 12.7 std dev = 5.6

37


Example – Subpixel Motion Estimation

std dev = 12.7 std dev = 4.4

std dev = 4 std dev = 3.8

Need to interpolate values

38


Video Compression Standards

P: prediction (forward) B: backward prediction

39

Documents

Image and Multidimensional Signal Processinginside.mines.edu/~whoff/courses/EENG510/lectures/26-CompressionLossy.pdfImage Compression - Lossy . Colorado School of Mines Image and Multidimensional