1-D Unitary Transform

ENEE631 Digital Image Processing (Spring'04)

Unitary TransformsUnitary Transforms

Spring ’04 Instructor: Min Wu

ECE Department, Univ. of Maryland, College Park

www.ajconline.umd.edu (select ENEE631 S’04) [email protected]

Based on ENEE631 Based on ENEE631 Spring’04Spring’04Section 9Section 9

ENEE631 Digital Image Processing (Spring'04) Lec12 – Unitary Transform [3]

Image Transform: A RevisitImage Transform: A Revisit

With A Coding PerspectiveWith A Coding Perspective

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

004)


Why Do Transforms?Why Do Transforms? Fast computation

– E.g., convolution vs. multiplication for filter with wide support Conceptual insights for various image processing

– E.g., spatial frequency info. (smooth, moderate change, fast change, etc.) Obtain transformed data as measurement

– E.g., blurred images, radiology images (medical and astrophysics)– Often need inverse transform– May need to get assistance from other transforms

For efficient storage and transmission– Pick a few “representatives” (basis) – Just store/send the “contribution” from each basis

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)


Basic Process of Transform CodingBasic Process of Transform CodingU

MC

P E

NE

E63

1 S

lides

(cre

ated

by

M.W

u ©

200

4)

Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 8)


1-D DFT and Vector Form 1-D DFT and Vector Form

{ z(n) } { Z(k) }n, k = 0, 1, …, N-1WN = exp{ - j2 / N }

~ complex conjugate of primitive Nth root of unity

Vector form and interpretation for inverse transformz = k Z(k) ak

ak = [ 1 WN-k WN

-2k … WN-(N-1)k ]T / N

Basis vectors – ak

H = ak* T = [ 1 WN

k WN2k … WN

(N-1)k ] / N

– Use akH as row vectors to construct a matrix F

– Z = F z z = F*T Z = F* Z

– F is symmetric (FT=F) and unitary (F-1 = FH where FH = F*T )

1

0

1

0

)(1)(

)(1)(

N

k

nkN

N

n

nkN

WkZN

nz

WnzN

kZ

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001/

2004

)


Basis Vectors and Basis ImagesBasis Vectors and Basis Images A basis for a vector space ~ a set of vectors

– Linearly independent ~ ai vi = 0 if and only if all ai=0– Uniquely represent every vector in the space by their linear combination

~ bi vi ( “spanning set” {vi} ) Orthonormal basis

– Orthogonality ~ inner product <x, y> = y*T x= 0 – Normalized length ~ || x ||2 = <x, x> = x*T x= 1

Inner product for 2-D arrays– <F, G> = m n f(m,n) g*(m,n) = G1

*T F1 (rewrite matrix into vector) !! Don’t do FG ~ may not even be a valid operation for MxN matrices!

2D Basis Matrices (Basis Images) – Represent any images of the same size as a linear combination of basis

images

A vector space consists of a set of vectors, a field of scalars, a vector addition operation, and a scalar multiplication operation.

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)


1-D Unitary Transform1-D Unitary Transform Linear invertible transform

– 1-D sequence { x(0), x(1), …, x(N-1) } as a vector – y = A x and A is invertible

Unitary matrix ~ A-1 = A*T – Denote A*T as AH ~ “Hermitian”

– x = A-1 y = A*T y = ai*T y(i)

– Hermitian of row vectors of A form a set of orthonormal basis vectorsai

*T = [a*(i,0), …, a*(i,N-1)] T

Orthogonal matrix ~ A-1 = AT – Real-valued unitary matrix is also an orthogonal matrix– Row vectors of real orthogonal matrix A form orthonormal basis vectors

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)


Exercise – Question for Today:Exercise – Question for Today:

Which is unitary or orthogonal?

2-D DFT– Write the forward and inverse transform– Express in terms of matrix-vector form– Find basis images

cossinsincos

2

12

12

12

1

2132

inv(A1) = [2, –3; -1, 2]

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001/

2004

)


Properties of 1-D Unitary Transform Properties of 1-D Unitary Transform y = A xy = A x

Energy Conservation– || y ||2 = || x ||2

|| y ||2 = || Ax ||2= (Ax)*T (Ax)= x*T A*T A x = x*T x = || x ||2

Rotation– The angles between vectors are preserved

– A unitary transformation is a rotation of a vector in an N-dimension space, i.e., a rotation of basis coordinates

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001/

2004

)


Properties of 1-D Unitary Transform (cont’d)Properties of 1-D Unitary Transform (cont’d) Energy Compaction

– Many common unitary transforms tend to pack a large fraction of signal energy into just a few transform coefficients

Decorrelation– Highly correlated input elements quite uncorrelated output coefficients– Covariance matrix E[ ( y – E(y) ) ( y – E(y) )*T ]

small correlation implies small off-diagonal terms

Example: recall the effect of DFT

Question: What unitary transform gives the best compaction and decorrelation?

=> Will revisit this issue in a few lectures

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001/

2004

)


Review: 1-D Discrete Cosine Transform (DCT)Review: 1-D Discrete Cosine Transform (DCT)

Transform matrix C– c(k,n) = (0) for k=0 – c(k,n) = (k) cos[(2n+1)/2N] for k>0

C is real and orthogonal– rows of C form orthonormal basis– C is not symmetric!– DCT is not the real part of unitary DFT! See Assignment#3

related to DFT of a symmetrically extended signal

Nk

N

NknkkZnz

NknknzkZ

N

k

N

n

2)(,1)0(

2)12(cos)()()(

2)12(cos)()()(

1

0

1

0

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001/

2004

)


Periodicity Implied by DFT and DCTPeriodicity Implied by DFT and DCTU

MC

P E

NE

E63

1 S

lides

(cre

ated

by

M.W

u ©

200

4)

Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 8)


Example of 1-D DCT Example of 1-D DCT

100

50

0

-50

-1000 1 2 3 4 5 6 7

n

z(n)

100

50

0

-50

-1000 1 2 3 4 5 6 7

k

Z(k)

DCT

From Ken Lam’s DCT talk 2001 (HK Polytech)U

MC

P E

NE

E63

1 S

lides

(cre

ated

by

M.W

u ©

200

1)


UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)

Example of 1-D DCT (cont’d)Example of 1-D DCT (cont’d)

k

Z(k)

Transform coeff.

1.0

0.0

-1.0

1.0

0.0

-1.0

1.0

0.0

-1.0

1.0

0.0

-1.0

1.0

0.0

-1.0

1.0

0.0

-1.0

1.0

0.0

-1.0

1.0

0.0

-1.0

Basis vectors

100

0

-100

100

0

-100

100

0

-100

100

0

-100

100

0

-100

100

0

-100

100

0

-100

100

0

-100u=0

u=0 to 1

u=0 to 4

u=0 to 5

u=0 to 2

u=0 to 3

u=0 to 6

u=0 to 7

Reconstructions

n

z(n)

Original signal

From Ken Lam’s DCT talk 2001 (HK Polytech)


2-D DCT2-D DCT Separable orthogonal transform

– Apply 1-D DCT to each row, then to each columnY = C X CT X = CT Y C = mn y(m,n) Bm,n

DCT basis images:– Equivalent to represent

an NxN image with a set of orthonormal NxN “basis images”

– Each DCT coefficient indicates the contribution from (or similarity to) the corresponding basis image

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)


2-D Transform: General Case2-D Transform: General Case

A general 2-D linear transform {ak,l(m,n)}– y(k,l) is a transform coefficient for Image {x(m,n)} – {y(k,l)} is “Transformed Image”– Equiv to rewriting all from 2-D to 1-D and applying 1-D transform

Computational complexity– N2 values to compute– N2 terms in summation per output coefficient– O(N4) for transforming an NxN image!

),(),(),(

),(),(),(

1

0

1

0,

1

0

1

0,

nmhlkynmx

nmanmxlky

N

k

N

llk

N

m

N

nlk

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001/

2004

)


2-D Separable Unitary Transforms2-D Separable Unitary Transforms Restrict to separable transform

– ak,l(m,n) = ak(m) bl(n) , denote this as a(k,m) b(l,n)

Use 1-D unitary transform as building block– {ak(m)}k and {bl(n)}l are 1-D complete orthonormal sets of basis vectors

use as row vectors to obtain unitary matrices A={a(k,m)} & B={b(l,n)}

– Apply to columns and rows Y = A X BT

often choose same unitary matrix as A and B (e.g., 2-D DFT)

For square NxN image A: Y = A X AT X = AH Y A* – For rectangular MxN image A: Y = AM X AN T X = AM

H Y AN*

Complexity ~ O(N3)– May further reduce complexity if unitary transf. has fast implementation

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)


Basis ImagesBasis Images X = AH Y A* => x(m,n) = k l a*(k,m)a*(l,n) y(k,l)

– Represent X with NxN basis images weighted by coeff. Y– Obtain basis image by setting Y={(k-k0, l-l0)} & getting X

{ a*(k0 ,m)a*(l0 ,n) }m,n

in matrix form A*k,l = a*k al*T ~ a*k is kth column vector of AH

trasnf. coeff. y(k,l) is the inner product of A*k,l with the image

Exercise– Unitary transform or not?– Find basis images– Represent an image X with basis images (Jain’s e.g.5.1, pp137)

21

21

21

21

A

4321

X

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)


UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)

Common Unitary Transforms and Basis ImagesCommon Unitary Transforms and Basis Images

DFT DCT Haar transform K-L transform

See also: Jain’s Fig.5.2 pp136


2-D DFT2-D DFT 2-D DFT is Separable

– Y = F X F X = F* Y F*

– Basis images Bk,l = (ak ) (al

)T

where ak = [ 1 WN-k WN

-2k … WN-(N-1)k ]T / N

1

0

1

0

1

0

1

0

),(1),(

),(1),(

N

k

N

l

mkN

nlN

N

m

N

n

mkN

nlN

WWlkYN

nmX

WWnmXN

lkY

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001/

2004

)


Summary and Review (1)Summary and Review (1) 1-D transform of a vector

– Represent an N-sample sequence as a vector in N-dimension vector space

– Transform Different representation of this vector in the space via different basis e.g., 1-D DFT from time domain to frequency domain

– Forward transform In the form of inner product Project a vector onto a new set of basis to obtain N “coefficients”

– Inverse transform Use linear combination of basis vectors weighted by transform coeff.

to represent the original signal

2-D transform of a matrix– Rewrite the matrix into a vector and apply 1-D transform– Separable transform allows applying transform to rows then columns

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)


Summary and Review (1) cont’dSummary and Review (1) cont’d Vector/matrix representation of 1-D & 2-D sampled signal

– Representing an image as a matrix or sometimes as a long vector

Basis functions/vectors and orthonomal basis– Used for representing the space via their linear combinations– Many possible sets of basis and orthonomal basis

Unitary transform on input x ~ A-1 = A*T – y = A x x = A-1 y = A*T y = ai

*T y(i) ~ represented by basis vectors {ai*T}

– Rows (and columns) of a unitary matrix form an orthonormal basis

General 2-D transform and separable unitary 2-D transform– 2-D transform involves O(N4) computation– Separable: Y = A X AT = (A X) AT ~ O(N3) computation

Apply 1-D transform to all columns, then apply 1-D transform to rows

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)


Optimal TransformOptimal Transform

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

004)


Optimal TransformOptimal Transform Recall: Why use transform in coding/compression?

– Decorrelate the correlated data– Pack energy into a small number of coefficients

– Interested in unitary/orthogonal or approximate orthogonal transforms Energy preservation s.t. quantization effects can be better understood

and controlled

Unitary transforms we’ve dealt so far are data independent– Transform basis/filters are not depending on the signals we are processing

What unitary transform gives the best energy compaction and decorrelation?– “Optimal” in a statistical sense to allow the codec works well with

many images Signal statistics would play an important role

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

004)


Review: Correlation After a Linear TransformReview: Correlation After a Linear Transform Consider an Nx1 zero-mean random vector x

– Covariance (autocorrelation) matrix Rx = E[ x xH ] give ideas of correlation between elements Rx is a diagonal matrix for if all N r.v.’s are uncorrelated

Apply a linear transform to x: y = A x

What is the correlation matrix for y ?

Ry = E[ y yH ] = E[ (Ax) (Ax)H ] = E[ A x xH AH ]

= A E[ x xH ] AH = A Rx AH

Decorrelation: try to search for A that can produce a decorrelated y (equiv. a diagonal correlation matrix Ry )

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

004)


K-L Transform (Principal Component Analysis)K-L Transform (Principal Component Analysis) Eigen decomposition of Rx: Rx uk = k uk

– Recall the properties of Rx Hermitian (conjugate symmetric RH = R); Nonnegative definite (real non-negative eigen values)

Karhunen-Loeve Transform (KLT) y = UH x x = U y with U = [ u1, … uN ]

– KLT is a unitary transform with basis vectors in U being the orthonormalized eigenvectors of Rx

– UH Rx U = diag{1, 2, … , N} i.e. KLT performs decorrelation

– Often order {ui} so that 1 2 … N

– Also known as the Hotelling transform or the Principle Component Analysis (PCA)

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001/

2004

)


Properties of K-L TransformProperties of K-L Transform

Decorrelation– E[ y yH ]= E[ (UH x) (UH x)H ]= UH E[ x xH ] U = diag{1, 2, … , N}

– Note: Other matrices (unitary or nonunitary) may also decorrelate the transformed sequence [Jain’s e.g.5.7 pp166]

Minimizing MSE under basis restriction – If only allow to keep m coefficients for any 1 m N, what’s the

best way to minimize reconstruction error? Keep the coefficients w.r.t. the eigenvectors of the first m largest

eigenvalues

Theorem 5.1 and Proof in Jain’s Book (pp166)

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001/

2004

)


KLT Basis RestrictionKLT Basis Restriction Basis restriction

– Keep only a subset of m transform coefficients and then perform inverse transform (1 m N)

– Basis restriction error: MSE between original & new sequences

Goal: to find the forward and backward transform matrices to minimize the restriction error for each and every m– The minimum is achieved by KLT arranged according to the

decreasing order of the eigenvalues of R

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

004)


K-L Transform for ImagesK-L Transform for Images

Work with 2-D autocorrelation function– R(m,n; m’,n’)= E[ x(m, n) x(m’, n’) ] for all 0 m, m’, n, n’ N-1– K-L Basis images is the orthonormalized eigenfunctions of R

Rewrite images into vector form (N2x1)– Need solve the eigen problem for N2xN2 matrix! ~ O(N 6)

Reduced computation for separable R– R(m,n; m’,n’)= r1(m,m’) r2(n,n’)– Only need solve the eigen problem for two NxN matrices ~ O(N3)– KLT can now be performed separably on rows and columns

Reducing the transform complexity from O(N4) to O(N3)

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)


Pros and Cons of K-L TransformPros and Cons of K-L Transform

Optimality– Decorrelation and MMSE for the same# of partial coeff.

Data dependent– Have to estimate the 2nd-order statistics to determine the transform– Can we get data-independent transform with similar performance?

DCT

Applications– (non-universal) compression– pattern recognition: e.g., eigen faces– analyze the principal (“dominating”) components

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)


Energy Compaction of DCT vs. KLTEnergy Compaction of DCT vs. KLT DCT has excellent energy compaction

for highly correlated data

DCT is a good replacement for K-L– Close to optimal for highly correlated data– Not depend on specific data like K-L does– Fast algorithm available

[ref and statistics: Jain’s pp153, 168-175]

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

004)


Energy Compaction of DCT vs. KLT (cont’d)Energy Compaction of DCT vs. KLT (cont’d) Preliminaries

– The matrices R, R-1, and R-1 share the same eigen vectors– DCT basis vectors are eigenvectors of a symmetric tri-diagonal matrix Qc

– Covariance matrix R of 1st-order stationary Markov sequence with has an inverse in the form of symmetric tri-diagonal matrix

DCT is close to KLT on 1st-order stationary Markov– For highly correlated sequence, a scaled version of R-1 approx. Qc

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

004)


Summary and Review on Unitary TransformSummary and Review on Unitary Transform Representation with orthonormal basis Unitary transform

– Preserve energy

Common unitary transforms– DFT, DCT, Haar, KLT

Which transform to choose?– Depend on need in particular task/application– DFT ~ reflect physical meaning of frequency or spatial frequency– KLT ~ optimal in energy compaction– DCT ~ real-to-real, and close to KLT’s energy compaction

=> A comparison table in Jain’s book Table 5.3

UM

CP

EN

EE

631

Slid

es (c

reat

ed b

y M

.Wu

© 2

001)


Documents

1-D Unitary Transform