Upload
saige-herdman
View
218
Download
3
Tags:
Embed Size (px)
Citation preview
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Unitary Transform Unitary Transform
Spring ’09 Instructor: Min Wu
Electrical and Computer Engineering Department,
University of Maryland, College Park
bb.eng.umd.edu (select ENEE631 S’09) [email protected]
ENEE631 Spring’09ENEE631 Spring’09Lecture 10 (2/25/2009)Lecture 10 (2/25/2009)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [2]
OverviewOverview
Last Time:– MMSE Quantizer for non-uniform and uniform source– Companding: Quantizer with pre- and post- nonlinear transformation– Quantizer in predictive coding
Today:– Vector vs. Scalar Quantizer – Revisit image transform from a coding and basis perspective
=> Unitary transform
– DCT transform
Logistics: (1) mid-term exam
(2) Assign#3 to be posted
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [3]
Recap:Recap:Scalar QuantizerScalar Quantizer
Quantize one sample at a time
Quantizer
…
quantization error
…
…
…
…Input/Output
response
Input x
Output Q(x)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [4]
Recap: MMSE Quantizer Recap: MMSE Quantizer Example – Gaussian SourceExample – Gaussian Source
Start with uniform quantizer Use iterative algorithm. optimum thresholds (red) and reconstruction values (blue)
From B. Liu PU EE488 F’06
Truncated Gaussian N(0,1) with L=16 quantizer
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [5]
Vector QuantizationVector Quantization
Encode a set of values together– Find the representative combinations– Encode the indices of combinations
Scalar vs. Vector quantization– SQ is simpler in implementation – VQ allows flexible partition of coding cells– VQ could naturally explore the correlation
between elements
Stages to build vector quantizer– Codebook design– Encoder– Decoder
From Bovik’s Handbook Sec.5.3
vector quantization of 2 elements
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
scalar quantization of 2 elements
Signal Sample-1
Sig
nal S
ampl
e-2
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [6]
Outline of Core Parts in VQOutline of Core Parts in VQ
Design codebook– Optimization formulation is similar to MMSE scalar quantizer– Given a set of representative points
“Nearest neighbor” rule to determine partition boundaries
– Given a set of partition boundaries “Probability centroid” rule to determine representative
points that minimizes mean distortion in each cell
Search for codeword at encoder– Tedious exhaustive search– Design codebook with special structures to
speed up encoding E.g., tree-structured VQ
Reference:A. Gersho and R. M. Gray, Vector Quantization and Signal Compression, Kluwer Publisher. R. M. Gray, ``Vector Quantization,'' IEEE ASSP Magazine, pp. 4--29, April 1984.
vector quantization of 2 elements
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [7]
Recap: List of Compression ToolsRecap: List of Compression Tools
Lossless encoding tools– Entropy coding: Huffman, Arithmetic coding, Lemple-Ziv, …– Run-length coding
Lossy tools for reducing bit rate– Quantization: scalar quantizer vs. vector quantizer– Truncations: discard unimportant parts of data
Facilitating compression via Prediction– Convert the full signal to prediction residue with smaller dynamic range– Encode prediction parameters and residues with less bits– Be careful: use quantized version available to decoder when designing encoder
Facilitating compression via Transforms– Transform into a domain with improved energy compaction
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [8]
Image Transform: A RevisitImage Transform: A Revisit
With A Coding PerspectiveWith A Coding Perspective
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [9]
Why Do Transforms?Why Do Transforms?
Fast computation– E.g., convolution vs. multiplication for filter with wide support
Conceptual insights for various image processing– E.g., spatial frequency info. (smooth, moderate change, fast change, etc.)
Obtain transformed data from measurement– E.g., blurred images, radiology images (medical and astrophysics)– Often need to perform an inverse transform to obtain the actual data
For efficient storage and transmission– Pick a few “representatives” (basis) – Just store/send the major “contribution” from some basis image/vector
=> Examine a segment of signal samples together
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [10]
Basic Process of Transform CodingBasic Process of Transform Coding
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 8)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [11]
Basis Vectors and Basis ImagesBasis Vectors and Basis Images
A basis for a vector space ~ a set of vectors that is– Linearly independent ~ ai vi = 0 if and only if all ai=0– Uniquely represent every vector in the space by their linear combination
~ bi vi ( “spanning set” {vi} )
Orthonormal basis– Orthogonality ~ inner product <x, y> = y*T x= 0 – Normalized length ~ || x ||2 = <x, x> = x*T x= 1
Inner product for 2-D arrays– <F, G> = m n f(m,n) g*(m,n) = G1
*T F1 (rewrite matrix into vector) !! Don’t do FG ~ may not even be a valid operation for MxN matrices!
2D Basis Matrices (Basis Images)– Represent any images of the same size as a linear combination of basis
images
A vector space consists of a set of vectors, a field of scalars, a vector addition operation, and a scalar multiplication operation.
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [12]
Standard / “Trivial” BasisStandard / “Trivial” Basis
Standard basis vectors
Standard basis images
1
0
0
1
0
1
0
3
0
0
1
6
1
3
6
10
000
01
003
00
102
00
012
03
22
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [13]
Matrix/Vector Form of 1-D DFT Matrix/Vector Form of 1-D DFT
{ z(n) } { Z(k) }
n, k = 0, 1, …, N-1, WN = exp{ - j2 / N } ~ complex conjugate of primitive Nth root of
unity
)1(
)1(
)0(
)1(
)2(
)1(
)0(
1
1
1
1 1 11
)1(
)2(
)1(
)0(
110
/)1(2/)1(22/)1(2
/)1(22/42/22
/)1(2/22/2
2 NZ
Z
Z
aaa
NZ
Z
Z
Z
eee
eee
eee
Nz
z
z
z
N
NNjNNjNNj
NNjNjNj
NNjNjNj
1
0
1
0
)(1
)(
)(1
)(
N
k
nkN
N
n
nkN
WkZN
nz
WnzN
kZ
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
z
a
a
a
Nz
z
z
z
eee
eee
eee
NZ
Z
Z
Z
TN
T
T
NNjNNjNNj
NNjNjNj
NNjNjNj
*1
*1
*0
/)1(2/)1(22/)1(2
/)1(22/42/22
/)1(2/22/2
)1(
)2(
)1(
)0(
1
1
1
1 1 11
)1(
)2(
)1(
)0(
2
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [14]
Matrix/Vector Form of 1-D DFT (cont’d)Matrix/Vector Form of 1-D DFT (cont’d)
{ z(n) } { Z(k) }
n, k = 0, 1, …, N-1, WN = exp{ - j2 / N }
“transform kernels” are complex exponentials
Vector form and interpretation for inverse transform
z = k Z(k) ak
where ak = [ 1, WN-k , WN
-2k , … WN-(N-1)k ]T /
N
Basis vectors – ak
H = ak* T = [ 1, WN
k , WN2k , … WN
(N-1)k ] / N
Use akH as row vectors to construct a matrix F
Z = F z z = F*T Z = F* Z
– F is symmetric (FT=F) and unitary (F-1 = FH where FH = F*T )
1
0
1
0
)(1
)(
)(1
)(
N
k
nkN
N
n
nkN
WkZN
nz
WnzN
kZ
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [15]
1-D Unitary Transform1-D Unitary Transform
Consider linear invertible transform– 1-D sequence { x(0), x(1), …, x(N-1) } as a vector – y = A x and A is invertible
Unitary matrix: A is unitary if A-1 = A*T = AH – Denote A*T as AH ~ “Hermitian”
– x = A-1 y = A*T y = ai*T y(i)
– Hermitian of row vectors of unitary matrix A form a set of orthonormal basis vectors {ai
*T }
Think: how about column vectors of A?
Orthogonal matrix ~ A-1 = AT – Real-valued unitary matrix is also an orthogonal matrix– Row vectors of real orthogonal matrix A form orthonormal basis vectors
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [16]
Exercise:Exercise:
Is each matrix here unitary or orthogonal?– If yes, what are the basis vectors?
cossin
sincos5A
2
1
2
12
1
2
1
4A
21
323A
1: n; 2: y. inv(A3) = [2, –3; -1, 2]; Check A A’ = I ?
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
1
1
2
12 j
jA
2
21
j
jA
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [18]
Properties of 1-D Unitary Transform Properties of 1-D Unitary Transform y = A xy = A x
Energy Conservation: || y ||2 = || x ||2
Proof: || y ||2 = || Ax ||2= (Ax)*T (Ax)= x*T A*T A x = x*T x = || x ||2
Interpretation:– The angles between vectors are preserved
– A unitary transformation is a rotation of a vector in an N-dimension space, i.e., a rotation of basis coordinates
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [19]
Properties of 1-D Unitary Transform (cont’d)Properties of 1-D Unitary Transform (cont’d)
Energy Compaction and Decorrelation– Many commonly used unitary transforms tend to pack a large fraction of
signal energy into just a few transform coefficients
– Highly correlated input elements quite uncorrelated output coefficients Covariance matrix E[ ( y – E(y) ) ( y – E(y) )*T ]
small correlation implies small off-diagonal terms
Example: recall the effect of DFT
Question: What unitary transform gives the best compaction and decorrelation?
=> Will revisit this issue
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [20]
1-D Discrete Cosine Transform (DCT)1-D Discrete Cosine Transform (DCT)
Linear transform matrix C
c(k,n) = (0) for k=0
c(k,n) = (k) cos[(2n+1)/2N] for k>0=> Compare “transform kernels” of DCT vs. DFT
C is real and orthogonal
– rows of C form an orthonormal basis
– C is not symmetric!– DCT is not the real part of unitary DFT! See Assignment#3
related to DFT of a symmetrically extended signal
1..., 1,kfor 2
)( ;1
)0(
2
)12(cos)()()(
2
)12(cos)()()(
1
0
1
0
NN
kN
N
knkkZnz
N
knknzkZ
N
k
N
n
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [21]
Periodicity Implied by DFT and DCTPeriodicity Implied by DFT and DCT
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 8)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [22]
Example of 1-D DCT Example of 1-D DCT
100
50
0
-50
-100
0 1 2 3 4 5 6 7n
z(n)
100
50
0
-50
-100
0 1 2 3 4 5 6 7k
Z(k)
DCT
From Ken Lam’s DCT talk 2001 (HK Polytech)U
MC
P E
NE
E6
31
Slid
es
(cre
ate
d b
y M
.Wu
© 2
00
1)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [23]
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
Example of 1-D DCT (cont’d): N = 8Example of 1-D DCT (cont’d): N = 8
k
Z(k)
Transform coeff.
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
Basis vectors
100
0
-100
100
0
-100
100
0
-100
100
0
-100
100
0
-100
100
0
-100
100
0
-100
100
0
-100u=0
u=0 to 1
u=0 to 4
u=0 to 5
u=0 to 2
u=0 to 3
u=0 to 6
u=0 to 7
Reconstructions
n
z(n)
Original signal
From Ken Lam’s DCT talk 2001 (HK Polytech)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [26]
2-D DCT2-D DCT Separable orthogonal transform
– Apply 1-D DCT to each row, then to each column
Y = C X CT X = CT Y C = mn y(m,n) Bm,n
– Set y(m,n)=1 and rest as zeros to obtain basis image Bm,n ~ outer product of C’s mth & nth rows
DCT basis images:– Equivalent to represent
an NxN image with a set of orthonormal NxN “basis images”
– Each DCT coefficient indicates the contribution from (or similarity to) the corresponding basis image
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [28]
2-D Transform: General Case2-D Transform: General Case
A general 2-D linear transform with kernel {ak,l(m,n)}
– y(k,l) is a transform coefficient for Image {x(m,n)} – {y(k,l)} is “Transformed Image”– Equiv to rewriting all from 2-D to 1-D and applying 1-D transform
Computational complexity– N2 values to compute– N2 terms in summation per output coefficient– O(N4) for transforming an NxN image!
),(),(),(
),(),(),(
1
0
1
0,
1
0
1
0,
nmhlkynmx
nmanmxlky
N
k
N
llk
N
m
N
nlk
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [30]
2-D Separable Unitary Transforms2-D Separable Unitary Transforms
Focus our attention to separable transform– ak,l(m,n) = ak(m) bl(n) , denote this as a(k,m) b(l,n)
Use 1-D unitary transform as building block– {ak(m)}k and {bl(n)}l are 1-D complete orthonormal sets of basis vectors
use as row vectors to obtain unitary matrices A={a(k,m)} & B={b(l,n)}
– Apply to columns and rows Y = A X BT
often choose same unitary matrix as A and B (e.g., 2-D DFT)
For square NxN image A: Y = A X AT X = AH Y A* – For rectangular MxN image A: Y = AM X AN T X = AM
H Y AN*
Complexity ~ O(N3)– May further reduce complexity if unitary transf. has fast implementation
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [31]
Basis Images for Separable TransformBasis Images for Separable Transform
X = AH Y A* => x(m,n) = k l a*(k,m)a*(l,n) y(k,l)– Represent X with NxN basis images weighted by coeff. Y– Obtain basis image by setting Y={(k-k0, l-l0)} and getting X
{ a*(k0 ,m)a*(l0 ,n) }m,n
Basis image in matrix form A*k,l = a*k al*T
~ a*k is kth column vector of AH transf. coeff. y(k,l) is the inner product of the basis A*k,l with
image X
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [32]
Exercise on Basis Images Exercise on Basis Images
For 2-D separable unitary transform: Y = A X AT => X = AH Y A*
– Represent X with NxN basis images weighted by coeff. Y– Obtain basis image by setting Y={(k-k0, l-l0)} & getting X
In matrix form A*k,l = a*k al*T ~ a*k is kth column vector of AH
Exercise:
– A is Unitary transform or not?– If so, find basis images– Represent an image X with basis images
(Jain’s e.g.5.1, pp137: A’ [5, –1; – 2, 0] A; outer product of columns of AH : [1,1]’[1 1]/2, …)
2
1
2
12
1
2
1
__
__
2
1
2
12
1
2
1
*AYAX H
43
21
11
11
2
1XA
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [35]
Review: 2-D DFTReview: 2-D DFT
Recall: 2-D DFT is Separable
Y = F X F X = F* Y F*
– Basis images Bk,l = (ak ) (al
)T ~ outer product of two vectorswhere ak = [ 1 WN
-k WN-2k … WN
-(N-1)k ]T / N
1
0
1
0
1
0
1
0
),(1
),(
),(1
),(
N
k
N
l
mkN
nlN
N
m
N
n
mkN
nlN
WWlkYN
nmX
WWnmXN
lkY
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [36]
Visualizing Fourier Basis ImagesVisualizing Fourier Basis Images
Fourier basis uses complex exponentials– Their real and imaginary parts give smoothly varying sinusoidal patterns
in different frequencies and orientations
exp[ j 2 (ux + vy) ] = cos[2 (ux + vy)] + j sin[2 (ux + vy)]
v
Real(cos) part
Imaginary(sin) part
(u, v) (1, 0) (0, 5)(1, 1)
Figures from Mani Thomas U.Del CISC489/689 2D FThttp://vims.cis.udel.edu/~mani/TACourses/Spring06/cv/index.html
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [37]
8x8 DFT Basis Images8x8 DFT Basis Images
Figures from John Woods’ book.
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [38]
Summary and ReviewSummary and Review
1-D transform of a vector– Represent an N-sample sequence as a vector in N-dimension vector space– Transform
Different representation of this vector in the space via different basis e.g., 1-D DFT from time domain to frequency domain
– Forward transform In the form of inner product Project a vector onto a new set of basis to obtain N “coefficients”
– Inverse transform Use linear combination of basis vectors weighted by transform
coefficents to represent the original signal
2-D transform of a matrix– Generally can rewrite the matrix into a long vector & apply 1-D transform– Separable transform allows applying transform to rows then columns
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [39]
Summary and Review (cont’d)Summary and Review (cont’d) Vector/matrix representation of 1-D & 2-D sampled signal
– Representing an image as a matrix or sometimes as a long vector
Basis functions/vectors and orthonormal basis– Used for representing the space via their linear combinations– Many possible sets of basis and orthonormal basis
Unitary transform on input x ~ A-1 = A*T – y = A x x = A-1 y = A*T y = ai
*T y(i) ~ represented by basis vectors {ai*T}
– Rows (and columns) of a unitary matrix form an orthonormal basis
General 2-D transform and separable unitary 2-D transform– 2-D transform involves O(N4) computation– Separable: Y = A X AT = (A X) AT ~ O(N3) computation
Apply 1-D transform to all columns, then apply 1-D transform to rows
– For non-square image of size MxN: Y = AMxM X ATNxN ; basis images MxN
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [40]
Summary of Today’s LectureSummary of Today’s Lecture
Unitary transform and properties– Basis vectors and Basis images
DCT transform
Next time: putting together …– Transform coding and JPEG compression standard
Readings– Gonzalez’s 3/e book 8.1; 8.2.1, 8.2.3-8.2.7, 8.2.9 (till before motion);
2.6.6-2.6.7; 8.2.8; – Jain’s book 5.1-5.7 (on unitary transform)
– To read more: Woods’ book 4.2, 4.3, 4.5
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)