Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Advanced Signal Processing 2 - 2007Reinisch Bernhard
Teichtmeister Georg
Series Expansion with Wavelets
2/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Introduction
● Series expansion● Fourier Series: Either periodic or bandlimited
signals● Timedomain: No frequency information● Fourierdomain: No time information● Is there something between?
3/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Contents
● Basics of signal representation● Wavelets
– Haar wavelet– Multiresolution analysis– Construction of the Sinc - Wavelet
● Wavelets derived from iterated filter banks– Haar case, Sinc case, general construction
● Wavelet series and its properties● Practical outlook (image processing)
4/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Recap of Series expansion
• Signals are points in a Vectorspace• Time-Domain: Basis functions are infinite short
impulses• Signals can be projected onto other basis
functions
∫
∑∞
∞−
∞
−∞=
=
=
duufuufu
tufutf
kk
kkk
)()()(),(
)()(),()(
*ϕϕ
ϕϕ
5/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Possible Basis Functions
• Fourierseries– periodic
• Fouriertransform– bandlimited
• STFT– Infinite set of Fourier Transforms
• Piecewise Fourier Series• Wavelets
∑∞
−∞=
=k
TktjekFtf /)2(][)( π
∫−
=π
π
ωω ωπ
deeXnx njj )(21][
6/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Short Time Fourier Transform
● Window Signal● Compute the Fourier Transform● Shift window and repeat⇒ Spectrogram, Periodogram
7/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Time and Frequency Resolution
• Window has Energydistribution in both: Frequency (σω) and Time (σn).
• Uncertainty principle:
• Optimality is only reached by Gaussian window21≥nσσω
8/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
STFT T/F-Resolution
● Constant over Time and Frequency
9/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Piecewise Fourierseries
● Fourier Series with non-overlapping rectangular windows in time and periodic expansion
● Why?– Overlapping windows are redundant information– Good Time Resolution– Representation of arbitrary functions
● Bad Frequency Resolution● Errors at boundaries
10/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Desired Features of Basis Functions
● Simple characterization● Localization Properties in Time and
Frequency● Invariance under certain operations● Smoothness properties● Moment properties
11/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Haar - Expansion
• Simplest Wavelet Expansion
• Scaled and shifted Wavelets:
• m … Scale• n … Timeshift
)2(2)( 2/, ntt mmnm −= −− φφ
12/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Dyadic Tiling
• Resolution depends on Frequency now• )2(2)( 2/
, ntt mmnm −= −− φφ
13/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Orthonormal Basis for L2?
● Two wavelets on the same Scale have no common support
● Shorter wavelet always averages to zero● Shifting so that jump matches, is not possible
14/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Proof: Definitions
• Consider functions which are constant on
• and have finite support on
• Can approximate L2 arbitrarily well• We call it
]2)1(,2[ 00 mm nn −− +
]2,2[ 11 mm−
)()( 0 tf m−
15/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Proof: Scaling function
• The scaling function
• Approximating the piecewise constant function⎪⎩
⎪⎨⎧
+<≤=−−
−
otherwisentnt
mmm
nm0
2)1(22)( 000
0
2,ϕ
)2(2
2
)(
0020
0
10
0
00
)()(
1
,)()(
mmmn
mm
N
Nnnm
mn
m
nff
N
ftf
m−−−
+
−
−=−
−−
−
=
=
= ∑ ϕ
17/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Proof: Keystep
• Examination of two adjacent Intervals
• Now can be expressed as
• For m0=0, n=1, this means
[ ) [ )0000 2)22(,2)12( and 2)12(,22 mmmm nnnn −−−− +++
)(3)(2)()( [3,4) and )3,2[
3,02,03,0)0(
32,0)0(
2 tttftf ϕϕϕϕ +=+
)()( 0 tf m−
)()( 12,)(
2,)(
0
0
120
0
2tftf nm
mnm
mnn +−−
−−
++ ϕϕ
18/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Proof: Average and Difference
The function can also be expressed as the average
and the difference
over two intervals
)()( 0 tf m−
)(22 ,1
)(12
)(2
0
00
tffnm
mn
mn
+−
−+
− + ϕ
)(22 ,1
)(12
)(2
0
00
tffnm
mn
mn
+−
−+
− − φ
19/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Proof: Coefficients
• With
• We get
( )
( ))(12
)(2
)1(
)(12
)(2
)1(
000
000
212
1
mn
mn
mn
mn
mn
mn
ffd
fff
−+
−+−
−+
−+−
−=
+=
)()(
)()(
,1)1(
,1)1(
12,)(
2,)(
0
0
0
0
0
0
120
0
2
tdtf
tftf
nmm
nmm
nmm
nmm
nn
nn
+−+−
+−+−
+−−
−−
+
=++
φϕ
ϕϕ
20/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Proof: Finalization
• Applying all the things we can write
• Repeating the Average/Difference scheme for higher scales leads to
∑∑−
−=+−
+−−
−=+−
+−
+−+−−
+=
=+=12/
2/,1
)1(12/
2/,1
)1(
)1()1()(
)()(
)()()(
0
0
0
0
000
N
Nnnm
mn
N
Nnnm
mn
mmm
tdtf
tdtftf
φϕ
M
Mm
mm nnm
mn
m
mm nnm
mn
mm
mm
mm
mm
mm
td
tdtftf
εφ
φ
+=
=+=
∑ ∑
∑ ∑+
+−=
−
−=
+−=
−
−=
−
−
−
−
−
1
0
1
1
1
0
1
1
0
1
12
2,
)(
1
12
2,
)()1()(
)(
)()()(
22/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Multiresolution
● Successive approximation● Coarse approximation + added details● Coarse and detail subspace are orthogonal● Leads to self-similar Wavelets in Scale● Useful for applications
23/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Axiomatic Definition (1)
• Sequence of embedded closed subspaces
• Upward Completeness
• Downward Completeness
KK 21012 −− ⊂⊂⊂⊂ VVVVV
)(2 RLVmZm
=∈U
}0{=∈
mZmVI
24/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Axiomatic Definition (2)
• Scale Invariance
• Shift Invariance
• Existance of a orthonormal Basis– Non-orthogonal Basis can be orthogonalized
0)2()( VtfVtf mm ∈⇔∈
ZnVntfVtf n ∈∀∈−⇒∈ )()( 0
25/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Orthogonal Complements
• Vm is a subspace of Vm-1
• We define Wm the orthogonal subset of Vm in Vm-1•• Vm is the space of the scaling functions• Wm the space of the wavelets• By repeating we get
mmm WVV ⊕=−1
MZmWRL
∈⊕=)(2
26/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Constructing the Sinc Wavelet
• Now the scaling functions will be the space of bandlimited functions
• V0 is bandlimited between [-π, π], V-1 between[-2π, 2π]
• W0 the functions bandlimited to [-2π, -π] combined with [π, 2π]
•001 WVV ⊕=−
27/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Scaling function
• The scaling function is given by
ttt
ππϕ sin)( =
28/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Representation of φ
• V0 belongs to V-1
• φ(t) can be represented by basis functions of V-1
• Without proof
)(),2(2][ ;1][
)2(][2)(
00
0
tntngng
ntngtn
ϕϕ
ϕϕ
−==
−= ∑∞
−∞=
∑∈
−=
+−−=
Zn
n
ntngt
ngng
)2(][2)(
]1[)1(][
1
01
ϕφ
29/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Construction Kernel
• g0 is given by
• And finally the wavelet⎩⎨⎧ ≤≤−−
=
=
−
otherwiseeeG
nnng
jj
02)(
2/)2/sin(
21][
220
0
ππωω ω
ππ
)2/3cos(2/
)2/sin()( tt
tt πππφ =
31/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Iterated filter banks
● Until now we constructed wavelets by scalingand shifting of orthonormal function families– Based on multiresolution analysis
● Different approach by filter banks– Iteration leads to a wavelet– Key proberties
● regularity● degree of regularity
32/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Haar case
• Low- and Highpass
• Iterate the filter bank on the lowpass channel
• Multirate signal processing results
⎥⎦⎤
⎢⎣⎡ −=⎥⎦
⎤⎢⎣⎡=
21,
21;
21,
21
10 gg
33/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Haar case
● size-8 discrete Haar transform
● Number of coefficients growth exponentially● Continuous time function
● Length bounded, piecewise constant
34/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Sinc case (1)
( ) ]1[1][;2/
)2/sin(2
1][ 010 +−−== ngngn
nng n
ππ
● Impulse responses (low and highpass filter)
● Fourier transform
● Now consider the iterated filter bank– Upsampling filter impulse– Emulate the Haar construction with g0[n], g1[n]– And define a scaling function
35/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Sinc case (2)
• For further analysis: important part is in the brackets• This product is 2i2π periodic -> in the end it‘s only a
perfect lowpass (sinc scaling function)
1
12/
10
(i)
00
20
200
)(0
1
12/2/)(
02/(i)
)(
2/)2/sin(
2)(
:rewritecan we
)(2
1)(M
:short)()...()()(
:where2/
)2/sin()(2)(
)(oftransformFourier
1
1
1
+
+−
=
+
+−−−
+
−
+
⎥⎦
⎤⎢⎣
⎡⎟⎠⎞
⎜⎝⎛=Φ
=
=
=Φ
∏ i
ij
i
kk
j
jjjji
i
ijjii
i
i
i
ii
eM
eG
eGeGeGeG
eeG
t
ωωωω
ω
ωωω
ϕ
ω
ω
ωωωω
ωω
36/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Sinc case (3)
● Cumbersome way ● But we have gained a more general
construction● The key is the infinite product
– Does this product converge and to what– Converge to what kind of scaling function
37/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Iterated filter banks cont. (1)
● General construction– Two channel orthogonal filter bank– g0[n], g1[n] are low- and highpass filter– Iterate on the branch of the lowpass filter and
process this to infinity– Express the two filters after i-steps– Multirate conclusions
● „Filtering with Gi(z) followed by upsampling by 2 isequivalent to upsampling by 2 followed by filtering wihGi(z2)„
38/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Iterated filter banks cont. (2)
● Discrete time iteratedfilters combined with thecontinuous time functions
● Normalization and rescaling
● Graphical function– piecewise constant– halving the intervall
39/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Iterated filter banks cont. (3)
● Fourier domain act as above● In the iteration scheme we are interesting in
convergence
● This will lead us to regularity discussion
40/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Regularity
● The existence of the limit are criticalconditions– Limits exist if g0[n] are regular– Regular filter leads through iteration to a scaling
function with some degree of smoothness(regularity)
– But not only convergence is sufficient we needalso L2 convergence to build orthonormal bases
– A lot of sufficient conditions, different approaches
41/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Wavelet series and properties
• Enumeration of some general properties of basisfunctions
• Wavelet– Linearity, Shift, Dyadic sampling and time frequency
tiling, Scaling, Localization, decay properties
∫
∑∞+
∞−
∈
==
=
dttfttftnmF
tnmFtf
nmnm
Znmnm
)(),()(),(],[
)(],[)(
,,
,,
ψψ
ψ
42/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Linearity
● The wavelet series is linear. The prooffollows from the linearity of the inner product
))(())(()]()([R ba,any for then
)(),(n]F[m,T[f(t)]
Toperator suppose
nm,
tgbTtfaTtgbtfaT
tft
+=+∈
== ψ
43/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Shift
• For Fourier transform– pair: f(t), F(ω) … f(t-tau), e-jωtau F(ω)
• Now for the wavelet series
[ ] ( ) ( )
[ ] ( ) ( )
[ ] mmknmFktfZkkZ
dttfntnmF
dttftnmF
mmm
mm
mmm
nm
<−↔−
∈=∈
+−=
−=
−
−
−−∞+
∞−
−
+∞
∞−
∫
∫
',2,)2(,2or 2
222,
,
''
2/'
,'
ττ
τψ
τψ
44/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Scaling
• For Fourier transform– pair: f(t), F(ω) … f(at), 1/a*F(ω/a)
[ ] ( ) ( )
[ ] ( )
[ ]nkmFtfZka
dttfna
tanmF
dtatftnmF
kk
k
mm
nm
,2)2(,2
22/1,
,
2/'
,'
−↔
∈=
⎟⎟⎠
⎞⎜⎜⎝
⎛−=
=
−
−
−∞+
∞−
−
+∞
∞−
∫
∫
ψ
ψ
45/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Dyadic sampling and time frequency tiling
• It is important to locate the basis functions in thetime-frequency plane
• sampling in time, at scale m, with period 2m
• The frequency is the inverse of scale, we find ifthe wavelet ist centered around ω0 then:
• This leads to dyadic sampling of time frequencyplane
( ) 2)( 0,, ntt mmnm −=ψψ
mnm 2/ around centered is )( 0, ωωΨ
46/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Dyadic sampling
● The dots indicate the center of the wavelets● The scale axis is logarithmic
47/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Time localization (1)
● Suppose we are interested in the signalaround t=t0
● Which values of F[m,n] carry informationabout signal f(t) at t0 => f(t0)
● Suppose wavelet ψ(t) is supported on theinterval [-n1, n2]
● Ψm,0(t) is supported on [-n12m, n22m]● Ψm,n(t) is supported on [(-n1+n)2m, (n2+n)2m]
48/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Time localization (2)
• At scale m, wavelet coefficients with index n satisfy( ) ( )
10m-
20m-
201
22rewritten becan
22
ntnnt
nntnn mm
−≤≤−
+≤≤+−
49/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Frequency localization (1)
• Suppose now in localization, but now in frequencydomain
( ) ( )
( )( ) ( )
( ) ( ) ωωωπ
ψ
ω
ψψ
ω
ω
deFnmF
dttftnmF
e
ntt
njmm
nm
njm
mmnm
m
m
∫
∫∞+
∞−
∞+
∞−
−
−−
Ψ=
=
Ψ
−=
22/
,
2m/2
2/,
2221],[
],[
22
is ansformFourier tr the22
50/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Frequency localization (2)
• Suppose that the wavelet vanishes in the Fourierdomain outside the region [ωmin,ωmax]
• At specific scale m, the support of Ψm,n(ω) will be[ωmin/2m,ωmax/2m]
• Therefore, a frequency component ω0 incluencesat scale m
⎟⎟⎠
⎞⎜⎜⎝
⎛≤≤⎟⎟
⎠
⎞⎜⎜⎝
⎛
≤≤
0
max2
0
min2
max0
min
loglog
rewrite22
ωω
ωω
ωωω
m
mm
51/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Decay properties
● Fourier series can be used to characterizethe regularity of a signal (decay of thetransfrom coefficients)– Global regularity
● The wavelet transform can be used in a similiar way – Local regularity
52/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Multidimensional wavelets
● A way to build multidimensional wavelets isto use tensor products of their onedimensional counterparts– This will lead to different „mother“ wavelets– Scale changes are now represented in matrices– offers diagonal scaling but is also more restricted
53/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Practical aspects
● Wavelets in matlab
● Images– Image compression– Edge detection– De-noising
54/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Matlab
● Matlab wavelet toolbox– Command line
● Help wavelet– Gui tool (wavemenu)
● 1D wavelets analysis● 2D wavelets analysis● De-noising● Image Fusion● Compression
55/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Example 1D wavelet transform
● Discrete/continuous wavelet transform● coefs = cwt(S, SCALES, ‚wname‘)
56/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Compression (1)
● Wavelet calculation● Find small coefficients and discard them● Store only remaining coefficients● Lossy compression
● Good compression with a fast convergencespeed of the wavelet and good decay of thecoefficients
58/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
Edge detection
original decomposition
approximationis set to zero
reconstruction
59/60ASP2 2007Teichtmeister G. Reinisch B.
Wavelet construction
References
● [1] Wavelets and Subband Coding, Martin Vetterli, JelenaKovacevic, ISBN 0-13-097080-8
● [2] Wavelets – praktische Aspekte, Markus Grabner, VO2006
● [3] AK Computergrafik Bildverarbeitung und Mustererkennung WS 2006/07