Upload
others
View
28
Download
0
Embed Size (px)
Citation preview
1
1
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Enabling Technologies for Sports
(5XSF0), Module 03
Discrete Fourier Transform and Filtering
Peter H.N. de With([email protected] )
slides version 1.0
2
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Overview Module 03
∗ Image Frequency Domain
– Fourier transform
– Properties
∗ Image filtering based on frequency domain
– Various filter types and their response (Gaussian, Laplace..)
– Noise reduction by frequency domain filtering
2
3
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Module 03 – Part 1
Frequency domain representation
Sampling and Fourier transform, 2-D
extensions, convolution, aliasing,
filtering concept
4
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Image waveform concept of Fourier
Image: essentially a 2-D
signal waveform
∗ Signal: decomposed in
a sum of components
– Fourier found in 1807
that any periodic or
‘limited energy’
waveform: can be a
weighted sum of sines
and cosines
3
5
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Fourier series and impulses
Characterization of a video signal f(t)∗ Assume function f(t) with period T, which can be expressed as sum
of sines and cosines, with coefficients (n=0, +/- 1, +/- 2, …)
∗ 2. Unit impulses
∗ 3. Sifting process
∗ Exercise: derive discrete versions!
+∞
−∞=
=n
Tntj
nectf
/2)(
πdtetf
Tc
T
T
Tntj
n +
−
−=2/
2/
/2)(
1 π
;)0();0(0)( ∞=≠= δδ tt 1)( =+∞
∞−
dttδ
)()()(00
tfdttttf =−+∞
∞−
δ
6
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Signal is series of (weighted) impulses
Characterization of a video signal
∗ Assume signal s(t) with sampling period T, which can be expressed as sum impulses, with signal coefficients(n=0, +/- 1, +/- 2, …)
∗ If weights are sn=1, then
equal to sampling function!
+∞
−∞=∆
∆−=n
nTTntsts )()( δ
4
7
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Fourier transform (one cont. variable)
Fourier transform of a signal f(t)∗ Assume function f(t) with of time t, which
∗ 2. Inverse Fourier transform
∗ 3. The equations form a transform pair. Note that with Euler, it also holds that
+∞
∞−
−= dtetfFftj πω 2
)()(
+∞
∞−
+= ωω πdeFtf
ftj 2)()(
)2sin()2cos(2 ftjfte ftj πππ +=
Note that ω is
replaced here to
frequency f in
many books!
If f(t) is real,
then the
transform is
complex!
8
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Fourier transform (cont.) - Example
[ ])(
)sin(
2)(
2/
2/
2
2/
2/
2
fW
fWAWe
fj
AdtAewF
W
W
ftj
W
W
ftj
π
π
πππ =
−==
+
−−
+
−
−
Frequency spectrum is magnitude of transform!
µ = f in the
drawing!
5
9
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Sampling of a signal – (1)
Sampling of
• continous
signal f(t)
• time period
∆T (or Ts)
• results in
weighted
impulse train
+∞
−∞=
∆−=∆=n
nTnttfTnff )()()( δ
10
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Sampling of a signal – (2)
Sampling
speed
such that
spectra fit!
• Signal
should be
bandwidth-
limited
µ=f in the
drawing!
6
11
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Sampling of a signal – (3) Theorem
Reconstruction is possible if one
spectrum can be extracted, thus
max22
12 f
T>
∆
Sampling at 2x
fmax is called the
Nyquist rate
µ=f in the
drawing!
12
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Sampling Theorem (& Nyquist rate)
Note that µ is
here frequency f
in the drawing!
Assume here
maxmax
)(
fff
TfH
+≤≤−
∆=
7
13
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Under-Sampling
Under-sampling
gives overlapping
frequency areas
• HF becomes LF!
• Overlap is called
aliasing
• Aliasing always
occurs but can be
suppressed with
low-pass pre-
filters
µ=f in the
drawing!
14
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Discrete Fourier Transform (one variable)
Discrete Fourier Transform of a signal f(t)∗ Assume function f(t) sampled at time ∆T
∗ 2. Inverse Discrete Fourier transform
∗ 3. This can be derived with taking M samples in interval of frequency
1,...,2,1,01
0
/2 −== −
=
−MmefF
M
n
Mmnj
nm
π
1,...,2,1,01
0
/2 −== −
=
+MneFf
M
m
Mmnj
mn
π
TM
mfMm
Tf
∆=−=
∆≤≤ ;1,...,2,1,0;
10
8
15
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
1-D Discrete Fourier Transform (mod. Not.)
Discrete Fourier Transform of an image signal I(x, y)∗ Assume function I(t) sampled at discr. times x∆T and discr. frequency u
∗ 2. Inverse Discrete Fourier transform
∗ 3. This is the common notation in many books and remainder of the course!
1,...,2,1,0)()(1
0
/2 −== −
=
−MuexfuF
M
x
Muxj π
1,...,2,1,0)(1
)(1
0
/2 −== −
=
+MxeuF
Mxf
M
u
Muxj π
16
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Properties of 1-D DFT
∗ 1. Both DFT and IDFT are infinitely periodic
∗ 2. Assuming that f(x) is real and m samples are taken, then the DFT gives a set of m complex numbers
∗ 3. It can be shown that the discrete equivalent of convolution is
∗ 4. If f(x) consists of M samples of f(t) taken ∆T units apart, then period duration T=M∆T and ∆u=1/(M∆T)=1/T
)()()()( kMxfxfkMuFuF +=+=
−
=
−=⊗1
0
)()()()(M
m
mxhmfxhxf
9
17
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
1-D DFT computation example∗ 1. Compute F(0), F(1) etc. of the DFT
– F(0)=1+2+4+4=11, F(1)=-3+2j,
– F(2)=-(1+0j), F(3)=-(3+2j)
∗ 2. The inverse DFT gives the computation of f(0)– f(0)=1/4[ 11-3+2j-1-3-2j ]=1
18
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Extensions to 2-D DFT – (1)
∗ 1. Sampling over 2-D area (image) instead of signal and impulse train becomes 2-D signal
∗ 2. DFT: summing over pixels (x, y), and 2 param’s m, n
∗ DFT periodic in 2-D frequency domain, param’s (u, v)
∗ 2-D sampling theorem: 2 constraints for band-limitations
maxmax 21
;21
vh fZ
fT
>∆
>∆
10
19
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Extensions to 2-D DFT – (2)
∗ 1. Ideal footprint of 2-D LPF becomes rectangular
∗ 2. Aliasing becomes 2-D pattern (e.g. Moire) µ=f h and
ν= fv in the
drawings!
20
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Extensions to 2-D DFT – (3) / Moire
11
21
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Extensions to 2-D DFT – (4) / Moire
22
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
2-D Discrete Fourier Transform
∗ 2-D Discrete Fourier Transform of image signal f(x, y) for 0<=u<=M-1 and 0<=v<=N-1 is
∗ 2. 2-D Inverse Discrete Fourier Transform
∗ 3. The above equations hold for an image of size M x Npixels and together they form a transform pair
−
=
+−−
=
=1
0
)//(21
0
),(),(N
y
NvyMuxjM
x
eyxfvuFπ
−
=
++−
=
=1
0
)//(21
0
),(1
),(M
u
NvyMuxjN
v
evuFMN
yxfπ
12
23
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Properties of 2-D DFT – (1)
∗ 2-D Spatial and frequency intervals using image signal f(x, y) of size M x N
∗ Translation and rotation
∗ Likewise, a rotation over an angle in f(x,y) gives an equal rotation of F(u, v) (this can be proven with polar coordinates with x=r cos(θ) etc., and u=w cos (ϕ), etc.)
ZNv
TMu
∆=∆
∆=∆
1;
1
0 0
0 0
2 ( / / )
0 0
2 ( / / )
0 0
( , ) ( , )
( , ) ( , )
j ux M vy N
j u x M v y N
f x y e F u u v v
f x x y y F u v e
π
π
+ +
− +
⇔ − −
− − ⇔
24
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Properties of 2-D DFT – (2)
∗ 2-D Periodicity of the DFT spectra
∗ As a special case (use Euler rule for this with u0=M/2)
∗ As a result, the spectrum shifts such that the F(0,0) is now
at the origin!
etcNkyxfyMkxfyxf
etcNkvuFvMkuFvuF
=+=+=
=+=+=
),(),(),(
.),(),(),(
21
21
)2/,2/()1)(,( NvMuFyxfyx −−⇔− +
13
25
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Properties
of 2-D DFT
– (3)
Centering
Multiplication
with (-1)x+y
relocates
F(0,0) to the
origin
µ=f in the
drawings!
26
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Properties
of 2-D DFT
– (4)
Symmetry
14
27
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Fourier spectrum and phase 2-D DFT
∗ 2-D DFT is complex, it can be expressed in polar form
∗ Where the magnitude is the Fourier spectrum and φ the
phase angle
∗ Finally, the power spectrum is
),(),(),(
vujevuFvuF
φ=
=
+=
),(
),(arctan),(
),(),(),(22
vuR
vuIvu
vuIvuRvuF
φ
),(),(),(),(222
vuIvuRvuFvuP +==
28
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Fourier spectrum and phase – Examples
∗ Typically, f(x,y) is real, which implies that 2-D DFT
spectrum is even symmetric about the origin
∗ and φ the phase angle is odd symmetric about the origin
∗ Finally, note that
),(),( vuFvuF −−=
),(),( vuvu −−= φφ
),(),(1
)0,0(1
0
1
0
yxfMNyxfMN
MNFM
x
N
y
== −
=
−
=
15
29
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Fourier spectrum – Example 1
30
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Fourier spectrum – Example 2a
16
31
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Fourier spectrum – Example 2b Phase
32
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Summary
2-D DFT
Proper’s 1
17
33
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Summary
2-D DFT
Proper’s 2
34
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Summary 2-D DFT Properties – (3)
18
35
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Summary 2-D DFT Properties – (4)
∗ And remember that due to Euler’s formula and the
cosine/sine rules:
∗ And
∗ Finally, note that
[ ]
[ ]),(),(2
1)22cos(
),(),(2
)22sin(
000000
000000
NvvMuuNvvMuuyvxu
NvvMuuNvvMuuj
yvxu
−−+++⇔+
−−−++⇔+
δδππ
δδππ
1),( ⇔yxδ
36
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Summary 2-D DFT Properties – (5)
∗ Exercise: derive one of those properties in a step by step
fashion
19
37
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Module 03 – Part 2
Image Filtering based on DFT
Filtering concept using DFT, filter
types, special cases
38
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Frequency-domain filtering fundament
∗ Filtering techniques in the frequency domain are based on modifying the Fourier transform
∗ Note that each value of F(u,v) contains information of all image samples!
∗ If a filter has a filter transfer function, we exploit that
Where F, H and g are all matrices of M x N. Assume that Hand F are centered, requiring pre-procesing!
[ ]),(),(),( vuFvuHIDFTyxg =
20
39
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Necessity of padding with zeros – (1)
∗ Without any measures, the use of the DFT creates a wraparound error when periodicity is created
∗ The solution is to enlarge the the window with extra samples, such as padding with zeros (creates a border)
∗ Padding can be used to decrease ringing in the frequency domain, employing the extended filter impulse response in the time domain.
40
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Necessity of padding zeros – (2)
Example
21
41
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Summary of steps for DFT-based filters
1. Determine padding parameters, typically P=2M and Q=2N
2. Form padded image fp(x,y) of size PxQ appending zeros
3. Multiply padded image with (-1)x+y for centering spectrum
4. Compute the DFT, F(u,v), of the centered padded image
5. Generate a real, symmetric filter, H(u,v), of size PxQ and center in the middle, and compute G(u,v)=H(u,v)F(u,v)
6. Obtain the processed image with the IDFT of G(u,v) and take the real part, and re-center it
7. Extracting the g(x, y) result by selecting the MxN region from the top-left quandrant
[ ][ ]{ } yx
pvuGIDFTrealyxg
+−= )1(),(),(
42
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Example of DFT-based filters / Gaussian2
2/2222
2)(2)(
σσπσπ uxAeuHAexh
−− =⇔=
Both real
functions!LPF HPF
22
43
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Example of DFT-based filters / Gaussian
44
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Sobel filter as DFT-based filter example
23
45
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Ideal 2-D LPF filter – (1) Concept• Passes all frequencies within circle of radius D0 : H(u,v)=1
• Rejects all frequencies outside circle with radius D0 : H(u,v)=0
[ ] 2/122)2/()2/(),( QvPuvuD −+−=
46
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Ideal 2-D LPF
filter –(2)Performance
comparison shows
• this is not very
practical, as
• most detail is
removed in only
13% of the power
• and ringing occurs
at already 2%
removal
24
47
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Ideal 2-D LPF filter – (3) Explanation
1. Blur comes from the main lobe in impulse response
2. Ringing comes from the side lobes in impulse response
3. Spread of sinc function is inversely proportional to pass-band of H(u,v). (Discuss the trade-off)
48
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Smooth 2-D LPF filter – (1) Gaussian1. Establish smooth roll-off in transfer function H(u,v)
2. Ringing comes from the side lobes in impulse response
):(),(0
202/),(
2
DnoteevuHDvuD == − σ
25
49
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
1. Blur can be regulated by variance
2. Ringing is absent
3. Spread of sinc function is smooth
4. Good visual results
Gaussian 2-D
LPF filter – (2)
50
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
2-D HPF filters –
various forms
),(1),( vuHvuHLPHP
−=
0
0
),(:0),(
),(:1),(
DvuDvuH
DvuDvuH
HP
HP
≤=
>=
HPF is opposite of
LPF filter!
26
51
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Smooth 2-D HPF filter – cf. Gaussian):(1),(
0
202/),(
2
DnoteevuHDvuD
HP=−= − σ
52
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
2-D HPF
filters –
perform.
Ideal HPF
vs.
Gaussian
HPF perfor-
mance
27
53
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Laplacian filtering - Concept
1. Laplacian can be implemented in frequency domain with
2. Or, with respect to center of frequency rectangle
3. Furthermore, it holds that for the filter in the freq. domain
4. Enhancement with subjective sharpness is obtained when adding this term (note the scaling), so that
),(4])2/()2/[(4),(
)(4),(
22222
222
vuDQvPuvuH
vuvuH
ππ
π
−=−+−−=
+−=
)},(),({),(2
vuFvuHIDFTyxf =∇
)},(),(),(2
yxfcyxfyxg ∇+=
)},()],(41{[)},()],(1{[
)},(),(),({),(
22vuFvuDIDFTvuFvuHIDFT
vuFvuHvuFIDFTyxg
π+=−
=−=
54
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Laplacian filtering - performance
28
55
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
Unsharp masking & highboost filtering
1. Concept: sharpen images by subtracting an unsharp image from the original. The difference is the mask
2. Steps are:1. Blur the original image
2. Subtract the blurred image from original (difference is the mask)
3. Add mask to the original
3. More formally, the specification is
4. If k=1, it is unsharp masking, if k>1, we have highboost filtering, with if k<1, we have high-freq. emphasis
),(*),(),(
),(),(),(
yxgkyxfyxg
yxfyxfyxg
mask
mask
+=
−=
56
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
High-frequency emphasis filtering
1. For the unsharp mask, we can take the freq.-domain approach
2. Function fLP(x,y) is a low-pass smoothed image from the filtering with HLP(u,v).
3. The unsharp masking equation from the previous slide becomes now in the frequency domain
4. This result can be respecified as a high-pass filter result
)],(),([),( vuFvuHIDFTyxfLPLP
=
)},()]],(1[*1{[),( vuFvuHkIDFTyxgLP
−+=
)},()],(*1{[),( vuFvuHkIDFTyxgHP
+=
High-frequency emphasis filter
29
57
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
High-freq. emphasis filtering / Example
1. For more general form, take coefficients k1, k2 such that
2. Where k1>0 gives controls of the offset of the origin and k2 >0 the high-frequency contribution
3. In medical applications, ringing artifacts from filters are not accepted. Because spatial and freq.-domain Gaussian filters are transform pairs, these filters give smooth response and avoid ringing
4. In the following example, HF emphasis filtering helps
)},()],(*{[),(21
vuFvuHkkIDFTyxgHP
+=
58
PdW-SZ / 2016
Fac. EE SPS-VCA
Enabling Technologies for Sports /
5XSF0 / Module 03 DFT & Filtering
High-freq. emphasis filtering / Example
Gaussian
filter with
D0=40
HF
emphasis
filter,
k1=0.5,
k2=0.75
HF em-
phasis &
histogram
equaliz.