Sistemes lineals(Filtrat lineal)

Felipe LumbrerasDept. Ciències de la Computació / Centre de Visió per Computador

Universitat Autònoma de Barcelonahttp://www.cvc.uab.es/shared/teach/a102784/

Linear Systems(images as 2D signals)

1. Signals2. Linear systems. Convolution.3. Linear filters

1. Smoothing2. Edges3. Pattern matching

signals

• continuous signal f(x)

• discrete space signal f(x1)

• digital signal f(x1) (quantized)

2D signals

Mathematically, what a discrete 2d-signal is?

A infinite sequence, defined at integer coordinates:

A special and simple signal:

Dirac delta function (unit impulse)

Dirac delta function

Any signal can be decomposed as a linear combination (weighted sum) of shifted impulses.

weights

(values of f at (k1,k2))Shifted unit impulses

Systems

What a system is?

T can be a lot of things, but we are only interested on systems that are:– Simple (= easy to study, characterize and compute)– Representing interesting transformations– Model real transformations applied to signals

Linear Shift-invariant Systems

• A system is linear if:

The output for a linear combination of input signals = the same linear

combination of the outputs for each of the input signals

• A systems is shift-invariant if: “do the same anywhere”

The output for a shifted input signal = The output of the non-shifted signal

shifted in the same way

Linear shift-invariant systems?

• How these transformations are?

• What does T1 do?

Convolution

a signal can be expressed as a

linear combination of Dirac deltas

Convolution

linearity

Convolution

h is defined as the centered unitary

impulse response of the system

Convolution

definition of the convolution operator (*)

Convolution

Convolution

Convolution

Fix (x1,x2) = (2,3) as example. Now, (k1,k2) can vary.

Convolution

Fix (x1,x2) = (2,3) as example. Now, (k1,k2) can vary.

Convolution

Convolution

What is the impulsive response h of the previous system?

Convolution

What could I do at borders?

Convolution

Why this operator is so important?

– Permits to characterize any linear shift-invariant through its impulsive response h.

– Characterize = compute, have only one property to define it.

– Very simple systems (products and additions).

– Field very well studied (signal processing).

– Changing h we can have several and very different behaviors.

– Some of them (useful): to finding contours, reducing noise, for pattern matching purposes.

– Model signal degradations, for example: defocused images.

– Needed if these degradations want to be removed.

CorrelationEqual to convolution, but without reflecting the h kernel.

Convolution

Concepts

• Padding: add extra pixels around the boundary. Different strategies:

0 (normal), value, Symmetric, Replicate, Circular

• Output size: same, full, valid

• Stride (DL): skip intermediate locations in convolution

• À trous (scale, wavelets, DL) : the convolution kernels increase its size adding intermediate zeros.

Image as a 2d signal

Images can be seen as digital 2d signals (discrete spacing and quantized values).

We can think that outside a certain range the values are zero, or the image (signal, function) is not defined, or has a periodicity, or doesn’t matter.

z grey level

Image convolution

Linear filtersSmoothing: reduce the intensity local variations, often due to

acquisition noise.

Result for correlation should be the same?

Linear filters

Linear filtersEdges: point out intensity local variations, when these variations are due to object contours.

shows (0,0) origin.

Linear filters

Linear filters

• Gradient

• Magnitude

• Orientation

Linear filters

Edge detectors

(1) Roberts

(2) Sobel

(3) Prewitt

(4) Laplacian of Gaussian (zero crossing)

(5) Canny

(1) (2)

(3) (4) (5)

Pattern Matching

Find patterns = find subimages, templates or structures, in an image, defining a similarity measure.

Correlation is a useful technique to start with this topic.

– Binary images: the result is the number of points that matches with the model

– Grey level images: we need a similarity measure:

well suited if Sf 2(x+i) is nearly constant (strong restriction)

iii

ixfxfxtitixfitxs )()()(2)()]()([)( 222

NCC (Normalized cross-correlation)

normxcorr2 (Matlab function)

Pattern Matching

Fourier Transform

1. Motivation

2. Definition

3. Interpretation

4. Linear systems characterization

5. Frequency filtering

6. Fast computation of correlation/convolution

Jean Baptiste Joseph Fourier

Motivation

What is FT used for in image processing?

1. Change in the representation of the images: from a spatial domain (x,y) to a frequency domain (u,v)

space / time frequency

Motivation

MotivationIn the frequency representation we can process (filter) in a different way than in the spatial one.

noise

mask

Motivation

2. Alternative characterization of the linear shift-invariant systems:

• Output computation of a system from an input

Motivation

2. Fast computation of convolution and correlations. using the FFT (Fast Fourier Transform).

Note:

Definition

• Continuous 2D signals

• Discrete space 2D signals

for

for integers, but

Definition

• Discrete space periodic M x N 2D signals

Discrete space periodic N x N 2D signals

direct

inverse

Properties

• Shift in space domain (x0 , y0) → phase changes

Properties

• Correlation and convolution in space: product

• Correlation and convolution with module and phase:

Properties

• Rotation

• Scale

• Periodicity

• Symmetry

• Dirac delta

Properties

• Derivatives

• Energy conservation (Parseval theorem)

Properties

• Separability: 2D FT computation, starts with 1D FT

Fourier Transform

Fourier Transform

Fourier Transform

Fourier Transform

Fourier Transform

Interpretation

Interpretation

Interpretation

Interpretation

Linear Systems: convolution

Convolution theorem:

if

f (x, y) ↔ F (u, v) ( F {f} = F )

h (x, y) ↔ H (u, v) ( F {h} = H )

then

f *h (x, y) ↔ F·H (u, v) ( F {f * h} = F·H )

• Convolution at space domain is product at frequency domain. And vice versa.

Linear Systems: convolution

Then, as there is only one FT of a function, and for each FT there is only one function.

H express how amplitude and phase

of F change at each u,v

Linear Systems: correlation

And also for definition:

Frequency domain filtering

Frequency domain filtering

Frequency domain filtering

Fast correlation / convolution

• Template matching by correlation

Fast correlation / convolution

– Space domain

– Frequency domainproducts

products

Convolution, correlation and Fourier transform in MatLab– Spatial domain

conv2

convn

imfilter

– Frequency domainfft2

ifft2

fftshift

conj, abs, atan2

single, double

.*

Related topics

1. Gaussian Derivatives

2. Laplacian of Gaussian (LoG)

3. Scale space

4. Gabor filters

5. Wavelets

6. Pyramids

Gaussian Derivatives

• Derivates are sensible to noise (higher orders worst).

• Smoothing with a Gaussian reduce noise.

• Radius of the Gaussian integrates local information (scale)

• Derivatives and convolution

Gaussian Derivatives

• Regularized derivatives

Gaussian Derivatives

Gaussian derivatives filter bank

Laplacian of Gaussian

• Second order derivative

s=0.5 s=1.0 s=2.0 s=4.0

LoG

Scale space

• Analyse Image structure at different scales.

• Human visual system extracts multiscale information.

• Representation with a family of smoothed images– linearity (no knowledge, no model, no memory)

– spatial shift invariance (no preferred location)

– isotropy (no preferred orientation)

– scale invariance (no preferred size, or scale)

Gabor filters

• Tunable filter on frequency and orientation.

• Similarities established with the human visual system.

• Well suited for texture analysis.

Gabor filter bank

frequency

representation

Wavelets

• Time frequency (scale) representation of a signal

• Multiresolution representation

Wavelets

• Related with Fourier analysis but …– Fourier bases (sin, cos) well localized in frequency and bad in space.

– Wavelets bases are well localized in frequency and space

mexican hat haargabordaubechies

Wavelets

• Discrete Wavelet Transform of an image

HH1HH1HL1

LH1

HH2

LL2 LH2

HL2

source: wikipedia

Pyramids

• Multiresolution image representation (redundant)

source: Brian A. Wandell, Foundations of Vision

Gaussian pyramid

Laplacian pyramid:

differences between one gaussian step

and its interpolation

Bibliography• Joan Serrat, Processament d’Imatges.

http://www.cvc.uab.es/shared/teach/a20380/c20380.htm

• D. A. Forsyth, J. Ponce. Computer Vision: A Modern Approach. Prentice Hall,

2003.

• Laplacian of Gaussian (LoG).

http://fourier.eng.hmc.edu/e161/lectures/gradient/node10.html

• Bart M. ter Haar Romeny, Introduction to Scale-Space Theory: Multiscale Geometric Image Analysis Tutorial VBC ’96, Hamburg, Germany