Image Processing - Lesson 6

• Linear Systems

• Definitions & Properties

• Shift Invariant Linear Systems

• Linear Systems and Convolutions

• Linear Systems and sinusoids

• Complex Numbers and Complex Exponentials

• Linear Systems - Frequency Response

Introduction to Fourier Transform - Linear Systems

• A linear system T gets an input f(t)and produces an output g(t):

• In the discrete caes:– input : f[n] , n = 0,1,2,…– output: g[n] , n = 0,1,2,…

f(t) g(t)

( ){ }tfTtg =)(

[ ] [ ]{ }nfTng =

T

• A linear system must satisfy two conditions:

– Homogeneity:

– Additivity:

[ ]{ } [ ]{ }nfTanfaT =

[ ] [ ]{ } [ ]{ } [ ]{ }nfTnfTnfnfT 2121 +=+

T

T

T

Homogeneity

Additivity

T

• Contrast change by grayscale stretching around 0:

– Homogeneity:

– Additivity:

T{bf(x)} = abf(x) = baf(x) = bT{f(x)}

T{f(x)} = af(x)

T{f1(x)+f2(x)} = a(f1(x)+f2(x)) = af1(x)+af2(x)= T{f1(x)}+ T{f2(x)}

• Convolution:

– Homogeneity:

– Additivity:

T{bf(x)} = (bf)*a = b(f*a) = bT{f(x)}

T{f(x)} = f*a

T{f1(x)+f2(x)} = (f1+f2)*a = f1*a+f2*a= T{f1(x)}+ T{f2(x)}

• Assume T is a linear system satisfying

• T is a shift-invariant linear system iff:

T

Shift Invariant

( ){ }tfTtg =)(

( ){ }00 )( ttfTttg −=−

T

• Contrast change by grayscale stretching around 0:

– Shift Invariant:

• Convolution:

– Shift Invariant:

T{f(x-x0)} = af(x-x0) =g(x-x0)

T{f(x)} = af(x) = g(x)

T{f(x-x0)} = f(x-x0)*a

T{f(x)} = f(x)*a =g(x)

� −−� =−−=j

0i

0 )xjx(a)j(f)ix(a)xi(f

= g(x-x0)

• Assume f is an input vector and T is a matrix multiplying f:

• g is an output vector. • Claim: A matrix multiplication is a

linear system:

– Homogeneity– Additivity

• Note that a matrix multiplication is not necessarily shift-invariant.

=

( ) 2121 TfTfffT +=+( ) aTfafT =

• An impulse signal is defined as follows:

• Any signal can be represented as a linear sum of scales and shifted impulses:

[ ]���

=≠

=−knwhereknwhere

kn10

δ

[ ] [ ] [ ]jnjfnfj

−=�∞

−∞=

δ

= + +

Proof: – f[n] input sequence– g[n] output sequence– h[n] the system impulse response:

h[n]=T{δ[n]}

[ ] [ ]{ } [ ] [ ]

[ ] [ ]{ }

[ ] [ ]

hf

nariancceshiftfromjnhjf

linearityfromjnTjf

jnjfTnfTng

j

j

j

∗=

−−=

−=

���

���

−==

�

�

�

∞

−∞=

∞

−∞=

∞

−∞=

)i(

)(δ

δ

The output is a sum of scaled and shifted copies of impulse responses.

Convolution as a sum of impulse responses

*H(n)f(n)

• The convolution (wrap around):

can be represented as a matrix multiplication:

– The matrix rows are flipped and shifted copies of the impulse response.

– The matrix columns are shifted copies of the impulse response.

[ ] [ ] [ ]283256123210021 −−−=∗−−

�������

�

�

�������

�

�

−−−

=

�������

�

�

�������

�

�

−−

�������

�

�

�������

�

�

283256

210021

210003321000032100003210000321100032

CirculantMatrix

• Commutative:

– Only shift-invariant systems are commutative.– Only circulant matrices are commutative.

• Associative:

– Any linear system is associative.

• Distributive:

– Any linear system is distributive.

fTTfTT ∗∗=∗∗ 1221

( ) ( )fTTfTT ∗∗=∗∗ 2121

( )( ) 2121

2121

fTfTffTandfTfTfTT∗+∗=+∗

∗+∗=∗+

The Complex Plane

Real

Imaginary

(a,b)

a

b

• Two kind of representations for a point (a,b) in the complex plane– The Cartesian representation:

12 −=+= iwherebiaZ– The Polar representation:

θReiZ =

Rθ

• Conversions:

– Polar to Cartesian:

– Cartesian to Polar

( ) ( )θsinθcosRe θ iRRi +=( )abiebaiba /tan22 1−

+=+

(Complex exponential)

• Conjugate of Z is Z*:

– Cartesian rep.

– Polar rep.

( ) ibaiba −=+ ∗

( ) θθ ReRe ii −∗ =

Reala

b

θ

-θ

-b

θReiiba =+

θRe iiba −=−

R

R

Imaginary

Algebraic operations:

• addition/subtraction:

• multiplication:

• Norm:

)()()()( dbicaidciba +++=+++

( )( ) ( ) ( )adbcibdacidciba ++−=++( )βαβα += iii ABeBeAe

( ) ( ) 222 baibaibaiba +=++=+ ∗

( ) 2θθθθ2θ ReReReReRe Riiiii === −∗

• The (Co-)Sinusoid as complex exponential:

Or

( )2eexcos

ixix −+=

( )i2eexsin

ixix −−=

( ) ( )ixex Realcos =

( ) ( )ixex Imagsin =

x 1

1

cos(x)

sin(x)eix

x

( )xπω2sin

ω1

– The wavelength of sin(2πωx) is 1/ω .– The frequency is ω .

1

x 1

1

cos(x)

sin(x)eix

The (Co-) Sinusoid- function

x

( )xA πω2sin

A

– Changing Amplitude:

– Changing Phase:

x

( )φπω2sin +xA

( )φsinA

( ) )(Imagφπω2sin πω2φ xii eAexA =+

Scaling and shifting can be represented as a multiplication with φiAe

• If we add a Sine wave to a Cosine wave with the same frequency we get a scaled and shifted (Co-) Sine wave with the same frequency:

( ) ( ) ( )φωωω +=+ xRxbxa sin cossin

��

���

�=+= −

abbaR 122 tanandwhere φ

( ) ( ) cossin xbxa ωω +

( )φω +xi

Two alternatives to represent a scaled and shifted Sine wave.

a2+b2

Combining Sine and Cosine - Proof

Linear Combination of sin(ωx) and cos(ωx) produces sin(ωx) with a change in Phase and Amplitude.

Proof:

a sin(ωx) + b cos(ωx) = R sin(ωx + θ)

a sin(ωx) + b cos(ωx) =

[ sin(ωx) + cos(ωx) ]ײa2+b2

a2+b2ײa

a2+b2ײb

Since

a2+b2ײa

ײb+

22= 1

there exists θ such that :

a2+b2ײa = cos(θ)

a2+b2ײb = sin(θ)

*

*Thus from we obtain:

[ cos(θ) sin(ωx) + sin(θ) cos(ωx) ]ײa2+b2

sin(ωx + θ)ײa2+b2=

AmplitudeR = ײa2+b2

θ = tg-1(b/a) Phase

The response of Shift-Invariant Linear System to a Sine wave.• The response of a shift-invariant linear system

to a sine wave is a shifted and scaled sine wave with the same frequency.

• The frequency response (or Transfer Function) of a system:

( )x2sin πω ( ) ( )( )ωφπω2sinω +xA

ω ω

A(ω) φ(ω)

x2ie πω ( ) ( ) x2ii eeA πωωφω

The response of Shift-Invariant Linear System to a Sine wave.• The response of a shift-invariant linear system

to a sine wave is a shifted and scaled sine wave with the same frequency.

• The frequency response (or Transfer Function) of a system:

ω ω

A(ω) φ(ω)

x2ie πω ( ) ( ) x2ii eeA πωωφω

( ) ( ) ( )ωω ωφ HeA i =

xie πω2

( ) ( ) ( )ωω ωφ HeA i =

( ) xieH πω2ω

)(δ x ( )xh

Impulse response

Frequency response

• If a function f(x) can be expressed as a linear sum of scaled and shifted sinusoids:

it is possible to predict the system response to f(x):

• The Fourier Transform:It is possible to express any signal as a sum of shifted and scaled sinusoids at different frequencies.

( ) xieFxf πω

ωω 2)( �=

( ){ } ( ) ( ) xieFHxfTxg πω

ωωω 2)( �==

( )�=ω

πω2ω)( xieFxf

( ) ωω)(ω

πω2 deFxf xi�=

Or

3 sin(x)

+ 1 sin(3x)

+ 0.8 sin(5x)

+ 0.4 sin(7x)

=

Every function equals a sum of scaled and shifted Sines

+

+

+

Linear System Logic

Input Signal

Express as sum of scaled

and shifted impulses

Express as sum of scaled

and shifted sinusoids

Calculate the response to

each impulse

Calculate the response to

each sinusoid

Sum the impulse

responses to determine the

output

Sum the sinusoidal

responses to determine the

output

Frequency Method

space/time Method

)()()( xhxfxg ∗=)ω()ω()ω( HFG =