Signal Processing

Signal ProcessingSignal Processing

COS 323COS 323

Digital “Signals”Digital “Signals”

• 1D: functions of space or time (e.g., 1D: functions of space or time (e.g., sound)sound)

• 2D: often functions of 2 spatial 2D: often functions of 2 spatial dimensionsdimensions(e.g. images)(e.g. images)

• 3D: functions of 3 spatial dimensions3D: functions of 3 spatial dimensions(CAT, MRI scans) or 2 space, 1 time (CAT, MRI scans) or 2 space, 1 time (video)(video)

Digital Signal ProcessingDigital Signal Processing

1.1. Understand analogues of Understand analogues of filtersfilters

2.2. Understand nature of Understand nature of samplingsampling

FilteringFiltering

• Consider a noisy 1D signal f(x)Consider a noisy 1D signal f(x)

• Basic operation: smooth the signalBasic operation: smooth the signal– Output = new function h(x)Output = new function h(x)

– Want properties: linearity, shift invarianceWant properties: linearity, shift invariance

• Linear Shift-Invariant FiltersLinear Shift-Invariant Filters– If you double input, double outputIf you double input, double output

– If you shift input, shift outputIf you shift input, shift output

ConvolutionConvolution

• Output signal at each point = weighted Output signal at each point = weighted average of local region of input signalaverage of local region of input signal– Depends on input signal, pattern of weightsDepends on input signal, pattern of weights

– ““Filter” g(x) = function of weights for linear Filter” g(x) = function of weights for linear combinationcombination

– Basic operation = move filter to some position Basic operation = move filter to some position x,x,add up f times gadd up f times g

ConvolutionConvolution

dttxgtfxgxf )()()()(

dttxgtfxgxf )()()()(

f(x)f(x)

g(x)g(x)

ConvolutionConvolution

• f is called “signal” and g is “filter” or f is called “signal” and g is “filter” or “kernel”, but the operation is symmetric“kernel”, but the operation is symmetric

• Usually desirable to leave a constant Usually desirable to leave a constant signalsignalunchanged: choose g such thatunchanged: choose g such that

1)(

dttg 1)(

dttg

Filter ChoicesFilter Choices

• Simple filters: box, triangleSimple filters: box, triangle

Gaussian FilterGaussian Filter

• Very commonly used filterVery commonly used filter

2

2

2

2

1)(

x

exG

2

2

2

2

1)(

x

exG

Gaussian FiltersGaussian Filters

• Gaussians are used because:Gaussians are used because:– Smooth (infinitely differentiable)Smooth (infinitely differentiable)

– Decay to zero rapidlyDecay to zero rapidly

– Simple analytic formulaSimple analytic formula

– Separable: multidimensional Gaussian =Separable: multidimensional Gaussian =product of Gaussians in each dimensionproduct of Gaussians in each dimension

– Convolution of 2 Gaussians = GaussianConvolution of 2 Gaussians = Gaussian

– Limit of applying multiple filters is GaussianLimit of applying multiple filters is Gaussian(Central limit theorem)(Central limit theorem)

2D Gaussian Filter2D Gaussian Filter

Sampled SignalsSampled Signals

• Can’t store continuous signal: instead store Can’t store continuous signal: instead store “samples”“samples”– Usually evenly sampled:Usually evenly sampled:

ff00=f(x=f(x00), f), f11=f(x=f(x00++x), fx), f22=f(x=f(x00+2+2x), fx), f33=f(x=f(x00+3+3x), …x), …

• Instantaneous measurements of continuous Instantaneous measurements of continuous signalsignal– This can lead to problemsThis can lead to problems

AliasingAliasing

• Reconstructed signal might be very Reconstructed signal might be very different from original: “aliasing”different from original: “aliasing”

• Solution: smooth the signal Solution: smooth the signal beforebefore samplingsampling

Discrete ConvolutionDiscrete Convolution

• Integral becomes sum over samplesIntegral becomes sum over samples

• Normalization condition isNormalization condition is

i

ixi gfgf i

ixi gfgf

1i

ig 1i

ig

Computing Discrete ConvolutionsComputing Discrete Convolutions

• What happens near edges of signal?What happens near edges of signal?– Ignore (Output is smaller than input)Ignore (Output is smaller than input)

– Pad with zeros (edges get dark)Pad with zeros (edges get dark)

– Replicate edge samplesReplicate edge samples

– Wrap aroundWrap around

– ReflectReflect

– Change filterChange filter

i

ixi gfgf i

ixi gfgf

Computing Discrete ConvolutionsComputing Discrete Convolutions

• If f has If f has nn samples and g has samples and g has mm nonzero nonzero samples,samples,straightforward computation takes timestraightforward computation takes time

O( O(nmnm))

• OK for small filter kernels, bad for large OK for small filter kernels, bad for large onesones

i

ixi gfgf i

ixi gfgf

Example: SmoothingExample: Smoothing

Original imageOriginal image Smoothed withSmoothed with2D Gaussian kernel2D Gaussian kernel

Example: Smoothed DerivativeExample: Smoothed Derivative

• Derivative of noisy signal = more noisyDerivative of noisy signal = more noisy

• Solution: smooth with a GaussianSolution: smooth with a Gaussianbeforebefore taking derivative taking derivative

• Differentiation and convolution both Differentiation and convolution both linear operators: they “commute”linear operators: they “commute”

dx

dgfg

dx

dfgf

dx

d dx

dgfg

dx

dfgf

dx

d

Example: Smoothed DerivativeExample: Smoothed Derivative

• Result: good way of finding derivative = Result: good way of finding derivative = convolution with derivative of Gaussianconvolution with derivative of Gaussian

Smoothed Derivative in 2DSmoothed Derivative in 2D

• What is “derivative” in 2D? Gradient:What is “derivative” in 2D? Gradient:

• Gaussian is separable!Gaussian is separable!

• Combine smoothing, differentiation:Combine smoothing, differentiation:

y

f

x

fyxf ,),(

y

f

x

fyxf ,),(

)()(),(

)()(),(

)()(),(

)()(),(),(),(

11

11

11

11

2yGxGyxf

yGxGyxf

yGxGyxf

yGxGyxfyxGyxf

)()(),(

)()(),(

)()(),(

)()(),(),(),(

11

11

11

11

2yGxGyxf

yGxGyxf

yGxGyxf

yGxGyxfyxGyxf

)()(),( 112 yGxGyxG )()(),( 112 yGxGyxG

Smoothed Derivative in 2DSmoothed Derivative in 2D

)()(),(

)()(),(

)()(),(

)()(),(),(),(

11

11

11

11

2yGxGyxf

yGxGyxf

yGxGyxf

yGxGyxfyxGyxf

)()(),(

)()(),(

)()(),(

)()(),(),(),(

11

11

11

11

2yGxGyxf

yGxGyxf

yGxGyxf

yGxGyxfyxGyxf

Smoothed Derivative in 2DSmoothed Derivative in 2D

Original ImageOriginal Image Smoothed Gradient MagnitudeSmoothed Gradient Magnitude

Canny Edge DetectorCanny Edge Detector

• SmoothSmooth

• Find derivativeFind derivative

• Find maximaFind maxima

• ThresholdThreshold

Canny Edge DetectorCanny Edge Detector

Original ImageOriginal Image EdgesEdges

Fourier TransformFourier Transform

• Transform applied to function to analyze its Transform applied to function to analyze its “frequency” content“frequency” content

• Several versionsSeveral versions– Fourier series:Fourier series:

• input = continuous, bounded; output = discrete, unboundedinput = continuous, bounded; output = discrete, unbounded

– Fourier transform:Fourier transform:• input = continuous, unbounded; output = continuous, input = continuous, unbounded; output = continuous,

unboundedunbounded

– Discrete Fourier transform (DFT):Discrete Fourier transform (DFT):• input = discrete, bounded; output = discrete, boundedinput = discrete, bounded; output = discrete, bounded

Fourier SeriesFourier Series

• Periodic function f(x) defined over [–Periodic function f(x) defined over [– .. .. ]]

wherewhere

1

021 )sin()cos()(

nnn nxbnxaaxf

1

021 )sin()cos()(

nnn nxbnxaaxf

dxnxxfb

dxnxxfa

n

n

)sin()(

)cos()(

1

1

dxnxxfb

dxnxxfa

n

n

)sin()(

)cos()(

1

1

Fourier SeriesFourier Series

• This works because sines, cosines are This works because sines, cosines are orthonormal over [–orthonormal over [– .. .. ]:]:

• Kronecker delta:Kronecker delta:

otherwise0

if1

0)cos()sin(

)sin()sin(

)cos()cos(

1

1

1

nm

dxnxmx

dxnxmx

dxnxmx

mn

mn

mn

otherwise0

if1

0)cos()sin(

)sin()sin(

)cos()cos(

1

1

1

nm

dxnxmx

dxnxmx

dxnxmx

mn

mn

mn

Fourier TransformFourier Transform

• Continuous Fourier transform:Continuous Fourier transform:

• Discrete Fourier transform:Discrete Fourier transform:

• F is a function of frequency – describes how F is a function of frequency – describes how much of each frequency much of each frequency ff contains contains

• Fourier transform is invertibleFourier transform is invertible

dxexfk xikxf

2)()(F )(F dxexfk xikxf

2)()(F )(F

1

0

2kF

n

x

xix

nk

ef

1

0

2kF

n

x

xix

nk

ef

Fourier Transform and Fourier Transform and ConvolutionConvolution

• Fourier transform turns convolutionFourier transform turns convolutioninto multiplication:into multiplication:

FF ((ff((xx) ) ** gg((xx)))) = = FF ((ff((xx)))) FF ((gg((xx))))

(and vice versa):(and vice versa):

FF ((ff((xx) ) gg((xx)))) = = FF ((ff((xx)))) ** FF ((gg((xx))))

Fourier Transform and Fourier Transform and ConvolutionConvolution

• Useful application #1: Use frequency Useful application #1: Use frequency space to understand effects of filtersspace to understand effects of filters– Example: Fourier transform of a GaussianExample: Fourier transform of a Gaussian

is a Gaussianis a Gaussian

– Thus: attenuates high frequenciesThus: attenuates high frequencies

==

FrequencyFrequency

Am

plit

ude

Am

plit

ude

FrequencyFrequency

Am

plit

ude

Am

plit

ude

FrequencyFrequency

Am

plit

ude

Am

plit

ude

Fourier Transform and Fourier Transform and ConvolutionConvolution

• Box function?Box function?

• In frequency space:In frequency space:sinc functionsinc function– sinc(x) = sin(x) / xsinc(x) = sin(x) / x

– Not as good at attenuatingNot as good at attenuatinghigh frequencieshigh frequencies

Fourier Transform and Fourier Transform and ConvolutionConvolution

• Fourier transform of derivative:Fourier transform of derivative:

• Blows up for high frequencies!Blows up for high frequencies!– After Gaussian smoothing, doesn’t blow upAfter Gaussian smoothing, doesn’t blow up

)()( 2 xfxfdx

dki FF

)()( 2 xfxfdx

dki FF

Fourier Transform and Fourier Transform and ConvolutionConvolution

• Useful application #2: Efficient Useful application #2: Efficient computationcomputation– Fast Fourier Transform (FFT) takes timeFast Fourier Transform (FFT) takes time

O( O(n n log log nn))

– Thus, convolution can be performed in timeThus, convolution can be performed in time O( O(nn log log nn + + mm log log mm))

– Greatest efficiency gains for large filtersGreatest efficiency gains for large filters