Algorithms for Reducing Noise in Synthetic Aperture Radar ...people.uncw.edu/tagliarinig/Courses/380/S2015...Troy Peterson Kling & Je rey Kidwell Algorithms for Reducing Noise in Synthetic

1/28

OutlineIntroduction

Image Filtering AlgorithmsConclusion

Algorithms for Reducing Noisein Synthetic Aperture Radar Images

Troy Peterson Kling& Jeffrey Kidwell

University of North Carolina Wilmington

April 24, 2015

Troy Peterson Kling & Jeffrey Kidwell Algorithms for Reducing Noise in Synthetic Aperture Radar Images

2/28

OutlineIntroduction


IntroductionProblem Description

Natural Language DescriptionFormal Description

Why CSC 380?

Image Filtering AlgorithmsLocal Filters

Boxcar FilterGaussian ConvolutionJ. S. Lee’s Filter

Non-local FiltersNon-local MeansFourier TransformModified ART-2

ConclusionThe NumbersResults for SAR ImagesFuture WorkQuestions


3/28

OutlineIntroduction


Problem DescriptionWhy CSC 380?

Natural Language Description

Images of Earth’s surface taken by synthetic aperture radar (SAR)instruments are inherently noisy. In order to perform any kind ofanalysis or classification on SAR images, it is desirable to firstreduce their granularity. This can be achieved by applying noisereduction filters to the images.

Problem description: Given a noisy synthetic aperture radarimage and a selection of image filters (Boxcar, Gaussianconvolution, Lee, non-local means, Fourier transform, and modifiedART-2 neural networks), the goal is to select the filter that bestreduces noise in the image while also retaining image clarity bypreserving edge details.


3/28

OutlineIntroduction



Natural Language Description

Images of Earth’s surface taken by synthetic aperture radar (SAR)instruments are inherently noisy. In order to perform any kind ofanalysis or classification on SAR images, it is desirable to firstreduce their granularity. This can be achieved by applying noisereduction filters to the images.

Problem description: Given a noisy synthetic aperture radarimage and a selection of image filters (Boxcar, Gaussianconvolution, Lee, non-local means, Fourier transform, and modifiedART-2 neural networks), the goal is to select the filter that bestreduces noise in the image while also retaining image clarity bypreserving edge details.


4/28

OutlineIntroduction



Formal Description

Given an n-by-m synthetic aperture radar image, X , with specklenoise modeled as a standard complex Gaussian distribution

p(z) =1

πexp(−|z |2) (1)

where z = Ae iθ is a vector with amplitude A and phase θ, andgiven a set of noise reduction filters F = {f1, f2, ..., fk}, find fp ∈ Fsuch that

MSE (fp(X )) ≤ MSE (fq(X )) ∀ fq ∈ F (2)

where

MSE (fq(X )) =1

nm

n∑i=0

m∑j=0

(fq(Xij)− Tij)2 (3)

and where T is the ideal, non-noisy image.


4/28

OutlineIntroduction



Formal Description


p(z) =1

πexp(−|z |2) (1)



where

MSE (fq(X )) =1

nm

n∑i=0

m∑j=0




4/28

OutlineIntroduction



Formal Description


p(z) =1

πexp(−|z |2) (1)



where

MSE (fq(X )) =1

nm

n∑i=0

m∑j=0




4/28

OutlineIntroduction



Formal Description


p(z) =1

πexp(−|z |2) (1)



where

MSE (fq(X )) =1

nm

n∑i=0

m∑j=0




4/28

OutlineIntroduction



Formal Description


p(z) =1

πexp(−|z |2) (1)



where

MSE (fq(X )) =1

nm

n∑i=0

m∑j=0




4/28

OutlineIntroduction



Formal Description


p(z) =1

πexp(−|z |2) (1)



where

MSE (fq(X )) =1

nm

n∑i=0

m∑j=0




4/28

OutlineIntroduction



Formal Description


p(z) =1

πexp(−|z |2) (1)



where

MSE (fq(X )) =1

nm

n∑i=0

m∑j=0


and where T is the ideal, non-noisy image.Troy Peterson Kling & Jeffrey Kidwell Algorithms for Reducing Noise in Synthetic Aperture Radar Images

5/28

OutlineIntroduction



Why CSC 380?

I Noise reduction has several real world applications, especiallyin the remote sensing and medical fields.

I We researched and implemented six different algorithms forsolving a complex problem.

I We analyzed the performance of each algorithm using Big-Onotation and run-time tests.

I We found the optimal image filter by comparing the rootmean squared error (RMSE) and peak signal-to-noise ratio(PSNR) statistics for each filter.


5/28

OutlineIntroduction



Why CSC 380?






5/28

OutlineIntroduction



Why CSC 380?






5/28

OutlineIntroduction



Why CSC 380?






6/28

OutlineIntroduction


Local FiltersNon-local Filters

Boxcar Filter

The Boxcar filter is a non-adaptive local filter. It reduces theoverall variation present in an image by setting each pixel’sintensity equal to the average of its neighborhood.

Let Xi be a pixel from the unfiltered image, and let Yi be thecorresponding pixel in the filtered image. Then the Boxcar filtercan be written as follows:

Yi = Box(Xi ) =1

|Wi |∑j∈Wi

Xj (4)

where Wi denotes the set of all pixels in pixel i ’s neighborhood.


6/28

OutlineIntroduction



Boxcar Filter



Yi = Box(Xi ) =1

|Wi |∑j∈Wi

Xj (4)



6/28

OutlineIntroduction



Boxcar Filter



Yi = Box(Xi ) =1

|Wi |∑j∈Wi

Xj (4)



6/28

OutlineIntroduction



Boxcar Filter



Yi = Box(Xi ) =1

|Wi |∑j∈Wi

Xj (4)



7/28

OutlineIntroduction



Boxcar Filter Results

Figure: Left: Original image. Center: Noise added programmatically.Right: Image after applying a 5x5 Boxcar filter.

RMSE: 10.444PSNR: 27.753


8/28

OutlineIntroduction



Gaussian Convolution

A Gaussian convolution filter is a signal processing technique thatconvolves a signal f with a Gaussian kernel. Mathematically, thisamounts to a Weierstrass transform

F (x) =1√4π

∫ ∞−∞

f (y)e−(x−y)2

4 dy (5)

Gaussian convolution can be expressed as a moving weightedaverage. The process of applying a Gaussian kernel KG of radius rto a single pixel, Xij , can be written as follows.

Yij =1∑KG

i+r∑p=i−r

j+r∑q=j−r

KG (p − r , q − r) · Xpq (6)


8/28

OutlineIntroduction





F (x) =1√4π

∫ ∞−∞

f (y)e−(x−y)2

4 dy (5)


Yij =1∑KG

i+r∑p=i−r

j+r∑q=j−r

KG (p − r , q − r) · Xpq (6)


8/28

OutlineIntroduction





F (x) =1√4π

∫ ∞−∞

f (y)e−(x−y)2

4 dy (5)


Yij =1∑KG

i+r∑p=i−r

j+r∑q=j−r

KG (p − r , q − r) · Xpq (6)


8/28

OutlineIntroduction





F (x) =1√4π

∫ ∞−∞

f (y)e−(x−y)2

4 dy (5)


Yij =1∑KG

i+r∑p=i−r

j+r∑q=j−r

KG (p − r , q − r) · Xpq (6)


9/28

OutlineIntroduction




The following is an example of a 5x5 Gaussian convolution filterused in the image processing domain.

Yij =1

159

2 4 5 4 24 9 12 9 45 12 15 12 54 9 12 9 42 4 5 4 2

∗W (7)

This process is repeated for each pixel in the image.


9/28

OutlineIntroduction





Yij =1

159

2 4 5 4 24 9 12 9 45 12 15 12 54 9 12 9 42 4 5 4 2

∗W (7)



9/28

OutlineIntroduction





Yij =1

159

2 4 5 4 24 9 12 9 45 12 15 12 54 9 12 9 42 4 5 4 2

∗W (7)



10/28

OutlineIntroduction



Gaussian Convolution Results

Figure: Left: Original image. Center: Noise added programmatically.Right: Image after applying a 5x5 Gaussian convolution filter, σ = 1.25.

RMSE: 9.506PSNR: 28.571


11/28

OutlineIntroduction



J. S. Lee’s Filter

The Lee filter is an adaptive local filter. Compared to the Boxcarfilter, the Lee filter performs better at preserving sharp edges andpoint scatterers. It can be written as follows.

Yi = Lee(Xi ) = Box(Xi ) + k(Xi − Box(Xi )) (8)

where k is an adaptive filtering coefficient defined by

k =Var(Y )

Var(W )=

Var(W )− E 2(W )σ2

Var(W )[1 + σ2](9)

where Var() is the variance operator, E () is the expectation, andσ2 is the a priori variance of the speckle noise.


11/28

OutlineIntroduction







k =Var(Y )

Var(W )=

Var(W )− E 2(W )σ2

Var(W )[1 + σ2](9)



11/28

OutlineIntroduction







k =Var(Y )

Var(W )=

Var(W )− E 2(W )σ2

Var(W )[1 + σ2](9)



11/28

OutlineIntroduction







k =Var(Y )

Var(W )=

Var(W )− E 2(W )σ2

Var(W )[1 + σ2](9)



11/28

OutlineIntroduction







k =Var(Y )

Var(W )=

Var(W )− E 2(W )σ2

Var(W )[1 + σ2](9)



12/28

OutlineIntroduction



J. S. Lee’s Filter Results

Figure: Left: Original image. Center: Noise added programmatically.Right: Image after applying a 5x5 Lee filter.

RMSE: 8.740PSNR: 29.300


13/28

OutlineIntroduction



Non-local Means

I Non-local filters considerall pixels in the entireimage.

I Similarity values aredetermined by comparingsmall windows.

I Using this knowledgeabout similarity, non-localfilters can more accuratelyestimate true pixelintensities.


13/28

OutlineIntroduction



Non-local Means





13/28

OutlineIntroduction



Non-local Means





13/28

OutlineIntroduction



Non-local Means





14/28

OutlineIntroduction



Non-local Means

In the non-local means paradigm, each pixel is assigned a newvalue equal to a weighted sum of all pixels in the image.

Yi = NL(Xi ) =∑j∈Wi

w(i , j)Xj (10)

where w(i , j) is a weight associated with each pair of pixels,calculated as follows.

w(i , j) =1

Z (i)e−

d2(i,j)

h2 (11)

where h is a tuning parameter, d2(i , j) is the Euclidean norm, andZ (i) is the normalizing constant

Z (i) =∑

je−

d2(i,j)

h2 (12)


14/28

OutlineIntroduction



Non-local Means



w(i , j)Xj (10)


w(i , j) =1

Z (i)e−

d2(i,j)

h2 (11)


Z (i) =∑

je−

d2(i,j)

h2 (12)


14/28

OutlineIntroduction



Non-local Means



w(i , j)Xj (10)


w(i , j) =1

Z (i)e−

d2(i,j)

h2 (11)


Z (i) =∑

je−

d2(i,j)

h2 (12)


14/28

OutlineIntroduction



Non-local Means



w(i , j)Xj (10)


w(i , j) =1

Z (i)e−

d2(i,j)

h2 (11)


Z (i) =∑

je−

d2(i,j)

h2 (12)


14/28

OutlineIntroduction



Non-local Means



w(i , j)Xj (10)


w(i , j) =1

Z (i)e−

d2(i,j)

h2 (11)


Z (i) =∑

je−

d2(i,j)

h2 (12)


14/28

OutlineIntroduction



Non-local Means



w(i , j)Xj (10)


w(i , j) =1

Z (i)e−

d2(i,j)

h2 (11)


Z (i) =∑

je−

d2(i,j)

h2 (12)


15/28

OutlineIntroduction



Non-Local Means Results

Figure: Left: Original image. Center: Noise added programmatically.Right: Image after applying a non-local means filters, using 5x5 similaritywindows, 35x35 search windows, and h = 13.5.

RMSE: 6.158PSNR: 32.342


16/28

OutlineIntroduction



Fourier Transform

Fourier transforms are used to transform a signal from the timedomain to the frequency domain.

f (x) =

∫ ∞−∞

F (k)e2πikxdk (13)

This transformation can be reverse with the inverse Fouriertransform

F (x) =

∫ ∞−∞

f (k)e−2πikxdk (14)

Noise reduction can be achieved by taking the Fast FourierTransform (FFT) of an image, filtering certain frequencies, andthen performing the inverse FFT.


16/28

OutlineIntroduction



Fourier Transform


f (x) =

∫ ∞−∞

F (k)e2πikxdk (13)


F (x) =

∫ ∞−∞




16/28

OutlineIntroduction



Fourier Transform


f (x) =

∫ ∞−∞

F (k)e2πikxdk (13)


F (x) =

∫ ∞−∞




16/28

OutlineIntroduction



Fourier Transform


f (x) =

∫ ∞−∞

F (k)e2πikxdk (13)


F (x) =

∫ ∞−∞




16/28

OutlineIntroduction



Fourier Transform


f (x) =

∫ ∞−∞

F (k)e2πikxdk (13)


F (x) =

∫ ∞−∞




17/28

OutlineIntroduction



Fourier Transform Results

Figure: Left: Original image. Center: Noise added programmatically.Right: Image after applying a band-pass Fourier transform.

RMSE: 10.197PSNR: 27.961


18/28

OutlineIntroduction



ART-2

I The Adapative ResonanceTheory (ART-2) neuralnetwork is specifically forpattern recognition.

I ART-2 is able tocategorize continuousinput patterns intodifferent groups based onsimilarity.

I F1 layer acts as short-termmemory, F2 as long-term.

I Black circles indicatenormalization.


18/28

OutlineIntroduction



ART-2






18/28

OutlineIntroduction



ART-2






18/28

OutlineIntroduction



ART-2






19/28

OutlineIntroduction



ART-2

Short-term memory equations

pi = ui +∑j

g(yj)zji qi =pi

e + ||pi ||

ui =vi

e + ||vi ||vi = f (xi ) + bf (qi )

wi = Ii + aui xi =wi

e + ||wi ||

Filter function

f (x) =

{0 if 0 ≤ x < θx if x ≥ θ (15)

And lots more!


19/28

OutlineIntroduction



ART-2


pi = ui +∑j

g(yj)zji qi =pi

e + ||pi ||

ui =vi

e + ||vi ||vi = f (xi ) + bf (qi )


e + ||wi ||

Filter function

f (x) =

{0 if 0 ≤ x < θx if x ≥ θ (15)

And lots more!


19/28

OutlineIntroduction



ART-2


pi = ui +∑j

g(yj)zji qi =pi

e + ||pi ||

ui =vi

e + ||vi ||vi = f (xi ) + bf (qi )


e + ||wi ||

Filter function

f (x) =

{0 if 0 ≤ x < θx if x ≥ θ (15)

And lots more!


19/28

OutlineIntroduction



ART-2


pi = ui +∑j

g(yj)zji qi =pi

e + ||pi ||

ui =vi

e + ||vi ||vi = f (xi ) + bf (qi )


e + ||wi ||

Filter function

f (x) =

{0 if 0 ≤ x < θx if x ≥ θ (15)

And lots more!


19/28

OutlineIntroduction



ART-2


pi = ui +∑j

g(yj)zji qi =pi

e + ||pi ||

ui =vi

e + ||vi ||vi = f (xi ) + bf (qi )


e + ||wi ||

Filter function

f (x) =

{0 if 0 ≤ x < θx if x ≥ θ (15)

And lots more!


20/28

OutlineIntroduction



Modified ART-2

I ART-2 is capable of performing basic noise reduction.

I We are in the process of modifying it so that it has similarproperties to the non-local means filter.

I The modified ART-2 neural network takes a sequence ofoverlapping windows as input patterns.

I Performing categorization on these input patterns tells uswhich windows are most similar.

I From there, principles from non-local means can be applied toreduce noise in the image.

I No results yet.


20/28

OutlineIntroduction



Modified ART-2






I No results yet.


20/28

OutlineIntroduction



Modified ART-2






I No results yet.


20/28

OutlineIntroduction



Modified ART-2






I No results yet.


20/28

OutlineIntroduction



Modified ART-2






I No results yet.


20/28

OutlineIntroduction



Modified ART-2






I No results yet.


21/28

OutlineIntroduction


The NumbersResults for SAR ImagesFuture WorkQuestions

Final Remarks

I Non-local meansoutperforms all otherfilters by a sizeablemargin.

I Non-local means isparticularly well-suited forSAR images.

I We expect our modifiedART-2 algorithm toperform better than Lee’sfilter.

Filter RMSE PSNR Big-O

NL-M 6.16 32.34 O(n2w 2k2)Lee’s 8.74 29.3 O(n2k2)Conv. 9.51 28.57 O(n2k2)Fourier 10.20 27.96 O(n2 log(n))Boxcar 10.44 27.75 O(n2k2)ART-2 N/A N/A O(?)

Note: Assuming the image is square.


21/28

OutlineIntroduction



Final Remarks








21/28

OutlineIntroduction



Final Remarks








22/28

OutlineIntroduction



Results for SAR Images

I So far we have seen imagefilters applied to standardphotographs.

I How well do these samefilters work on syntheticaperture radar images?

I Since we don’t know whatthe “true” de-noisedimage looks like, we haveto judge it by eye.


22/28

OutlineIntroduction








22/28

OutlineIntroduction








23/28

OutlineIntroduction




Figure: Top row: Original SAR image; Boxcar; Gaussian convolution.Bottom row: Lee’s filter; non-local means; FFT filter.


24/28

OutlineIntroduction




Figure: Top row: Original SAR image; Boxcar; Gaussian convolution.Bottom row: Lee’s filter; non-local means; FFT filter.


25/28

OutlineIntroduction



Future Work

I Finish implementing ART-2.

I Try wavelet transforms instead of Fourier transforms.

I Make filters work for polarimetric and/or interferometric SARimages.


25/28

OutlineIntroduction



Future Work





25/28

OutlineIntroduction



Future Work





26/28

OutlineIntroduction



Question #1

1. What is the difference between local and non-local filters, andwhy do non-locals filters perform better in most situations?

Answer: Local filters estimate each pixel’s true intensity bylooking at the pixels immediately surrounding it. Non-localfilters estimate each pixel’s true intensity by looking at everypixel in the entire image and performing a weighted averagebased on the degree of similarity between windows.


26/28

OutlineIntroduction



Question #1

1. What is the difference between local and non-local filters, andwhy do non-locals filters perform better in most situations?

Answer: Local filters estimate each pixel’s true intensity bylooking at the pixels immediately surrounding it. Non-localfilters estimate each pixel’s true intensity by looking at everypixel in the entire image and performing a weighted averagebased on the degree of similarity between windows.


27/28

OutlineIntroduction



Question #2

2. The non-local means filter gave the lowest RMSE out of allthe filters we tried, but it’s not the best in every respect.What is one major drawback of the non-local meansalgorithm? (Hint: Think Big-O notation)

Answer: The non-local means algorithm has complexityO(n2 · w2 · k2), where n is the width of the image (assumingit is square), w is the width of the search window, and k isthe width of similarity window. For even modestly large valuesof n, w , and k , non-local means takes a very long time tofinish running.


27/28

OutlineIntroduction



Question #2

2. The non-local means filter gave the lowest RMSE out of allthe filters we tried, but it’s not the best in every respect.What is one major drawback of the non-local meansalgorithm? (Hint: Think Big-O notation)

Answer: The non-local means algorithm has complexityO(n2 · w2 · k2), where n is the width of the image (assumingit is square), w is the width of the search window, and k isthe width of similarity window. For even modestly large valuesof n, w , and k , non-local means takes a very long time tofinish running.


28/28

OutlineIntroduction



Question #3

3. For the Lee filter, how does the adaptive filtering coefficient,k, affect the filtering procedure?

Answer: Homogeneous regions of the image produce k valuesclose to zero, which causes the Lee filter to act more like aBoxcar filter. Heterogeneous regions of the image thatcontain a lot of variation produce k values close to one, whichsets the filtered pixel value equal to the unfiltered pixel value.


28/28

OutlineIntroduction



Question #3

3. For the Lee filter, how does the adaptive filtering coefficient,k, affect the filtering procedure?

Answer: Homogeneous regions of the image produce k valuesclose to zero, which causes the Lee filter to act more like aBoxcar filter. Heterogeneous regions of the image thatcontain a lot of variation produce k values close to one, whichsets the filtered pixel value equal to the unfiltered pixel value.


29/28

OutlineIntroduction



Question #4

4. What was the ART-2 system originally designed for, and howcan it be applied to noise reduction?

Answer: The ART-2 neural network was originally designedfor categorization/classification of signal patterns. ART-2 canbe applied to noise reduction in several ways - one of which isby categorizing overlapping windows in the image and thenapplying a filter function for each category.


29/28

OutlineIntroduction



Question #4

4. What was the ART-2 system originally designed for, and howcan it be applied to noise reduction?

Answer: The ART-2 neural network was originally designedfor categorization/classification of signal patterns. ART-2 canbe applied to noise reduction in several ways - one of which isby categorizing overlapping windows in the image and thenapplying a filter function for each category.


Documents

Algorithms for Reducing Noise in Synthetic Aperture Radar ...people.uncw.edu/tagliarinig/Courses/380/S2015...Troy Peterson Kling & Je rey Kidwell Algorithms for Reducing Noise in Synthetic