Upload
others
View
27
Download
0
Embed Size (px)
Citation preview
!"#$"#%&
#&
Image Enhancement in the Spatial Domain
Jesus J. Caban
Outline
1. Final Project / Paper Presentation
2. Questions about homework / OpenCV?
3. Review topics from last class
4. Finish talking about spatial image enhancement
5. Start Fourier Analysis
!"#$"#%&
'&
Paper Presentation ! Presenters:
! Prepare a 30 – 35 mins talk about your topic ! Try to “connect” the two papers ! If needed, read other papers ! Undergraduates: 20 – 30mins ! Evaluation Form
! Rest of the class: ! Read each of the papers ! Prepare and submit a thoughtful question about the
papers
Graduate students will be given 10 additional minutes (30 – 35 mins in total) for their paper presentation. The additional time should be used to better cover some of the previous work as well as explain the formulation of the paper.
Recall:
Homework / OpenCV
1. Download and install cmake from cmake.org (default location)
2. Download OpenCV
3. unzip OpenCV-2.1.0-win.zip && cd OpenCV-2.1.0
4. cmake .
5. Make
6. sudo make install
7. Add theses two lines to my .bash_profile
export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python2.6/site-packages
export DYLD_LIBRARY_PATH=/usr/local/lib OR
export LD_LIBRARY_PATH=/usr/local/lib
8. Open a new terminal
9. cd OpenCV-2.1.0/samples/python/
10. python edge.py
!"#$"#%&
(&
OpenCV ##Load / Create
src_image = cv.LoadImage(“image.jpg”, cv.CV_LOAD_IMAGE_GRAYSCALE) editted_image = cv.CreateImage(cv.GetSize(src_image), 8, 1);
##Processing / Filter cv.Smooth(src_image, editted_image,cv.CV_BLUR, 3,3,0)
#cv.Laplace(src_image, float_image, 3)
#cv.Merge(src_image, src_image, src_image, None, rgb_image)
##Show / Save
cv.SaveImage(“out.jpg”, editted_image) cv.NamedWindow("Result", 0)
cv.ShowImage( "Result", image) cv.WaitKey(0)
Make sure you understand the following examples: �
• Fitellipse.py�• Edge.py�• Contours.py�• Cv20squares.py�
Final Project ! Draft Proposal (10%)
! Oct 4th
! Revised Proposal (10%) ! Oct 25th
! Literature survey (20%) ! Nov 3rd
! Final paper (40%) ! Dec 6th
! Final presentation (20%)
• When: Draft due Oct 4th
• What: One to two pages: 1. Abstract 2. Motivation 3. Plan: What you plan to do 4. How: Resources & APIs you plan to use 5. Timeline
!"#$"#%&
)&
Need to talk to other students?
! VANGOGH Lab ! Dr. Penny Rheingans and Dr. Marc Olano
! Lab meetings: ! Wednesday, 1pm
! ITE 365
Outline
1. Final Project / Paper Presentation
2. Questions about homework / OpenCV?
3. Review topics from last class
4. Finish talking about spatial image enhancement
5. Start Fourier Analysis
!"#$"#%&
$&
Point Processing ! Based on the intensity of a single pixel only as opposed to a
neighborhood or region
Histogram Equalization
!"#$"#%&
*&
Histogram Equalization 1) Probability of Occurrence
2) Create Cumulative Distribution Function
3) Map old value to new value
Histogram Equalization
!"#$"#%&
+&
Histogram Matching
! The histogram processing methods discussed previously are global, in the sense that pixels are modified by a transformation function based on the gray-level content of an entire image.
! However, there are cases in which it is necessary to enhance details over small areas in an image.
original
Local Enhancement
global Local (7x7)
!"#$"#%&
,&
! Moments can be determined directly from a histogram much faster than they can from the pixels directly.
! Let r denote a discrete random variable representing discrete gray-levels in the range [0,L-1], and p(ri) denote the normalized histogram component corresponding to the ith value of r, then the nth moment of r about its mean is defined as
where m is the mean value of r
! For example, the second moment (also the variance of r) is
Use of Histogram Statistics for Image Enhancement
! Two uses of the mean and variance for enhancement purposes: ! The global mean and variance (global means for the entire image) are
useful for adjusting overall contrast and intensity.
! The mean and standard deviation for a local region are useful for correcting for large-scale changes in intensity and contrast
Use of Histogram Statistics for Image Enhancement
!"#$"#%&
!&
Enhancement based on local statistics
Image Operations
! Just as with numbers, we can apply different operations to an image ! Addition: + ! Subtraction: - ! Multiplication: * ! And: && ! Or: || ! Derivative:
! Average ! Mean
!
dxdy
!"#$"#%&
#%&
! In general, linear filtering of an image f of size MxN is given by
! This concept called convolution. Filter masks are sometimes called convolution masks or convolution kernels.
Basics of Spatial Filtering
Smoothing
1/9 1/9 1/9
1/9 1/9
1/9
1/9 1/9
1/9
1/16 2/16 1/16
2/16 4/16
2/16
1/16 2/16
1/16
!"#$"#%&
##&
1st Order Derivative The formula for the 1st derivative of a function is as follows:
It’s just the difference between subsequent values and measures the rate of change of the function
2nd Order Derivative
The formula for the 2nd derivative of a function is as follows:
Simply takes into account the values both before and after the current value
!"#$"#%&
#'&
The Laplacian So, the Laplacian can be given as follows:
We can easily build a filter based on this
0 1 0
1 -4 1
0 1 0
Laplacian Image Enhancement
!"#$"#%&
#(&
Variants On The Simple Laplacian There are lots of slightly different versions of the Laplacian that can be used:
0 1 0
1 -4 1
0 1 0
1 1 1
1 -8 1
1 1 1
-1 -1 -1
-1 9 -1
-1 -1 -1
Simple Laplacian
Variant of Laplacian
1st Order Derivative
5 5 4 3 2 1 0 0 0 6 0 0 0 0 1 3 1 0 0 0 0 7 7 7 7
0 -1 -1 -1 -1 0 0 6 -6 0 0 0 1 2 -2 -1 0 0 0 7 0 0 0
!"#$"#%&
#)&
1st Derivative Filtering – The gradient Implementing 1st derivative filters is difficult in practice
For a function f(x, y) the gradient of f at coordinates (x, y) is given as the column vector:
1st Derivative Filtering The magnitude of this vector is given by:
For practical reasons this can be simplified as:
!
= Gx2 +Gy
2[ ]12
!
="f"x#
$ %
&
' ( 2
+"f"y#
$ %
&
' (
2)
* + +
,
- . .
12
!"#$"#%&
#$&
Gradient Estimation
1. Create orthogonal pair of filters,
!
h1(x,y)
!
h2(x,y)
!
"f (x,y) = f12(x,y) + f2
2(x,y)
3. Estimate the gradient magnitude:
2. Convolve image with each filter:
!
f1(x,y) = f (x,y)"h1(x,y)
!
f2(x,y) = f (x,y)"h2(x,y)
1st Derivative Filtering There is some debate as to how best to calculate these gradients:
which is based on these coordinates
z1 z2 z3
z4 z5 z6
z7 z8 z9
!"#$"#%&
#*&
Roberts Operator
! Small kernel, relatively little computation ! First difference (diagonally) ! Very sensitive to noise ! Origin not at kernel center ! Somewhat anisotropic
!
h2(x,y) =1 00 "1#
$ %
&
' (
!
h1(x,y) =0 1"1 0#
$ %
&
' (
Noise
! Noise is always a factor in images. ! Derivative operators are high-pass filters. ! High-pass filters boost noise!
! Effects of noise on edge detection: ! False edges ! Errors in edge position
Key concept: Build filters to respond to edges and suppress noise.
!"#$"#%&
#+&
Prewitt Operator
! Larger kernel, somewhat more computation ! Central difference, origin at center ! Smooths (averages) along edge, less sensitive to noise ! Somewhat anisotropic
• 3 x 3 kernel, same computation as Prewitt • Central difference, origin at center • Better smoothing along edge, even less sensitive to noise • Still somewhat anisotropic
Sobel Operator
-1 -2 -1
0 0 0
1 2 1
-1 0 1
-2 0 2
-1 0 1
!"#$"#%&
#,&
Sobel Example
Sobel filters are typically used for edge detection
An image of a contact lens which is enhanced in order to make defects (at four and five o’clock in the image) more obvious
Roberts cross-gradient operators
Sobel operators
1st Derivative Filtering
!"#$"#%&
#!&
Example
Original Roberts
Roberts Prewitt
!"#$"#%&
'%&
Prewitt Sobel
1st & 2nd Derivatives Comparing the 1st and 2nd derivatives we can conclude the following:
! 1st order derivatives generally produce thicker edges ! 2nd order derivatives have a stronger response to fine detail e.g.
thin lines ! 1st order derivatives have stronger response to grey level step ! 2nd order derivatives produce a double response at step changes
in grey level
!"#$"#%&
'#&
Effect of Noise
Where is the edge?
Multi-Filter Processing ! LoG: (Laplacian of Gaussian)
! Because second derivative measurements on an image are very sensitive to noise. Images are often Gaussian smoothed before applying the Laplacian filter.
! This pre-processing step reduces the high frequency noise components prior to the differentiation step.
! DoG: (Difference of Gaussians) ! Given f(x,y), compute f’(x,y) using a Gaussian
blur ! Subtract images ! Threshold image
!"#$"#%&
''&
Canny edge detector
• Based on the theoretical model that edges are corrupted by additive Gaussian noise
• The first derivative of the Gaussian closely approximates the operator that optimizes the product of signal-to-noise ratio and localization (1986)
Canny edge detector
1. Filter image with derivative of Gaussian
2. Find magnitude and orientation of gradient
(Round orientations: 0o, 45o, 90o, 135o)
3. Non-maximum suppression: ! Thin multi-pixel wide “ridges” down to single pixel width
!"#$"#%&
'(&
Non-maximum suppression At q, we have a maximum if the value is larger than those at both p and at r. Interpolate to get these values.
Non-maximum suppression
! A(0o): edge if I(x,y) > I(x+1,y) && I(x-1,y)
! A(90o): edge if I(x,y) > I(x,y+1) && I(x,y-1)
! A(135o): edge if I(x,y) > I(x-1,y-1) && I(x+1,y+1)
! A(45o): edge if I(x,y) > I(x+1, y-1) && I(x-1, y+1)
!"#$"#%&
')&
Canny edge detector
1. Filter image with derivative of Gaussian
2. Find magnitude and orientation of gradient
(Round orientations: 0o, 45o, 90o, 135o)
3. Non-maximum suppression: ! Thin multi-pixel wide “ridges” down to single pixel width
4. Linking of edge points
1. Assume the marked point is an edge point.
2. Construct the tangent to the edge curve (which is normal to the gradient at that point)
3. Use tangent to predict the next points
Edge linking
!"#$"#%&
'$&
Example
Gradient magnitude
Canny
!"#$"#%&
'*&
Summary: Image Enhancement – Spatial Domain
! Point Processing ! Contrast
! Operations ! Convolution ! Derivatives ! Edge detection
Image Enhancement: Frequency Domain
!"#$"#%&
'+&
! The spatial domain: ! The image plane ! For a digital image is a Cartesian coordinate system of discrete rows and columns.
At the intersection of each row and column is a pixel. Each pixel has a value, which we will call intensity.
! The frequency domain : ! A (2-dimensional) discrete Fourier transform of the spatial domain
! Enhancement : ! To “improve” the usefulness of an image by using some transformation on the
image. ! Often the improvement is to help make the image “better” looking, such as
increasing the intensity or contrast.
Recall: Image Enhancement - Spatial Domain
Jean Baptiste Joseph Fourier
! Fourier was born in Auxerre, France in 1768
! Published his work in 1822: “La Théorie Analitique de la Chaleur”
! His work was translated into English over 55+ years later (1878): “The Analytic Theory of Heat”
! Researchers didn’t pay too much attention to his work when it was first published
! Fourier Theory then became one of the most important mathematical theories in modern engineering
!"#$"#%&
',&
The General Idea
=
• Any function that periodically repeats itself can be expressed as a sum of sines and cosines of different frequencies each multiplied by a different coefficient • Resulting collection is called Fourier series
Fourier Series - Example
First 60 Fourier approximations
!"#$"#%&
'!&
Fourier Transform and Image Frequency
! Determines frequency components that fit evenly across an image of input size L
! The highest frequency that can be contained in a signal of length L is L/2
! At least two pixels are required to fully represent a cycle
Fourier Transform: Definition
! ! Changes in u and v obtain different basis functions of different periods
(frequencies) ! Therefore, F(f(x,y)) is really a weighted sum of complex sinusoids of
varying frequency ! Point in the new image FT(u,v) represent the contribution of that
frequency to the original image
Sum of sine and cosine basis functions
!"#$"#%&
(%&
The Discrete Fourier Transform (DFT) 1. Given an image f(x,y) or size MxN
x = 0, 1, 2…M-1 y = 0,1,2…N-1
2. Discrete Fourier Transform of f(x, y) is given by
u = 0, 1, 2…M-1 v = 0, 1, 2…N-1 i= sqrt(-1)
N
M
Origin
Acknowledgements ! Some of the images and diagrams have been taken from the
Gonzalez et al, “Digital Image Processing” book.