Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Digital Image ProcessingDigital Image ProcessingDigital Image ProcessingChapter 11:
Digital Image ProcessingChapter 11:p
Image Description andRepresentation
pImage Description and
RepresentationRepresentationRepresentation
Image Representation and Description? Image Representation and Description? Objective:Objective:
To represent and describe information embedded inan image in other forms that are more suitable than thean image in other forms that are more suitable than the image itself.
fiBenefits:- Easier to understand
R i f f t t b d- Require fewer memory, faster to be processed- More “ready to be used”
What kind of information we can use?- Boundary, shapey, p- Region- Texture- Relation between regions
Shape Representation by Using Chain Codes Shape Representation by Using Chain Codes Why we focus on a boundary?Why we focus on a boundary?The boundary is a good representation of an object shapeand also requires a few memoryChain codes: represent an object boundary by a connected sequence of straight line segments of specified length
and also requires a few memory.
sequence of straight line segments of specified lengthand direction.
4 di i l4-directionalchain code
8-directionalchain code
(Images from Rafael C.Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Examples of Chain Codes Examples of Chain Codes
Object BoundaryObjecboundary
(resampling)
Boundaryvertices
4-directional 8-directionald ec ochain code
8 d ec ochain code
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
The First Difference of a Chain Codes The First Difference of a Chain Codes Problem of a chain code:Problem of a chain code:
a chain code sequence depends on a starting point.Solution: treat a chain code as a circular sequence and redefine thestarting point so that the resulting sequence of numbers forms an integer of minimum magnitude.
The first difference of a chain code: counting the number of directionchange (in counterclockwise) between 2 adjacent elements of the code.
Example:1
Example: - a chain code: 10103322
Chain code : The first difference
2 0- The first difference = 3133030- Treating a chain code as a
circular sequence we get
0 1 10 2 20 3 3
3circular sequence, we getthe first difference = 33133030
0 3 32 3 12 0 22 1 3 The first difference is rotational
invariant.
Polygon Approximation Polygon Approximation
Represent an object boundary by a polygon
Object boundary Minimum perimeterpolygonpolygon
Minimum perimeter polygon consists of line segments thatMinimum perimeter polygon consists of line segments that minimize distances between boundary pixels.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Polygon Approximation:Splitting Techniques Polygon Approximation:Splitting Techniques
1. Find the line joiningtwo extreme points
0 Object boundary0. Object boundary 2. Find the farthest pointsfrom the linefrom the line
3. Draw a polygon(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
DistanceDistance--VersusVersus--Angle Signatures Angle Signatures Represent an 2-D object boundary in term of a 1-D functionRepresent an 2 D object boundary in term of a 1 D function of radial distance with respect to .
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Boundary Segments Boundary Segments Concept: Partitioning an object boundary by using verticesConcept: Partitioning an object boundary by using vertices of a convex hull.
Partitioned boundary
Convex hull (gray color)
Object boundary
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Input : A set of points on a cornea boundary
Convex Hull Algorithm Convex Hull Algorithm Input : A set of points on a cornea boundaryOutput: A set of points on a boundary of a convex hull of a cornea1 Sort the points by x-coordinate to get a sequence p p … p1. Sort the points by x-coordinate to get a sequence p1, p2, … ,pnFor the upper side of a convex hull2. Put the points p and p in a list L with p as the first point2. Put the points p1 and p2 in a list Lupper with p1 as the first point3. For i = 3 to n4. Do append pi to L4. Do append pi to Lupper5. While Lupper contains more than 2 points and the last 3
points in L do not make a right turnp upper g6. Do delete the middle point of the last 3 points from Lupper
TurnRight
TurnRight
TurnLeftNOK!Right
OK! OK!
For the lower side of a convex hullConvex Hull Algorithm (cont.) Convex Hull Algorithm (cont.)
7. Put the points pn and pn-1 in a list Llower with pn as the first point8. For i = n-2 down to 18. For i n 2 down to 19. Do append pi to Llower10. While L contains more than 2 points and the last 3 points10. While Llower contains more than 2 points and the last 3 points
in Llower do not make a right turn11. Do delete the middle point of the last 3 points from L11. Do delete the middle point of the last 3 points from Llower
12. Remove the first and the last points from Llower
13 Append L to L resulting in the list L13. Append Llower to Lupper resulting in the list L14. Return L
TurnTurnRight
TurnRightOK!
TurnOK!
TurnLeftNOK!
Skeletons Skeletons Obtained from thinning or skeletonizing processesObtained from thinning or skeletonizing processes
Medial axes (dash lines)Medial axes (dash lines)
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Thinning Algorithm Thinning Algorithm Concept: 1. Do not remove end pointsp p
2. Do not break connectivity3. Do not cause excessive erosion
Apply only to contour pixels: pixels “1” having at least one of its 8neighbor pixels valued “0”
iNeighborhoodarrangement
p2p9 p3
Notation:
arrangementfor the thinningalgorithm
p8
p7
p1 p4
p6 p5
=Let
E lgp7 p6 p5
Let 98321)( pppppN 00 1
Example
T(p1) = the number of transition 0-1 in the ordered sequence p2, p3, …
11
p1 00 1
, p8, p9, p2. N(p1) = 4T(p1) = 3
Thinning Algorithm (cont.) Thinning Algorithm (cont.)
Step 1. Mark pixels for deletion if the following conditions are true.a) b) T( ) 1
6)(2 1 pNp2p9 p3(Apply to all border pixels)b) T(p1) =1
c)d)
0642 ppp0ppp
p2p9 p3
p8
p7
p1 p4
p6 p5
(Apply to all border pixels)
d) 0864 ppp p7 p6 p5
Step 2. Delete marked pixels and go to Step 3.
Step 3. Mark pixels for deletion if the following conditions are true.a) 6)(2 1 pN (Apply to all border pixels)b) T(p1) =1c) 0842 ppp
0
(Apply to all border pixels)
d) 0862 ppp
Step 4. Delete marked pixels and repeat Step 1 until no changep p p p goccurs.
Example: Skeletons Obtained from the Thinning Alg. Example: Skeletons Obtained from the Thinning Alg.
Skeleton
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Boundary Descriptors Boundary Descriptors 1 Simple boundary descriptors:1. Simple boundary descriptors:we can use
Length of the boundary- Length of the boundary- The size of smallest circle or box that can totallyenclosing the objectenclosing the object
2 Shape number2. Shape number
3 Fourier descriptor3. Fourier descriptor
4 Statistical moments4. Statistical moments
Shape Number Shape Number Shape number of the boundary definition: 1Shape number of the boundary definition:
the first difference of smallest magnitudeThe order n of the shape number: 2 0p
the number of digits in the sequence3
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Shape Number (cont.) Shape Number (cont.)
Shape numbers of order S pe u be s o o de4, 6 and 8
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Example: Shape Number Example: Shape Number
2. Find the smallest rectanglethat fits the shape
1. Original boundarythat fits the shape
Chain code: 0 0 0 0 3 0 0 3 2 2 3 2 2 2 1 2 1 1
First difference:First difference:3 0 0 0 3 1 0 3 3 0 1 3 0 0 3 1 3 0
Shape No4. Find the nearest Shape No.0 0 0 3 1 0 3 3 0 1 3 0 0 3 1 3 0 33. Create grid
4. Find the nearest Grid.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Fourier Descriptor Fourier Descriptor Fourier descriptor: view a coordinate (x,y) as a complex number p ( y) p(x = real part and y = imaginary part) then apply the Fourier transform to a sequence of boundary points.
)()()( kjykxks Let s(k) be a coordinate of a boundary point k :
1
/2)(1)(K
KukeksK
ua Fourier descriptor : 0
)()(kK
p
Reconstruction formula
1
/2)(1)(K
Kukeuaks
Reconstruction formula
0
)()(k
euaK
ks
d(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Boundarypoints
Example: Fourier Descriptor Example: Fourier Descriptor Examples of reconstruction from Fourier descriptorsExamples of reconstruction from Fourier descriptors
1
/2)(1)(ˆP
Kukeuaks
0
)()(k
euaK
ks
P is the number ofP is the number of Fourier coefficients used to reconstruct th b dthe boundary
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Fourier Descriptor Properties Fourier Descriptor Properties
Some properties of Fourier descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Statistical Moments Statistical Moments Definition: the nth moment Example of moment:
1
)()()(K
in
in rgmrr
Definition: the n moment Example of moment:The first moment = meanThe second moment = variance
0i
where
1
)(K
ii rgrm 0
)(i
ii g
Boundarysegment 1D graph
1. Convert a boundary segment into 1D graph2 View a 1D graph as a PDF function
(Images from Rafael C.Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
2. View a 1D graph as a PDF function3. Compute the nth order moment of the graph
Regional Descriptors Regional Descriptors Purpose: to describe regions or “areas”Purpose: to describe regions or areas
1. Some simple regional descriptorsarea of the region- area of the region
- length of the boundary (perimeter) of the region- Compactness )(RAp
)()(
2 RPRAC
where A(R) and P(R) = area and perimeter of region RExample: a circle is the most compact shape with C = 1/4
2 T l i l D i t2. Topological Descriptors3. Texture4 Moments of 2D Functions4. Moments of 2D Functions
Example: Regional Descriptors Example: Regional Descriptors
White pixels represent “light of the cities”
% f hit i l% of white pixelsRegion no. compared to the
total white pixels
1 20.4%2 64.0%3 4.9%4 10.7%
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.Infrared image of America at night
Topological Descriptors Topological Descriptors Use to describe holes and connected components of the regionUse to describe holes and connected components of the region
Euler number (E):
HCE
C = the number of connected components
H = the number of holes
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Topological Descriptors (cont.) Topological Descriptors (cont.)
E = -1
E = 0 Euler Formula
EHCFQV
V = the number of verticesh b f dQ = the number of edges
F = the number of faces
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
E = -2
Example: Topological Descriptors Example: Topological Descriptors
Original image:Infrared imageOf Washington
After intensityThresholdingg
D.C. area (1591 connectedcomponents with 39 holes)Euler no. = 1552
The largestconnected After thinning
area (8479 Pixels)
(Hudson river)(Hudson river)
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Texture Descriptors Texture Descriptors Purpose: to describe “texture” of the region.p gExamples: optical microscope images:
B C
A
B C
Superconductor Cholesterol Microprocessor
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
p(smooth texture) (coarse texture)
p(regular texture)
Statistical Approaches for Texture Descriptors Statistical Approaches for Texture Descriptors We can use statistical moments computed from an image histogram:We can use statistical moments computed from an image histogram:
1
)()()(K
in
in zpmzzz = intensityp(z) = PDF or histogram of z
0i
where
1
)(K
ii zpzm
p(z) PDF or histogram of z
0
)(i
ii p
Example: The 2nd moment = variance measure “smoothness”Example: The 2 moment variance measure smoothnessThe 3rd moment measure “skewness”The 4th moment measure “uniformity” (flatness)
AB
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
C
Fourier Approach for Texture Descriptor Fourier Approach for Texture Descriptor
C t t 2D t i t 1D h
F i
Concept: convert 2D spectrum into 1D graphs
Divide into areasby angles
Originalimage
Fouriercoefficient
imageFFT2D
+FFTSHIFT image
S ll i lDivide into areas Sum all pixelsin each area
Divide into areasby radius
0
)()(R
rSS 1r
r
Sum all pixelsin each area
0
)()( rSrS
Fourier Approach for Texture Descriptor Fourier Approach for Texture Descriptor
Originalimage
2D Spectrum(Fourier Tr.)( )
S(r) S()S( ) S()
Anotherimage
Another S()
(Images from Rafael C.Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Moments of TwoMoments of Two--D Functions D Functions The moment of order p + q
x y
qppq yxfyxm ),(
00
10
mmx
00
01
mmy
The moment of order p + q
00 00
The central moments of order p + q
x y
qppq yxfyyxx ),()()(
0000 m 01001
1011011111 mymmxm
102020 mxm 010202 mym 102020 mxm 010202 mym
012
20112121 22 mxmymxm 102
203030 23 mxmxm
102
02111212 22 mymxmym 012
020303 23 mymym
Invariant Moments of TwoInvariant Moments of Two--D Functions D Functions
Th li d t l t f d +
pq where 1
qp
The normalized central moments of order p + q
00pq where 1
2
Invariant moments: independent of rotation translation scalingInvariant moments: independent of rotation, translation, scaling,and reflection
22 4 02201 1102202 4
203212
12303 33 203212
12304
22
20321
21230123012305
33
33
20321
2123003210321 33
20321
2123002206
0321123011
0321123002206
4
Example: Invariant Moments of TwoExample: Invariant Moments of Two--D Functions D Functions
3 Mi d1. Original image 2. Half size 3. Mirrored
4. Rotated 2 degree 5. Rotated 45 degree
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Example: Invariant Moments of TwoExample: Invariant Moments of Two--D Functions D Functions
Invariant moments of images in the previous slide
Invariant moments are independent of rotation, translation,scaling, and reflection
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Principal Components for Description Principal Components for Description Purpose: to reduce dimensionality of a vector image while maintainingPurpose: to reduce dimensionality of a vector image while maintaining
information as much as possible.
Let Tnxxx ]...[ 21x
Mean: K
kx KE 1}{ xxm
kK 1
C i t i K
TTTE 1}))({(CCovariance matrix
k
Txx
Tkk
Txxx K
E1
1}))({( mmxxmxmxC
Hotelling transformation Hotelling transformation
)(Let )( xmxAy
Where A is created from eigenvectors of C as followsWhere A is created from eigenvectors of Cx as followsRow 1 contain the 1st eigenvector with the largest eigenvalue.Row 2 contain the 2nd eigenvector with the 2nd largest eigenvalue.
00Then we get
….
1
1
0...00...0
C0}{ ym Ey
and
1......0............
yCT
xy AACC and
Then elements of are uncorrelated. The component of y with the largest is called the principal
)( xmxAy component of y with the largest is called the principal component.
Eigenvector and Eigenvalue Eigenvector and Eigenvalue
Eigenvector and eigenvalue of Matrix C are defined as
Let C be a matrix of size NxN and e be a vector of size Nx1Let C be a matrix of size NxN and e be a vector of size Nx1.If
eCe eCe
Where is a constantWhere is a constant
We call e as an eigenvector and as eigenvalue of CWe call e as an eigenvector and as eigenvalue of C
Example: Principal ComponentsExample: Principal Components
6 spectral imagesfrom an airborneScanner.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Example: Principal Components (cont.)Example: Principal Components (cont.)
Component 1 32102 931.43 118.54 83 884 83.885 64.006 13 40
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
6 13.40
Example: Principal Components (cont.)Example: Principal Components (cont.)
Original image After Hotelling transform
Principal Components for Description Principal Components for Description
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Relational Descriptors Relational Descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Relational Descriptors Relational Descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Relational Descriptors Relational Descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Relational Descriptors Relational Descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Relational Descriptors Relational Descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Structural Approach for Texture Descriptor Structural Approach for Texture Descriptor
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.