Robust Image Topological Feature Extraction
Kārlis Freivalds, Paulis Ķikusts
Theory Days at Jõulumäe
October 2008
University of Latvia
Image Analysis
Image analysis steps Pixels features Features objects Objects knowledge Knowledge decision
Features Edge Ridge Valley (ravine) Corner
Topological Features
Image is treated as 3D surface Ridge Valley Junction Peak Pit Saddle (Edge)
Ridges in Images
Ridge Image Examples
Text Line drawings
Natural objects Industrial applications
Mathematics
Image – smooth function f(x, y)
Derivatives
Directional derivative in direction p = (px, py)
Second derivative in two directions (unit vectors) p and q
xy
ff
x
ff xyx
2
,
2222 pypx
pyf
pypx
pxff yxp
qypyfqxpyfqypxfqxpxff yyyxxyxxpq
Principal Orthogonal Directions Two directions play the key role Gradient direction
Tangent direction h = (hx, xy) = (–gy, gx). fh = 0
Decision is based on fgg, fhh and fhg
2222 ,),(
yx
y
yx
x
ff
f
ff
fgygxg
22yxg fff
Ridge Definition
Point (x, y) is called a ridge point if and only if
K1: 0hhf (f has local maximum in direction h)
K2: || || gghh ff (ridge strength in direction h is larger than in g)
K3: 0ghf (gradient has an extremum in the perpendicular direction)
Analytical example
f(x, y) = y2 – x2
22
22
2yx
yxf hh
22
22
2yx
yxf gg
224
yx
xyf gh
K3: xy = 0 gives x = 0 and y = 0 K1: K2: always true
Ridge: x = 0
022 yx || || yx
Another example
f(x, y) = –y2 – 10x2
22
22
100
1020
yx
yxfhh
22
22
100
10002
yx
yxf gg
22100180
yx
xyf gh
K3: xy = 0 gives x = 0 and y = 0 K1: always trueK2:
Ridge: x = 0Theorem: Ridge of a quadratic surface is always a line.
0 100
100396
22
22
yx
yx || || yx
Naive implementation fx = (f(k + 1, l) – f(k – 1, l)) / 2, fy = (f(k, l + 1) – f(k, l – 1)) / 2, g = gx = fx / g, gy = fy / g, hx = – gy, hy = gx, fxx = f(k + 1, l) – 2f(k, l) + f(k – 1, l), fyy = f(k, l + 1) – 2f(k, l – 1) + f(k, l – 1), fxy = (f(k + 1, l + 1) + f(k – 1, l – 1) – f(k + 1, l – 1) – f(k – 1, l + 1)) / 4, fgg = fxxgxgx + 2fxygxgy + fyygygy, fhh = fxxhxhx + 2fxyhxhy + fyyhyhy, fgh = fxxgxhx + fxy(gxhy + gyhx) + fyygyhy.
Ridge conditions:
|| and || || and 0 ghhhgghh ffff
22yx ff
Naive implementation
Equivalent Formulation
Hessian matrix
Eigenvalues of H Eigenvectors of H
Ridge:
yyyx
xyxx
ff
ffH
11,21,
0,0 11 ge
21,ee
R. Haralick, “Ridges and Valleys on Digital Images,” Computer Vision, Graphics, and Image Processing, vol. 22, no. 10, pp. 28–38,Apr. 1983.
Algorithm
Calculate eigenvalues and eigenvectors See if See if there is a sign change of
in the direction of
Problems: Unreliable Requires interpolation Relatively Slow
01
ge 1
2e
Lee and Kim Algorithm
4 directions of discrete grid are analyzed Orthogonal directions of greatest curvature
are selected Point is classified as ridge, valley, peak or pit
based on sign changes of gradients in these directions.
Problems: Unreliable No clear distinction between ridges and peaks
Lee, S. and Kim, Y. J. 1995. Direct Extraction of Topographic Features for Gray Scale Character Recognition. IEEE Trans. Pattern Anal. Mach. Intell. 17, 7 (Jul. 1995), 724-729.
New Discrete Algorithm
Consider 4 directions of discrete grid Find direction of minimum second derivative (that
will be direction perpendicular to ridge), let fhh be that second derivative.
Test if the current point is a local maximum in that direction
Let fgg be the second derivative in the perpendicular direction.
Test if abs(fhh) >= abs(fgg) If all tests are true return abs(fhh) as ridge strength Else return that this is not a ridge point
Implementation // Calculation of ridge pixel strength at the given point, 0 if the point is not a ridge point // assumes that int *image holds the image values in row by row.
int RidgePixelStrengthOptimized(int x, int y){ int *p = image + (x + y*width); int f = *p; int fpp[4]; int fhh,fhhI, fhh_tmp, fhhI_tmp;
fpp[0] = 2*(f - p[1] - p[-1]); // calculate magnitudes of directional second derivatives fpp[1] = - p[width+1] - p[-width-1]; fpp[2] = 2*(f - p[width] - p[-width]); fpp[3] = - p[width-1] - p[-width+1];
if (fpp[1]>fpp[0]){fhh = fpp[1];fhhI = 1;} else {fhh = fpp[0];fhhI = 0;} // find the maximum one if (fpp[2]>fpp[3]){fhh_tmp = fpp[2];fhhI_tmp = 2;} else {fhh_tmp = fpp[3];fhhI_tmp = 3;} if(fhh_tmp > fhh){fhh = fhh_tmp;fhhI = fhhI_tmp;} fhh += 2*f; // diagonal
if(fhh<lowerThreshold) return 0 ;// optimization: early exit
int fgg = fpp[(fhhI+2)&3] + 2*f; // second derivative in the orthogonal direction int f1Ofs[4] = {1, width+1, width, width-1}; int ofs = f1Ofs[fhhI]; if(p[ofs] < f && p[-ofs] < f && abs(fgg) <= fhh) return fhh;
return 0; }
Results
Sensitivity to Noise
Sensitivity to Noise (1)
Sensitivity to Noise (2)
Sensitivity to Noise (3)
Note: All images were processed with the same parameters
Results
Connected lines Thin lines No artifacts Low noise sensitivity High performance
45MPix/s = 108 video frames(720 × 576) per second (AMD Athlon 2.6GHz)
Compared to Lee and Kim Algorithm Simpler Faster No spurious ridges
Original image Lee & Kim Freivalds & Ķikusts
Scale Selection
Gaussian blurring (Geusebroek et.al. 2002) Scale space and automatic scale selection
(Lindeberg 1996) Adaptive blurring (Elder, Zucker 1998)
Geusebroek, J., Smeulders, A. W., and Weijer, J. v. 2002. Fast Anisotropic Gauss Filtering. In Proceedings of the 7th European Conference on Computer Vision-Part I. LNCS, vol. 2350. pp. 99-112.
T. Lindeberg: "Edge detection and ridge detection with automatic scale selection", International Journal of Computer Vision, vol 30, number 2, pp. 117--154, 1998. Earlier version presented at IEEE Conference on Pattern Recognition and Computer Vision, CVPR'96, San Francisco, California, pages 465--470, june 1996
Elder, J. and Zucker, S. 1998. Local scale control for edge detection and blur estimation. IEEE Pattern Anal. Machine Intell., 20(7): 699–716
Topological Edge Detection
Edge conditions: fgg = 0
fggg < 0
Algorithm Find the discrete direction of maximum gradient See if the second derivative has sign change from
positive to negative in that direction
Comparison to Canny Detector Similar quality
Simpler Faster
Original Canny Proposed
Results – Shooting Target
Conclusion
New robust ridge, edge detection algorithms High quality High performance; suitable for real-time
applications Simple implementation Straightforward hardware implementation