527

Multi-Scale Orientation Estimation for Unstructured Sample Points

X.D. Yangt, W. Chautt. and S.K.M. Wongt

tDepartment of Computer Science University of Regina

Regina, Saskatchewan S4S OA2

Abstract

Orientation of sampled points is a very useful information at various levels of computer vision tasks. Given a set of unstruc- tured points sampled from a curve in an image, we consider the problem estimating orientation, i.e. finding tangent direction (or equivalently normal direction), at each point. The points are discrete samples of curves, and can be perturbated severely by noise. lloppe, et at (lYY2) suggests the use of the covariance matrix, constructed from a k-neighborhood of a point x , to esti- mate the orientation at X . A problem with this approach is the fixed k value, which imposes implicitly a very restrictive condition on the density of sampled points in order to work properly. To solve this problem, we present a multi-scale method for reliable estimation of orientation and curvature for a set of unstructured points. Furthermore, we derive a structural representation for the point set.

1. Introduction

Orientation of sampled points is a very useful information at various levels of computer vision tasks. For example, in Hough transform I41 and its generalized version 111, the use of orientation information can greatly reduce the size of the accumulation array and improve significantly the efficiency of computation. Orienta- tion is also very often used for computing approximation of curva- ture from discrete data, and the curvature in turn has been a very important feature in object representation for model-based recogni- tion 121, [3], (61, 171 and 181. Orientation plays also important roles in several surface reconstruction methods [ 121, 151, and [ IO] .

Given a set of unstructured points sampled from a curve in an image, we consider the problem of estimating orientation, i.e. finding the tangent direction (or equivalently the normal direction), at each point. The points are discrete samples of curves, and can be perturbated severely by noise. In the worst case, the sampled points may look like a thick layer of "cloud" around the underly- ing curve.

If a prior model for the curve is available, the curve fitting techniques (e.g. Hough transform, least mean-squared error, etc) may be applied. Flowever, such a prior niodel is, very often, not available. In this situation, an attempt to estimate orientation is likely to use local neighbor points, since the points are the only information given. A naive approach for orientation estimation may use pixel connectivity to find neighbors along a curve. This technique fails when the sampled points are noisy and the connec- tivities among the points are ambiguous.

A recent paper 151 suggests an algebraic approach to solve the problem. In this paper, the k points nearest to a point x under the consideration are selected to form a k-neighborhood of x , denoted by /VbM(x ) . A center o is taken to be the centroid of N h h d ( x ) . To compute the orientation at x , the covariance matrix of Nhhd(x) is constructed. The unit eigenvector associated with the smallest eigenvalue of the matrix is chosen to be the normal vec- tor. As indicated in 151. a problem with this approach is the fixed k neighborhood. Consider the situations given in Figure 1. In the situation (a), a circle of radius r centered at p contains k points, thus this set of points fomi the N b h d ~ , ) . If extra sample points are added into the image around the curve (the situation (b)), a much smaller circle with the radius r' will contain k points. The conse-

ttDepartment of Computer Science University of Waterloo

Waterloo. Ontario N2L 3G1

quence is that while a reasonable orientation estimation may be obtained in the situation (a), but i t could easily fail in the situation (b) because a too small range of neighborhood is looked at and its result could be very ambiguous and unreliable. Thus the applica- tion of this method relies on an underlying assumption that the density of sampling points in a local region should be proper for the related k. This is a very restrictive condition, and i t is difficult (if not impossible) to be ensured during the sampling process without additional knowledge about the curve.

(a) (b)

Figure I . k -neighborhood at different densities of sample points

To solve the above problem, we studied a geometrically based multi-scale method for estimating the orientation of a set of unstructured sample points. This method considers the estimation problem at different levels of scales. The orientation variance is computed at each scale. This variaiice is then used to guide a search for a proper scale at which reliable orientation estimation can be obtained. We show also that the value of the variance is related to the curvature of the underlying curve. Furthermore, we derive a high level structure of the curve based on the estimated curvature.

2. Multi-Scale Orientation Estimation

Given a set of unstructured points sampled from a curve, our first task is to estimate the orientation at each point, i.e. ihe tangent direction (or equivalently the normal direction). Since the points are the only available information, any attempt to estimate orientation is likely to use the local neighborhood of a point. In general. sampled data in an image are on discrete grids, thus there are quantization errors. In addition, their location can be pertur- bated by noise. I t is often difficult to distinguish between fine features of a curve and noise presented in a curve. For example, given a set of points as shown in Figure 2.(a), if it is assumed to contain fine features, the expected orientation for each point should be like the one shown in Figure 2.(b). On the other hand, if it is considered to be noisy sample data, the orientation shown in Figure 2.(c) would be a better choice. Clearly, the need for scale-specific estimations to deal with the noise problem causes an inherent preference for certain ranges of curvature values and involves strong implicit assumptions about the underlying curve. The actual orientations at those points depend on what wc call noise and what we call features, and hence may take on differing values dependinR on our goal.

The problem can be further complicated by the situation in

CCECElCCGEI '93 0-7803-143-3193 $3.00 0 1993 IEEE 31.2

*... .. . . - e , ] .

(b) ( c ) Figure 2. Ambiguity between features and noise.

which points form a thick layer of “cloud” (Figure 3). Local neighbors cannot be trivially traced out in order to estimate an orientation.

Figwe 3. A curve with large amount of noise.

To deal with the above problems, we studied a multi-scale estimation method i n order to handle regions that have more than one type of characters. This method is attempt to target the smoothing technique for this particular type of unstructed point set, such that i t handles noise in a natural and automatic manner.

Given an arbitrary point p i n the image, we first define the local neighborhood of p as the following. We select a scale inter- val [ d , d+k*d l , for d > 0, and k a constant usually between 0.5 and 1.0. A point belongs to the neighborhood set of p . if its distance from the point p is less than or equal to (d+k*d) , i.e. within the circle of radius (d+k*d) centered at the point p .

For any pair of points within the neighborhood set of p , if the distance between the two points falls in the scale interval ( d , d+k*d) , we draw a line segment between them. We also define a unit vector perpendicular to each line segment to be a normal vec- tor. There are two choices for the direction of a normal vector. We start with a vector in an arbitrary direction. The rests are determined by making a smaller angle with the first vector. The mean vector of all normal vectors within the neighborhood set is defined as the unnormalized normal vector at the point p : :

where N is the number of vectors within the neighborhood set.

A nomialized nomial vector at the point p is the uni t length vector along h p .

(a) (b) Figure 4. Magnitudes of unnormalized normal vectors.

There are two important properties associated with the l ip .

( I ) When all normal vectors within the neighborhood set are in parallel, hp has a unit length (Figure 4.(a)).

(2) When there is a large variance among the normal vectors in the neighbor set, the length of A, will be small (Figure 4.(b)).

Thus, the magnitude of fip characterizes the fuzzyness of the orientation at the point p by using this estimation method. We can define a fuzzyness index to be 1 1 I f i , I .

By varying the range of the scale interval. we can obtain a set of orientations estimated at different levels of scales. A n orien- tation is considered to be reliable, if it has a small fuzzyness index value.

Our estimation of orientation can be adaptively refined with the guidance of the fuzzyness index. This process starts with a default large scale interval, and the fuzzyness index is evaluated. Then the scale is reduced and the fuzzyness index is evaluated again. If the new fuzzyness index value is smaller than the previ- ous one, this process is repeated. The process stops when a minimum fuzzyness index value is reached.

The above idea is illustrated by the example shown in Fig- ure S. Figure S.(a) is a set of noisy sample points from a curve. Figure 5.(b) is the estimated orientation at a relatively large scale (d = 15 pixel length) for the small rectangular region shown in Figure S.(a). In this case, a large fuzzyness index value is obtained, indicating that there is a large variance in the estimated orientation. By reducing the scale to d = 10 pixel length, the vai- ance of orientation decreases (Figure 5.(c)). By further reducing the scale (d = 5 pixel length), the fuzzyness index reaches approx- imately its minimum (Figure 5.(d)). If the scale continues to be reduced, the variance of orientation will increase sharply. This is because a too small range of neighborhood is computed, and the result is essentially dominated by the noise. Figure 5.(e) shows the final estimated orientation.

3. Curvature Estimation

Assume that a curve is expressed parametrically as x ( s ) and y ( s ) , where s is a path length variable along the curve. The curvature k at any point p on the curve can be defined as the instantaneous rate of change of e, which is the angle subtended by the tangent at point p with the x axis, with respect to arc-length 5

(Figure 6.(a)), given by

k ( s ) = * As

Unfortunately, the analytical definition for curves in an image is generally not available. Here we present a method to compute curvatures for the sampled points based on the previously obtained orientation result.

The following derivation is based on an observation from Figure 6.(b). The angular difference between the two neighboring

5 2 9

I I I

I I I

\ \

\ \ \ \ \ ' \ \ \

I t I \ \ \ \ / / \ I \ \

I I I

Figure 5 . An example of multi-scale orientation estimation.

(a) (b) Figure 6. Curvature centers.

orientation vectors is I$, and the arc length is s . When the angle $ is small, s can be approximated by the distance d between the two points. Hence, k w d .

Given the orientation estimated at a certain scale interval, we can use the same neighborhood set used for orientation estima- tion to estimate curvatures. First we define the mean orientation at a point p to be

(3) E p -1" -

" p = - ptip-&)l

I =o

The orientation variance at the point p is defined as

(4) 1 "

I =I

where $(Ep-t?,) is the angular difference between the two vectors. Finally,

( 5 )

where d is the current scale. 'I'nble 1 shows orientation variances computed for the points in a sampled region in Figure 5 at three different scales.

2 v

d k + P

row col Scale 1

144 175 5.46 145 175 5.73 146 173 6.49 146 174 6.55 147 174 6.40 148 170 9.60 148 171 8.01 148 172 7.63 149 167 14.88 149 169 11.86 149 170 9.60 149 172 7.46 149 173 6.39 150 163 16.31 150 164 16.20 150 166 15.74 150 167 14.88 150 168 13.98 150 169 11.86 150 170 9.39

_ _ _ _ _ _ ____- -_ Scale 2

11.25 12.27 14.72 14.75 14.53 14.32 15.10 15.87 9.63

13.21 14.32 15.50 14.50 5.42 6.00 7.10 9.63

11.89 13.21 13.55

- - - - - - - Scale 3

29.93 32.92 39.12 36.93 37.54 49.53 39.84 39.59 29.10 46.91 49.53 39.85 39.62 29.72 28.07 22.13 29.10 40.69 46.91 48.63

------_

Table 1. Variances of orientation.

The curvature information derived here can facilitate many existing curvature-based curve segmentation techniques [2j. [3], 161, 171 and 181.

4. Curve Structure

Abstract representations about a scene are always desired for performing model-based recognition. Various types of elemen- tary shapes are employed in different systems. Among existing systems, line segments, circular arcs, and elliptic arcs are the most commonly used elementary shapes (e.g. 191).

5 30

We have estimated orientations and curvatures from a set of unstructured points without actually constructing underlying curves. In the next, we derive an abstract, high level structural for the point set. The objective here is to partition the point set into subgroups such that each group can potentially define a underlying line segment, or circular arc segments. This task can be performed in two steps.

(1) Start with an arbitrary point p . Let k p be the curvature at p . Then specify a small 6. for 6 9 . We propagate from p to extend the group. A neighboring point p , is included in the group, if Ikpj-k,1<6. This process is performed repeatedly unt i l no more points can be added into the group.

Start for a new group with an non-partitioned point, and perform the above process again. This process is repeated until no more non-partitioned points are left in the image.

In each partitioned group, all points have a siniilar curva- ture. Thus, they either correspond an underlying line segment, if the k A 0, or an underlying circular arc. if the k>O.

(2) For a group which has a k > O , we can estimate its correspond- ing curvature center as follows.

A group is divided into two subgroups by a line which passes the centroid and is parallel to the mean orientation of the group. Then, the mean orientation for each subgroup is calculated and placed at the center of the subgroup. By extending these two mean orientation vectors, we obtain an intersection point. This point serves as the curvature center for the corresponding group.

along the curve Figure 7 shows several detected centers of circular arcs

Figure 7. Detected circular arcs.

This structural representation can also facilitate many exist- ing shape recognition techniques 121, [7] and [SI.

5. Conclusions

We presented a multi-scale method for reliable estimation of orientation and curvature for a set of unstructured points. Furth- ermore, we derived a structural representation for the point set. The results computed from this method may be incorporated with many existing shape recognition techniques. The extension of this method to 3D surface problems is greatly interested.

Acknowledgement

This work is supported i n part by the NSERC of Canada under the research grant OGPO105708.

References

Barllard, D.H. "Generalizing the Hough transform to detect arbitrary shapes." Pattern Recognition, Vol. 13: 1 I 1 - 122, 1981. Bruckstein, A.M., N. Katzir, M. Lindenbaum, and M. Porat. "Similarity-invariant signatures for partially occluded planar shapes." Intl. J. Computer Vision. 7(3):27 1-285, 1992. Dudek, G., and J.K. Tsotsos. "Shape representation and recognition from curvature". Proc. IEEE CVPR'91, 1991.

Gonzalez, R.C. and P. Wintz. Digital image processing. 2nd ed. Addison-Wesley, Reading, MA, 1987. Hoppe, H. T. DeRose, T. Duchamp, J. McDonald, and W. Stuetzle. "Surface reconstruction from unorganized points." Computer Graphics, 26(2):71-78. 1992. Kalvin, A., E. Schonberg. J.T. Schwartz, and M. Sharir. "Two dimensional model based boundary matching using footprints." Tech. Rep. 162, Dept. of CS, NYU. May 1985. Lowe, D. "Organization of smooth image curves at multiple scales." Proc. 2nd ICCV, Tampa, FL. 1986, pp.558-567. Mokhtarian, F., and A.K. Mackworth. "A theory of multis- cale, curvature-based shape representation for planar curves."

pp.35-37.

IEEE PAMI. 14(8):789-805, 1992. Rosin, P.L., and G.A. West. "Segmenting curves into elliptic arcs and straight lines." Proc. 3rd ICCV, 1990 pp.75-78.

[IO] Sziliski, R. and D. Tonnesen. "Surface modeling with oriented particle systems." Computer Graphics, 26(2): 185- 194, 1992.

[ 1 I ] Witkin, A.P. "Scale space filtering." Proc. IJCAI,, Karlsruhe, W. Germany, 1983, pp.1019-1023.

[I21 Yang, X.D. "Fuzzy disk modeling and rendering of textured complex 3D surfaces of real objects." Tech Rep-414, Courant Institute, New York University, Nov. 1988.