Representation and recognition of surface shapes in range images: A differential geometry approach

COMPUTER VISION, GRAPHICS, AND IMAGE PROCESSING 52, 78-109 (1990)

Representation and Recognition of Surface Shapes in Range Images: A Differential Geometry Approach

PING LIANG

School of Computer Science, Technical lJnil:ersity of Nova Scotia, Halifax, Nova Scotia

AND

JOHNS.TODHUNTER

Department of Electrical Engineering, University of Pittsburgh, Pittsburgh, Pennsylvania 15260

Received August 5, 1987; revised August 21, 1989

Theory and matching algorithms are developed for accurate orientation determination and recognition of 3D surface shapes in range images. Two corollaries to the fundamental theory of surface theory are proved. The first corollary proves the invariance of the fundamental coefficients when lines of curvature are used as the intrinsic parameter curves. The second corollary proves that a diffeomorphism which preserves the intrinsic distance along the principal directions, in addition to preserving the eigenvectors and eigenvalues of the shape operator (Weingarten map), is necessarily an isometry. Based on these two corollaries, a set of geometric descriptors which satisfy the uniqueness and invariance requirements are theoretically identified for all classes of surfaces, namely, hyperbolic, elliptic, and developable surfaces. The unit normal and shape descriptors list array (UNSDLA) representation and the corresponding matching algorithm are developed. The UNSDLA is a generalization of the extended Gaussian image (EGI). The EGI has a fundamental limitation; that is, it can only uniquely represent convex shapes. The new representation overcomes this limitation of the EGI and extends the scope of unique representation to all classes of surfaces. Moreover, it still has all the advantages of the EGI. This is achieved by preserving the connectivity of the original data. Connectivity here should include not only the adjacency relation of points or patches on a surface, but also the direction and order in which the points or patches are traversed in a connected path. The importance of the direction and order of connectivity is emphasized. Surface matching can be performed more accurately using the UNSDLA than the EGI. Based on the UNSDLA representations, surfaces can be matched via the Gaussian map by optimization over all possible rotations of a surface shape. The representation and matching algorithm can deal with hyperbolic and elliptic surfaces whose Gaussian maps are not one-to-one. Developable surfaces whose Gaussian maps of lines of curvature with nonzero principal curvature are not one-to-one can also be accommodated. Two theorems on developable surfaces are proved. 0 1YYO Academic Press, Inc.

1. INTRODUCTION

Accurate orientation determination and recognition of 3D surface shapes are of fundamental importance in computer vision and robotics studies and have a wide variety of applications.

A range image provides a sampled version of visible object surfaces represented by a 2D array of numbers. The numbers represent the distance from the sensor to points on surfaces of objects. A range image is a 3D image of the 3D world, The use of a range image greatly facilitates the description, recognition, and 3D measurement of object shapes by their surface geometric characteristics. Various techniques for acquisition of range images have been developed and improved in recent years [l-13]. A survey of range finding techniques has been made by Jarvis 1141.

78 0734-189X/90 $3.00 Copyright 0 1990 by Academic Press, Inc. All rights of reproduction in any form reserved.

DIFFERENTIAL GEOMETRY APPROACH 79

1.1. Problem Statement

In this paper we are interested in accurately determining orientation and recognizing:

(1) smooth curved surfaces, such as engine blades, ship propellers, surface patches of aircraft, and precision industrial parts, metal sheets, and

(2) 3D shapes consisting of multiple smooth, especially curved, surface patches, such as light bulbs, precision industrial parts, etc. The 3D shapes may have discontinuities in orientation along junctions between different smooth surface patches. This class includes most types of objects in practical applications.

For the purpose of measurement, orientation determination, and recognition of surface shapes in range images, proper descriptions and representations for object models, and surfaces measured by a range sensor, referred to as sensed surfaces hereafter, should be developed. They should be computed inexpensively and, desirably, the computation should admit implementation by parallel processing. It is assumed that a range image is sampled over a regularly spaced 2D rectangular grid.

A matching algorithm should be developed to match the representations of a sensed surface shape and an object model. If a sensed surface is recognized, the matching algorithm should determine its orientation and the scaling factor between the object model and the sensed surface, if they are similar shapes of different sizes. The orientation, scale, and translation are necessary information for robot manipulation. If no match between a sensed surface shape and object models is found, the description of the sensed surface shape should be stored. This information will be used for construction of the representation of the object model for this newly encountered object. Even when no match is found, the description of the sensed surface shape should uniquely describe the surface and be used as measurement of the shape of the sensed surface.

1.1.1. Surface Description and Representation

It is known from the fundamental theorem of surface theory that the six coefficients of the first and second fundamental forms uniquely determine a smooth surface up to a proper Euclidean motion. However, the coefficients of the first and second fundamental forms are dependent on the choice of the parameterization. They are not directly applicable to surface shape characterization. The fundamental theorem of surface theory can also be formulated in terms of isometry of surfaces and shape operators [151. It asserts that two surfaces are congruent if and only if there is an isometry between the two surfaces that preserves the shape operator. This result needs to be translated to conditions that could be tested with a small number of computations for it to be applicable to surface shape description and recognition.

Therefore, only descriptors related to geometric quantities that do not rely on the parameterization, such as principal curvatures, Gaussian and mean curvatures, had been used in differential geometric approaches to surface shape description. In some applications, these descriptors might be sufficient. In general, these descriptors do not uniquely characterize the shape of a surface.

80 LIANG AND TODHUTER

A set of surface shape descriptors are desired which uniquely describe a surface shape and can be defined geometrically so that they are invariant to change of parameterization. Based on this set of surface shape descriptors, the form of representations of the model objects and sensed surfaces should be developed.

The surface shape descriptors and the representation should apply to all types of surface shapes, namely, surface shapes with negative, positive, and zero Gaussian curvatures, often referred to as hyperbolic, elliptic, and developable’ surfaces, respectively. Research previously reported [16-181 most similar to this paper scarcely dealt with hyperbolic and developable surfaces because the basic theorem used, Minkowski’s theorem, applies only to convex objects.

The representation should be properly normalized so that surfaces with the same shape but different sizes can be matched. It is also desired that the surface shape descriptors can be used for accurate shape measurement of a sensed surface.

1.1.2. Criteria for the Matching Algorithm A matching algorithm should consider hyperbolic, elliptic, and developable

surfaces and surface shapes with a combination of them. In many applications, the objects of interest are often occluded, or only parts of the objects are sensed. A model object contains the description of the whole surface shape, which often is constructed from multiple range images. A matching algorithm should be capable of matching surface shapes with occlusion and matching part of a surface shape to the whole surface shape.

To completely specify the position and orientation of an object in a 31) Euclidean space, three translational and three rotational variables are needed. It is desired that a matching algorithm deal separately with translation and orientation, thus simplifying the matching problem by dealing with three variables at a time. The translation of a sensed surface from the sensor can be determined from the input range image. A matching algorithm should determine the orientation of an object. The scaling factor should be determined if the surfaces matched are similar shapes of different sizes. A matching algorithm should fully exploit the accuracy of the input range image. It should have a simple implementation and should admit parallel processing for acceleration of the matching process.

1.2. Literature Review The computer vision and robotics research community has shown increasing

interest in range image based 3D shape description and recognition [l-4, 13, 16-331. Differential geometry provides precise and complete local description for smooth surfaces. The differential geometric approach to surface shape description and recognition has been investigated by many researchers, for example, [16-18, 22-26, 34, 351.

The extended Gaussian image (EGO has been studied by many researchers for shape description, orientation determination, and object recognition [16-18, 35, 361. Matching algorithms, which match the EGI of an object model and that of sensed range data to recognize a surface and to determine the orientation, have

‘A developable surface is a surface with Gaussian curvature equal to zero everywhere [46]. A surface is planar if both principal curvatures are zero.


been developed [16-181. The matching is implemented by minimizing a matching error. The translational and rotational variables are separated.

The EGI representation is advantageous over most other types of 3D representations in many aspects. However, it has the following limitations.

(1) The most fundamental limitation of the EGI representation is that it can only uniquely represent convex shapes. This is because the EGI is based on Minkowski’s theorem which only applies to convex objects. There exists a one- to-one onto mapping between a closed convex 3D shape and the unit sphere using the unit normal mapping. However, for a nonconvex shape, this mapping is not one-to-one, and the connectivity of the original data is lost through the mapping unless it is explicitly represented. In this paper, connectivity includes both the adjacency relation of points on a surface, and the direction and order in which the adjacent points are traversed in a connected path. Preserving the direction and order of connectivity is equally as important as preserving the adjacency relation. Extension of the EGI to nonconvex objects with positive Gaussian curvatures are discussed [16, 351, but it cannot deal with hyperbolic and developable surfaces.

(2) In EGI-based approaches, the goal of a matching algorithm is to find the rotation in the space of all possible rotations that minimize a matching error. However, with EGI, this minimization cannot be carried out analytically for all possible orientations, since the rotation space is discretized. This limits the accuracy of the matching algorithm.

Vemuri et al. [221 developed a principal curvature-based representation of objects from the range image. Besl and Jain [23, 241 used the signs of both the Gaussian and mean curvatures to classify and segment range image surfaces into regions of one of the eight basic surface types. Medioni and Nevatia [25] developed a set of shape descriptors based on the zero crossings of the Gaussian curvature and the maximal principal curvature, and the maxima of the maximal principal curvature. Fan et al. [26] updated the approach in 1251.

The differential geometric descriptors developed in [22-261 are useful in surface shape description and recognition. When coupled with a priori information they may be sufficient in some applications. However, in general, they cannot uniquely describe a surface shape.

There are basically two types of representations for 3D shapes: surface representations and volumetric representations [ 19, 201. Volumetric representations, such as the generalized cylinder model, can only describe a certain class of objects and are suited for certain applications.

Surface representations seem ideal for recognizing and positioning objects since object surfaces are what can be actually measured by sensors. The volume of an object can be easily inferred from the representation of its bounding surfaces.

Approaches to description of 3D surface shapes from range data are categorized into three classes [26]: approximation by simple surface patches, such as planar and quadratic surfaces [19, 27, 281, extraction of edges in range images [22, 291, and 3D surface shape characterization [16-18, 22-261. The approach presented in this paper is in the category of 3D surface shape characterization by geometric descriptors.


In Section 2, two corollaries to the fundamental theorem of surface theory are proved which provide a theoretical basis for surface shape description and recognition using a differential geometry approach. Section 3 incorporates the results from Section 2 with the properties of the Gaussian map of surfaces to develop a representation and matching scheme for surface shapes. Section 4 considers the representation and matching algorithm for three types of developable surfaces based on the two corollaries proved and an analysis of the characteristics of developable surfaces. Also, two theorems on the description of developable surfaces are proved. Conclusion and discussion of future research directions arc presented in Section 5.

2. CONDITIONS ON CONGRUENCE OF SURFACES APPLICABLE TO SURFACE SHAPE DESCRIPTION AND RECOGNITION

2.1. Lines of Curvature as Parameter Curves

The tangent space T,M and the unit normal v(P) at a point P on surface M can be defined independently of the parameterization of the surface. Thus the first and second fundamental forms, Z(X, Y) = (X, Y), ZZ(X, Y) = - ( Vxv, Y) can be defined independently of the parameterization, where X, Y E Tr M, - 0, v: T,M + T,M is the Weingarten map, and ( . , . > is the usual Riemannian metric on R3. Once the second fundamental form is defined, the principal curvatures and the principal directions can be identified. The principal curvatures are the maximum and minimum of ZZ(X, XI, X E TpM, subject to (X, X) = 1, and the principal directions are the directions in which the maximum and minimum are achieved. A line of curvature is a curve whose tangent always points along a principal direction.

Therefore, from a computational point of view, it is possible to find the principal directions and lines of curvature without knowing the parameterization of the surface. Once the lines of curvature are obtained, the surface can be parameterized by the curvilinear coordinates formed with the lines of curvature as the parameter curves [46].

The invariance of the fundamental coefficients when the lines of curvature are taken as the parameter curves is first investigated. We proved the following theorem on the invariance of the fundamental coefficients when lines of curvature are the parameter curves. It is a corollary to the fundamental theorem of surface theory in terms of the first and second fundamental coefficients [46]. The proof is omitted here and is given in [37].

Let gii, Lti, i, j = 1,2, be the coefficients of the first and second fundamental forms, and f,, i = 1,2, be the derivatives of f with respect to the ith variable in f<., . 1.

THEOREM 1. Let M and &f be two connected oriented surfaces (without umbilic points) embedded in R3. Let 4: M -+ ii? be an orientation preserving diffeomorphtkm . There exists an open cover 0 = {N, c M} of M and embeddings f a: U, c R2 --+ N, with lines of curvature on N, as the parameter curves and origin in R2 mapped to P, E N,, and pa: V, c R2 -+ 4(N,) with th e mes of curvature on r#r( NJ as the 1’ parameter curves and origin in R2 mapped to #r( P,) E +(N,>. TTaen M and R are congruent if and only if by properly choosing the orientations of the surfaces and the


positive directions of the principal vectors, there exists a 4 such that

4*fW> =J?ia(4(Q)), G(Q) =&X4(Q)), -G(Q) = t:(4(Q>) i = 1,2, forallQ EN, andallN, E 0. (1)

Then 4 is a proper Euclidean motion A restricted to M such that 4 = AIM and A,v = V.

When lines of curvature are the parameter curves, fi, i = 1,2, are the tangent vectors of the lines of curvature, and the principal curvatures are given by Ki = LJg,,. Moreover, gii = (fi, fi>, i = 1,2. Therefore, it is equivalent to say that when the lines of curvature are the parameter curves, the tangent vectors to the lines of curvature, fi, i = 1,2, and the principal curvatures K~, i = 1,2, uniquely describe a surface shape.

In applications, all the surfaces we are interested in are compact surfaces. Therefore there exist open covers of A4 and a consisting of finitely many bounded open sets. Suppose 0 = (Nk c M, k = 1,2,. . . , n}, then the congruence needs only to be tested on those n bounded open sets.

2.2. Conditions Based on Principal Vectors and Principal Curvatures

The fundamental theorem of surface theory can be formulated in terms of isometry of surfaces and shape operators [15, 421. The theorem asserts that two surfaces are congruent if and only if there is an isometry between the two surfaces that preserves the shape operator. This result needs to be translated to conditions that could be tested with a small number of computations for it to be applicable to surface shape description and recognition.

It is known that the shape operator is uniquely specified by its eigenvectors (not necessarily unit vectors) and eigenvalues-the principal directions and the principal curvatures. Therefore, only the principal directions and the principal curvatures need to be tested for the shape operator condition to hold. However, the isometry condition is still not guaranteed. We proved the following theorem. It is a corollary to the fundamental theorem of surface theory in terms of isometry and shape operators [15, 421. The proof is given in [37].

THEOREM 2. Let M and a be two connected oriented z&aces (without umbilic points) embedded in R3. Let 4: M -+ a be an orientation preserving di’eomorphism. To test the conditions for isometry and shape operator in the fundamental theorem of s&ace theory given in [15, 421, it is necessary and suficient to test only the following conditions :

4*PPi = *Pi, K,(P) = zi(4(P>)7 i = 1,2, forallP EM, (2)

where pi, pi, i = 1,2, are the principal vectors with the maximal and minimal principal curvatures, K~, Ki, i = 1,2, on M and a, respectively. Su$ace M and mare congruent if and only if by properly choosing the orientation of M and fi, such that the conditions in Eq. (2) hold. Then 4 is a proper Euclidean motion A restricted to M and 4 = AIM.

Previous literature in computer vision discussed the shape operator (Weingarten map), for example in [23, 241, and it is obvious that the eigenvalues and eigenvectors of the map uniquely determine it. However, the Weingarten map alone cannot


uniquely determine a surface. Past papers in computer vision applying differential geometry fell short of identifying an applicable surface isometry condition.

The contribution of Theorem 2 is that it asserts that if a diffeomorphism preserves the intrinsic distance along the principal directions, in addition to preserving the eigenvectors and eigenvalues of the shape operator, it is necessarily an isometry.

Both Theorems 1 and 2 apply to hyperbolic, elliptic, and developable surfaces. Since the principal vectors and principal curvatures are defined geometrically, they are invariant to change of parameterization and can be used in surface shape description and recognition.

2.3. Umbilic Points

Now let us consider umbilic points. There exists no umbilic point on surfaces with strictly negative Gaussian curvature and surfaces with zero Gaussian curvature where the nonzero principal curvature does not change sign. Umbilic points are possible only on surface patches with strictly positive Gaussian curvature and on surfaces with zero Gaussian curvature where the nonzero principal curvature’ changes sign.

The principal vectors are not needed in the description of surfaces with strictly positive Gaussian curvature. Such surfaces are strictly locally convex [46]. Gaussian curvature and unit normal can be used as the descriptors for description of surfaces with K > 0 [38, 461.

The description of surfaces with zero Gaussian curvature is discussed in Section 4. Surfaces with zero Gaussian curvature should be segmented into regions where the nonzero principal curvature does not change sign.

Therefore, the descriptors and the representation developed in this paper do not place any restriction on the existence of umbilic points on a surface.

2.4. Description of Discrete Surfaces

When Theorem 2 is applied to a discrete surface, only the principal vectors and principal curvatures at discrete sampling points are available. The surface patch on a discrete surface around a sampling point is only determined to within a class of surface patches with the given principal vectors and principal curvatures at the sampling point. If the principal vectors and principal curvatures are given continu- ously at all points on a surface, by Theorem 2, the surface is completely specified.

It can be shown from an approximation point of view how the principal vectors and principal curvatures at sampling points describe the shape of a discrete surface. Suppose the discrete surface is sampled from a smooth connected oriented surface M. Let P be a sampling point. Let the moving orthonormal frame (pl, pZ, v(P)) at point P be the (x, y, z) axes of a Euclidean space, P be the origin of the R3 coordinate system, and the tangent plane be the n: - y plane. Then it can be shown that the quadratic approximation of M near P is given by

K1X2 + K2y2 iZ=

2 (3)

*By the nonzero principal curvature, we mean the principal curvature that is not constantly zero.


Therefore, in describing a discrete surface by the principal curvatures, we are approximating the smooth surface shape by small surface patches t = (K,x* + K2y2)/2 around the sampling points with the moving orthonormal frames as the (x, y, 2) axes.

Local approximation of a small surface patch by Eq. (3) is a well-known result [15, 461. The principal curvatures at a point determine the local shape of the surface patch around the point. Therefore, the principal curvatures have been used extensively for surface description [22-261. However, the above approximation scheme cannot be extended to an entire surface, because the relative orientation between the neighboring surface patches is not specified. That is, the information on how to put all the small surface patches together to approximate a larger surface is not given.

What is new in applying Theorem 2 to discrete surface description is the extension of the above local approximation scheme to an entire surface by inclusion of the principal vectors. The principal vectors at sampling points specify the relative orientations between surface patches at neighboring sampling points, while the principal curvatures determine the shapes of the surface patches.

In this paper, the surface normals and the principal vectors of a discrete surface shape are computed by a method generalized from Anderson and Bezdek’s work [39] on curvature and tangential deflection of planar discrete curves. The idea is similar to dimension reduction by Karhunen-Loeve expansion in statistical pattern recognition. First, the surface normal at a point is defined as the cross product of the eigenvectors with the largest and second largest eigenvalues of the covariance matrix of the discrete surface patch around the point. The principal vectors at a point are defined as the unit eigenvectors with the largest and second largest eigenvalues of the covariance matrix of the unit normals of the surface patch around the point. The principal curvatures are computed by the unit surface normals along the principal directions. In the current implementation, the principal curvatures at a point are derived by a least square fit to a neighborhood of the point using Eq. (3). The parameters of the least square fit is computed in a coordinate system determined by the principal vectors and the unit normal. The algorithm for finding the principal directions can be easily implemented by calling a routine to compute the eigenvalues and eigenvectors of a symmetric positive definite matrix, such as the Jacobian rotation algorithm. The definitions and computations are based on the geometric properties of a surface and the statistical properties of the sets of the sampling points. They are independent of the parameterization.

If the orientation of a surface is reversed, that is, if the unit normal field is taken as -v, the two principal curvatures both change their signs. Thus, in implementation, only the principal curvatures with an arbitrary orientation need to be computed. The principal curvatures with the reversed orientation are obtained by changing the signs of the two principal curvatures computed with the original orientation.

3. REPRESENTATION AND MATCHING OF SURFACE SHAPES

To apply Theorem 2 to surface shape description and recognition, the proper diffeomorphism C$ in Theorem 2 must be identified to establish the mapping of the


descriptors of a model surface to those at the corresponding point of a sensed surface shape.

Let S* be the unit sphere, V: M -+ S* c R3 and ? M -+ S* c R”, be the unit normal fields determined by the orientations. Let R be a 3D rotation such that after this rotation the Gaussian maps of the two surface shapes coincide. The mappings between surfaces can be established by 4 = (i7-’ . R . v: M + a?, provided 4 is one-to-one.

V: M + S2 c R3 is a well-defined function. In general, however, the inverse mapping from the unit sphere to surface M cannot be defined, since it is one-to-many for a nonconvex shape. It is this one-to-many mapping that causes the EGJ to lose the connectivity of the original data and therefore limits its scope of unique representation.

Observe that all the advantages of the EGI come from the presence of the normal direction in the representation; while its shortcoming is a result of the direct use of the unit normal mapping from a surface to the unit sphere. A way out of this dilemma is to keep the normal directions in the representation while avoiding the straightforward mapping of the descriptors to the unit sphere in order to preserve the connectivity. A surface is inherently 2D. It can be parameterized by a curvilinear coordinates and represented by a 2D array, or by a 2D edge-node graph. The 2D array or the edge-node graph representations of surfaces preserves the connectivity. Note that connectivity here means not only the adjacency relation of points, but also includes the direction and order of traversing the points. That is, if a connected path traverses a set of adjacent points on the surface in a given direction and order, the corresponding nodes and edges, or the elements, in the 2D array should also form a a connected path that is traversed in the same direction and order. We emphasize the importance of the direction and order of the connectivity. If the representation also contains the unit normals and the shape descriptors of the surfaces, surfaces can still be matched through the Gaussian maps, no matter whether the Gaussian maps of the surfaces are one-to-one.

From Theorem 2, it is easy to prove that a surface M is similar to another surface i;? (both with K > 0 or with K < 0 and have a one-to-one Gaussian map) if and only if by properly choosing the orientations of the two surfaces there exists a rotation R of M such that after the rotation, the equations

W,(P) = ,i(Cf’,(p>)> +*pi = -+-Pi i = 1,2, for all P E M. (41

hold, where 4 = (V)-’ . R . v: M -+ a. Note that Theorem 2 applies to all types of surfaces. Elliptical surfaces can be

described by the principal vectors and principal curvatures as well. However, based on the Minkowski’s theorem, they can be described more efficiently by Gaussian curvatures and unit normals. If M and %? have strictly positive Gaussian curvature, Eq. (4) can be simplified to

p*K(P) = ~b#@‘))> for all P E M. (5) where p > 0 in Eqs. (4) and (5) is the scaling factor in both cases. If Eq. (4) or (5) hold, then M and M differ only by a proper Euclidean motion and a scale factor p, denoted as pa = R(M). Hereafter, surface M represents a sensed surface, and surface m represents an object model surface.


3.1. Segmentation of S&ace Shapes

To build a representation using geometric descriptors, a surface shape should first be segmented into regions with the same sign of Gaussian curvature. A surface shape is segmented into regions with Gaussian curvature K < 0, K > 0, and K = 0. Surface shapes in a range image are also segmented along discontinuities of surface orientation and depth. When K = 0, the surface should be further segmented into regions with the nonzero principal curvature K~ < 0, K~ > 0, and K2 = 0.

A relational graph is then constructed to represent the connectivity of the segmented surface regions. Connectivity here has the same meaning as that for each of the surface regions. That is, it includes both the adjacency relation and the direction and order in which the segmented regions are traversed in a connected path. A relational graph (or connectivity graph) is an edge-node graph. Each node represents a segmented surface region having the same characteristics, that is, the same sign of Gaussian curvature in this case. Each node has a sign (+, -, or 01, same as the sign of the Gaussian curvature of the surface region it represents. If a node has K = 0, it is further labeled by the sign of the nonzero principal curvatures K~. Two nodes are connected if the segmentation boundaries of the surface regions they represent share a common segment.

The segmentation approach is based on the following reasoning:

(1) Such a segmentation is a powerful first step in matching surface shapes. A relational graph is a higher level representation and provides a rough description of a surface shape. Matching at this level first avoids the heavy computational load of matching objects that cannot be congruent nor similar at the lower representation levels. Examples of previous research along this line of approach are [22-24, 40, 41, 431. Also, it has been proposed that the human visual system decompose shapes into parts for the object recognition purpose [45].

(2) Hyperbolic, elliptic, and, especially developable, surfaces have different characteristics. Segmenting a surface shape into regions with constant signs of Gaussian curvature allows them to be dealt accordingly. Representation and matching for each type of surface region will be simplified. As will be shown later, the representation and matching of developable surfaces are very simple and efficient. Elliptic surfaces can be represented by the unit normal and the Gaussian curvature. This also saves computation compared with the hyperbolic case. Surface shapes with smooth surface patches but discontinuities of orientation, which represent a large percentage of objects in practical applications, can be dealt with by segmenting along discontinuities of orientation. Also, as pointed out in Section 2.3, umbilic points are easily taken care of by segmenting a surface into regions with constant signs of Gaussian curvature and describing elliptic surfaces using Gaussian curvature and unit normals.

(3) Even after segmenting a surface shape into regions with constant signs of Gaussian curvature, there may still be some points in the same region having the same unit normal. A simple example is a spiral cylinder or a spiral cone, such as the one given by Brou [16]. But after the segmentation, for a large percentage of surface shapes, the Gaussian maps will be one-to-one. A matching algorithm can be made much simpler when both surfaces to be matched have one-to-one


Gaussian maps. A flag can be set if at least one of the Gaussian maps is not one-to-one. Thus, the matching of surfaces with both Gaussian maps one-to-one and with at least one Gaussian map not one-to-one can be dealt separately, resulting in more efficient overall surface shape matching.

After the segmentation, two surface shapes are congruent (or similar) if and only if

(1) the two relational graphs are the same, or one is the same as a subgraph of the other, in the case of occlusion and matching part of a surface shape to the whole surface shape;

(2) in every pair of matching nodes of the relational graphs, the regions they represent are congruent or similar;

(3) the proper Euclidean motions which bring the congruent (or similar) regions into coincidence (or correspondence) are the same.

If the two surface shapes are similar, the scaling factors for all the regions should be the same. The above three conditions actually are the three steps for a matching algorithm. Note that in matching the relational graphs, only nodes with the same sign can be matched.

Besl and Jain [23, 241 have sufficiently addressed the problem of segmenting range images using Gaussian and mean curvatures. They have developed a range image segmentation algorithm and have demonstrated the feasibility in experiments. Therefore, we will not address the implementation of range image segmentation algorithms in this paper.

3.2. The Unit Normal and Shape Descriptor List Array Representation

In the following, only the representation and matching of surfaces with Gaussian curvature K < 0 and K > 0 are considered. The representation and matching of developable surface shapes are considered in Section 4.

A surface shape is represented at two levels. A relational graph represents the connectivity between segmented surface regions. Each segmented surface region is represented by a 2D list array of unit normals and surface shape descriptors associated with the unit normals. This representation is referred to as the unit normal and shape descriptors list array WNSDLA). It is a generalization of the EG1.3

For the representation to be compact, it is not possible to store the unit normals and shape descriptors for all the points on a range image surface. The unit normal orientation space needs to be subdivided and sampled. This corresponds to a tessellation of the unit sphere. Note that, however, increasing compactness of the representation implies a lower sampling rate, and thus, a less accurate representation and lower accuracy in orientation determination.

A UNSDLA is represented as a 2D edge-node graph and can be stored as a multi-link list in computer implementation. Each node represents a connected surface patch whose unit normals fall within a tessellated cell on the unit sphere. The mapping between the connected surface patch represented by a node and the

3The continuous version of the UNSDLA may be referred to as the generalized Gaussian image.


corresponding tessellated cell is one-to-one and onto.4 Two nodes are connected if the boundaries of the two surface patches they represent share a common segment. The connectivity of nodes should preserve the direction and order in which the surface patches they represent are traversed in a connected path. A node stores the average of one of the unit principal vectors and the average principal curvatures, or the average Gaussian curvatures, of the surface patch. The tessellated cell on the unit sphere corresponding to a node specifies the unit normals of the surface patch represented by the node. The connectivity of the surface patches corresponding to the nodes is represented by the links of the graph. The connectivity is preserved by avoiding directly mapping the shape descriptors to the unit sphere as was done in the EGI.

Only one principal vector is needed in the UNSDLA, because the two principal vectors and the unit normal always form a moving orthonormal frame. In a UNSDLA, the unit normals are already specified by the corresponding tessellated cells on the unit sphere. When one principal vector is known, the other can always be obtained by taking the cross product of the known principal vector and the unit normal. Either the principal vector with the maximum principal curvature or the one with the minimum principal curvature could be chosen in the UNSDLA.

By storing the average of the shape descriptors, instead of the integration of them over a connected surface patch corresponding to a tessellated cell, the requirement that all the tessellated cells on the unit sphere must have the same area is avoided.

After segmentation of a surface into regions with constant signs of Gaussian curvature and along discontinuities of orientation, the mapping from a small connected neighborhood on a surface region to a tessellated cell on the unit sphere is always one-to-one. However, for a whole segmented surface region, there may exist several non-adjacent patches whose unit normals fall within the same tessellated cell on the unit sphere. In other words, there may exist several nodes in a UNSDLA whose unit normals fall within the same tessellated cell on the unit sphere.

Even after segmentation of the surfaces of an object model, a segmented surface region may still not be totally visible by the sensor from a single direction. The UNSDLA of a surface of an object model can be constructed using range images of the surface from multiple viewing directions. Range images from different directions should be registered so that identical points that have already occurred in one range image are removed from other range images. Also, the connectivity of points between range images should be identified. The UNSDLA of surfaces of an object model may also be constructed from other representations as well, such as analytical representation, planar and quadratic surface patch approximations.

Two possible tessellations of the unit sphere for constructing the UNSDLA are discussed below.

3.2.1. UNSDLA by Regular Polyhedra Tessellation of the Unit Sphere

A tessellation of the unit sphere can be obtained by projecting a regular polyhedron onto the unit sphere [16-l& 351. In this tessellation, all the cells have

4The unit normal image of a surface patch near the boundary of a surface may not fully occupy a tessellated cell, although the mapping will still be one-to-one.


the same area, the same shape, and the same relation to their neighboring cells. There exist rotations that can bring the cells of the tessellation into coincidence with themselves.

However, there are only five regular polyhedra. It is difficult to increase the sampling rate, that is, to make a tessellation with a finer angular resolution. In other words, it is difficult to increase the accuracy of the representation.

In EGI approaches, both the model and the sensed surfaces are represented by EGIs. Similarly, both model and sensed surfaces can be represented by UNSDLAs. The UNSDLA of a sensed surface can be matched to the UNSDLA of a model by rotating one of the UNSDLAs to minimize a matching error. This can be done because there exist rotations that can bring the cells of the tessellation on the unit sphere into coincidence with themselves. However, this results in a discretization of the rotation space. For example, an icosahedron is commonly used which has 20 triangular faces, leading to only 60 possible rotations. As a result, the space of all possible rotations is discretized and the match of two surfaces can only be searched over these 60 rotations [16]. The discretization of the space of all possible rotations results in low accuracy in matching and orientation determination. The number of rotations can be increased by obtaining offset rotations. The price paid is that the algorithm then calls for the generation of one object model for each offset rotation, which is computationally very expensive.

3.2.2. UNSDLA by Approximately Uniform Tessellation of the Unit Sphere

The unit normal orientation space can be sampled by a much simpler tessellation that is approximately uniformly distributed on the unit sphere. With this simple tessellation, to be presented in the next section, the sampling rate, and thus the accuracy of the representation, can be easily adjusted.

To overcome the shortcoming of discretization of the rotation space, we do not construct the UNSDLA for a sensed surface. Only the model surfaces are represented by the UNSDLA. The unit normals, one of the principal vectors, and the principal curvatures, or the Gaussian curvatures, are computed for every point of the range image of a sensed surface. They only need to be computed once. It is assumed that a sensed surface shape is sampled over a regularly spaced 2D rectangular grid in the range image. We associate the unit normal and the shape descriptors with the corresponding point of the 2D rectangular grid. This 2D array can be indexed in any way as long as the connectivity of the points on the surface is not changed. This array of unit normals and shape descriptors is then matched to the UNSDLA of a model surface. This allows continuous rotation of the sensed surface and accurate orientation determination, since rotation space is not discretized.

In the following, we will discuss in more detail this simple tessellation and the matching algorithm.

3.3. UNSDLA by Approximately Uniform Tessellation of the Unit Sphere

The matching algorithm to be described in the following sections requires that a tessellation ofthe unit sphere should satisfy the following requirements:

(1) It should tessellate the unit sphere finely enough. The sampling rate, that is, the angular resolution of the tessellation, should be easy to adjust, so that it can be adapted to different application requirements.


FIG. 1. An example of the distribution of the tessellation cells on the upper hemisphere with N = 40.

(2) The tessellation cells should be uniformly distributed. (3) When matching, it should be easy to determine to which cell a point on the

unit sphere belongs. It should be determined without search. (4) The tessellation cells should be easily put into a 2D array form to simplify

the data structure and the matching process.

Regular polyhedra tessellations such as those in [16-X31 satisfy requirements (2). But as pointed out earlier [16-H], such a tessellation is too coarse and there are only five possible tessellations. It needs to be further subdivided to satisfy (1). Once it is subdivided, the tessellation is only approximately uniform. Moreover, a regular polyhedron tessellation does not satisfy requirements (3) and (4). A simple tessellation is chosen which satisfies Cl), (31, and (4) and approximately satisfies (2).

Let r = [ri, r2, rs], r3 2 0, be a unit vector located at the center of the unit sphere pointing to a point on the upper hemisphere. Let 0 = arccos(r,), A = arccos(r2), r 2 8, A 2 0. A tessellation of the upper hemisphere is given by

An example of the distribution of this set of tessellation cells on the upper hemisphere with N = 40 is shown in Fig. 1. It can be seen that this set of tessellation cells are approximately uniformly distributed over the upper hemisphere. Similarly, the lower hemisphere can be tessellated. If the equator is included in the upper hemisphere, it will be excluded from the lower hemisphere. It is obvious that the sampling rate of this tessellation can be easily adjusted.

3.3.1. Mapping Matrices

The above simple tessellation of the unit sphere can by easily represented in a matrix form with one matrix for the upper hemisphere and one matrix for the


lower hemisphere. During surface matching, the two mapping matrices are used to establish the correspondence between a model surface and a sensed surface.

The mapping matrices are constructed for every segmented region of an object model surface. Every element in the mapping matrices corresponds to a cell of the tessellation on the unit sphere. If there exist nodes in the UNSDLA whose unit normals fall within a cell on the unit sphere, the corresponding element in the mapping matrices wiI1 contain pointer(s). The pointer points to the location of the node in the UNSDLA where the associated shape descriptors are stored. The unit normals of a surface patch represented by a node are specified by the tessellated cell associated with the element in the mapping matrices. Note that an element in the matching matrices may contain more than one pointer pointing to multiple nodes when the Gaussian map of the surface is not one-to-one.

When matching a sensed surface to an object model surface, every unit normal of the sensed surface is mapped to a tessellated cell on the unit sphere. The average shape descriptors and vectors are computed for all connected surface patches whose unit normals fall within a tessellated cell on the unit sphere. The corresponding node in the UNSDLA of the object model surface is found through the mapping matrices. A matching error between the corresponding shape descriptors and vectors is then computed.

In the matching process, the unit normal vectors of the sensed surface are rotated. After every rotation, the correspondence between the model and the sensed surfaces established through the mapping matrices is changed, and so is the matching error. Note that for every rotation of the unit normal vectors of the sensed surface, the principal vectors, if they are present, should also be rotated in the same way.

For this simple tessellation, there is no rotation that could bring the tessellated cells into coincidence with themselves. Therefore, a UNSDLA for a sensed surface is not constructed. Instead, the unit normals (and the associated principal vectors) of a sensed surface are rotated and are sampled every time after a rotation of the unit normal vectors. With a regular polyhedron tessellation, when only rotations that could bring the tessellated cells into coincidence with themselves are considered, the space of all possible rotations is discretized as a result [16].

The advantages of this simple tessellation and matching scheme are that the rotation space is not discretized and the tessellation of the unit normal space is very simple. In addition, the angular resolution of the tessellation, thus the accuracy of the UNSDLA representation, can be easily adjusted. The match can be searched over the space of all possible rotations. This allows more accurate orientation determination and matching. It also facilitates the searching of the minimum of a matching error by a minimization algorithm.

In summary, the UNSDLA representation and matching scheme described above have the following features:

(1) The representation is unique for elliptic and hyperbolic surfaces even when the Gaussian map of the surface is not one-to-one. This is because the Gaussian maps of elliptic and hyperbolic surfaces are always one-to-one in a small neighborhood, and the connectivity of points on the surface is preserved.

(2) A sensed surface can be rotated by any 3D rigid rotation to minimize a matching error. The rotation space is not discretized.

DIFFERENTIAL GEOMETRY APPROACH

FIG. 2. An example using a 2D curved shape to demonstrate the adaptive sampling characteristics of the UNSDLA. (a) is the boundary of an F-19 aircraft. The UNSDLA of the shape is constructed using sampling intervals Aa = 5” and lo”, respectively. The UNSDLA sampling points are superimposed on the original discrete boundary, as shown in (a) and (b); (cl and (d) show the shapes reconstructed from the UNSDLAs with A(Y = 5” and lo”, respectively.

(3) The tessellation of the unit sphere is very simple. The accuracy of the UNSDLA representation and the matching result can be easily adjusted by adjusting the angular resolution of the tessellation.

(4) It simplifies the matching of surfaces by separating the translational and rotational variables.

(5) The representation provides proper normalization of surface shape descriptions to match similar surface shapes. Since the unit normals are invariant under scaling, correspondence between surfaces with same shape but different sizes can be easily established.

3.3.2. Adaptive Sampling

Sampling the unit normal space by a set of (approximately) uniformly spaced points on the unit sphere has the effect of adaptive sampling of a surface. As a result, a surface is sampled with the sampling rate adapted locally to the angular changes of the unit normals. Slowly curving regions are sampled sparsely, and fast curving regions are sampled densely. A range image is a sampling of surfaces in a scene over a regularly spaced grid. In a range image, a nearly flat surface patch is sampled at the same rate as a fast curving surface patch. In constructing a UNSDLA, a surface is re-sampled adaptively.

For easy illustration, an example using a 2D curved shape is given in Fig. 2 to demonstrate the adaptive sampling characteristics of the UNSDLA. The UNSDLA can be easily applied to 2D shapes 1471. In the 2D case, the Gaussian map is on a


unit circle instead of a unit sphere, and the UNSDLA for planar curve is only 1D. The unit normal orientation space is sampled with evenly spaced sampling points on the unit circle. Let the angular interval of two adjacent sampling points on the unit circle be A.cr. Shown in Fig. 2a is the boundary of an F-19 aircraft. There are originally 1306 sampling points. The UNSDLA of the shape is constructed using sampling intervals Acr = 5” and lo”, respectively. The UNSDLA sampling points are superimposed on the original discrete boundary, as shown in Figs. 2a and b. The UNSDLA of the shape has only 183 and 126 terms in the two cases, respectively. They are only 14.01 and 9.65% of the number of points in the original discrete boundary. As can be seen from Figs. 2a and b, slowly curving regions are only sampled sparsely, whereas fast curving regions are sampled densely. To illustrate the uniqueness of the representation, Figs. 2c and d show the shapes reconstructed from the UNSDLAs with Aa = 5” and lo”, respectively.

3.4. Matching of Su$ace Shapes rlia Gaussian Map

A sensed surface is matched to an object model by using the Gaussian map, namely, the mapping matrices, to establish the correspondence. A matching error is computed between the principal vectors and the principal curvatures, or the Gaussian curvatures, of the sensed surface patches and the corresponding nodes in the UNSDLA of the model surface. A matching error should be a monotonically increasing function of the absolute angular difference between the corresponding principal vectors and the absolute difference between the corresponding curvatures. The unit normals of the sensed surface are rotated to change the correspondence so as to minimize the matching error. If a match is found, the rotation gives the orientation of the sensed surface relative to the object model surface. Since the descriptors are local, part of a surface can be matched to the whole surface shape. Note that only matching of surfaces with the same sign of Gaussian curvature needs to be considered.

When matching a sensed surface to an object model surface, we choose to rotate the unit normals (and the principal vectors) of the sensed surface, instead of those of the model surface, to minimize the matching error. In this way, the UNSDLA and mapping matrices of the object model surfaces need not be changed in the matching and recognition phase. Also, if parallel matching with multiple models is possible, one only needs to rotate the sensed shape, instead of rotating all the models.

Matching of surfaces with Gaussian maps not one-to-one can be accomplished using the UNSDLA, since a UNSDLA is still a unique representation even when the Gaussian map is not one-to-one. An adjacency constraint is used to resolve the multiple correspondence problem in matching surfaces with Gaussian maps not one-to-one. That is, surface patches adjacent in the array of unit normals and shape descriptors of a sensed surface should be matched to nodes that are connected in the UNSDLA of an object model surface. This is because (1) the connectivity of patches on the two surfaces are preserved in the array and the UNSDLA, and (2) the mapping 4 = (VI-’ . R . V: M -+ &? is continuous and locally one-to-one for hyperbolic and elhptic surfaces.

A gradient descent or a quasi-Newton type of minimization algorithm can then be applied to minimize the matching error. Further details of the mapping matrices, the matching process, and the minimization formulation are given in (371.


We make the following two comments on the matching process:

(1) In robotics applications, quaternions are widely used because they reduce computation and storage. However, one should note that this depends on the computation involved. For example, Faugeras and Hebert [19] used quaternions to compute a rotation from a set of primitive pairs. The primitives included points, line segments, planes, and quadrics (they did not address the matching of quadrics using quaternions). However, we are not matching a set of primitives. We are not

FIG. 3. (a) shows an object model surface S,; (b) is the sensed surface S,.


seeking a rotation to bring a set of vectors into coincidence with another set of vectors. Instead, the normal vectors are only used as a local parameter to establish correspondence between two sets of vectors and shape descriptors. The matching algorithm involves rotating a large set of 3D vectors. It is known that to rotate a 3D vector requires 12 additions and 22 multiplications for a quaternion representation and only requires 6 additions and 9 multiplications with a rotation matrix representation [44]. Naturally, rotation matrix, instead of quaternion, is used in the current matching algorithm. Computation of the rotation will be further discussed in Section 5.

FIG. 4. (a) shows the principal vectors of surface S, mapped on to the unit sphere. The principal curvatures of St are shown in (b) and Cc). The upper hemisphere has been plotted on a R* coordinate system.


(2) Since recognizing objects with the same shape and different scales is important, matching errors should be defined using ratios between principal curvatures, or Gaussian curvatures. Attention should be paid to the fact that if the denominator curvature value is near zero at a point, it may blow up the whole match. This can be taken care of by adding a constant to both the numerator and the denominator.

3.5. Examples

In the following examples, it is assumed that surfaces have been segmented, for example, using the segmentation algorithm in Besl and Jain [24]. Only examples of matching the same type of surfaces are presented. Experiments using real range images are currently underway.

An example of matching two hyperbolic surfaces is first presented, followed by an example of matching two elliptic surfaces. The size of the sensed range images are 128 X 128. The mapping matrices are 41 X 41. Matching is performed at multiresolution levels to speed up the convergence of the matching algorithm. Three resolution levels, coarse, medium, and fine, are used. The surfaces used in the examples have constant signs of Gaussian curvature and correspond to surfaces represented by the matching nodes of the relational graphs.

Figure 3a shows an object model surface S,, and Fig. 3b shows the sensed surface S,. Surface S, is the visible part viewed from the negative z-axis direction of the surface obtained by scaling surface S, by a factor of i, rotating (Y = 45” in the x - y plane, /? = 20” in the y - z plane, and y = 0” in the x - z plane. Because of the 20” rotation in the y - z plane, there is occlusion, for some points on the surface are no longer visible from the negative z-axis direction. The matching algorithm finds the rotation matrix R and the scaling factor. The scaling factor is found to be k = 0.4893561. The rotation angles are computed as LY = 42.3897410”, p = 23.1093218”, y = -2.4139287”.

FIG. 4-Continued


To illustrate the matching results, the principal vectors and principal curvatures of surface S, and S, are mapped onto the unit sphere and are shown in Figs. 4 and 5. The tessellation of the upper hemisphere is the same as that described for the mapping matrices. For the principal curvatures, the upper hemisphere has been plotted on a R* coordinate system with 8 and A as the coordinate axes in the range ( - rr, ~1. The unit normals and principal vectors of surface S, is rotated by the rotation matrix R which minimize the matching error. After the rotation, the principal vectors and principal curvatures are again mapped onto the unit sphere and are shown in Fig. 6. Note that both principal vectors are shown in the figures,

b *x

FIG. 5. (a) shows the principal vectors of surface S, mapped on to the unit sphere. The principal curvatures of S, are shown in (b) and Cc) with the upper hemisphere plotted on a R* coordinate system.


FIG. 5-Continued

but actually only one of them is used in the representation and matching. If Fig. 6a is superimposed on Fig. 4a, it can be seen that the rotated principal vectors of surface S, mapped onto the unit sphere matches the principal vectors of surface S, mapped onto the unit sphere. It can also be seen that some principal vectors that are in Fig. 4a are no longer in Fig. 6a because of occlusion. If Figs. 6b and c are superimposed on Figs. 4b and c, it can be seen that the values of the principal curvature of surface S, rotated by R are approximately twice those of surface S, in height at the corresponding points.

The next example illustrates the matching of two elliptic surfaces. The model surface is a whole ellipsoid as shown in Fig. 7a. The sensed surface is the range image of only part of the visible surface of an ellipsoid similar to the ellipsoid in Fig. 7a but twice as large and oriented differently. It is rotated (Y = 45” in the x - y plane, p = -25” in the y - z plane, and y = 15” in the x - z plane. The sensed surface is shown in Fig. 7b. Because of the symmetry in this example, the sensed surface may be matched to different part of the object model surface. Actually, two different rotation matrices are found by the matching algorithm (depending on the initial guesses provided). The scaling factors are 2.3517865 and 1.8013971 respectively for the two rotations found. The rotation angles for the two cases are computed as CY~ = 41.2682391”, p, = -21.7528903”, yr = 17.4253372”, and CX* = - 133.1738902”, & = - 27.8340510”, y2 = 18.0170037”, respectively.

4. REPRESENTATION AND MATCHING OF DEVELOPABLE SURFACE SHAPES

Developable surfaces are one of the most commonly encountered type of surfaces in industrial environment and everyday life. The classical classification of developable surfaces [461 are planes, generalized cylinders,5 generalized cones and

‘The generalized cylinder here is different from that used in volumetric representation of objects.

100

a

LIANG AND TODHUTER

FIG. 6. The unit normals and principal vectors of surface S, is rotated by the rotation matrix R which minimize the matching error. After the rotation, the principal vectors and principal curvatures are again mapped on to the unit sphere. (a) shows the rotated principal vectors of surface Sz mapped on to the unit sphere. The principal curvatures of S, after the rotation are mapped on to the upper hemisphere shown in (b) and (c).

tangent developables. There are other more complex developable surfaces [46]. But the above four types are the most common developable surfaces in apphca- tions. Both principal curvatures of a planar surface patch are zero and its representation is easy. Only the rest three types are considered in this section.

The representation and matching algorithm developed in the preceding section do not directly apply to developable surfaces, since the Gaussian map of any


FIG. 6-Continued

neighborhood of a point on a developable surface is not one-to-one. Actually, the Gaussian map of a developable surface is a curve on the unit sphere. The Gaussian map of any line of curvature with nonzero principal curvature coincides with this curve. Lines of ruling, instead of lines of curvature with nonzero principal curvature, should be used in computation of the representation. A line of ruling is much easier to determine than a line of curvature with nonzero principal curvature. A line of ruling is a straight line and the unit normal along it is constant. Figure 8 shows the lines of ruling detected on simulated range images of a generalized cone and a generalized cylinder.

Once the lines of ruling are determined, a developable surface can be classified into one of the three types. Generalized cylinder, generalized cone, and tangent developable have quite different properties. Description, representation, and matching of developable surfaces will be simplified by first classifying a developable surface into one of the three types. Methods for such classification are provided [37].

The nonzero principal curvature along a line of ruling of a generalized cylinder is a constant. A generalized cone can be parameterized as

f(s,t) = I/+ t(c(s) - v), (7)

where I/ E R3 is the vertex and c(s) is a 3D curve. A tangent developable can be parameterized as

f(s,t) = c(s) + w’(s), (8)

where c(s) is a 3D curve and is called the generating curve of a tangent developable.

A common feature in the above parameterization of generalized cone and tangent developable is that a line of ruling is given by fixing s and allowing only t


FIG. 7. An example illustrating the matching of two elliptic surfaces. (a) is a model ellipsoid. (b) shows the sensed range image of only part of the visible surface of an ellipsoid twice as large as (a) and oriented differently.


FIG. 8. Lines of ruling detected on a generalized cone and a generalized cylinder. The generalized cylinder also illustrates that even after segmenting a developable surface into regions with the sign of the nonzero principal curvature strictly positive or negative, there may still be points on a line of curvature with nonzero principal curvature having the same unit surface normal.

to vary. It can be proved that for a generalized cone and a tangent developable (without umbilic points), the nonzero principal curvature can be expressed as

where B(s) is a certain function depending only on S.


THEOREM 3. The nonzero principal curvature of a generalized cone or a tangent developable at a point of a line of ruling is inversely proportional to the distance from the point to the vertex of the cone or to the generating curve of the tangent developable along the line of ruling.

This can be shown as follows. Consider a generalized cone. Let D denote the Euclidean length from the vertex to the curve c along a given line of ruling. Let d(P) be the distance from point P on the line of ruling to the vertex. The parameter t in Eq. (7) can be redefined as t = d(P)/D, and the result is obtained by substituting it into Eq. (91,

K2 = B(s)D/d(P).

The parameter t in Eq. (8) can be similarly defined to prove the same result for a tangent developable surface.

Based on Theorem 3 and the above analysis, shape descriptors, and representations for the three types of developable surfaces are developed. They should satisfy the same criteria for those of the hyperbolic and elliptic surfaces. The representation developed is also a unit normal and shape descriptors list array representation. The UNSDLA for a developable surface is a 1D array. The ith element of the array contains the unit normal n(i), the principal vector p,(i) with zero principal curvature, which is the direction of the line of ruling, the nonzero principal curvature KJi). For generalized cone and tangent developable surfaces, it also contains d(i), the distance from the point where the above descriptors are computed to the vertex or the generating curve, and (b(i), e(i)>, the distance from the beginning and ending points of the ith line of ruling to the vertex or the generating curve. Each element in the 1D UNSDLA corresponds to a line of ruling sampled on the developable surface.

A UNSDLA representation uniquely describes a developable surface shape. An indicator of the type of the developable surface should also be included in the representation of a model developable surface.

Both model and sensed developable surface shapes are also segmented into regions with nonzero principal curvature K~ > 0, K* < 0, and K~ = 0. Even after segmenting a developable surface into regions with the sign of the nonzero principal curvature strictly positive or negative, there may still be points on a line of curvature with the nonzero principal curvature having the same unit surface normal. An example is the generalized cylinder shown in Fig. 8. However, after the segmentation, for a large percentage of developable surface shapes, the Gaussian maps of a line of curvature with nonzero principal curvature will be one-to-one.

For a sensed developable surface to be congruent or similar to a model developable surface, they must be of the same type of developable surfaces and the nonzero principal curvatures must have the same sign. These two conditions are higher level descriptions than the UNSDLA. By matching these two descriptors, a large number of objects that are impossible to match are easily screened out. This can result in a significant saving of computation.

Unlike the hyperbolic and elliptic surfaces, where the UNSDLA is 2D, the matching of the 1D UNSDLA is easier. Both model and sensed developable surfaces are represented by their UNSDLAs. This is the same as the representa-


tion and matching for planar curves using UNSDLA [47]. The matching can be done without explicitly using the Gaussian map, although, conceptually, the mapping between the two surfaces is still established via the Gaussian map.

The representation of a developable surface is still unique even when there are points having the same unit surface normal on a line of curvature with nonzero principal curvature. Therefore, such developable surfaces can be matched. No adjacency constraint is necessary in the matching algorithm. This is because the UNSDLA for a developable surface is lD, the propagation of the mapping relation between elements of two UNSDLAs is one-directional and sequential. Implemen- tation of the matching algorithm for developable surfaces and experiments using real range images are currently underway.

Another theorem on the description of developable surfaces is proved. A merit of this theorem is that its proof is constructive. The proof is given in [37].

THEOREM 4. A nonplanar developable surface patch is uniquely determined up to a translation by the curvature K and torsion r, as junctions of the arclength s, of a single line of curvature with nonzero principal curvature, and the unit normal along this line of curvature, also as a function of the arclength. The boundary of the su$ace patch is determined by the distance from the boundary to the given line of curvature along the lines of ruling.

This theorem provides another efficient method for data compression and representation for developable surface shapes. It shows that it is possible to reconstruct a developable surface from the description of a single line of curvature and the unit surface normals.

5. CONCLUSION AND DISCUSSION

The major contributions and conclusion of this paper are:

(1) Two corollaries to the fundamental theorem of surface theory are proved. Theorem 1 proves the invariance of the first and second fundamental coefficients when lines of curvatures are the parameter curves. Theorem 2 proves that if a mapping preserves the intrinsic distance along the principal directions, in addition to preserving the eigenvectors and eigenvalues of the shape operator, it is necessarily an isometry. The two theorems apply to hyperbolic, elliptic, and developable surfaces. They provide a theoretical basis for general surface shape description and recognition by a differential geometry approach.

Based on the analysis, geometric descriptors that uniquely determine a surface shape and are invariant to change of parameterization are theoretically identified. From Theorem 2 it is known that the principal vectors and the principal curvatures can be used as the desired shape descriptors. In applying Theorem 2 to describe a discrete surface in a range image, the principal curvatures serve to determine the shape of the surface patch around a sampling point, while the principal vectors specify the relative orientations of surface patches at neighboring sampling points.

(2) Unit normal and shape descriptors list array (UNSDLA) representations are developed. The UNSDLA preserves not only the adjacent relation of points or patches of a surface, but also the direction and order in which the points or patches are traversed in a connected path. The importance of the direction and order of the connectivity is emphasized. Because the connectivity is preserved, the


UNSDLA can represent all types of surfaces in an unified representation. It extends the scope of unique representation of the EGI to all types of surfaces, in addition to preserving the advantages of the EGI.

Matching of surfaces is formulated as an optimization problem. The minimization can be carried out in the space of all possible rotations. The orientation and scaling factor of a sensed surface shape relative to an object model are determined by the matching algorithm.

(3) Descriptors, representations, and matching procedures for developable surfaces are developed. Three types of developable surfaces are considered. In matching developable surfaces, the type of the developable surface and the sign of the nonzero principal curvature are first matched. The UNSDLA representation of a developable surface is 1D. Two theorems on developable surfaces are proved.

Some features of the representation and matching algorithm presented in this paper include:

(1) The UNSDLA of a hyperbolic or elliptic surface is still a unique representation even when the Gaussian map is not one-to-one. The UNSDLA of a developable surface is also a unique representation even when the Gaussian map of a line of curvature with nonzero principal curvature is not one-to-one.

(2) The representations developed separate the translational and rotational variables. The representation is normalized in the sense that surfaces with same shape but different sizes can be matched.

(3) For hyperbolic and elliptic surfaces, the rotation space is not discretized. The tessellation of the unit sphere is very simple and facilitates the matching process. The match can be searched over the space of all possible rotations. This allows more accurate orientation determination and matching. It also facilitates searching for the minimum of the matching error by minimization algorithms. Discretization of the space of unit normal orientation has the advantage of adaptive sampling and can produce a more efficient and compact representation.

(4) Since the geometric descriptors used are local characteristics, the matching algorithms can match surfaces with occlusion and match part of a surface to the whole surface.

We believe that a surface, no matter elliptic, hyperbolic, or developable, could be uniquely reconstructed from its UNSDLA with a simple algorithm. This is a topic for future investigation.

In this paper, different descriptors are used for elliptic and hyperbolic surfaces. Unit normals and associated Gaussian curvatures are used to represent elliptic surfaces. One could use the principal vectors and principal curvatures to describe both elliptic and hyperbolic surfaces. Although using Gaussian curvatures is faster in matching elliptic surfaces, Gaussian curvature computation tends to be more sensitive to noise than the principal curvatures. This is because the Gaussian curvature is the product of the two principal curvatures.

A New Matching Algotithm

The UNSDLAs are matched in this paper using a minimization algorithm. A more efficient and robust matching algorithm described briefly below is currently under implementation.


After the relational graphs of the sensed and model surfaces are matched, the congruence or similarity of the surfaces represented by a pair of matching nodes is determined as follows:

1. Find the surface patches, corresponding to a tessellated cell, with the maximum and minimum of the two principal curvatures on the sensed surface M, denoted as max{K,}, min{Kr}, max{KJ, min{rc,). At least two patches can be identified. They are referred to as candidate matching patches on M. To simplify the presentation, assume there is no multiple patch achieving the same maximum or minimum. The algorithm can be easily modified if this is not the case.

2. Find nodes, within a given tolerance, in the UNSDLA of the model surface M with principal curvatures having the same value as IEiX(K1), min{rc,}, max{K2}, min{rc,}, or having a constant scaling factor with them. Those nodes and the corresponding candidate matching patches on M are referred to as candidate matching pairs.

If such nodes cannot be found in the UNSDLA of a, the two surfaces cannot be congruent or similar.6

If candidate matching pairs are found, then a rotation is to be found to match the tessellated cells representing the unit normals, and the unit principal vector pr, on M to their counterparts on m. If the two surfaces are congruent or similar, there must exist a single rotation that maps all the tessellated cells and the vectors on M to the corresponding tessellated cells and vectors on &?. If a single rotation cannot be found within a given tolerance, the two surfaces cannot be congruent or similar.

3. If there exists such a rotation, additional candidate matching pairs are identified. There are many possible ways to choose candidate matching patches on M. If all the points on surface M is exhausted, it is concluded that the two surfaces are congruent or similar. The process will stop at any stage when candidate matching pairs cannot be found, or a single rotation to match all the vectors cannot be found. In either case, the two surfaces cannot be congruent or similar.

Attention should be paid to surfaces with Gaussian maps not one-to-one. The adjacency constraint should be applied to deal with matching of points on a surface having the same unit normal.

ACKNOWLEDGMENTS

The first author thanks Dr. Paul Besl of General Motor Research Laboratories for his helpful comments on an earlier version of this paper. Most of this work was performed at the Image Processing and Pattern Recognition Laboratory at the University of Pittsburgh, and was supported in part by NIH Grants DE01697-25 and NS16337-07.

REFERENCES 1. D. Nitzan, A. E. Brain, and R. 0. Duda, The measurement and use of registered reflectance and

range data in scene analysis, Proc. IEEE 65, 1977, 206-220.

‘Note that we picked the maximum and minimum of the principal curvatures from the sensed surface. Therefore, if the sensed surface is congruent or similar to the model surface, there must exist nodes in the UNSDLA of the model surface with the same principal curvatures (within a given tolerance). However, one could not say the same if the maximums and minimums are picked on the model surface.


2. R. A. Lewis and A. R. Johnston, A scanning laser rangefinder for a robotic vehicle, in Proceedings, 5th IJAI., 1977, QQ. 762-768.

3. R. A. Jarvis, A laser time of flight range scanner for robotic vision, IEEE Trans. Pattern Anal. Mach. Intell. PAMI-5, 1983, 505-511.

4. T. Okada, Development of an optical distance sensor for robot, ht. J. Rob. Res. 1, No. 4. 1982, 3-14.

5. P. W. Will and K. S. Pennington, Grid coding: A novel technique for image processing, Proc. IEEE 60, 1972, 669-680.

6. J. L. Posdamer and M. D. Altschuler, Surface measurement by space-encoded projected beam system, Comput. Graphics Image Process 18, 1982, 1-17.

7. J. Le Moigne and A. M. Waxman, Projected light grids for short range navigation, in Proceedmgs, 7th Int. Conf. Pattern Recog., July, 1984, QQ. 203-206.

8. H. K. Nishihara, Practical real time imaging stereo matcher, Opt. Eng. 23, 1984, 536-545. 9. Y. Yakimovsky and R. Cunningham, A system for extracting three dimensional measurements from

a stereo pair of TV cameras, Comput. &ion Graphics Image Process. 7, 1978, 152-210. 10. K. Prazdny, Detection of binocular disparities, Biol. Cybem. 52, 1985, 93-99. 11. H. H. Baker, Edge based stereo correlation in Proceedings, Image Understanding Workshop,

DARPA, University of Maryland, Apr. 1980, pp. 86-91. 12. R. J. Woodham, Photometric method for determining surface orientation from multiple images.

Opt. Eng. 19, 1981, 1399144. 13. M. Magee, B. A. Boyter, C. C. Chen, and J. K. Aggarwal, Experiments in intensity guided range

sensing recognition of three dimensional objects, IEEE Trans. Pattern Anal. Mach. Intell. PAMI-7, 1985, 629-637.

14. R. A. Jatvis, Perspective on range finding techniques for computer vision, IEEE Trans. Pattern Anal. Mach. Intell. PAMI-5, No. 2, 1983, 122-139.

15. B. O’Neill, Elementary Differential Geometry, Academic Press, New York, 1966. 16. P. Brou, Using the Gaussian image to find the orientation of objects, Int. J. Rob. Res. 3, No. 4,

1984, 89-125. 17. B. K. P. Horn and K. Ikeuchi, Mechanically manipulating randomly oriented parts, Sci. Amer. 251,

August 1984, 100-113. 18. K. Ikeuchi et al., Picking Up an Object from a Pile of Objects, MIT AI Memo 726, May 1983. 19. 0. D. Faugeras and M. Hebert, The representation, recognition and positioning of 3-D shapes from

range data, in Techniques for 3-D Machine Perception (A. Rosenfeld, Ed.), pp. 13-52, North- Holland, Amsterdam, 1986.

20. D. T. Kuan and R. J. Drazovich, Model based interpretation of 3-D range data, in Techniques for 3-D Machine Perception, (A. Rosenfeld, Ed.), pp. 219-230, North-Holland, Amsterdam, 1986.

21. W. E. L. Grimson and T. Lozano-Perez, Model-based recognition and localization from sparse range data, in Techniques for 3-D Machine Perception (A. Rosenfeld, Ed.) pp. 133-148, North-Holland, Amsterdam, 1986.

22. B. C. Vemuri, A. Mitiche and J. K. Aggarwal, Curvature-based representation of objects from range data, Image Vision Comput. 4, No. 2, 1986, 107-114.

23. P. J. Besl and R. C. Jain, Invariant surface characteristics for 3D object recognition in range images, Comput. Vision Graphics Image Process. 33, No. 1, 1986, 33-80.

24. P. J. Besl and R. C. Jain, Segmentation through symbolic surface descriptions, in Proceedings, Comput. I’,ion Pattern Recog. ‘86, Miami Beach, Florida, June 1986, pp. 77-85.

25. G. Medioni and R. Nevatia, Description of 3-D surfaces using curvature properties, in Proceedings, Image Understanding Workshop, New Orleans, LA., DARPA, Oct. 1984, pp. 291-299.

26. T. J. Fan, G. Medioni and R. Nevatia, Description of surfaces from range data using curvature properties, in Proceedings Comput. Vision Pattern Recog. ‘86, Miami Beach, Florida, June 1986, QQ. 86-91.

27. T. C. Henderson, Efficient 3-D object representation for industrial vision systems, IEEE Trans. Pattern Anal. Machine Intell. PA&II-S, 1983, 609-617.

28. M. Oshima and Y. Shirai, Object recognition using three-dimensional information, IEEE Trans. Pattern Anal. Mach. Intell. PAMI-5, No. 4, 1983, 353-361.

29. A. Mitiche and J. K. Aggarwal, Detection of edges using range information, IEEE Trans. Pattern Anal. Mach. Intell. P&II-S, 1983, 174-178.


30. G. J. Agin and T. 0. Binford, Computer description of curved objects, IEEE Trans. Comput. C-25, 1976, 439-449.

31. R. Nevatia and T. 0. Binford, Description and recognition of curved objects, Art& Intell. 8, 1977, 77-98.

32. K. Sugihara, Range data analysis guided by a junction dictionary, Art& Intell. 12, 1979, 41-69. 33. 0. D. Faugeras, et al., Object representation, identification and positioning from range data, in

Robotics Research: 1st Int. Symp., (M. Brady and R. Paul, Eds.), pp. 413-424, MIT Press, Cambridge, MA, 1984.

34. M. Brady, J. Ponce, A. Yuille and H. Asada, Describing surfaces, Comput. Ksion Graphics Image Process. 32, No. 1, 1985, l-28.

35. B. K. P. Horn, Extended gaussian image, Proc. IEEE 72, 1984, 1671-1686. 36. J. J. Little, Recouering Shape and Determining Attitude from Extended Gaussian Images, Ph.D. thesis,

Dept. of Comput. Sci., University of British Columbia, 1985. 37. P. Liang, Measurement, Orientation Determination and Recognition of Surjace Shapes in Range

Images, Ph.D. dissertation, Dept. of Elect. Eng., University of Pittsburgh, 1987. 38. A. V. Pogorelov, Extrinsic Geometry of Conuw Surfnces, Translation of Math. Monographs, Vol. 35,

Amer. Math. Sot., Providence, RI, 1978. 39. I. M. Anderson and J. C. Bezdek, Curvature and tangential deflection of discrete arcs, IEEE Trans.

Pattern Anal. Mach. Intell. PAMI-6, 1984, 27-40. 40. M. Brady, H Asada, Smoothed local symmetries and their implementation, in Robotics Research:

1st Int. &np., (M. Brady and R. Paul, Eds.), pp. 331-354, MIT Press, Cambridge, MA, 1984. 41. M. Brady, Criteria for shape representations, Human and Machine Vision (J. Beck and

A. Rosenfeld, Eds.), Academic Press, New York, 1983. 42. J. A. Thorpe, Elementary Topics in Differential Geometry, Springer-Verlag, New York, 1979. 43. J. Koenderink and A. van Doorn, The shape of smooth objects and the way contours end,

Perception 11, 1982, 129-137. 44. K. S. Fu, R. C. Gonzalez and C. G. Lee, Robotics: Control, Sensing, Vision and Intelligence,

McGraw-Hill, New York, 1987. 45. D. D. Hoffman and W. A. Richards, Parts of recognition, in Visual Cognition (S. Pinker, Ed.), pp.

65-96, MIT Press, Cambridge, MA, 1984. 46. M. Spivak, A Comprehensive Introduction to Differential Geometry, Vols. I, II, III, Publish or Perish,

Boston, 1979. 47. P. Liang and J. F. Lee, 2D shape representation and recognition by 1D unit normal and shape

descriptors list array: A new approach, to appear.

Documents

Representation and recognition of surface shapes in range images: A differential geometry approach