This document contains the draft version of the following paper: T. Peng and S.K. Gupta. Algorithms for generating adaptive projection patterns for 3-D shape measurement. ASME Journal of Computing and Information Science in Engineering, 8(3), 2008. Readers are encouraged to get the official version from the journal’s web site or by contacting Dr. S.K. Gupta (skgupta@umd.edu).

Algorithms for Generating Adaptive Projection Patterns

for 3-D Shape Measurement

Tao Peng ∗ Satyandra K. Gupta †

Abstract

Point cloud construction using digital fringe projection (PCCDFP) is a non-contact

technique for acquiring dense point clouds to represent the 3-D shapes of objects.

Most existing PCCDFP systems use projection patterns consisting of straight fringes

with fixed fringe pitches. In certain situations, such patterns do not give the best

results. In our earlier work, we have shown that for surfaces with large range of

normal directions, patterns that use curved fringes with spatial pitch variation can

significantly improve the process of constructing point clouds. This paper describes

algorithms for automatically generating adaptive projection patterns that use curved

fringes with spatial pitch variation to provide improved results for an object being

measured. We also describe the supporting algorithms that are needed for utilizing

adaptive projection patterns. Both simulation and physical experiments show that

adaptive patterns are able to achieve improved performance, in terms of measurement

accuracy and coverage, as compared to fixed-pitch straight fringe patterns.

Keywords: 3-D shape measurement; point cloud; digital fringe projection; adaptive pro-

jection pattern.

∗Department of Mechanical Engineering, University of Maryland, College Park, MD 20742. Email:pengtao@umd.edu

†Department of Mechanical Engineering and Institute for Systems Research, University of Maryland,College Park, MD 20742. Email: skgupta@eng.umd.edu

1

1 Introduction

Accurate and rapid measurement of the 3-D shapes of objects is needed in many applications.

Representative applications include reverse engineering, 3-D digital replication, and part

inspection. In these applications, dense point clouds (often containing hundreds of thousands

of points) that represent the shape of object need to be acquired for surface reconstruction

and error analysis. Existing methods for point cloud acquisition can be divided into two

categories: contact methods and non-contact methods. Examples of contact methods include

Coordinate Measuring Machine (CMM), articulated coordinate measuring arm, and laser

tracking system. These methods require touch probes to make physical contact with the

objects being measured, which leads to the following limitations: slow measuring speed,

difficulty in dealing with surfaces that are soft or fragile, and the presence of normal offsets

in the measured data (i.e. the measured 3-D points are not the contact points with the surface

but the center of the touch probe’s spherical tip). Non-contact measurement methods, on

the other hand, have eliminated these limitations. Some popularly used methods in industry

include laser scanning [1], photogrammetry [2, 3, 4], and structured light [5, 6].

In the community of Computer-Aided Design (CAD), intensive research has been done on

topics related to the processing of massive point cloud data acquired by varied means. Some

of the most popular topics include: representation of point cloud data [7, 8, 9], registration

and merging of multiple point clouds [10], surface reconstruction (from point clouds) [11,

12, 13, 14], extraction of surface features [15], and scan planning [16, 17]. In this paper, we

focus on the acquisition of point cloud data instead of discussing the above topics.

Point cloud construction using digital fringe projection (PCCDFP) is a non-contact

measurement technique based on structured light [18]. Compared to laser scanning and

photogrammetry, the most significant advantage of PCCDFP technique is its high data

rate. Depending on camera resolution and hardware synchronization, PCCDFP systems can

achieve data rates in the order of million points per second. While laser scanners need to be

2

Figure 1: Schematic diagram of a PCCDFP system with one projector and one camera

mounted on CMM or similar devices to perform area scan, due to their line scanning nature,

PCCDFP systems can perform area scans by themselves. For photogrammetry based tech-

niques, it is generally needed to place reflective targets on surfaces being measured in order

to achieve good accuracy. There is no such need for PCCDFP technique. For these reasons,

PCCDFP is the preferable technique for many industrial applications where an accurate,

fast, and inexpesive shape measurement tool is needed.

The simplest PCCDFP system contains one digital projector and one camera (as shown

in Fig. 1). The basic principle behind the approach is to project known patterns on the

object using the projector, by which the object’s surface is encoded with phase values. The

camera is used to take images of the object with the known patterns imposed on it. By

using techniques such as phase-shifting to design the projection patterns, the phase map of

the surface can be acquired accurately from the resulting images. A dense point cloud that

represents the surface being measured can then be constructed using triangulation method.

Readers are referred to Ref. [19, 20, 21] for more details on PCCDFP.

PCCDFP technique uses a set of projection patterns to perform a measurement. Different

designs of patterns could result in different measurement accuracy and coverage [22]. For

applications that demand high measurement speed, e.g. 100% on-line inspection of parts,

the number of projection patterns that can be used in a measurement is constrained by the

speed of system hardware, e.g. the speed of the projector and the camera, as well as the

sychronization between the two. In such a case, an interesting research issue is how to design

3

(a) Sinusoidal fringe pattern(shown with uniformly increasedfringe pitch than actual)

(b) Image (portion view) of thesphere under the projection ofthe pattern

Figure 2: Measurement of a sphere using straight sinusoidal fringes with fixed pitch

projection patterns such that measurement accuracy and coverage can be optimized when

the number of patterns to be used is limited.

A popularly used projection pattern for PCCDFP is sinusoidal fringe pattern (as shown

in Fig. 2(a)), in which the fringes are straight and the fringe pitch is fixed. Here, term “fixed

fringe pitch” refers to the pitch of fringes in a single pattern. For systems using temporal

phase unwrapping techniques [23, 24], multiple fringe patterns with different fringe pitches

are used. However, the fringe pitch in each pattern is still a constant, although it may vary

from pattern to pattern. In certain situations, fixed-pitch fringe patterns do not give ideal

results. Fig. 2(b) shows the image (portion view) of a sphere under the projection of a

sinusoidal fringe pattern. As can be seen, at the left side of the sphere fringes are crowded

together and indistinguishable. This will result in an unresolvable area in the constructed

point cloud. This is mainly caused by the change of surface normal direction from the left

side of the sphere to the right. On the other hand, while using a carefully designed pattern

with variable fringe pitch, the resulting image shows appropriately spaced fringes and the

accuracy of measurement can be possibly improved (see Fig. 3(a) and 3(b)). We should

note that the problem with crowding of fringes in the image cannot be simply solved by

using sinusoidal fringe pattern with a much larger fringe pitch, because that will reduce the

4

(a) Adaptive pattern (shownwith uniformly increased fringepitch than actual)

(b) Image (portion view) of thesphere under the projection ofthe pattern

Figure 3: Measurement of a sphere using curved fringes with spatially varying pitch

measurement accuracy in the middle portion of the sphere.

Digital projectors based on Digital Micro-mirror Device (DMD) or Liquid Crystal Display

(LCD) are capable of generating geometrically complex projection patterns. They provide

the hardware foundation for using variable-pitch fringe patterns in PCCDFP technique.

However, the potential improvement in measurement performance that can be achieved by

using adaptive projection patterns has not been utilized so far.

In our prior work [21], we have described a mathematical framework that can use ge-

ometrically complex projection patterns. The traditional phase-shifting method has been

generalized to handle patterns with spatially variable fringe pitches. A new triangulation

algorithm was developed to convert phase maps to point clouds. By utilizing a reference

phase map, this algorithm is able to use generalized projection patterns in a seamless man-

ner and at the same time handles the projector’s lens distortions automatically. Since the

new triangulation algorithm was derived strictly from the optical geometry model of the

PCCDFP system, it provides high accuracy in point cloud constructions.

In this paper, we describe algorithms for automatically generating adaptive projection

patterns to achieve improved accuracy and coverage in point cloud acquisition. The algo-

rithm utilizes knowledge on the rough shape of the object, which can be acquired from a

5

preliminary measurement. For applications such as inspection of parts on the production

line, this method involves negligible overhead, because the adaptive pattern needs to be

constructed only once and can be used for all successive parts of approximately the same

shape.

It is worth mentioning that, the accuracy and coverage in point cloud acquisition is a

complicated issue and the possible improvement that can be achieved by using adaptive

patterns is constrained by many factors, e.g. specifications of the hardware and the shape

of the object. It is possible that in certain cases adaptive patterns do not provide signif-

icant advantage over traditional fixed-pitch fringe patterns. The use of adaptive patterns,

nevertheless, provides a new tool for achieving better measurement performance.

2 Background

When fixed-pitch fringe patterns are used, the measurement of an object using PCCDFP

includes the following steps (see Fig. 4):

• Projects phase-shifted straight fringe patterns on the object and records the images.

• Constructs an absolute phase map of the object from the images acquired.

• Constructs a point cloud from the absolute phase map of the object.

Using adaptive projection patterns requires modifications to the above procedure and

introduces additional steps. The primary new step is generation of adaptive projection

pattern based on the shape of the object being measured. A schematic diagram of the new

procedure is shown in Fig. 5. A detailed explanation of it is given below.

• Generation of adaptive projection pattern: If objects have different shapes or

they are placed at different positions, the adaptive projection patterns that could give

optimal measurement performance may be different. Hence, if the shape or position of

6

Figure 4: Schematic diagram of the measurement workflow when fixed-pitch fringe patternis used

Figure 5: Schematic diagram of the measurement workflow when adaptive projection patternis used

7

the object being measured has changed, the adaptive projection pattern needs to be

regenerated. The algorithm for the generation of adaptive patterns requires knowledge

of the approximate shape of the object, as well as its position and orientation w.r.t.

the sensor. These information can either be provided to the system in the form of

a geometric model of the object, or acquired by a preliminary measurement. In the

latter case, the measurement can be conducted using a fixed-pitch fringe pattern with a

medium number of fringes to avoid any possible unresolvable area. This measurement

is not intended to give accurate result of the object’s shape; however it is appropriate

for the generation of adaptive patterns. A detailed description of the algorithm for

generating adaptive projection patterns is given in Section 3.

• Construction of new reference phase map: The point cloud construction algo-

rithm we have used requires a reference phase map, which is an absolute phase map of a

flat plane under the projection pattern used for measurement. When a new projection

pattern is generated and is to be used in future measurements, a new reference phase

map needs to be constructed. The construction of the new reference phase map can

be done through an interpolation based approach which takes the adaptive projection

pattern as input as well as two “regular” reference phase maps that are obtained using

fixed-pitch fringe patterns. A detailed description of this algorithm is presented in

Section 4.

• Measurement of the objects: Once the adaptive projection pattern has been gen-

erated and the corresponding reference phase map has been acquired, the procedure

of measurement by using the adaptive pattern is the same as with fixed-pitch fringe

patterns. In other words, a set of phase-shifted adaptive patterns are projected on

the object and the corresponding images are recorded. From these images an absolute

phase map of the object, and hence a point cloud that represents the object’s surface,

can be constructed.

8

To use adaptive projection patterns, we have generalized the conventional phase-shifting

technique [21]. The following two step approach is used to generate fringe patterns:

• Step 1: A phase function Φ(ξ, η) is defined in the image plane of the projector, i.e.

the ξ–η plane. Φ(ξ, η) must be continuous in the ξ–η plane and monotonic in either

ξ or η direction, depending on the position of the camera w.r.t. the projector.

• Step 2: The phase function Φ(ξ, η) is converted to light projection pattern(s) by some

encoding method. Light properties that can be used for encoding include intensity,

color, etc. In practice, light intensity, without the involvement of color, is most pop-

ularly used since the intensity of light can be measured accurately by photo sensors

such as CCD. A widely used modulation function for converting phase values to light

intensities is the sinusoidal function. For digital projectors, in which light intensity is

presented in gray-levels, the sinusoidal modulation can be described using the following

equation:

I(P )(ξ, η) =I

(P )max

2[1 + sin(Φ(ξ, η))] (1)

where I(P )(ξ, η) is the gray-level in the projection pattern and I(P )max is the maximum

gray-level of the projector. Fig. 6 shows an example phase function Φ (gray-levels

represent phase values) and the corresponding projection pattern constructed using

Eqn. 1.

3 Algorithm for Automated Generation of Adaptive

Projection Patterns

In PCCDFP techniques, projection patterns are used to acquire phase maps of surfaces.

The accuracy and coverage of the phase maps acquired determine the accuracy and coverage

of the final measurement result. The main idea behind adaptive projection patterns for

9

100 200 300 400 500

50

100

150

200

250

300

350

400

450

500

(a) Phase function Φ(ξ, η) (gray-levels represent phase values)

(b) Projection pattern con-structed, I(P )(ξ, η)

Figure 6: An example of the construction of generalized projection pattern

measuring a particular surface is to manipulate the local fringe pitch in the pattern such

that the accuracy and coverage of the phase map can be optimized.

Considering the presence of error sources such as image noise, the accuracy of the phase

maps can be improved by two approaches: maintaining high fringe contrast in the images,

and using the smallest fringe pitch possible in the projection pattern. There are inherent

limitations on how small a fringe pitch and high a contrast can be used. In addition, these

two approaches may lead to conflicting parameter settings.

The smallest fringe pitch that can be used in projection patterns is limited by the res-

olution of the digital projector. Most commercial digital projectors have relatively large

apertures and hence small depth of focus. The light projection in the measurement volume

of a PCCDFP system generally experiences some degree of defocusing [25]. When the fringe

pitch in the projection pattern is smaller than a certain value, the defocusing issue can cause

a significant drop of the projected fringe contrast and hence a poor fringe contrast in the

images. Based on our experience with three different DMD projectors, we found that 8 pixels

is the practical minimum for the fringe pitch to be used in projection patterns.

The finite resolution of the digital camera also limits the fringe pitch that could be used

in projection patterns. As demonstrated earlier, the local fringe pitch in the images depends

10

on the surface shape. For fixed fringe pitch in the pattern, the local fringe pitch in certain

areas of the images might be too small such that fringe crowding problem would occur and

result in unresolvable areas in the constructed phase map.

Based on the above guidelines for designing projection patterns, we developed an algo-

rithm to automatically generate adaptive fringe patterns for point cloud acquisition. The

basic idea behind the algorithm is to always use the smallest local fringe pitch in the pattern

which could provide good projection contrast and at the same time not to result in fringe

crowding in the images. A detailed description of the algorithm is as follows:

1. Acquisition of two absolute phase maps, Φ(V ) and Φ(H), of the surface being

measured:

Two absolute phase maps of the surface being measured are acquired by using vertical

and horizontal fixed-pitch fringe patterns respectively. Both patterns use large fringe

pitches to avoid possible unresolvable areas in the phase maps (due to fringe crowding

in the images). Let p(P )fc denote the smallest fringe pitch in the pattern that could

provide satisfactory projection contrast. In practice, we choose p(P )fc to be 8 pixels.

The fringe pitch used in the vertical fringe pattern can be written as cp0 · p(P )fc , where

cp0 is a coefficient that is generally selected to be greater than 2.

Let Φ(V )P (ξ, η) and Φ

(H)P (ξ, η) denote the phase distributions in the vertical and the

horizontal fringe patterns respectively. Φ(V )P and Φ

(H)P can be expressed using the

following equations:

Φ(V )P (ξ, η) =

[

2π/(cp0 · p(P )fc )

]

· ξ

Φ(H)P (ξ, η) =

(

2π/p(H)P

)

· η(2)

where p(H)P is the fringe pitch used in the horizontal fringe pattern and is generally

selected to be close to the value of cp0 · p(P )fc .

Let Φ(V )(u, v) and Φ(H)(u, v) denote the phase maps of the surface obtained by using

the patterns given by Eqn. 2. With these two phase maps, each point (u1, v1) in the

11

image can be associated with two phase values, Φ(V )(u1, v1) and Φ(H)(u1, v1). On the

other hand, a point in the projection pattern can be located uniquely by the same

phase value pair. If (ξ1, η1) denote this point, then the following equations hold:

Φ(V )P (ξ1, η1) = Φ(V )(u1, v1)

Φ(H)P (ξ1, η1) = Φ(H)(u1, v1)

(3)

By combining Eqns. 2 and 3, the coordinates (ξ1, η1) can be calculated as follows:

ξ1 =[

cp0 · p(P )fc /(2π)

]

· Φ(V )(u1, v1)

η1 =[

p(H)P /(2π)

]

· Φ(H)(u1, v1)(4)

2. Construction of the gradient field of the adaptive pattern, [∇Φ](A)ξ :

Let Φ(A)P (s, t) denote the phase distribution of the adaptive projection pattern to be

built, which is a 2-D array defined in the ξ-η space. (s, t) are indices to elements in

Φ(A)P . Let [∇Φ]

(A)ξ (s, t) be the gradients of Φ

(A)P (s, t) along ξ-axis, i.e.

[∇Φ](A)ξ (s, t) = Φ

(A)P (s, t + 1) − Φ

(A)P (s, t)

for s = 1, · · · , S and t = 1, · · · , (T − 1)

(5)

where S ×T has the dimensions of Φ(A)P . To build the adaptive projection pattern, the

gradient field [∇Φ](A)ξ needs to be constructed using the following approach:

• Initialize all elements in [∇Φ](A)ξ to 2π/p

(P )fc , where p

(P )fc is the critical fringe pitch

of projection pattern as defined earlier.

• Let p(I)fc denote the smallest local fringe pitch in the image that can provide

satisfactory phase accuracy. Let [∇Φ]u(i, j) denote the gradients of phase map

Φ(V )(u, v) along u-axis. Create a bitmap mask, M (I), for [∇Φ]u(i, j) to mark the

12

pixels whose phase gradients are larger than a given threshold. In other words,

for all pixels (i, j) in the phase map of the surface that satisfy

[∇Φ]u(i, j) >2π

cp0 · p(I)fc

(6)

mark the corresponding pixels in M (I) as 1s and all others as 0s. where cp0

is the coefficient defined in Eqn. 2. The 1-pixels in M (I) form a number of

connected components, which can be isolated and labeled by using region growing

algorithm [26].

• Let R(I)M (l) (l = 1, . . . , L) denote the connected components in M (I). Create a

bitmap mask M (A) for [∇Φ](A)ξ to mark the pixels whose gradients need to be

adjusted. This is done by the following procedure. For each region R(I)M (l) in

M (I), find its boundary B(I)M (l); map this boundary to M (A) using Eqn. 4; fill the

mapped boundary in M (A) to form a region R(A)M (l), which corresponds to the

region R(I)M (l) in M (I).

• Consider a marked region R(I)M (l) (l = 1, . . . , L) in M (I). For a pixel (i1, j1) in the

region, compute its corresponding coordinates in the projection pattern by using

Eqn. 4. Let (s1, t1) denote the calculated coordinates. Set the ξ-gradients of this

pixel in the adaptive projection pattern as follows:

[∇Φ](A)ξ (s1, t1) =

4π2

cp0 p(I)fc p

(P )fc [∇Φ]u(i1, j1)

(7)

Perform this operation for all pixels in region R(I)M (l). Depending on the mapping

from R(I)M (l) to R

(A)M (l), there might be missed pixels in R

(A)M (l) whose gradients

were not updated. Hence, a search in R(A)M (l) for missed pixels needs to be done

as the final step and the values of these pixels can be updated by means of

interpolation.

13

The above process is conducted for all marked regions R(I)M (l). The result of the

process is that, if the local fringe pitch in an area of the image is too small, the

corresponding area in the adaptive projection pattern is set with larger fringe

pitches.

• Smoothen [∇Φ](A)ξ to avoid steep changes in gradient values. This is done by

performing a Gaussian filtering over [∇Φ](A)ξ .

3. Construction of the phase distribution of the adaptive pattern, Φ(A)P :

Once the gradient field [∇Φ](A)ξ is constructed, the phase distribution of the adaptive

projection pattern, Φ(A)P , can be calculated from discrete integration of [∇Φ]

(A)ξ as

follows:

Φ(A)P (s, t) = Φ

(A)P (s, t − 1) + [∇Φ]

(A)ξ (s, t − 1) ,

for s = 1, · · · , S and t = 2, · · · , T

(8)

This process requires the initialization of the first column of Φ(A)P , i.e. Φ

(A)P (s, 1) (s =

1, · · · , S), which is generally set to zeros.

4. Building the adaptive pattern:

The adaptive projection pattern is built from its phase distribution, Φ(A)P , by applying

a sinusoidal modulation. The intensity distribution of the pattern can be expressed

using the following equation:

I(P )(s, t) =I

(P )max

2

[

1 + sin(

Φ(A)P (s, t)

)]

(9)

where I(P )(s, t) is the intensity of pixel (s, t) and I(P )max is the maximum intensity of the

projection pattern.

The process of generating an adaptive projection pattern for the measurement of a sphere

is demonstrated in Fig. 7. Fig. 7(a) shows the phase map of the sphere Φ(V ) (shown as iso-

phase value contours), which is acquired by using the vertical fringe pattern Φ(V )P . The

14

100 200 300 400 500

50

100

150

200

250

300

350

400

450

500

(a) Contours of the sphere’s phasemap, Φ(V ) (pixels with large phasegradients, [∇Φ]u, are highlighted)

100 200 300 400 500

50

100

150

200

250

300

350

400

450

500

(b) Pixels in the projection pat-tern whose phase gradients, [∇Φ]ξ,need to be set smaller than theirinitial values

100 200 300 400 500

50

100

150

200

250

300

350

400

450

500

(c) Contours of the constructedphase distribution of the adaptive

pattern, Φ(A)P

(d) Adaptive pattern generatedat the end (shown with uni-formly increased fringe pitchthan actual)

Figure 7: Generation of an adaptive projection pattern for the measurement of a sphere

15

highlighted region in the phase map indicates the 1-pixels in the mask M (I). The corre-

sponding region in the projection pattern, represented by mask M (A), is shown in Fig. 7(b).

Fig. 7(c) shows the constructed phase distribution of the adaptive pattern Φ(A)P in the form

of contours. The adaptive projection pattern generated at the end is shown in Fig. 7(d).

4 Algorithm for Construction of New Reference Phase

Map

Recall that the construction of point clouds requires a reference phase map which depends on

the projection pattern used for measurements (see Section 2 and Fig. 5). When a newly gen-

erated projection pattern has to be used, the reference phase map needs to be reconstructed.

The algorithm for construction of new reference phase map requires two absolute phase maps

of the reference plane as well as the generated adaptive pattern. Let Φ(V )P (ξ, η) and Φ

(H)P (ξ, η)

denote the phase distributions of the projection patterns used to obtain the required phase

maps. Φ(V )P (ξ, η) and Φ

(H)P (ξ, η) can be expressed using the following equations:

Φ(V )P (ξ, η) = cV · ξ

Φ(H)P (ξ, η) = cH · η

(10)

where cV and cH are constants. Φ(V )P is a vertical fringe pattern and Φ

(H)P is a horizontal

fringe pattern. Both patterns are of fixed fringe pitch.

Let Φ(V )R (u, v) and Φ

(H)R (u, v) be the corresponding phase maps of the reference plane

acquired by using fringe patterns Φ(V )P (ξ, η) and Φ

(H)P (ξ, η) respectively. Let Φ

(A)P (ξ, η) denote

the phase distribution of the adaptive projection pattern. Notice that when the reference

plane is fixed w.r.t. the PCCDFP system, each point (u, v) in the phase map has a unique

corresponding point in the projection pattern, (ξ, η), which shares same phase value as (u, v)

no matter what pattern is projected. Utilizing this fact, the construction of the new reference

16

phase map Φ(A)R can be completed by the following procedure:

1. For each pixel (u1, v1) in Φ(A)R (u, v), compute its corresponding point (ξ1, η1) in the

projection pattern in the following manner:

ξ1 = (1/cV ) · Φ(V )R (u1, v1)

η1 = (1/cH) · Φ(H)R (u1, v1)

(11)

where cV and cH are constants defined in Eqn. 10.

2. Using the calculated coordinates (ξ1, η1), compute the phase value Φ(A)P (ξ1, η1) by in-

terpolating over the phase distribution of the generated adaptive pattern Φ(A)P (ξ, η).

3. The calculated phase value Φ(A)P (ξ1, η1) is then assigned to Φ

(A)R (u1, v1).

An example of constructing new reference phase map for an adaptive projection pattern

is shown in Fig. 8. Fig. 8(a) and 8(b) are the contour plots of the phase maps Φ(V )R and Φ

(H)R

respectively. The blue crosses drawn in the figures are example pixels in the u-v space, i.e.

pixels (u1, v1) referred in the description of the algorithm above. Fig. 8(c) shows the phase

distribution of the adaptive pattern Φ(A)P in contours. The red crosses drawn in the figure are

points in the ξ-η space that correspond to the example pixels. Fig. 8(d) shows the contour

plot of the new reference phase map Φ(A)R that is constructed.

5 Results and Discussion

The two algorithms presented in this paper, namely the algorithm for automated generation

of adaptive patterns and the algorithm for construction of new reference phase map, were

implemented in Matlab. Using a PCCDFP system (hardware and software) we have devel-

oped in our prior work [21], a number of experiments were performed on objects with various

shapes. For each object, several measurements were made by using automatically generated

17

100 200 300 400 500

50

100

150

200

250

300

350

400

450

500

(a) Contours of phase map Φ(V )R

(blue crosses are example pixels inthe (u, v) space)

100 200 300 400 500

50

100

150

200

250

300

350

400

450

500

(b) Contours of phase map Φ(H)R

(example pixels drawn in bluecrosses)

100 200 300 400 500

50

100

150

200

250

300

350

400

450

500

(c) Phase distribution of the adap-tive pattern shown in contours,

Φ(A)P (red crosses are points in the

(ξ, η) space that correspond to theexample pixels shown in (a), (b)and (d))

100 200 300 400 500

50

100

150

200

250

300

350

400

450

500

(d) Contours of the new reference

phase map computed, Φ(A)R (exam-

ple pixels drawn in blue crosses)

Figure 8: Construction of new reference phase map for an adaptive pattern

18

adaptive patterns as well as traditional sinusoidal patterns with various fringe pitches. The

measurement accuracy and coverage were then compared as indicators of the performance of

different patterns. Two types of experiments were conducted. The first type of experiments

were on CAD models using a simulation software. The second type of experiments were

conducted on physical objects using a physical PCCDFP system.

5.1 Performance Tests Using Measurements on CAD Models

We have developed a simulation software for PCCDFP system. The software is basically a

scene rendering engine which generates images (from the camera’s view) for given object(s),

projection pattern, and scene description (e.g. projector and camera parameters, ambient

light condition, etc.). The projector and the camera are modeled using the pinhole camera

model with Brown’s lens distortion model [27]. The scene rendering is done by ray tracing

with the consideration of the pixel integration effect of CCD/CMOS cameras. The ACIS

geometric kernel is used to facilitate the ray tracing process so that the software can handle

objects of virtually any shapes provided their ACIS models are given. The classic Phong

model is used to model the surface reflection of the objects.

To simulate the influence of error sources, noise was added to the images of objects which

were obtained under the projection of fringe patterns. For all simulation runs, noises were

generated with a uniform distribution and a magnitude of 2.5% (plus and minus) of the

maximum image intensity.

The developed simulator allows us to quickly conduct experiments on virtual objects (i.e.,

CAD models). Four representative experiments are reported in this section: a block with a

cone-shaped hole (shown in Fig. 9), a part with a sawtooth profile (shown in Fig. 13(a)), a

spherical surface (shown in Fig. 14(a)), and a randomly generated spline surface (shown in

Fig. 15(a)). The X-Y dimensions of the parts are equal to or bigger than 200 mm×200 mm.

The Z-dimensions of the parts are as follows: the cone-shaped hole, 160 mm; the part with

19

Figure 9: Part with a cone-shaped hole

Figure 10: One of the images acquired (portion view) by using fixed-pitch fringe pattern,nF = 100

sawtooth profile, 27 mm; and the spline surface, 55.554 mm. All parts have diffuse surfaces.

The automatically generated adaptive projection patterns for the measurements of differ-

ent parts are shown in Fig. 11(a), 13(b), 14(b) and 15(b). When fixed-pitch fringe patterns

(e.g. nF = 100) were used, the images of all the parts have certain degree of fringe over-

crowding in some portions of the images. As a result, the construction of point clouds failed

at these regions. Take the measurement of the cone-shaped hole as an example. As can be

seen from Fig. 10, there is a severe fringe overcrowding in the region that corresponds to

the center-right part of the cone. In the constructed point cloud as shown in Fig.12(a), that

part of the cone is missing. The same problem was observed in the measurements of other

parts as well. However, the use of adaptive projection patterns eliminated this problem (see

Fig. 11(b)). This is because of the fact that, in the middle region of the projection pattern

(see Fig. 11(a)), which is projected on the right slope of the cone, the fringe pitch has been

20

(a) Adaptive fringe pattern gen-erated (shown in larger fringepitch than actual)

(b) One of the images acquired (por-tion view)

Figure 11: Simulated measurement of a cone-shaped hole using adaptive fringe pattern

(a) Point cloud obtainedby using fixed-pitchfringe pattern, nF = 100(pseudo-color representsz-coordinate)

(b) Point cloud obtained byusing the adaptive fringepattern

0 1000 2000 3000 4000 50000.03

0.035

0.04

0.045

0.05

0.055

0.06

0.065

Size of Unresolvable Area (pixels)

RM

S o

f Mea

sure

men

t Err

or (

mm

) Fixed−pitch frg. ptn., θPart

=0o

Fixed−pitch frg. ptn., θPart

=7o

Adaptive fringe pattern

49

64

81

100

121

49

64

81 100121

(c) Measurement performance:adaptive fringe pattern vs. fixed-pitch fringe patterns (nF representsfringe number)

Figure 12: Results of the measurement of a cone-shaped hole

21

extended dramatically so that a reasonable fringe pitch is forged when observed from the

camera’s angle.

The measurement performance achieved by using adaptive and fixed-pitch fringe patterns

are shown in Fig. 12(c), 13(d), 14(c) and 15(c). The horizontal axes of the plots represent

the number of pixels that are unable to be resolved in a measurement. The total number of

pixels is 512 × 512 = 262, 144. The vertical axes represent the RMS value of measurement

error. nF is the number of fringes in the fixed-pitch fringe patterns. As can be seen from the

plots, when fixed-pitch fringe patterns (with varied fringe numbers) were used, any single

measurement could not achieve good accuracy and good coverage at the same time. A high

fringe number (e.g. 121) is able to achieve very good measurement accuracy but the surface

area it fails to measure is quite large; On the other hand, a low fringe number (e.g. 49) has

excellent measurement coverage but the accuracy of measurement is fairly poor. With the

use of adaptive patterns, a compromise can be achieved. In all measurements conducted, the

adaptive patterns achieved an accuracy that is comparable to a fixed-pitch fringe pattern

with a fringe number between 100 to 121. At the same time, it achieved a measurement

coverage that is comparable to (a fixed-pitch fringe pattern with) a fringe number between

49 to 64.

In the measurements of the cone-shaped hole by using fixed-pitch fringe patterns, the

part was placed at a few slightly different tilt angles around Y (W )-axis. Fig. 12(c) shows the

results of the part at two different tilt angles, θPart = 00 and θPart = 70, where θPart = 00

corresponds to the orientation that the part’s top surface is parallel to the X(W ) − Y (W )

plane. The measurement results indicate that, different tilt angles of the part yield different

measurement accuracies and coverages for the same fringe pattern. However, it only changes

the balance of the measurement performance between accuracy and coverage but makes no

improvement to the measurement performance as a whole.

22

(a) Part with a sawtooth profile (b) Adaptive fringe patterngenerated (shown using largerfringe pitch than actual)

(c) Point cloud obtained by usingthe adaptive fringe pattern (pseudo-color represents z-coordinate)

0 500 1000 1500 2000 2500 3000 35000.044

0.046

0.048

0.05

0.052

0.054

0.056

0.058

0.06

Size of Unresolvable Area (pixels)

RM

S o

f Mea

sure

men

t Err

or (

mm

) Fixed−pitch fringe patternAdaptive fringe pattern

nF=49

nF=64

nF=81

nF=100

nF=121

(d) Measurement performance:adaptive fringe pattern vs. fixed-pitch fringe patterns

Figure 13: Simulated measurements of a part with a sawtooth profile

(a) Spherical surface (b) Adaptive fringe patterngenerated (shown in largerfringe pitch than actual)

0 500 1000 1500 2000 25000.03

0.035

0.04

0.045

0.05

Size of Unresolvable Area (pixels)

RM

S o

f Mea

sure

men

t Err

or (

mm

) Fixed−pitch fringe patternAdaptive fringe pattern

nF=49

nF=64

nF=81

nF=100

nF=121

(c) Measurement performance:adaptive fringe pattern vs.fixed-pitch fringe patterns

Figure 14: Simulated measurements of a spherical surface

23

(a) Randomly generatedspline surface

(b) Adaptive fringe patterngenerated (shown in largerfringe pitch than actual)

0 100 200 300 400 500 6000.036

0.038

0.04

0.042

0.044

0.046

0.048

0.05

0.052

Size of Unresolvable Area (pixels)

RM

S o

f Mea

sure

men

t Err

or (

mm

) Fixed−pitch fringe patternAdaptive fringe pattern

nF=121

nF=64

nF=49

nF=100

nF=81

(c) Measurement performance:adaptive fringe pattern vs.fixed-pitch fringe patterns

Figure 15: Simulated measurements of a spline surface

5.2 Performance Tests Using Measurements on Physical Parts

The performance test of adaptive projection patterns was also conducted on physical parts,

using a PCCDFP hardware we have built. The major parameters of the hardware are as

follows: the angle between the optical axis of the projector and the optical axis of the camera

is 270; the distance between the sensor and the center of the measurement volume is around

600 mm; the projector’s field-of-view angles are 43.60 (Horizontal) and 33.40 (Vertical); the

camera’s field-of-view angles are 22.280 (Horizontal) and 16.770 (Vertical); the resolution of

the projector is 1024× 768 pixels; the resolution of the camera is 640× 480 pixels; and both

the projector and the camera have a gray-depth of 8 bits. All measurements were conducted

under regular indoor lighting conditions.

Fig. 16 shows the experiment results of a plastic flowerpot, which has a maximum di-

ameter of 198 mm and a depth of 149 mm. The measurement coverages achieved by using

adaptive projection pattern and fixed-pitch fringe patterns are shown in Fig. 16(c). The

horizontal axis of the plot represents the number of fringes in the projection pattern (nF ),

and the vertical axis represents the number of pixels that could not be resolved in the mea-

surement. For fixed-pitch fringe patterns, the size of unresolvable area increases as the fringe

number increases. This is caused by fringe overcrowding as the fringe pitch in the projection

24

(a) Photograph of theflowerpot

(b) Adaptive fringe pattern gen-erated

0 30 60 90 120 150 1800

5

10

15

20

Number of Fringes in Projection Pattern

Siz

e of

Unr

esol

vabl

e A

rea

(x10

3 pix

els) Fixed−pitch fringe pattern

Adaptive fringe pattern

(c) Measurement coverageachieved: adaptive pattern vs.fixed-pitch fringe patterns

Figure 16: Measurements of a plastic flowerpot

pattern gets smaller. In the case of adaptive pattern, a measurement coverage comparable to

a (fixed-pitch fringe pattern with a) fringe number of 50 was achieved, although the adaptive

pattern has an equivalent fringe number of 118. Since the accurate shape of the flowerpot is

not known, the study on measurement accuracy was not possible.

Fig. 17 shows the experiment results of a plastic tube, which has a diameter of 127.34 mm

and a height of 95 mm. The adaptive pattern generated for the measurement of the tube

is shown in Fig. 17(b). In order to evaluate the measurement accuracy, the point cloud

acquired in a measurement was fitted to a cylinder and the residual deviation was analyzed.

Fig. 17(c) shows the measurement coverage and measurement accuracy achieved by using

different projection patterns. The horizontal axis of the plot represents the number of pixels

that could not be resolved in the measurement, and the vertical axis represents the RMS value

of the divergence of the point cloud from a perfect cylinder. As can be seen, the adaptive

projection pattern achieved a better overall measurement performance than fixed-pitch fringe

patterns. Also notice that, for fix-pitch fringe patterns, the RMS value of measurement error

increased as the fringe number (nF ) grew from 100 to 144, which is different from the test

results with CAD models (see Sec. 5.1). This is because, as the fringe number gets larger

than around 100, the defocusing issue of the projector [25] causes a significant drop in fringe

contrast, which in turn causes an increase in measurement error. However, this phenomenon

25

(a) Photograph of thetube

(b) Adaptive fringe pattern gen-erated

0 5 10 150.06

0.065

0.07

0.075

0.08

0.085

0.09

0.095

0.1

Size of Unresolvable Area (x103 pixels)

RM

S o

f Mea

sure

men

t Err

or (

mm

) Fixed−pitch fringe patternAdaptive fringe patternn

F=32

nF=64

nF=81

nF=100

nF=125

nF=144

(c) Measurement performance:adaptive fringe pattern vs.fixed-pitch fringe patterns

Figure 17: Measurements of a plastic tube

was not captured in the tests with CAD models since the defocusing issue of projector was

not considered in simulations.

Similiar tests were also conducted on a variety of parts. The developed algorithms worked

well in generating adaptive projection patterns to eliminate fringe overcrowdings in images.

Fig. 18 shows the result of measurements made on a plastic human face model. When fixed-

pitch fringe patterns were used, as shown in Fig. 18(a), the left boundary of the face and

the left side of the nose have fringe overcrowding problems. The automatically generated

adaptive pattern, shown in Fig. 18(b), has increased fringe pitches in these regions. The

resulting image (shown in Fig. 18(c)) obtained from adaptive pattern has no such problem.

Similar improvement was also observed in the measurements of a plastic base of a telephone

handset (results shown in Fig. 19).

As a summary of the above, when the number of fringes in the projection pattern is

fixed, e.g. in order to maintain a certain measurement accuracy, adaptive projection pattern

is able to achieve better measurement coverage than fixed-pitch fringe pattern.

26

(a) Image (trimmed) acquiredusing fixed-pitch fringe pattern

(b) Adaptive fringe pattern gener-ated (shown in larger fringe pitchthan actual)

(c) Image (trimmed) acquiredusing adaptive pattern

Figure 18: Measurements of a plastic human face model

6 Conclusions

Most existing PCCDFP systems use projection patterns consisting of straight fringes with

fixed fringe pitches. When measuring objects with a large range of surface normal directions,

it is hard to achieve high measurement coverage and good measurement accuracy at the

same time by using fixed-pitch straight fringe patterns. Patterns that use curved fringes

with spatial pitch variation can significantly improve the point cloud acquisition process in

such cases. Hence, a possible way to achieve good measurement accuracy and coverage is

to use projection patterns that have been customized for the object being measured. Such

customization involves curving fringes at certain locations and spatially varying fringe pitch.

This paper describes algorithms for automatically generating adaptive projection patterns

that use curved fringes with spatial pitch variation to provide improved measurement results

for an object being measured. In addition, we also describe the algorithm for constructing the

reference phase map. Both simulation and physical experiments show that adaptive patterns

are able to achieve improved performance. Specifically, adaptive projection patterns provide

27

(a) Image (trimmed) acquiredusing fixed-pitch fringe pattern

(b) Adaptive fringe pattern gener-ated (shown in larger fringe pitchthan actual)

(c) Image (trimmed) acquiredusing adaptive pattern

Figure 19: Measurements of a plastic base of a telephone handset

better overall measurement performance (coverage and accuracy) than fixed-pitch fringe

patterns, especially when the object being measured has a large range of surface normal

directions.

In applications such as on-line parts inspection doing batch processes, e.g. sheet metal

stamping, parts being measured are of virtually the same shape and are placed approxi-

mately at the same position and orientation with respect to the measuring device. Hence,

the adaptive patterns optimized for measuring these parts need to be generated only once

and can be repeatedly used thereafter. The overhead of using adaptive patterns (i.e. the

procedure to generate adaptive patterns) is negligible, which makes it an ideal solution for

such applications.

As a part of the future work, an in-depth understanding of the relationship between

the fringe pitch in projection pattern and the resulting measurement performance needs

to be established. Furthermore, the influence of other measurement factors need to be

characterized.

28

7 Acknowledgments

This work has been supported by NSF grant DMI-0093142. However, the opinions expressed

here are those of the authors and do not necessarily reflect that of the sponsor.

References

[1] Z. Ji and M. C. Leu. Design of optical triangulation devices. Optics and Laser Tech-

nology, 21(5):335–338, 1989.

[2] C. S. Fraser and D. C. Brown. Industrial photogrammetry - new developments and

recent applications. Photogrammetric Record, 12(68):197–217, 1986.

[3] S. N. Lane, T. D. James, and M. D. Crowell. Application of digital photogramme-

try to complex topography for geomorphological research. Photogrammetric Record,

16(95):793–821, 2000.

[4] L. M. Galantucci, R. Ferrandes, and G. Percoco. Digital photogrammetry for facial

recognition. Journal of Computing and Information Science in Engineering, 6(4):390–

396, 2006.

[5] B. Breuckmann, F. Halbauer, E. Klaas, and M. Kube. 3D-metrologies for industrial

applications. In Proceedings of SPIE 3102, pages 20–29, 1997.

[6] F. Chen, G. M. Brown, and M. Song. Overview of three-dimensional shape measurement

using optical methods. Optical Engineering, 39(1):10–22, 2000.

[7] P. Benko, R. R. Martin, and T. Varady. Algorithms for reverse engineering boundary

representation models. Computer-Aided Design, 33(11):839–851, 2001.

[8] W. Sun, C. Bradley, Y. F. Zhang, and H. T. Loh. Cloud data modelling employing a

unified, non-redundant triangular mesh. Computer-Aided Design, 33(2):183–193, 2001.

29

[9] J. Huang and C. H. Menq. Combinatorial manifold mesh reconstruction and opti-

mization from unorganized points with arbitrary topology. Computer-Aided Design,

34(2):149–165, 2002.

[10] J. Jamshidi, G. W. Owen, and A. R. Mileham. A new data fusion method for scanned

models. Journal of Computing and Information Science in Engineering, 6(4):340–348,

2006.

[11] J. Barhak and A. Fischer. Parameterization and reconstruction from 3D scattered points

based on neural network and PDE techniques. IEEE Transactions of Visualization and

Computer Graphics, 7(1):1–16, 2001.

[12] S. Azernikov and A. Fischer. A new volume warping method for surface reconstruction.

Journal of Computing and Information Science in Engineering, 6(4):355–363, 2006.

[13] G. Moroni and M. Rasella. Application of regression spline to reverse modeling. Journal

of Computing and Information Science in Engineering, 7(1):95–101, 2007.

[14] R. J. Urbanic, W. H. ElMaraghy, and H. A. ElMaraghy. An integrated systematic design

recovery framework. Journal of Computing and Information Science in Engineering,

6(4):318–330, 2006.

[15] K. Schreve, C. L. Goussard, A. H. Basson, and D. Dimitrov. Interactive feature modeling

for reverse engineering. Journal of Computing and Information Science in Engineering,

6(4):422–424, 2006.

[16] C. Mehdi-Souzani, F. Thiebaut, and C. Lartigue. Scan planning strategy for a gen-

eral digitized surface. Journal of Computing and Information Science in Engineering,

6(4):331–339, 2006.

30

[17] P. Singh, Y. Wu, R. Kaucic, J. Chen, and F. Little. Multimodal industrial inspection

and analysis. Journal of Computing and Information Science in Engineering, 7(1):102–

107, 2007.

[18] S. Toyooka and Y. Iwaasa. Automatic profilometry of 3-D diffuse objects by spatial

phase detection. Applied Optics, 25(10):1630–1633, 1986.

[19] R. Sitnik, M. Kujawinska, and J. Woznicki. Digital fringe projection system for large-

volume 360-deg shape measurement. Optical Engineering, 41(2):443–449, 2002.

[20] R. Legarda-Saenz, T. Bothe, and W. P. Juptner. Accurate procedure for the calibration

of a structured light system. Optical Engineering, 43(2):464–471, 2004.

[21] T. Peng and S. K. Gupta. Model and algorithms for point cloud construction using digi-

tal projection patterns. Journal of Computing and Information Science in Engineering,

7(4):372–381, 2007.

[22] E. U. Wagemann, M. Schonleber, and H. J. Tiziani. Grazing holographic projection

of object-adapted fringes for shape measurements with enhanced sensitivity. Optics

Letters, 23(20):1621–1623, 1998.

[23] H. Zhao, W. Chen, and Y. Tan. Phase-unwrapping algorithm for the measurement of

three-dimensional object shapes. Applied Optics, 33(20):4497–4500, 1994.

[24] W. Nadeborn, P. Andra, and W. Osten. A robust procedure for absolute phase mea-

surement. Optics and Lasers in Engineering, 24(2–3):245–260, 1996.

[25] C. R. Coggrave and J. M. Huntley. Optimization of a shape measurement system based

on spatial light modulators. Optical Engineering, 39(1):91–98, 2000.

[26] Kenneth R. Castleman. Digital Image Processing. Prentice Hall, second edition, 1995.

31

[27] D. C. Brown. Close-range camera calibration. Photogrammetric Engineering, 37(8):855–

866, 1971.

32