Finger Sculpting with Digital Clay

Technical Report GIT-GVU-02-2230 October 2002

Joshua Gargus Byungmoon Kim Ignacio Llamasschwa@cc.gatech.edu bmkim@cc.gatech.edu llamasi@cc.gatech.edu

Jarek Rossignac Chris Shawjarek@cc.gatech.edu cdshaw@cc.gatech.edu

GVU Center and College of Computing

Georgia Institute of Technology

October 30, 2002

Abstract

Digital Clay is a term that signifies a computer-controlled physical surface, capable of taking any of awide variety of possible shapes in response to changesin a digital 3D model or changes in the pressure ex-erted upon it by bare hands. The physical propertiesof such a device impose design and user-interface con-straints not encountered in traditional, tracker-basedsoftware for the manipulation of virtual models. Thispaper describes the interaction techniques we havedeveloped to work with this future medium. In par-ticular, we present our solution for tracking the user’sfingers using a local deformation of the surface, whichwe call a blister, that senses the tangential and nor-mal displacements of the finger. We also present a so-lution for creating variable-height bosses and creaseswith the simple sweep of a finger. Since the DigitalClay hardware is not yet operational, we have haveimplemented a haptic simulation framework based ona PHANTOM device.

Figure 1: Sculpting with real clay

1

Figure 2: Mock-up using “Pinhead” sculpture toy . The clay raises a blister (see Section 4.3) to follow theuser’s finger as a ridge is raised from the surface (see Section 5).

1 Introduction

Shape is a key element in successful communication,interpretation, and understanding of complex data invirtually every area of engineering, art, science, andmedicine. While in recent years the communicationof both form and complex data have been greatlyenhanced by visualizations based on planar images,computational power and manufacturing technolo-gies have reached the point where it is possible to con-sider interactive three-dimensional tactile input andoutput devices. The NSF/ITR Digital Clay project[1] aims at developing a medium that allows thissort of interaction; it is an instrumented, actuated,computer-controlled physical volume bounded by anactuatable surface that acts as a haptic interface. Wefreely abbreviate “Digital Clay” to “clay” throughoutthe paper.

Digital Clay is conceived as a collaborative tool: asone manipulates the surface, the shape change canbe sensed and transmitted to another piece of clayor other computational device. For example, a geo-graphically dispersed team of engineers could inter-actively explore various shape modifications to a partthey are designing.

The goal of the research presented here is to de-velop user interface techniques that are a good fit forthe affordances and constraints of Digital Clay; thechallenges inherent in the fabrication of such a de-vice are beyond the scope of this work, and will bediscussed elsewhere. Since the Digital Clay hardware

is not yet operational, we have implemented a hapticsimulation framework based on a PHANTOM device.We are careful to ensure that our clay simulation doesnot have access to more information about the userthan the final hardware will. For example, the clayhardware will only be able to sense the position ofthe user’s finger by the deformations it causes in theclay’s surface. We use the PHANTOM only to sim-ulate the effects of the finger on the clay surface; theinformation available to the clay model is the same,regardless of whether the finger is real or virtual.

Physical interaction between user and clay consistsprimarily of the forces applied by each to the other.In addition, the user can inspect the shape visually,and by touching it lightly without modifying it. Wedecided against multimodal strategies such as com-bined touch and voice input, instead opting to limitourselves to “finger sculpting”: interactions that con-sist of touching the surface with one or more fingers.The focus of this research is to learn how to inter-act with a medium whose affordances and limitationsare not yet understood. At this early stage, exper-imenting with multimodal interactions would morelikely hinder than facilitate insight into the essentialnature of interactions with Digital Clay. The tem-porary restriction to single-finger interactions ratherthan whole-hand manipulations results from both thedesire to simplify the interaction space that we areexploring, as well as the nature of the PHANTOMdevice.

It should be emphasized that the clay metaphor

2

is imperfect. Digital Clay offers manipulation possi-bilities beyond those of common modelling clay. Forexample, Digital Clay may change its volume, pro-vide for multiresolution shape editing [7], or allowcut-and-pasting and procedural generation of surfacefeatures [19]. By taking advantage of its ability tochange shape, Digital Clay can facilitate completionof a task by actively responding to the user. We be-lieve that the most interesting and useful techniquesfor interacting with Digital Clay will be of this type.

The research contributions of this paper can be cat-egorized as either interaction elements, or completeinteractions composed of these elements. The firstcategory includes our solution for tracking the user’sfingers using a local deformation of the surface, whichwe call a blister, that senses the tangential and nor-mal displacements of the finger. Other contributionsin this category include detecting the user’s intent tobegin or end an edit operation. The second categorycontains our methods for accomplishing specific usertasks, such as creating variable-height bosses on theclay surface. In his thesis [17], Pierce argues that bytaking a principled approach to exploring the possi-bilities of a new medium, we can more quickly buildup a lexicon of interactions for that medium. Webelieve that our techniques represent a first step to-wards such a lexicon for Digital Clay interactions,analogous to the rich toolkit available for creatingGUI applications.

1.1 Kinematic Properties of Antici-pated Initial Prototype of DigitalClay

Many of the design problems encountered during thisresearch could not have been resolved without consid-ering the physical properties of Digital Clay. Further-more, several designs for clay prototypes have beenproposed; our research has focused on the one dubbedthe “Bed of Nails”, since it is most likely to be thefirst one fabricated. We describe aspects of this de-sign that have relevance to the research presented inthis paper.

The Bed of Nails design consists of a rectangularmatrix of individually actuated rods. It may be help-

ful to imagine a Pinhead toy (shown in Figure 2)modified so that each pin is computer-controlled. Wemodel the clay as a heightfield S(x, y), and use theterm grid point to refer to (x, y) positions in a height-field’s domain.

The motive power for each rod comes from a sharedhydraulic reservoir, and is controlled by a hydraulicactuator. Using hydraulics allows higher actuatordensity (and therefore, surface resolution) and gen-eration of stronger forces than achievable by minia-turized electrical motors. These actuators are ableto generate forces approximating a damped spring,described by the equation F (d) = −kd+rd′. The co-efficients k and r represent the spring and dampingcharacteristics of the surface, and d is the displace-ment of the actual, measured surface configurationfrom the internal model of where the surface shouldbe. Furthermore, each rod has an individually spec-ified equilibrium height e from which the value of dis computed. Manipulating the values of k, r, ande allows us to control the shape of the clay and theforces exerted by the clay on the finger. Constraintson the acceptable values for k, r, and e depend onother design decisions which are not discussed here.

1.2 Organization of Remainder of Pa-per

Section 2 describe the work of other researchers thatis similar to, or has provided inspiration for our re-search. Section 3 describes layering, which is usefulboth for describing the desired behavior of the clay, aswell as for implementing this behavior. The next twosections contain our primary research contributions.Section 4 explores how our system handles interac-tion elements such as deciding whether the user istouching the surface to feel or to modify its shape,and tracking a finger as it pushes into, slides along,or pulls away from the surface. These basic buildingblocks can be combined to create entire operationsfor finger sculpting. Section 5 describes several suchoperations, as well as the task scenarios they are de-signed to address. We also present solutions to issuesunique to each task. Section 6 describes aspects of thearchitecture of our simulation, in order to facilitatereplication of our research. Finally, Section 7 sum-

3

marizes our contributions, and suggests directions forfurther research.

2 Previous Work

Many haptic systems have been created to manip-ulate virtual surfaces and volumes. Physical accu-racy is important for some applications, such as tis-sue simulation for web-based surgical training [5].Physically-based volumes are also used for sculptingbecause of their intuitive behavior, but tend to becomputationally expensive. Recent research seeks toenable interactive response rates by using multiscaletechniques [4, 7, 16], or by developing material mod-els that lend themselves to precomputation [14].

Other sculpting systems (including ours) choosenot to use physically-based deformations either be-cause of the computational expense, or because cer-tain characteristics (for example, volume preserva-tion) are undesirable. Free-form deformation [3, 18]is an early and very successful approach of this type.It has since been extended to allow the use of dif-ferent deformation “tools” [6], and to allow directmanipulation of the surface instead of by adjustinga control lattice [11]. Haptic feedback is added in[10]; as with our system, forces are generated using aspring model.

Several researchers have noted that human touchperception is actually comprised of a number of dis-tinct mechanoreceptive systems [15, 2]. Haptic de-vices such as the PHANTOM only target the kines-thetic channel, which gathers information from sen-sory receptors in the muscles and tendons. The cu-taneous channel, which senses pressure informationwith mechanoreceptors embedded in the skin, is ig-nored by most haptic devices.

Such observations have motivated researchers todevelop hardware that can take advantage of the cu-taneous channel. A device that laterally stretches thefinger’s skin is described in [9]. Shape memory alloyis used as a tactile output device by [20]. Digital Claycan also be included in this category, since it providesan actual surface that can be felt with bare hands.

FEELEX [13] describes a hardware device thatbears a striking resemblance to the Bed of Nails

design, although the latter will have a significantlyhigher resolution. However, few details are givenabout the implementation of specific interactionswith their system.

3 Layered Clay Representation

In this section, we introduce layering, which is piv-otal to both conceptualizing and implementing theclay’s behavior. We discuss here only the layers ofconceptual significance; layers existing only for im-plementational reasons are discussed in Section 6. Anunderstanding of layering is assumed by the discus-sions in Sections 4 and 5.

The mathematical representation of the volumeis decomposed into n layers Li=0..n−1(x, y), each ofwhich is a heightfield. The height S(x, y) of the claysurface is the sum of the heights of the layers:

S(x, y) =n−1∑i=0

Li(x, y)

Another way to look at this is that each layer rep-resents a delta function to be applied to the layerbeneath it. These delta values may be either positiveor negative.

We also define intermediate heights Sj as the sum

Figure 3: Conceptual and Architectural Layering.Physical deformations are specified as offsets from UIsurface features. Similarly, the UI surface featuresare specified as offsets from the base surface. Thenames in parentheses are those by which the layersare denoted in the rest of the paper.

4

of the heights of Li and all of the layers beneath it:

Sj(x, y) =j∑

i=0

Li(x, y)

Figure 3 shows the three main conceptual layersin our system, which play a large role in the follow-ing sections. The base layer Lb reflects the shape ofthe surface as it was just before the last time it wastouched. The UI layer Lu is used to locally and tem-porarily change the shape of the surface to facilitateuser interactions with the clay. The touch layer Lt

represents deformations in the surface caused by userpressure; in physical clay, values for this layer wouldbe computed by measuring the differences betweenthe sensed physical state of the surface and the modelof what the surface would be if no external force werebeing applied (given by Lu in this example).

Extra functionality can easily be added by addingnew layers. Section 6 describes one example of howlayers have aided in the implementation of our sys-tem.

4 User Interaction

Section 1.1 describes the physical characteristics ofDigital Clay. When the clay is not being edited, thespring and damping coefficients are fixed, and it feelslike an elastic surface that always returns to the sameequilibrium shape. Once an edit operation begins, itactively changes shape in response to the user’s inputas described in the following subsections.

4.1 How does the clay know where itis being touched?

Digital Clay has no direct way of obtaining the po-sition of the user’s finger, and must compute this in-formation based on deviations between the measuredand expected surface configurations. Although oursimulation has access to the location of the PHAN-TOM, we refrain from using this information directlyin order to remain true to the constraints inherent inDigital Clay. Instead, we deform the touch layer tosimulate the reaction to finger pressure.

(a) discontinuous tran-sition

(b) continuous transi-tion

Figure 4: Two force profile transitions for switchingfrom touch mode to edit mode(damping effects areomitted for simplicity). Blue (resp. red) curves de-scribe the force profile before (resp. after) the tran-sition. dt is the threshold displacement at which thetransition occurs, and ft is the corresponding force.Note that in both diagrams, the edit mode profilehas a minimum force of fm, even for negative dis-placements; this is discussed in Section 4.3.1.

Since we are concerned only with single-finger in-teractions, we are able to use a simple algorithm todetermine where the user is touching the clay. Whenthe user presses on the surface, the heights of severalrods will be displaced from the heights dictated bythe internal model of the surface’s shape. The posi-tion of the user’s touch is computed as the centroidof the affected rods. The centroid is calculated bysumming the weighted (x, y) coordinates of each rod,where the weight for each rod is proportional to themagnitude of its displacement.

4.2 How does the user indicate an in-tent to edit?

When touching the surface, the user will in somecases intend to modify it and sometimes only wishto feel its shape. The clay maintains a finite statemachine that determines how to react to each ofthese cases. For example, in touch mode, the clay be-haves elastically, resisting with a spring force withoutchanging its desired equilibrium shape. In edit mode,

5

the clay’s surface is modified in response to the user’stouch. There are many types of modifications thatcan occur; a few of them are described in Section 5.

How does the user indicate when to make the tran-sition from touching to editing, or vice-versa? Wediscuss the first case here, and the second in Sec-tion 4.4. The transition between touching and edit-ing mode occurs when the user presses deeper intothe surface than a threshold displacement dt. Theuser is notified to the transition by a change in theclay’s force profile. The surface becomes softer, andthe user feels as though she has broken through someresistance into editing mode. In Figure 4, we de-pict the two force profile transitions that we experi-mented with. We name them the discontinuous andcontinuous transitions, according to whether there isa change in applied force at the instant of the tran-sition. During the discontinuous transition, only theslope of the force-displacement curve is changed. As aresult of the force discontinuity, the user often pushesunexpectedly far into the surface. This prompted usto develop the continuous transition, where the slopeand offset of the force profile are simultaneously ad-justed so that the curves of the editing and touchingprofiles cross at dt. Users of the system were still dis-tinctly aware of the transition, but did not suddenlypush deeper than intended, as they tended to withthe discontinuous transition.

4.3 How can the user specify input“above the surface”?

In Section 4.1, our discussion of how to determinethe position of the user’s finger assumes that the userwants to specify input points beneath the surface ofthe clay. However, it is easy to devise a scenariowhere the user wishes to specify an input point abovethe surface. In one such scenario (explored in greaterdepth in Section 5) the user desires to raise a ridgewhose height follows the trajectory of her finger asit moves above the surface. Since the clay can onlylocate the user’s finger when they are in contact, itmust actively seek to maintain contact as the usermoves her finger toward a point above the base sur-face Lb.

We call our solution a blister : a temporary bump,

raised artificially from the surface to follow the user’sfinger. The blister is created by modifying the offsetsin the UI layer Lu, and is cylindrical in shape.

The following subsections discuss the vertical andhorizontal behavior of the blister separately beforedescribing how they are combined in the full behav-ior.

4.3.1 Vertical blister behavior only

We begin our discussion by keeping the promise madein Figure 4 to explain why the edit force profiles,depicted there in red, extend to negative displace-ments. The magnitude of a negative displacement isthe height above the base layer Lb. However, thisdoes not imply that the finger is not in contact withthe clay, since the blister is defined in a layer aboveLb, namely the UI layer Lu. The blister adjusts itsheight to maintain contact with the user in such away that the force applied to the finger matches theforce required by the profile.

This section describes how the height of the blis-ter varies in the simplified case where the finger mayonly move vertically. Our goal is to derive an expres-sion for the blister height that is general enough toaccommodate the edit force profiles for both the con-tinuous and discontinuous cases. We proceed by firstfinding an expression that generates heights matchinga very simple force profile. This expression is twicegeneralized to encompass more complicated profiles,resulting in an expression capable of describing thecases of interest. Figure 5 depicts these force pro-files, each of which will be discussed in the followingparagraphs. The figure also shows the blister heightscorresponding to each profile.

We start with the simplest force profile, which hasthe equation f = kd (drawn in red). Since the claycannot stick to the finger, it cannot apply a negativeforce; we therefore restrict the domain of this profileto d ≥ 0. Since the curve passes through the origin,the force profile can be satisfied by adjusting only thespring constant k. No blister is necessary.

Next, we modify the force profile to f = kd + fb

(drawn in green). This is nearly the same as thecontinuous profile, except that we extrapolate alongthe same slope (dashed line) instead of clamping to

6

Figure 5: Three force profiles (solid lines), andthe corresponding blister heights (dashed lines). fb

raises the blue and green force profiles by a constantamount, and fm is the minimum force allowed bythe blue profile; Section 4.3.1 uses these values to de-rive the blister height for a given displacement. Theshaded area represents negative displacements: posi-tions above the original surface height. The scales ofthe “Force” and “Blister Height” axes are such thatthe height of a blister is equivalent to the extra forceapplied for a given displacement from the base sur-face.

a minimum force. Since the surface can only applyspring forces, we must add a blister to the surface toapply the extra force fb = ft − kdt. Dividing by theslope k gives us the height of the blister:

h1 = ft/k − dt

Finally, we consider the continuous profile (drawnin blue), which is the same as the last, except that theminimum force is clamped to fm. The displacementdm, where the minimum force is first seen, is found bydividing the force by the slope. To see the effect thatfm has on the blister height, consider some d < dm,and look at the difference between the minimum forceand the extrapolated (dashed line) force:

∆f = fm − (kd + fb)

Figure 6: Tracking of Finger by Blister. The user’sfinger starts in the center of the blister (top), and ismoved to the left and up (middle). The clay detectsthat the blister is no longer centered on the finger,and adjusts the position and height of the blister ac-cordingly(bottom).

This extra force must be generated by raising theblister by the appropriate amount; dividing by theslope k and rearranging terms gives ∆h = fm−fb

k −d.The diagram shows that the blister is not affected ford > dm, so we can write the effect of the minimumforce constant as

h2 = max(fm − fb

k− d, 0)

The total height of the blister is then given by

h = h1 + h2

It is evident from the figure that the discontinuouscase is the same as the continuous case with the addedconstraint fb = 0. Therefore, the expression for h iscapable of describing both cases.

7

4.3.2 Combined horizontal and vertical blis-ter behavior

In order to track the finger horizontally, the blistercompares its center with the position of the finger asdescribed in Section 4.1. If there is a discrepancy,then the blister is moved so that it is centered on thefinger position.

After the blister is centered on the finger position,the height of the blister may need to be adjustedso that the force applied by the blister to the fin-ger matches the chosen force profile. Note that evencompletely horizontal finger movement can result ina change of the height of the blister, since horizontalmovement will change the displacement of the fingerwith respect to a non-horizontal clay surface. If theblister height needs to be adjusted, the new height iscomputed as though the finger motion is purely ver-tical. Figure 6 shows how the blister tracks the fingerwhen it is moved diagonally.

4.4 How does the user end an edit op-eration?

If the clay is continually maintaining contact withthe user with a raised blister, how does the user no-tify the clay that the edit operation should end? Theanswer is to set a maximum rate of change for theblister height. If the user moves her finger up rapidlyenough, the blister is unable to follow. As soon asthe clay no longer senses the user’s touch (see Sec-tion 4.1), the edit operation is ended.

Since the user cannot instantaneously break con-tact with the clay, the last part of the input trajec-tory will consist of positions that the user does notintend to be part of the edit operation. Our solu-tion is to discard the last part of the input. We havefound that discarding 0.2 seconds works quite well;the user need only pause for an instant before raisingher finger in order to ensure that no desired input isdiscarded.

5 Applications

We now suggest several applications of the techniquesdescribed in the previous section. The first two havebeen implemented in our system, and the last twoshow the broad applicability of our techniques.

Before proceeding, we introduce notation for as-signing values in layers. We write Li(x, y) ← h toassociate a new value with a grid point in a particu-lar layer. We also define assignment for intermediateheights:

Si(x, y)← h ≡ Li(x, y)← h− Si−1(x, y)

We refer to these as the default assignment behaviorsfor layers.

5.1 Raising Bumps and Digging Holes

The user wishes to dig a hole or raise a bump witha circular cross-section of radius r. After enteringthe editing mode by pressing into the surface, theposition of her fingertip is tracked as described inthe previous section. Once a satisfactory positionp0 = (x0, y0, z0) has been chosen, the edit operationis completed by quickly raising the finger from thesurface.

The new shape of the base surface is determined asfollows. Let d = z0 − Lb(x0, y0) be the height differ-ence between the finger and its projection on the baselayer. Also, let p be an arbitrary grid point no far-ther than r from (x0, y0). The base layer heights aremodified by a Gaussian function whose steepness isdetermined by σ, so that Sb(p)← Sb(p)+e−σ‖p0−p‖2

.We call this offset function a Gaussian bump. Notethat raising and digging are accomplished identically;if d is positive, a bump is raised, otherwise a hole iscreated.

5.2 Embossing

Instead of creating a single bump or hole, the goalis to raise a ridge or dig a ditch. Once can switchbetween digging a ditch and raising a ridge simplyby changing whether the input position is below orabove the surface. As with bumps and holes, ridges

8

Figure 7: Result of immediately incorporating offsetsinto base surface. The user has specified a level tra-jectory for the apex of the ridge, but the resultingsurface is above this trajectory. Note that the sever-ity of the undesired effect is exaggerated by the largedistance between the two adjacent bumps.

and ditches are created similarly; this observation al-lows us to simplify our presentation by allowing amention of one to refer to both whenever the intendedmeaning is clear.

Our first attempt at an embossing operation mod-ified the base surface only after the entire trajectorywas known. This proved unsatisfactory, as users com-plained loudly about the lack of intermediate feed-back. We addressed these concerns by incrementallyraising the ridge as the trajectory is specified. Thefollowing discussion is concerned only with the re-vised behavior.

While defining a trajectory of sampled points, theuser will occasionally move quickly enough that thereis a significant gap between points. To avoid thejaggy ridge that would result, we add additionalpoints along the line segment between widely spacedpoints, until there are no points that are more than asingle grid unit apart. This works fairly well becausewhen the user moves quickly, the motion is usuallyquite linear. Had this assumption proved to be in-correct, it would be straightforward to sample newpoints from a spline fit to the sensed positions.

As points are added to the trajectory, we computetheir effect on the ridge by applying a Gaussian bumpbased on the displacement of the point from Lb. How-ever, we do not take the naive approach of mergingthe effect of each point into the base surface before

the next point is handled. Figure 7 shows that doingso results in a ridge that is higher than the trajectoryprovided by the user. Our solution involves addinga new layer, the embossing layer Le, which storessurface modifications that have not yet been incor-porated into the base surface. The embossing layerfits between Lb and Lu (i.e. 0 = b < e < u < t),and differs from the previously introduced layers intwo ways. The first is that it overrides the default as-signment operator, and the second is that it mergessurface modifications into the base layer once theyexceed some threshold age.

5.2.1 Overriding the Default AssignmentOperator

We modify the default assignment behavior (definedat the beginning of this section) so that the new valueh is only assigned if its magnitude is greater than thatof the existing value. This is formalized in the follow-ing equation, where ←o and ←d respectively denotethe overridden and default assignment operators:

Le(x, y)←o h ≡{ |h| > |Le(x, y)| ⇒ Le(x, y)←d h|h| ≤ |Le(x, y)| ⇒ unchanged

(1)Regardless of whether the assignment makes a

change, the age associated with Le(x, y) is reset tozero.

5.2.2 Aging of Surface Modifications

Before discussing the details of how surface modifi-cations are aged, we first justify why we do it at all.Figure 8 depicts the desired behavior when a ridge,after being reduced in height, intersects itself. As thelower part of the ridge encounters the higher part, afurrow is cut through the higher part. In other words,we want the older part of the ridge to be treated as ifit were part of the base surface, even though we havediscussed why this is unsatisfactory for the leadingedge of the ridge. Our solution has two components.First, we increment the age of all edited grid points,and merge the offsets into the base layer when theage exceeds a threshold ta. Next, we reset the age ofany grid point to which we attempt to assign a new

9

Figure 8: Desired behavior of self-intersecting ridgewhen the height of the ridge is lower at the time ofintersection than at the start of the manipulation. Onthe left is a top-down view of such a ridge. On theright is a cross-sectional view of the intersection. Theheight of the ridge is reduced in the area bounded bythe blue rectangle.

value h, even if the assignment does not affect theheightfield because magnitude of h is less than theexisting value of Le(x, y).

5.2.3 Fixing Mistakes

The user will sometimes raise a ridge higher thandesired, and be unable to lower it because the de-sired height has a smaller magnitude than the exist-ing height. We solve this problem by not resettinggrid point ages when the input position is station-ary. We allow a small amount of leeway, since theuser will not be perfectly steady. Let ps representthe stationary point : the point that other points arecompared to to determine whether the user is sta-tionary. ps is initially set to the first input point. Weuse a threshold ts to define the region in which thefinger is considered to be stationary. If ‖ps − p‖ < ts,then no ages are reset with the edit region. Other-wise, the ages are reset, and the value of ps is set top. If the input point remains stationary for longerthan ta, then this aging behavior will have the resultof incorporating the entire ridge into the base layer.ps is now beneath the base surface.

Does this give the desired behavior? Consider abump raised to a height h1 that is higher than thedesired height h2. For simplicity, the height of thebase surface is initially zero. The desired heightof the bump at a point p in the neighborhood ofp2 = (x0, y0, h2) is given by h2(e−σ‖p0−p‖2

). If wefirst raise the bump too high to h1 and then correctit as described, the result will be

h1(e−σ‖p0−p‖2) + (h2 − h1)(e−σ‖p0−p‖2

)

which simplifies to the desired result:

h2(e−σ‖p0−p‖2)

.After experimenting with a variety of values for ta,

we found 0.5 seconds to be satisfactory. This providesthe stated benefit of aging without making the userwait annoyingly long to fix a mistake.

5.3 Finger Painting

The 3D haptic painting functionality of [8] allows theuser to paint directly on a virtual clay-like surface.Digital Clay could act as an input mechanism for fin-ger painting, where the color of the clay is changed bydrawing on it with the finger. As in the FEELEX sys-tem [13], a projector directed at the clay could showthe results of the painting. Alternately, advances inLEP (light emitting polymer) and OLED (organiclight emitting device) allow us to imagine a futureversion of Digital Clay that can change the color ofits surface; such a device would be perfect for digitalfinger painting.

5.4 Extrusion of Region

Teddy [12] interprets the user’s freeform 2D strokesto define plausible 3D models. Of particular interestto us are the multi-step operations such as extrusion.The user first draws on the surface the outline of thebase of the region that is to be extruded. A secondstroke defines how region is raised from the surface.

Multi-stroke operations can also be implementedfor Digital Clay. We again use extrusion as an ex-ample. The area surrounded by a closed ridge could

10

Figure 9: The simulation in use by a user, who hasjust finished raising a serpentine boss from the sur-face.

be interpreted as the base for an extrusion operation.The shape of the extruded volume could depend onthe point at which the extrusion is initiated, as well asthe subsequent trajectory of the finger. Such multi-stroke operations will allow greater control over theshape of Digital Clay.

6 Implementation

6.1 Control Flow: Haptic and ControlLoops

GHOST, the standard framework for developingPHANTOM applications, structures applications bysplitting execution into two loops. The haptic loop isin charge of generating feedback forces. The controlloop is responsible for updating the graphic display,and for computing the clay’s response to user input.

In order to maintain the illusion that a surface ex-ists, the haptic loop must have an update rate of atleast 1000Hz. This strict performance requirementmeans that the computations done by this loop mustbe kept simple. For example, operations that mod-ify entire regions of the surface run too slowly forthe haptic loop. Such computationally intensive op-erations are instead processed in the control loop,

which has a frequency of approximately 20Hz. Un-fortunately, changing the surface’s height at this rel-atively low rate results in perceptible jerkiness. Thenext section describes a new layer that addresses thisproblem.

6.2 Layers Revisited

The layers described thus far have both a conceptualand an implementational role. Each plays a part indescribing how the clay behaves, and each is includedin our implementation in a straightforward manner.

Other layers do not have a place in the concep-tual understanding of Digital Clay, and exist solelyfor implementation reasons. For example, we haveimplemented a type of layer that linearly interpo-lates between two reference surface shapes. This layeraddresses the problem of jerky height transitions bystoring the previous and current surface heights, andblending them. The interpolation parameter rangesfrom zero at the instant that the current height is set,to one just before the next height is set. The percep-tion is now that the height is changing smoothly.

7 Conclusion

This paper has introduced the concept of DigitalClay, and described the salient characteristics of theplanned physical prototype. We have described ourPHANTOM-based haptic simulation of a finger inter-acting with Digital Clay, emphasizing that we haverefrained from using any information that the physi-cal prototype will not have access to; in doing so, weare confident that the techniques we have developedcan be smoothly transferred to the physical proto-type.

We have coined the term finger sculpting to de-scribe interactions that involve manipulation of thesurface with one or more fingers. Due to the hard-ware at our disposal, we have developed techniquesthat work with a single finger.

Our research represents the first steps towards aninteraction lexicon for Digital Clay. We have de-scribed interaction elements that allow the clay tosense the position of the user’s touch, and to deter-

11

mine when the user intends to start and end editing.We also presented the blister, which allows the clayto maintain contact with the finger, even when thefinger is raised above the original level of the surface.

These interaction elements were combined to formtwo complete clay manipulation operations. We de-scribe how we have implemented digging holes andraising bumps, as well as a variable-height embossingoperation.

8 Acknowledgements

This work was supported by the National Sci-ence Foundation under grant NSF-ITR/PE+SYAward#:0121663, and by a Seed Grant from theGVU Center at the Georgia Institute of Technology.The authors thank Norberto Ezquerra for providingaccess to a Phantom.

References

[1] Allen, M., Book, W., Ebertuphoff, I.,

Rosen, D., and Rossignac, J. Digital Clayfor Shape Input and Display: NSF-ITR/PE+SYAward#:0121663, September 2001.

[2] Ambrosi, G., Bicchi, A., Rossi, D. D., and

Scilingo, P. The role of contact area spreadrate in haptic discrimination of softness. In Pro-ceedings of IEEE International Conference onRobotics and Automation (May 1999), IEEE,pp. 305–310.

[3] Barr, A. H. Global and local deformationsof solid primitives. In Proceedings of ACMSIGGRAPH (July 1984), ACM SIGGRAPH,pp. 21–30.

[4] Capell, S., Green, S., Curless, B.,

Duchamp, T., and Popovic, Z. A multireso-lution framework for dynamic deformations. InProceedings of ACM SIGGRAPH (2002), ACMSIGGRAPH.

[5] Choi, K. S., Sun, H., Heng, P. A., and

Cheng, J. C. Y. A scalable force propaga-

tion approach for web-based deformable simula-tion of soft tissues. In Proceeding of the seventhinternational conference on 3D Web technology(2002), ACM Press, pp. 185–193.

[6] Coquillart, S. Extended free-form deforma-tion: A sculpting tool for 3d geometric model-ing. In Proceedings of ACM SIGGRAPH (Au-gust 1990), ACM SIGGRAPH, pp. 187–196.

[7] Debunne, G., Desbrun, M., Cani, M.-P.,

and Barr, A. H. Dynamic real-time defor-mations using space time adaptive sampling. InProceedings of ACM SIGGRAPH (2001), ACMSIGGRAPH, pp. 31–36.

[8] Ehmann, S. A., Gregory, A. D., and Lin,

M. C. A touch-enabled system for multi-resolution modeling and 3D painting. The Jour-nal of Visualization and Computer Animation12, 3 (2001), 145–157.

[9] Hayward, V., and Cruz-Hernandez, J. M.

Tactile display device using distributed lateralskin stretch. In Proceedings of the Haptic In-terfaces for Virtual Environment and Teleoper-ator Systems Symposium (ASME IMECE2000)(2000), pp. 1309–1314.

[10] Higashi, M., Aoki, N., and Kaneko, T.

Application of haptic navigation to modifyfree-form surfaces through specified points andcurves. In Proceedings of the seventh ACM sym-posium on Solid modeling and applications (June2002).

[11] Hsu, W. M., Hughes, J. F., and Kaufman,

H. Direct manipulation of free-form deforma-tions. Computer Graphics 26, 2 (1992), 177–184.

[12] Igarishi, T., Matsuoka, S., and Tanaka,

H. Teddy: A sketching interface for 3d freeformdesign. In Proceedings of ACM SIGGRAPH(1999), ACM SIGGRAPH, pp. 409–416.

[13] Iwata, H., Yano, H., Nakaizumi, F., and

Kawamura, R. Project feelex: Adding hap-tic surface to graphics. In Proceedings of ACM

12

SIGGRAPH (August 2001), ACM SIGGRAPH,pp. 469–476.

[14] James, D. L., and Pai, D. K. A unified treat-ment of elastostatic contact simulation for realtime haptics. Haptics-e, The Electronic Journalof Haptics Research 2, 1 (September 2001).

[15] Johnson, K. O. The roles and functions ofcutaneous mechanoreceptors. Current Opinionin Neurobiology 11 (2001), 455–461.

[16] McDonnell, K. T., and Qin, H. FEM-basedsubdivision solids for dynamic and haptic in-teraction. In Proceedings of 6th Symposium onSolid Modeling and Application, pp. 312–313.

[17] Pierce, J. S. Expanding the Interaction Lex-icon for 3D Graphics. PhD thesis, CarnegieMellon University, School of Computer Science,Pittsburgh, Pennysylvania 15213, 2001.

[18] Sederberg, T. W., and Parry, S. R. Free-form deformation of solid geometric models.In Proceedings of ACM SIGGRAPH (August1986), ACM SIGGRAPH, pp. 151–160.

[19] Velho, L., Perlin, K., Ying, L., and Bier-

mann, H. Procedural shape synthesis on sub-division surfaces. In SIBGRAPI 2001 - XIVBrazilian Symposium on Computer Graphicsand Image Processing (October 2001).

[20] Wellman, P. S., Peine, W. J., Favalora,

G. E., and Howe, R. D. Mechanical designand control of a high-bandwidth shape memoryalloy tactile display. In International Symposiumon Experimental Robotics (June 1997).

13