Rendering detailed haptic texturesvcg.isti.cnr.it/vriphys05/material/paper45.pdfWorkshop On Virtual Reality Interaction and Physical Simulation (2005) F. Ganovelli and C. Mendoza (Editors)

Workshop On Virtual Reality Interaction and Physical Simulation (2005)F. Ganovelli and C. Mendoza (Editors)

Rendering detailed haptic textures

Víctor Theoktisto† Marta Fairén Isabel Navazo Eva Monclús2

Dept. of Llenguatges i Sistemes Informàtics 2 Institute of Robotics and Informatics{vtheok,mfairen,isabel}@lsi.upc.edu [email protected]

Universitat Politècnica de Catalunya

AbstractRendering haptic textures seamlessly out of triangle meshes just by using geometry requires heavy work and doesnot allow high sampling rates for detailed, rugged models. Better approaches simulate surface texture withoutincreasing much the complexity or processing cost, at the expense of fidelity of perception. We propose a methodfor rendering local height field maps out of an underlying triangle mesh, which relies in a space subdivisionrepresentation based on octrees for collision detection, and rendering individual surface detail by modulatingforce response using local height fields. We compare our method against a force mapping implementation forrendering/perceiving the same models with textures using normal maps. The proposed technique allows for realtime perception of 3D surface detail, allowing the user to perceive the best haptic rendering alternative for a givenmodel. Some experimental results are presented to show the goodness of the approach.

Categories and Subject Descriptors(according to ACM CCS): I.3.3 [Computer Graphics]: Haptic Rendering

Figure 1: Height field texture rendering

1. Introduction

Haptic systems provides unique and bidirectional communi-cation between humans and virtual environments in a man-ner much closer to usual physical environment manipulation.Haptic interfaces enables direct interaction with computer-generated objects, and when coupled with an intuitive vi-

† On leave from Universidad Simón Bolívar, Caracas, Venezuela

sual display of complex data raise applications to new levels;these applications include molecular docking, nanomaterialmanipulation, surgical training, virtual prototyping, machineassembly and digital sculpting.

Haptics drives the development of new algorithms forphysical object and surface modeling, volume processing,and the visualization of novel data structures able to encodeshape and material properties. From single one-point based,single person operation to multipoint, multihand, and multi-person interaction scenarios, an enticingly rich interactivityis within reach of many computer graphics applications.

One of the interesting applications on haptic perception isto be able to feel, with a haptic device, variations in texture,roughness or details of the surface being contacted. Therehas been some work done in this direction, but none com-pare the results of applying different techniques to the samemodel. The general idea, as in visualization, would be thesimulation of roughness and other surface features withouthaving to increase the geometric density of the model.

In this paper, we compare three possible solutions, anal-izing their advantages and disadvantages, being the last oneour solution for haptic perception. The contribution is a spe-cific model for simulating haptic perception of surface de-

c© The Eurographics Association 2005.

V. Theoktisto, M. Fairén, I. Navazo & E. Monclús / Rendering detailed haptic textures

tails, by using a per-face height field as a texture to bemapped onto the objects’ geometry. We achieve a good cor-respondence between surface detail visualization and hapticperception (figure 1).

The paper is organized as follows: in section 2 we presentrelated work recently done in haptic rendering. Section 3 ex-plains the solutions we have compared, with one of thembeing the approach we present in this paper. In section 4 re-sults are presented and discussed, and finally the conclusionsand future work.

2. Related work

The term haptic rendering is applied to the generation andrendering of haptic virtual objects [LS04]. Although therehas been some work inpseudo hapticsby simulating sur-face properties using common computer mice [LBE04], it ismore common the use of a specialized haptic interface de-vice producing a force-feedback response. Users can man-ually navigate, explore and feel the shape and surface de-tails of virtual objects in the growing field ofcomputer hap-tics [BS02]. As sampling rates for sensing devices standard-izing in the 1000 Hz range, as in the Phantom or HapticMas-ter device [FCS02], efficient haptic-interaction techniquesmay go beyond the simple detection of geometric primitives,towards real-time rendering of arbitrary surfaces of irregulardetail or convexity, conveying surface geometry and materialproperties, without forgetting its role of user-interaction de-vice for high level event input, recognition of haptic “icons”and general haptic user interfaces or HUIs [ME03].

The most simple haptic model uses simple surfaces, apoint-based device and a force-collision detection algo-rithm [OL05]. A haptic cursor representing a force-feedbackdevice is placed into a 3D environment, and a high speedevent loop checks whether it collides with a surface of an ob-ject, after which it produces a repulsing force of varying di-rection and magnitude, which physically acts upon the forceexerted by the user in the haptic device, correcting any pen-etration ruled out by the object’s geometry [MH04].

Using a ray-based rendering algorithm with the samesetup allows perceiving torque and force-torque feedbackmechanisms [MS05], while using a third object as aprobe allows also texture diferentiation and shape percep-tion [OL03].

There is an ongoing effort for developing libraries andframeworks to incorporate haptic capabilities to applica-tions, such as OpenHaptics(TM) [IHZ05] with separatemodules for haptic device and user interaction. Other ap-proach is connecting a haptic rendering library, its graphicaltwin, and a collision detection module in a generic frame-work [LBDM04].

These efforts choose among several alternatives for mod-eling and rendering surfaces. Johnson [JW04] uses a pure

geometric modeling approach for haptically render arbitrarypolygons using neighborhood proximities to reduce compu-tational load. Some haptic techniques and approaches can befound in [BS02], with a treatment of “force mapping” thatsimulates detail at higher resolutions by perturbing surfacenormals, therefore modifying the surface texture perceivedby the haptic probe. Potter [PJC04] provides a simple modelto perceive the haptic texture of large heightfield objects.Multiresolution techniques [AB01] are also used to generatesmooth geometry from large heightfield objects, allowing abetter simulation of terrain models.

Based on all these works we can summarize a taxonomyfor haptic detail rendering, which determines the particularalgorithm to be used.

• Geometry, rendering the surface as detailed polygonalmeshes, NURBS, or point clouds, and detecting collisionsagainst the surfaces.

• Surface Relief Detail, in which a haptic texture is sampledin lieu of the actual surface. On its own, haptic texturesmay be based on bump maps or height fields.These approaches may also allow managing LOD in thevisual and haptic resolutions, and having a repository ofassorted relief textures.

In the next section, we present a comparison among threedifferent techniques for thesurface relief detailhaptic per-ception (see figure 2).

3. Models for haptic perception of surface details

The simulation of surface details in very complex models hasnot been a problem from the visualization point of view sincethe late 70’s. Algorithms such as the use of colored texturesor bump-mapping are well known in the literature [Bli78].

In the case of haptic perception, as we have seen in sec-tion 2, any simulation algorithm should be efficient enoughto achieve the high frequency updates (1000 Hz) required bythe force feedback devices.

Our objective is to find an algorithm to allow haptic per-ception of surface details in objects represented by trianglemeshes, and to compare it to other known solutions.

3.1. Generation of geometry

A direct algorithm simulates surface details by generatingthe local geometry corresponding to these details. This ge-ometry can be calculated as a preprocess for the wholemodel, or locally for each triangle of the mesh at run time,by deciding which triangles of the mesh are inside the hapticinfluence area.

A collision detection algorithm then tracks the hapticprobe (orgod-object), which is represented in the screenby a proxy. When moving freely in empty space, its coor-dinates coincide. When thegod-objecthits a triangle, the



(a) Geometry (b) Force mapping (c) Height fields

Figure 2: Different approaches to simulate surface details in haptic perception

proxyclamps to the collision point, and a force is exerted inthe haptic device to push thegod-objectto the nearest pointin the hit triangle, with theproxy following suit along thesurface.

In the afore-mentioned cases, the cost in time and/or spaceof modeling detailed surface using just geometry is not af-fordable given the frequency updates requirement. Havingmany small triangles, the haptic probe may well be at an-other point in the surface (a far-off triangle) after detecting acollision, and the resulting force interplay can cause unstablebehavior, adding spurious force feedback, inducing vibrationor simply wrong sampling (haptic aliasing).

3.2. Force mapping

One of the algorithms used in visualization to simulate sur-face details is thebump-mappingalgorithm. It consists of atexture map where each value corresponds to a normal vec-tor direction (RGB corresponds to XYZ coordinates respec-tively), which can be different for each point in the map. Adiscretization of the triangle’s surface is achieved by map-ping texture points to corresponding points within the meshtriangle, having a different normal vector at each point. Thevisualizationbump-mappingalgorithm uses these normalvectors at the discrete points to calculate illumination val-ues inside the triangle, simulating the surface details.

The haptic perceptionbump-mappingalgorithm (alsocalled force mapping) [BS02] is based on the same princi-ple. It uses the normal vector at discretized surface points tocalculate the force direction and magnitude to be applied tothe haptic device when it collides with the triangle (see fig-ure 3). The algorithm used in this case can be described asfollows:

for eachnew haptic device positiondodetectcollision with the meshif collision existsthen

// we have the collision point in the trianglecalculateforce direction from the map at that pointapplythis force to the haptic device

endifendfor

By using this haptic perception algorithm and implement-ing the bump-mappingvisualization as a GPU shader, weachieve a correct perception of surface roughness when thedistortion is not very high. This is because the collision is al-ways detected against the triangle of the mesh, so a percep-tion of displacement upward or downward from the trianglesurface is not possible (more detailed results in section 4).

Figure 3: In 2D: correspondence between simulated surface(up) and normal vectors (down).

3.3. Height field haptic rendering

Height field rendering has been used primarily as a techniquefor imaging large datasets out of GIS data. These are usuallyhuge files stored as RGBα images, with the RGB providingtexture information and theα channel providing the verticalheight. A grid is used to scan the height field, interpolatingand filtering values as needed, and creating a correspondinggeometry.

For haptic rendering in this sort of applications (terrainmodels), theproxy’s calculated position must correspondaccurately to that of the trackedgod-object’s, because themagnitude of the height field map is usually very big andsmall differences may cause wrong perception. These mod-els generally render a whole terrain as a gigantic height fieldmap [PJC04].

The requirements can change slightly when the objectiveis not a GIS application. In our case, our interest is usingheight field mapping applied to haptically render a trianglemesh model, in which different height field textures can beapplied to different triangles. Another requirement in thissort of applications is that the triangles are not as big asthe whole model, and actually in many cases they are small



enough to be represented in a small area of the screen. Thisimplies that the accuracy required for computing theproxy’sposition as explained above is not so strict in this case, sincewe want to simulate small surface details in asingletriangleof the mesh. On the contrary, it becomes much more impor-tant to be able to determine, at a low cost, which triangleis being collided by the haptic probe, so the haptic renderalgorithm can apply the appropriate height field mapping.

In order to fulfil this requirements our algorithm combinesdifferent techniques to make this haptic rendering as efficientas possible:

• the triangle mesh is represented by an octree space de-composition, which allows a very quick detection of thearea where the haptic probe is located, discarding early inthe process a very high amount of geometry of the modelwhich does not need to be considered in the rest of thealgorithm;

• the collision with a triangle of the mesh is detected againsta prism created between the triangle,<V1, V2, V3>, anda copy of the triangle,<W1, W2, W3>, displaced over acertain maximum distance,mh, for the height field mapvalues (see figure 5(a)). All triangles created to define thisprism share the same identifier with the original surfacetriangle of the mesh, so the computation of the right trian-gle is inmediate;

• a potential collision against any side of the prism triggersthe height field haptic rendering of a small grid cell justlying in a triangle under the probe, shadowing the probe’smovement (Ct ). If at any time the probe descends belowthe signaled height for that cell, a repealing force is ap-plied to the haptic probe along the surface normal at theprojected point, proportional to the height difference (orpenetration). This forces the god-object onto the surface atthat point, at which the force ceases to be (see figure 5(b));

• the height field maps are forced to displacement zero onthe edges of the mapped triangle, avoiding texture discon-tinuities on these edges that can cause instability in thehaptic device;

The algorithm used for this approach works as follows(see figure 5):

for eachnew haptic device positiondodetectthe octree nodedetectcollision with a triangle prismif collision existsthen

// thegod-objectposition is inside the triangle prismproject thegod-objectposition to the triangle of the meshobtainthe height of this point from the height fieldif (distance(god-object, triangle) < height field)then

// we had a collisioncalculatean outward force proportional to therelative height differencesapplythis force to the haptic device

endifendif

endfor

By forcing the height field to distance zero on the edges ofeach triangle, we have C0-continuity on the triangle edges.This solution avoids the problem of having different heightscomming from the two triangles sharing the edge. In case ofconvex edges, this solution avoids possible instabilities at theedges of triangles, but this is not enough for concave edges.

When two triangles share a concave edge thegod-objectposition can be inside two triangle prisms at the same time.In this case, the algorithm compares the heights in the tex-tures of both triangles with thegod-objectposition. We as-sume the model will be correct, so thegod-objectcannothave contact with both heights at the same time. However, ifthe model is correct but the heights are quite near in space(as ilustrated in figure 4), the user can still perceive some in-stability because of the different forces applied to very nearpoints. However, these cases are rare in the most commonapplications.

Triangle A

Area of instability

Triangle B

Figure 4: Possible case of instability

4. Results

In order to compare the quality of haptic perception usingboth force mapping and height fields, we decided to set upthe environment experimental conditions as close as possiblefor each run. This means using the same base models forboth runs, and an exact correspondence between the reliefmap used as a height field and the normals map used forforce mapping.

First, a sphere, built recursively at several resolutions outof a perfect icosahedron was chosen as the base model. Forthe sake of enhancing the features detailed in the article, ithas been set to the simplest possible resolution for the fig-ures appearing in this article, to better appreciate visuallythe rendered surface texture.

Secondly, we generated synthetic height field maps, andused it to compute a corresponding normal map, using a plu-gin for the open source GIMP application. In this way we ob-tained corresponding pairs of height field and normal mapsrepresenting the same 3D geometry, as shown on figure 6. Inthe end we settled for the three textures shown here, sinceeach allows to perceive different surface characteristics.



rv 1 v 2

v 3 w 2w i = v i + n * m h w 3

m h w 1 c tc t

(a) Triangle box collision area

c tf p r o x y

H F u v p e n e t r a t i o ng o d � o b j e c t p o s i t i o n

(b) Collision point computation

Figure 5: Height field collision mapping

(a) Normalmap (ovals)

(b) Normalmap (cross)

(c) Normalmap (warts)

(d) Heightfield (ovals)

(e) Heightfield (cross)

(f) Heightfield (warts)

Figure 6: Normals and Heights maps

The textures chosen are one with alternating polished andvariable bumpy areas, one with gently sloping circles with asoft gradient, and one with an embossed cross with only flatand vertical surfaces. Additionally, taking advantage of thegraphics card hardware, we implemented the shading partcorresponding to our bump mapping as a GPU shader, justto allow a better haptic sampling rate.

The equipment used is a HapticMASTER from FCS, hav-ing a built-in wide haptic 3D space, able to exert forces froma delicate 0.01 N upto a very heavy 250 N, with a wide 3Dperception field, equivalent to a wedge of 40 cm x 36 cm x 1radian. The PC is a 2GHz Pentium IV with an ATI Radeon9700 graphics card.

We show on figure 8, different renderings using forcemapping and height fields. It is evident that bump mappingmakes a smoother visualization. However, in terms of haptic

perception, the comparisons are quite different and dependon the characteristics of the texture map.

We studied the haptic perception of the user in both meth-ods, force mapping and our height field algorithm, with sev-eral texture maps with diferent characteristics:

• In the case of a texture map like the one shown in fig-ures 8(a) and 8(b), where the texture image shows twobumpy areas, the resulting perception is somehow similarin both methods. The user perceives a certain roughnessin the fine bumpy area and a bumping feeling and certainguidance among bumps in the coarse bumpy area.

• In the case of a texture like the one shown in figures 8(c)and 8(d), where the texture image shows a big oval andtwo smal bumps over the surface, the resulting perceptionis a bit different beetween the two methods. In the partof small bumps there is almost no difference, but in thebig oval part, when the user is inside the oval, althoughin both cases there is some similar resistance to go out,with the force mapping method the user only perceives theresistance for going up to the oval while with the heightfield method the perception is clearly going up to the ovaland down from it.

• In the case of a texture like the one shown in figures 8(e)and 8(f), where the texture image shows a cross step overthe surface, the perception is clearly different between thetwo methods. With the force mapping method the useronly perceives resistance on the going up and a jumpgoing down, but no height differences can be perceived.Moreover, there are some instabilities on this perceptionin the areas of the middle of the cross, because there areneighbour points with very different normal directions.With the height field method the user perception is muchbetter in this case, because the parts of the cross goingup and down give the feeling of going up and down withdifferent height on the top of the cross than on the base.There is no instability either in those areas in the middleof the cross.

In order to apply our method for perceiving scratches onthe surface [BPMG04], we have extended our method in or-



der to accept texture height field maps having negative val-ues, so we can represent negative height fields (see figure 7).In these cases, the force mapping is not able to give the cor-rect perception because there are neighbour points with verydifferent normals which cause haptic instability. Our heightfield algorithm allows a correct perception of this sort ofcharacteristics as well.

Figure 7: Example of a negative height field

As a summary, we can conclude that the force mappingmethod can be a good approximation for modeling an appar-ent roughness of material, but is not sufficient for irregularnormal maps where the perception has to be tight to the tex-ture shape we want to simulate. This problem is solved withour height field algorithm, which gives an accurate sense ofthe surface characteristics.

5. Conclusions

We have developed a practical method for rendering localhaptic textures in triangle meshes, which allows a user toperceive exact surface details at several resolutions. This ex-tends the use of height field haptics beyond the usual fieldof gigantic terrain textures and allows perceiving surface de-tails without modeling them. This approach can be used forlocally mapping relief textures in triangular meshes and hap-tically render them in real time.

We are extending further this research by exploring a su-perposition of approaches: a force map rendering applied toa height field on top of a triangle mesh. This is possible byusing only one texture, combining normal and height valueshaving 3 values for the normal x, y, z coordinates and thefourth one for the height value. This extension would allow abetter perception of those height fields simulating relativelyhigh slopes, because the collision force applied to the hapticdevice would have the direction of the normal to the simu-lated surface, instead of the normal to the base triangle.

Another work for the future is the study of using a globaltexture, ie. a texture mapped to more than one triangle, al-lowing continuity on the height values on the edges of trian-gles.

Acknowledgements

We want to thank Iban Lozano for his valuable help onthe implementation of the force mapping method, which al-lowed us to compare it with our solution.

This work has been partially co-financed by the projectTIN2004-08065-C02-01 of the spanish government (MEC)and FEDER funding.

References

[AB01] A SGHAR M. W., BARNER K. E.: Nonlinear Multi-resolution Techniques with Applications to ScientificVisualization in a Haptic Environment.IEEE Trans-actions on Visualization and Computer Graphics 7, 1(January/March 2001), 76–93.

[Bli78] B LINN J. F.: Simulation of wrinkled surfaces. InPro-ceedings of SIGGRAPH’78(1978), ACM.

[BPMG04] BOSCHC., PUEYO X., MÉRILLAU S., GHAZANFAR-POUR D.: A physically-based model for renderingrealistic scratches.Computer Graphics Forum 23, 3(2004).

[BS02] BASDOGAN C., SRINIVASAN M. A.: Haptic renderingin virtual environments. InHandbook of Virtual Envi-ronments, Stanney K., (Ed.). London, Lawrence Earl-baum, Inc., 2002, ch. 6, pp. 117–1343, 2212–2218.

[FCS02] FCS CONTROL SYSTEMS: HapticMASTER Installa-tion Manual. The Netherlands, December 2002.

[IHZ05] I TKOWITZ B., HANDLEY J., ZHU W.: The OpenHap-tics? Toolkit: A Library for Adding 3D Touch? Navi-gation and Haptics to Graphics Applications. InPro-ceedings of the First Joint Eurohaptics Conference andSymposium on Haptic Interfaces for Virtual Environ-ment and Teleoperator Systems(2005).

[JW04] JOHNSON D. E., WILLEMSEN P.: Accelerated Hap-tic Rendering of Polygonal Models through Local De-scent. InProceedings of the 12th International Sympo-sium on Haptic Interfaces for Virtual Environment andTeleoperator Systems (HAPTICS?04)(2004).

[LBDM04] L UCIANO C., BANERJEE P., DEFANTI T., MEHRO-TRA S.: Realistic Cross-Platform Haptic ApplicationsUsing Freely-Available Libraries. InProceedings of the12th International Symposium on Haptic Interfaces forVirtual Environment and Teleoperator Systems (HAP-TICS?04)(2004).

[LBE04] LÉCUYER A., BURKHARDT J.-M., ETIENNE L.:Feeling Bumps and Holes without a Haptic Interface:the Perception of Pseudo-Haptic Textures. InProceed-ings of CHI 2004(2004).

[LS04] LIN M., SALISBURY K.: Haptic rendering – beyondvisual computing.IEEE Computer Graphics and Ap-plications(March/April 2004), 22–23.

[ME03] MACLEAN K., ENRIQUEZ M.: Perceptual design ofhaptic icons. InProceedings of the Eurohaptics 2003(Dublin, Ireland, 2003).



(a) Warts (using force maps) (b) Warts (using height fields)

(c) Ovals (using force maps) (d) Ovals (using height fields)

(e) Cross (using force maps) (f) Cross (using height fields)

Figure 8: Height field collision mapping

[MH04] M ELDER N., HARWIN W. S.: Extending the FrictionCone Algorithm for Arbitrary Polygon Based HapticObjects. InProceedings of the 12th International Sym-posium on Haptic Interfaces for Virtual Environmentand Teleoperator Systems (HAPTICS?04)(2004).

[MS05] MARTÍN J., SAVALL J.: Mechanisms for Haptic

Torque Feedback. InProceedings of the First JointEurohaptics Conference and Symposium on Haptic In-terfaces for Virtual Environment and Teleoperator Sys-tems(2005).

[OL03] OTADUY M. A., L IN M. C.: Sensation preserving sim-plification for haptic rendering.ACM SIGGRAPH 2003



/ ACM Transactions on Graphics 22(July 2003), 543–553.

[OL05] OTADUY M. A., L IN M. C.: Stable and Respon-sive Six-Degree-of-Freedom Haptic Manipulation Us-ing Implicit Integration. InProceedings of the FirstJoint Eurohaptics Conference and Symposium on Hap-tic Interfaces for Virtual Environment and TeleoperatorSystems(2005).

[PJC04] POTTER K., JOHNSON D., COHEN E.: Height FieldHaptics. InProceedings of the 12th International Sym-posium on Haptic Interfaces for Virtual Environmentand Teleoperator Systems (HAPTICS?04)(2004).


Documents

Rendering detailed haptic texturesvcg.isti.cnr.it/vriphys05/material/paper45.pdfWorkshop On Virtual Reality Interaction and Physical Simulation (2005) F. Ganovelli and C. Mendoza (Editors)