Tactile Perception of Textiles in a Virtual-Reality System (Cognitive Systems Monographs)

Cognitive Systems Monographs

Volume 10

Editors: Rüdiger Dillmann · Yoshihiko Nakamura · Stefan Schaal · David Vernon

Dennis Allerkamp

Tactile Perception of

Textiles in a Virtual-Reality

System

ABC

Rüdiger Dillmann, University of Karlsruhe, Faculty of Informatics, Institute of Anthropomatics,

Humanoids and Intelligence Systems Laboratories, Kaiserstr. 12, 76131 Karlsruhe, Germany

Yoshihiko Nakamura, Tokyo University Fac. Engineering, Dept. Mechano-Informatics, 7-3-1 Hongo,

Bukyo-ku Tokyo, 113-8656, Japan

Stefan Schaal, University of Southern California, Department Computer Science, Computational Learn-

ing & Motor Control Lab., Los Angeles, CA 90089-2905, USA

David Vernon, Khalifa University Department of Computer Engineering, PO Box 573, Sharjah, United

Arab Emirates

Author

Dipl.-Math. Dennis Allerkamp

Gottfried Wilhelm Leibniz Universität HannoverInstitut für Mensch-Maschine-KommunikationFachgebiet Graphische DatenverarbeitungWelfengarten 130167 HannoverGermanyE-mail: [email protected]

ISBN 978-3-642-13973-4 e-ISBN 978-3-642-13974-1

DOI 10.1007/978-3-642-13974-1

Cognitive Systems Monographs ISSN 1867-4925

Library of Congress Control Number: 2010929202

c© 2010 Springer-Verlag Berlin Heidelberg

This work is subject to copyright. All rights are reserved, whether the whole or part of the material isconcerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting,reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publicationor parts thereof is permitted only under the provisions of the German Copyright Law of September 9,1965, in its current version, and permission for use must always be obtained from Springer. Violationsare liable for prosecution under the German Copyright Law.

The use of general descriptive names, registered names, trademarks, etc. in this publication does notimply, even in the absence of a specific statement, that such names are exempt from the relevant protectivelaws and regulations and therefore free for general use.

Typeset & Cover Design: Scientific Publishing Services Pvt. Ltd., Chennai, India.

Printed on acid-free paper

5 4 3 2 1 0

springer.com

Preface

This monograph originates from my work on the HAPTEX project. In De-cember 2004 Prof. Franz-Erich Wolter, the head of the Institute of Man–Machine Communication of the Leibniz Universitat Hannover, offered methe opportunity to participate in that EU funded project. Being a mathe-matician I had only very little experience in the field of haptic simulationin those days, but Prof. Wolter trusted in my ability to become acquaintedwith new fields of research in a very short time. I am still thankful for theconfidence he has shown me.

Since then I indeed learned and found out a lot. With this monograph Itry to pass on the knowledge I gained. Having a reader in mind who—likeme at the beginning of the project—has no background in psychophysics,neurophysiology or textile engineering I will provide the necessary basics. Theskilled reader may safely skip these parts. Nevertheless I presume some basicknowledge in mathematics. I hope that this thesis might help a newcomer todiscover the fascinating field of tactile simulation.

This work would not have been possible without the funding of the project“HAPtic sensing of virtual TEXtiles” (HAPTEX) under the Sixth Frame-work Programme (FP6) of the European Union (Contract No. IST-6549).The funding is provided by the Future and Emerging Technologies (FET)Programme, which is part of the Information Society Technologies (IST)programme and focuses on novel and emerging scientific ideas. For invent-ing this project and accordingly requesting the funding I thank Prof. Na-dia Magnenat-Thalmann, Dr. Harriet Meinander, Prof. Franz-Erich Wolter,Dr. Ian Summers and P.Eng. Fabio Salsedo.

Special thanks go to my thesis advisor Prof. Franz-Erich Wolter for hisguidance and support throughout my work on my PhD thesis which hasgreatly benefited from all his feedback. I hope I inherited some of his goodsense for interesting and profound research.

My research was influenced a lot by Dr. Ian Summers who originallyworked on the subject of tactile simulation. He not only provided several

VI Preface

tactile displays for the experiments but also shared a lot of his knowledgeand experience. I really appreciate I was able to work with him.

I also thank Prof. Nadia Magnenat-Thalmann for the coordination of theHAPTEX project. She always applied very high standards to our researchwork. This was sometimes very stressful but the project would not have beennearly as successful as without her leadership. I appreciate her willingness tobe a member of my examination committee.

I really enjoyed the interdisciplinary collaboration within the HAPTEXproject. I thank all the team members for the interesting discussions and thepositive working atmosphere. I would like to give a special mention to GuidoBottcher; we worked closely together on the haptic rendering software andcomplemented each other very well. It was a pleasure to work with him.

I had the pleasure of working with some very talented students whocontributed to my research: I thank (in alphabetical order) Steffen Blume,Daniel Glockner, Tjard Kobberling, Karin Matzke and Natalya Obydenna. Ialso thank Steffen Blume, Wiebke Frey and Iris Lieske for proofreading themanuscript and for the helpful discussions. The experiments conducted formy research were only possible because numerous persons voluntarily partic-ipated as probands. I thank all of them for their support. Finally, I reallyappreciate the moral support of my colleagues, friends and especially of myfamily. Thank you!

Hannover, March 2010 Dennis Allerkamp

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Human Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1 Introduction to Psychophysics . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1 Just Noticeable Differences and Weber’s Law . . . . . . . 62.1.2 Weber-Fechner Law and Stevens’ Power Law . . . . . . . . 72.1.3 Measuring Psychometric Distances . . . . . . . . . . . . . . . . 82.1.4 Statistical Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.1.5 Signal Detection Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 102.1.6 Multidimensional Scaling . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2 The Human Nervous System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.2.1 Neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.2.2 Neural Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.2.3 Sensory Reception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.3 Human Tactile Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.3.1 Mechanoreceptive Afferents . . . . . . . . . . . . . . . . . . . . . . . 182.3.2 Perception of Form and Texture . . . . . . . . . . . . . . . . . . . 192.3.3 Perception of Fine Textures . . . . . . . . . . . . . . . . . . . . . . . 202.3.4 Perceptional Model of Tactile Simulation . . . . . . . . . . . 20

2.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3 Devices for Tactile Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.1 Survey of Existing Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.1.1 Electromagnetic Displays . . . . . . . . . . . . . . . . . . . . . . . . . 263.1.2 Pneumatic Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.1.3 Displays with Shape Memory Alloys . . . . . . . . . . . . . . . 273.1.4 Piezoelectric Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

VIII Contents

3.1.5 Other Actuator Mechanisms . . . . . . . . . . . . . . . . . . . . . . 283.2 The Tactile Displays Used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.2.1 The Piezoelectric Effect . . . . . . . . . . . . . . . . . . . . . . . . . . 293.2.2 Mechanical Behaviour . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.2.3 Geometrical Configurations . . . . . . . . . . . . . . . . . . . . . . . 31

3.3 Drive Electronics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.3.1 USB Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.3.2 Data Bus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373.3.3 Variable Voltage Supply . . . . . . . . . . . . . . . . . . . . . . . . . . 393.3.4 Digital–Analogue Conversion . . . . . . . . . . . . . . . . . . . . . 393.3.5 Amplification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.4 Force-Feedback Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423.4.1 The Haptic Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.4.2 Modeling the Contact Forces . . . . . . . . . . . . . . . . . . . . . . 433.4.3 Different Degrees of Freedom . . . . . . . . . . . . . . . . . . . . . 44


4 Generation of Virtual Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.1 Sample Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

4.1.1 Fibres, Yarn and Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . 504.1.2 Selection of Sample Fabrics . . . . . . . . . . . . . . . . . . . . . . . 52

4.2 Kawabata Evaluation System for Fabrics . . . . . . . . . . . . . . . . . 534.2.1 KES-F Roughness Test . . . . . . . . . . . . . . . . . . . . . . . . . . . 544.2.2 KES-F Friction Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

4.3 Spatial Frequency Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564.3.1 Selection of Appropriate Sections . . . . . . . . . . . . . . . . . . 574.3.2 Discrete Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . 584.3.3 Two-Dimensional Composition . . . . . . . . . . . . . . . . . . . . 60

4.4 The Correlation–Restoration Algorithm . . . . . . . . . . . . . . . . . . 614.4.1 Data Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624.4.2 Two-Dimensional Composition . . . . . . . . . . . . . . . . . . . . 63

4.5 Surface Reconstruction from an Optical Surface Scan . . . . . . 644.5.1 Symmetry Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654.5.2 Shape from Shading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

4.6 Artificial Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 684.6.1 Brownian Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69


5 Tactile Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755.1 Rendering Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

5.1.1 Position Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 775.2 Experiment with a Force-Feedback Device . . . . . . . . . . . . . . . . 80

5.2.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Contents IX

5.2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815.3 Vibrotactile Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

5.3.1 Computation of Resulting Vibrations . . . . . . . . . . . . . . 855.3.2 Decomposition of Vibrations into Base

Frequencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 865.3.3 First Results with Brownian Surfaces . . . . . . . . . . . . . . 885.3.4 Further Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

5.4 Integration with a Force-Feedback Device . . . . . . . . . . . . . . . . . 915.4.1 Physical Simulation and Haptic Rendering . . . . . . . . . . 935.4.2 Integrated Interface Hardware . . . . . . . . . . . . . . . . . . . . 945.4.3 Software Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 955.4.4 Evaluation of the Integrated System . . . . . . . . . . . . . . . 96


6 Summary and Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

A Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Acronyms

ACM autocorrelation matrixCRA correlation–restoration algorithmDAC digital–analogue converterDFT discrete Fourier transformDIP dual in-line packageDOF degrees of freedomFFT fast Fourier transformfps frames per secondIC integrated circuitIPC inter-process communicationjnd just noticeable differenceKES-F Kawabata evaluation system for fabricslpi lines per inchMDS multidimensional scalingOPA operational amplifierPC Pacinian mechanoreceptive afferentRA rapidly adapting mechanoreceptive afferentrms root mean squareSA1 slowly adapting type 1 mechanoreceptive afferentSA2 slowly adapting type 2 mechanoreceptive afferentSFM statistical feature matrixSFS shape from shadingUSB Universal Serial BusVR virtual reality

Chapter 1

Introduction

Virtual Reality (VR) has a lot of applications ranging from entertainment to mechan-

ical design and medical training. VR systems are often used in training situations

where training in a real environment would be inappropriate and possibly even dan-

gerous. Pilots, for example, often practise on a flight simulator before flying with

a new type of aeroplane. Interestingly, flight simulators are also sold as games for

personal computers. Today, computer games are probably the most common VR

systems.

VR systems can be categorised by the modalities they support. In today’s systems

the modalities of seeing and hearing are the most commonly employed as these are

also the modalities in which we as human beings mostly exchange information.

They require least effort in terms of energy transfer, the corresponding sensory re-

ceptors are concentrated in the retina and the cochlea and can be excited remotely

with light and sound waves respectively.

In contrast to seeing and hearing the creation of appropriate haptic stimuli de-

mands very sophisticated hardware. Firstly, the skin with its size of 1.5–2 m2 is a

very large organ. Therefore most haptic devices focus on a rather small part of the

human body—usually the fingertip. Secondly, forces cannot be transmitted contact-

free with current technology. Thus haptic devices always need direct contact to the

parts of the skin where the forces are applied. Thirdly, the amount of energy is rel-

atively high compared to other modalities, e.g. if one wants to simulate the lifting

of an object with a mass of 500 g the haptic device has to create a force of ap-

proximately 5 N. All these properties make haptic simulation a complex task still

presenting a lot of problems awaiting a good solution.

Today’s VR systems provide haptic interaction with virtual objects only indi-

rectly via tools like a thimble or a pen-like probe or more specialised tools like, for

example, the yoke of a plane, the handle of a scalpel or the handles of a pair of scis-

sors. These tools usually appear twice in a VR system: as end effector transmitting

the forces from the haptic device and as a virtual representation of the tool in the

simulated virtual environment. The VR system is responsible of matching the phys-

ical state of the end effector and its virtual representation. Therefore, these tools can

be seen as link between the real and the virtual world. However, this rather simple

solution has two drawbacks. Firstly, special end effectors have to be designed for

D. Allerkamp: Tactile Perception of Textiles in a Virtual-Reality System, COSMOS 10, pp. 1–4.

springerlink.com c© Springer-Verlag Berlin Heidelberg 2010

2 1 Introduction

different applications, e.g. an end effector imitating the handle of a scalpel is prob-

ably useless for flight training of aviators. Secondly, it is not possible to directly

touch the virtual objects with the hand, which reduces the possible realism of a VR

system.

In order to simulate direct touch the parts of the skin that are in contact with

virtual objects have to be appropriately stimulated with a tactile display. Although

there exists a large variety of tactile displays only very little research has been done

on the tactile simulation of real objects. The latter is the topic of the work at hand.

In order to further narrow the research question, the virtual objects to be simulated

have been restricted to textiles and only one of the many different types of tactile

displays was investigated.

This work was part of the EU funded HAPTEX project, which aimed at develop-

ing a VR system for the visual and haptic presentation of textiles (cf. [4, 1, 2]). The

project was coordinated by the MIRALab at the University of Geneva which also

contributed the physical based simulation system of the fabrics. The Biomedical

Physics Group at the University of Exeter developed the tactile stimulator hard-

ware and was responsible for the multimodal integration and validation. The PER-

CRO Laboratory at the Scuola Superiore di Studi Universitari e di Perfezionamento

Sant’Anna in Pisa developed the force-feedback hardware which also carried the

tactile stimulator hardware. The SmartWearLab at the Tampere University of Tech-

nology provided a selection of fabrics together with measurements appropriate for

the project. And the Welfenlab at the Leibniz Universitat Hannover developed the

haptic rendering software computing appropriate signals for the force feedback and

the tactile devices.

The target scenario of the project is depicted in Fig. 1.1. The virtual cloth is

attached to a fixed stand. The user can touch, squeeze, rub and stretch the fabric

with the thumb and index finger, feeling appropriate tactile stimulation at the finger-

tips. Reaching the target scenario is a very challenging task, because force-feedback

and tactile simulation have to be integrated, posing more than only mechanical

Fig. 1.1 HAPTEX final

scenario from [1] (courtesy

of PERCRO Laboratory)

1 Introduction 3

difficulties. Therefore, an intermediate scenario has been defined as a test bed for

the tactile rendering. Here the fabric is completely fixed on a rigid support. The user

can tactually explore the fabric’s surface with one fingertip. Force feedback is only

available to define the plane of the surface.

This research has been conducted at the Welfenlab of the Leibniz Universitat

Hannover. The workgroup can draw upon many years of experience in the field of

haptics in VR environments. Already in the Nineties a cooperation with the Virtual

Reality Research Laboratory at the Volkswagen AG in Wolfsburg existed. From this

cooperation the doctoral theses [3] and [5] emerged.

Figure 1.2 shows the structure of this monograph. Chapter 1 provides an introduc-

tion to the problem tackled in this work and describes the context of each chapter.

Chapters 2, 3 and 4 build the foundation for the following chapters.

In Chap. 2 human perception is described from two different angles: in psy-

chophysics perception is regarded as a black box and reactions to stimuli are

!"#$%&'()*$&+,-.$/+*

!"#$%&'01/2.-22/+*'"*,'3-$4++5

!"#$%&'67".$/4%'8%*,%&/*9

!"#$%&':;-<"*'=%&.%#$/+*

!"#$%&'>?%*%&"$/+*'+@A/&$-"4'B-&@".%2

!"#$%&'C1%D/.%2'@+&

7".$/4%'B/<-4"$/+*

Fig. 1.2 Structure of this monograph

4 1 Introduction

investigated whereas in neurophysiology the functionality of single neurons is re-

garded. The former provides methods for the evaluation of the tactile renderers de-

veloped within this work and the latter provides more insight into human perception

and leads to a theoretical model that builds the theoretical frame of the research

described in this book. Finally, tactile perception is described in more detail.

In Chap. 3 an overview of existing tactile displays is given and the display used

within this work is presented. In order to generate appropriate electric signals for the

tactile display an electronics module had to be developed, which is also described in

the chapter. Since also force-feedback devices are used in Chap. 5 an introduction

to this class of devices is given.

The generation of virtual surfaces is the topic of Chap. 4. As the virtual surfaces

are intended to be representations of real fabrics the production and measurement

of fabrics is treated first. Then three different methods for the generation of virtual

representations from real samples are presented. The chapter concludes with the

generation of artificial virtual surfaces that do not have a real counterpart.

After all these preparations the main topic of this monograph, the tactile ren-

dering of virtual surfaces, is presented in Chap. 5. Several different approaches have

been investigated within this work which share a common software framework. This

framework manages the virtual fabrics positioned on a virtual workspace and con-

trols the device tracking the position of the tactile display. After the description of

the framework several experiments on tactile rendering and also the integration with

a force-feedback device are described in the chapter.

Chapter 6 summarises the results of this work and suggests directions for future

research on tactile rendering.

References

[1] Fontana, M., Marcheschi, S., Tarri, F., Salsedo, F., Bergamasco, M., Allerkamp, D.,

Bottcher, G., Wolter, F.E., Brady, A.C., Qu, J., Summers, I.R.: Integrating force and

tactile rendering into a single VR system. In: 2007 International Conference on Cyber-

worlds, pp. 277–284. IEEE Computer Society, Los Alamitos (2007)

[2] Magnenat-Thalmann, N., Volino, P., Bonanni, U., Summers, I.R., Bergamasco, M.,

Salsedo, F., Wolter, F.E.: From physics-based simulation to the touching of textiles: The

HAPTEX project. International Journal of Virtual Reality 6(3), 35–44 (2007)

[3] Rabatje, R.: Virtuelle Montage- und Demontageuntersuchungen mit Hilfe von flexiblen

Bauteilen unter Verwendung geeigneter Kraftruckkopplungsgerate. PhD thesis, Welfen-

lab, Universitat Hannover (2002)

[4] Salsedo, F., Fontana, M., Tarri, F., Ruffaldi, E., Bergamasco, M., Magnenat-Thalmann,

N., Volino, P., Bonanni, U., Brady, A.C., Summers, I.R., Qu, J., Allerkamp, D., Bottcher,

G., Wolter, F.E., Makinen, M., Meinander, H.: Architectural design of the HAPTEX sys-

tem. In: Haptex 2005 – Workshop on Haptic and Tactile Perception of Deformable Ob-

jects, Welfenlab, Universitat Hannover, pp. 1–7 (2005)

[5] Schulze, M.: Von computergraphischen zu haptischen Texturen. Virtual Reality fur den

Entwicklungsbereich Design/Styling in der Automobilindustrie. PhD thesis, Welfenlab,

Universitat Hannover (2005)

Chapter 2

Human Perception

Perception is the process of attaining awareness or understanding of sensory infor-

mation. Together with our motor skills it is nothing less than our gate to the reality

outside our psyche. As the scope of this monograph is the simulation of a certain

facet of reality, namely the tactile properties of textiles, a good insight into human

perception is a necessary prerequisite. The intention of this chapter is to cover the

aspects of human perception that are important for this task.

In Sect. 2.1 an introduction to psychophysics is given. It provides theories and

methods for the measurement of a subjects internal response to physical stimuli.

Section 2.2 describes the biological background of our perception. It covers neurons,

the structure of our nervous system and different methods for the measurement of

neural activity. After these rather broad treatments of perception Sect. 2.3 focuses

on tactile perception. It justifies the way the virtual textiles are rendered in Chap. 5.

2.1 Introduction to Psychophysics

Psychophysics is a discipline of psychology that establishes a relationship between

a physical stimulus and its sensation (percept). Like in physics attainment of knowl-

edge is achieved by finding theoretical models describing experimentally gained

data.

The term “psychophysics” was coined by Gustav Theodor Fechner in his work

“Elemente der Psychophysik” released in 1860. Fechner states that bodily and con-

scious facts are different sides of one reality and that a mathematical relation can be

established between them: In order that the intensity of a sensation may increase in

arithmetical progression, the stimulus must increase in geometrical progression.

The latter is known as the Weber-Fechner law and, together with Stevens’ power

law, will be described in more detail in Sect. 2.1.2.



6 2 Human Perception

2.1.1 Just Noticeable Differences and Weber’s Law

A just noticeable difference (jnd) is the smallest difference in the intensity of two

stimuli that is still detectable by subjects. Its measurement leads to a psychometric

function as shown in Fig. 2.1. Such a function usually resembles a sigmoid function

with the percentage of the correctly noticed differences displayed on the ordinate

and the difference in the intensity of the stimuli on the abscissa. The jnd is by defi-

nition the point where the function crosses 50%.

0

20

40

60

80

100

0 5 10 15 20

co

rre

ctly n

otice

d d

iffe

ren

ce

s in

%

difference in intensity

psychometric curvetheoretical curve

Fig. 2.1 An example of a psychometric function

A special case of a jnd is the absolute threshold which is the smallest detectable

level of a stimulus.

In 1834 Ernst Heinrich Weber studied jnds between stimuli and found the fol-

lowing relation, today known as Weber’s law:

ΔR = cwR (2.1)

where ΔR denotes the jnd to a given stimulus intensity R. The constant cw depends

on the kind of stimulus.

2.1 Introduction to Psychophysics 7

2.1.2 Weber-Fechner Law and Stevens’ Power Law

Weber’s law only relates jnds to stimulus intensities. It is not a psychophysical scale

E(R) relating the intensity of a stimulus R to a value E indicating the intensity of

the corresponding percept. E(R) is assumed to be a continuous function. Fechner

constructed such a psychophysical scale based on Weber’s law. Its construction is

described below.

We define the following sequence of stimuli, where R0 denotes the absolute

threshold:

R1 = R0 + ΔR0 = R0 + cwR0 = (1 + cw)R0

R2 = R1 + ΔR1 = (1 + cw)2R0

...

Rn = (1 + cw)nR0

Assuming that the difference in perceived intensity is the same for all jnds we have

E(Ri+1)− E(Ri) = E(R + ΔR)− E(R) =: cp. (2.2)

We claim E(R0) = 0, i.e. the perceived intensity is zero at the absolute threshold,

and we arbitrarily set cp = 1. Now we can replace Rn by R(E) and use (2.2) to

construct the inverse function of the psychophysical scale:

R(E) = (1 + cw)ER0 (2.3)

By inverting this function and setting c = 1ln(1+cw) we obtain the Weber-Fechner law:

E = c lnR

R0

(2.4)

The Weber-Fechner law is not only a theoretical construct. It is also part of our

everyday life. A well-known example is the keyboard of a piano. An octave is always

perceived as the same interval and therefore there are always 12 semitones in an

octave. However, from the lower to the higher pitch the corresponding frequency

is always doubled. In other words: As the keys represent the sensation increasing

in arithmetical progression, the corresponding frequencies increase in geometrical

progression. (Most of today’s keyboards are well temperamented, so the situation is

a little more complicated, but this is definitely out of the scope of this work.)

One hundred years after Fechner constructed his famous psychophysical scale

the psychologist Stanley Smith Stevens designed a different method for the con-

struction of such scales: Subjects were presented a reference stimulus which was

assigned a number value reflecting the perceived intensity. The subjects then had

to rate the intensity of successive stimuli with respect to the reference stimulus, i.e.

if the reference had an intensity value of for instance 100 another stimulus with only


half the intensity of the reference should be rated with an intensity value of 50 by

the subject. According to Stevens the results could be well approximated by

E = kRb (2.5)

where E is again the perceived intensity and R is the intensity of the stimulus. The

exponent b depends on the type of stimulation and k is a proportionality constant

depending on the units used. Equation (2.5) is known as Stevens’ Power Law.

For some modalities, e.g. the intensity of light, the Power Law resembles the

Weber-Fechner Law, while for others, e.g. electric shocks, Stevens’ Power Law

turns out to be superior. Stevens’ law also proved of value because it is applica-

ble to nearly every dimension of physical perception. Psychologists used it for a

variety of scales, e.g. tone pitch, length, beauty or weight of crimes.

Human perception is a very complicated process. Approximating it with simple

formulas such as the Weber-Fechner Law or Stevens’ Power Law appears to be very

naive. But their applicability to a lot of practical examples proves their usefulness.

Perhaps they might be compared to Newton’s laws of motion which later turned

out to be too simplistic to accurately reflect reality. However, with some limitations

these laws are still applicable and are indeed also applied in practice.

2.1.3 Measuring Psychometric Distances

In Sect. 2.1.1 the concept of jnds is introduced. How can the ability of a subject to

detect a difference be measured? The subject could of course simply be asked but

this is prone to response bias: the systematic tendency of a subject to answer in a

certain way that has nothing to do with sensory factors. This biased behaviour can

be measured with so called catch trials and then used to offset the original measured

data. See Sect. 2.1.5 for a detailed discussion. The result is the theoretical probability

that an arbitrary subject is able to detect the given difference. As shown in Sect. 2.1.1

these probability values are used to produce the psychometric function.

According to [4] we distinguish between six standard discrimination methods:

2-Alternative Forced-Choice method (2-AFC) Two different objects are presented

to the subject, who has to select the object with the strongest (or weakest) sensory

characteristic.

3-Alternative Forced-Choice method (3-AFC) Three objects are presented to the

subject, with two of them being identical. The subject has to select the object

with the strongest (or weakest) sensory characteristic.

Duo-Trio method Three objects are presented to the subject, with two of them

being identical. One of the objects is labeled as the “control” sample. The subject

has to decide which of the remaining two objects matches the control object. In

this configuration it is possible that the odd object is chosen as control sample.

Triangular method Three objects are presented to the subject, with two of them

being identical. The subject then has to tell, which one of the three objects is

different from the two other ones.


A–Not A method The subject is familiarised with two different objects labeled as

“A” and “Not A”. One of these objects is presented to the subject, who has to

decide whether it is “A” or “Not A”.

Same–Different method Two objects, which can either be identical or different,

are presented to the subject, who has to decide whether they differ or not.

For our experiments we mostly used the triangular method, also referred to as the

odd-man-out forced-choice procedure. Here the subject either really detects the

difference or just randomly draws an object. In the latter case the subject might

accidentally draw the right object with a probability of 13.

In our experiments we observed that subjects thinking they were just randomly

drawing an object still performed better than the theoretical probability suggests.

This might indicate that rather than measuring the conscious ability to detect dif-

ferences, the odd-man-out procedure also measures the sub-conscious ability. How-

ever, this assumes that subjects really tried to intuitively draw an object when not

being able to consciously detect the different object. Some subjects tended to sys-

tematically draw for instance the first object, so the results were in danger of be-

ing biased by the way subjects “randomly” drew objects. Therefore we generally

advised all subjects to intuitively draw the object when being unsure rather than

systematically choose one.

2.1.4 Statistical Analysis

For a given pair of stimuli the discrimination experiment is usually repeated several

times. Let us assume the experiment is repeated n times, then the subjects will detect

a difference in k of n times. Such an experiment is called Bernoulli experiment and

the probability of k is given by a binomial distribution

Pp(k) = bn,p(k) :=

(

n

k

)

pk(1 − p)n−k. (2.6)

The letter p denotes the probability that a subject detects the difference or chooses

the right object, respectively.

For given n and k we want to estimate the probability p, such that Lk(p) := Pp(k)becomes maximal:

Lk(p) := sup{Lk(p) : 0 ≤ p ≤ 1} (2.7)

p(k) is called maximum likelihood estimator. For a Bernoulli experiment we find

p(k) =k

n. (2.8)

Depending on the number of experiments n we can have more or less confidence

in the estimated probability p. For large n the error |p − p|is probably small. We


can define a confidence interval C(k) for which we can assume that p ∈C(k) with a

certain probability, called confidence level 1 − α:

Pp(p ∈C(k)) ≥ 1 − α. (2.9)

For many domains α = 0.05 is considered to be sufficiently low. This definition is

not unique as there are various intervals fulfilling (2.9), a trivial example is C(k) =[0,1]. For Bernoulli experiments a table of confidence intervals for confidence levels

0.95 and 0.99 that is suitable for our needs is provided in [5].

Related to confidence intervals is the concept of statistical hypothesis testing.

Here we want to decide if an estimated parameter belongs to a previously defined

interval. For our experiments we mostly want to know whether the subjects just

guessed, denoted by p = p0. This is called the null hypothesis. If we can falsify

the null hypothesis we can assume the alternative hypothesis p > p0 which means

that the subjects sometimes were able to detect a difference. As it is not possible

to measure the probability p directly we have to rely on a Bernoulli experiment

again. Assuming that the null hypothesis holds, we can explain the outcome k of

the experiment with the binomial distribution bn,p0(k). We now define t to be the

smallest value that fulfills

Pp0(x ≥ t) =

n

∑x=t

bn,p0(x) ≤ α. (2.10)

The value t is called critical value to the significance level α . Like for the confidence

intervals α = 0.05 is usually considered to be sufficiently low. If k ≥ t we may reject

the null hypothesis and assume the alternative hypothesis.

We are interested in the probability pd that a subject detects the difference of a

given pair of stimuli. This is not equal to the probability pm that a subject chooses the

right object in an odd-man-out experiment, because, as already stated in Sect. 2.1.3,

when the subject does not detect the difference he or she still chooses the right object

with a probability of 13. Given an estimation pm of pm we can estimate pd with

pd = max{0, pm −1

2(1− pm)}. (2.11)

2.1.5 Signal Detection Theory

In Sect. 2.1.3 experiments for the measurement of psychometric distances were in-

troduced. The subsequent statistical discussion in Sect. 2.1.4 only dealt with the

probability that a subject detects a difference between two stimuli. However, we are

more directly interested in the perceived distance between two stimuli. The Weber-

Fechner law described in Sect. 2.1.2 assigned a perceived distance value of cp to a

discrimination probability of 50% in (2.2). Under some assumptions it is possible

to assign a distance value to every discrimination probability. Signal-detection the-

ory provides a well-established framework for this kind of problems. Only a short


introduction to this theory can be given here. The interested reader is referred to

[19].

In a classical signal-detection experiment an observer listening to white noise has

to decide whether a faint tone (the signal) is added to the noise or not. There are four

cases: there is no signal and the observer does not report a signal (correct rejection),

there is no signal and the observer reports a signal (false alarm), there is a signal

and the observer does not report a signal (miss) or there is a signal and the observer

reports a signal (hit). The hit rate h is defined as

h =number of hits

number of signal trials(2.12)

and the false alarm rate f is defined as

f =number of false alarms

number of noise trials. (2.13)

In the same way we can define the miss rate and the correct rejection rate, but these

do not carry any additional information.

As stated in Sect. 2.1.3 such an experiment is vulnerable to bias, so the hit rate

alone does not suffice to describe the result of the experiment. But also together with

the false alarm rate the situation does not become immediately evident. It would be

much better to have a single value describing the detectability of the signal and

another value describing the bias.

Signal-detection theory makes three assumptions:

• The internal response of the observer to a stimulus can be represented by a single

number.

• The internal response is subject to random variation.

• The choice is based on a simple decision criterion applied to the magnitude of

the internal response.

Figure 2.2 shows an example of these assumptions. The internal response is repre-

sented as a value on the abscissa. It is a random variable drawn from one of two

probability distributions, each belonging to one of the two stimuli. The decision

criterion is described by λ . If the internal response is larger than λ the signal is

detected otherwise not.

A detection model has to specify the form of the probability distributions. In the

equal-variance Gaussian model both distributions are assumed to be normal distri-

butions with a variance of one. The mean of one distribution is set to zero, the mean

of the other to d′. With these assumptions λ and d′ can be estimated from f and h:

λ = −Z( f ) (2.14)

d′ = Z(h)−Z( f ). (2.15)

Z denotes the inverse of the cumulative normal distribution. A derivation of (2.14)

and (2.15) is given in [19]. λ and d′ describe the result of the experiment much


0

No Yes

dλ

Stimulus 1 Stimulus 2

internal response

Fig. 2.2 Decision based on probability distributions for two different stimuli

clearer than f and h: d′ represents the detectability of the signal whereas λ repre-

sents the bias.

The detectability d′ can not be influenced by the observer, but λ is freely cho-

sen. One can assume that an unbiased observer chooses a criterion such that the

likelihood of a correct answer is maximised (see Chap. 9 of [19]). For many of our

discrimination experiments the triangular method was used, for which [8] provides

a table assigning a d′ to a given probability. In such an experiment the internal re-

sponse to two of the objects is drawn from one probability distribution, because

these objects create similar stimuli. The remaining object creates a different stimu-

lus and so the corresponding internal response is drawn from another distribution.

Craven assumes that the observer chooses the object maximising the likelihood that

the remaining objects are drawn from a single distribution. He shows that this rule

is equivalent to simply choosing the object which is farthest from the mean of all

objects. Because of the difficulty of analytically expressing the relationship between

probability and d′ the table was generated by the Monte Carlo method. Figure 2.3

depicts Craven’s result.

2.1.6 Multidimensional Scaling

So far the considered scales related only one-dimensional stimuli and their subjec-

tive correlates, which were also expressed with only one dimension. The scale was

thus expressed as a function E : IR → IR. Nevertheless, in many stimulus domains

the dimensions are not known. Often even the number of relevant dimensions is not

clear. Here, a versatile tool called multidimensional scaling (MDS) turns out to be

very useful. Given measurements of similarity among pairs of objects, MDS repre-

sents these measurements as distances between points of an embedding multidimen-

sional space. Depending on the dimension of this space the distances between the


0

1

2

3

4

5

6

7

0 0.2 0.4 0.6 0.8 1

dis

tan

ce

probability

Fig. 2.3 Distances d′ for given probabilities according to [8]

points can be consistent—to a greater or lesser extend—with the measured similari-

ties. The overall error is called stress and decreases with an increase of dimensions.

It vanishes for a dimensionality m ≥ n− 1 where n is the number of objects. How

the positions of these points are computed is not described here. As we only use the

MDS, we do not bother with such details. The interested reader is referred to [6].

The following example might be enlightening: In [10] 14 colours differing only

in their wavelength but not in their brightness or saturation were used. Each of all

possible pairs of different colours was projected onto a screen and 31 subjects were

asked to rate the similarity on a scale from 0 (no similarity) to 4 (identical). The

averaged and normalised ratings are summarised in Table 2.1. The first row and

column show the wavelengths of the colours in nanometers.

The same data is depicted in Fig. 2.4, where MDS was used to express proxim-

ities as geometrical distances. The result reminds of the CIE chromaticity diagram

shown in Fig. 2.5. This similarity is quite interesting, because the CIE diagram has

been constructed with a very different, more complicated experiment (for details see

Chap. 13.2.2 of [11]).

This example not only shows how MDS works in principle but also illustrates a

case where a one-dimensional physical entity is represented in a two-dimensional

perceptual space. The colour circle becomes a one-dimensional manifold embedded

in this perceptual space.


Table 2.1 Similarity matrix of colours from [10]

nm 434 445 465 472 490 504 537 555 584 600 610 628 651 674

434

445 .86

465 .42 .50

472 .42 .44 .81

490 .18 .22 .47 .54

504 .06 .09 .17 .25 .61

537 .07 .07 .10 .10 .31 .62

555 .04 .07 .08 .09 .26 .45 .73

584 .02 .02 .02 .02 .07 .14 .22 .33

600 .07 .04 .01 .01 .02 .08 .14 .19 .58

610 .09 .07 .02 .00 .02 .02 .05 .04 .37 .74

628 .12 .11 .01 .01 .01 .02 .02 .03 .27 .50 .76

651 .13 .13 .05 .02 .02 .02 .02 .02 .20 .41 .62 .85

674 .16 .14 .03 .04 .00 .01 .00 .02 .23 .28 .55 .68 .76

Fig. 2.4 MDS representa-

tion for color proximities in

Table 2.1 (from [6])

The dimensionality m of the embedding space has not been chosen arbitrarily. As

stated above setting m >= n−1 = 13 would be an uninteresting choice, because the

colours then can be perfectly represented in these spaces. We are more interested

in the minimal number of dimensions that still results in an acceptable error. In

this example the stress of the 1D, 2D and 3D MDS representations is 0.272, 0.023,

0.018, respectively. So two dimensions are clearly preferable to only one dimension.

But higher dimensions do not have much of an effect, because the stress of the 2D

solution is so close to zero already.

2.2 The Human Nervous System 15

Fig. 2.5 CIE 1931 xy chro-

maticity diagram (Source:

Wikimedia Foundation)

2.2 The Human Nervous System

In order to understand human perception it is necessary to gain some insight into the

human nervous system as this is the system that processes all information acting on

the sensory organs. It performs three tasks: sensory input, integration of information

and motor output. The content of this section is mainly based on [7].

2.2.1 Neurons

The basic unit of the nervous system is the neuron. Although the functionality of a

single neuron is relatively simple, the interaction of several million neurons—every

cubic centimetre of the human brain contains more than 50 million—is able to fulfil

very complex tasks. The complexity is caused by the large number of intercon-

nections, every neuron communicates with thousands others. Together the neurons

control all aspects of our perception and motor activity. They allow us to learn, to

remember and to consciously perceive our environment and ourselves.

Neurons are electrically excitable cells that are composed of a cell body (called

soma), dendrites, and an axon (see Fig. 2.6). The dendrites collect signals from other

cells and forward these information as an electrical signal to the soma. Information

sent by the neuron is transmitted by its axon. These are usually much longer than

the dendrites, some more than a metre. At its end the axon branches out into several

terminals which release neurotransmitters to transfer information to other cells. The

contact where this information is transmitted is called synapse.

While a neuron is in a resting state it has an electrical potential of −70 mV com-

pared to the outside of the cell. This so called membrane potential can be changed by


Fig. 2.6 Signal propagation

in neurons (Source: U.S.

National Institute on Aging)

outside influences, usually by the neurotransmitters from other cells (see Fig. 2.7).

If the membrane potential reaches a threshold potential of usually −50 to −55 mV,

an action potential of about +35 mV is triggered. After this spike the membrane

potential returns to its resting potential of about −70 mV.

Fig. 2.7 A triggered action

potential

2.2 The Human Nervous System 17

The amplitude of the action potential is independent of the intensity of the stim-

uli. The action potential can thus be seen as a binary information: it is either there

or not. The intensity is rather coded by the frequency of the occurrences of the ac-

tion potentials, typically denoted by impulses per second. The maximal frequency is

given by the length of the refractory period, i.e. the time until the neuron recovered

its resting potential after an action potential. The action potential is transmitted to

other parts of the nervous system by the axon, which enables signal speeds of up to

150 m s−1.

2.2.2 Neural Integration

A single neuron can receive information from other neurons via thousands of

synapses. Each synapse influences the neuron either in an inhibitory way (decrease

of the membrane potential) or in an excitatory way (increase of the membrane po-

tential). Depending on the location of the synapse it has more or less effect on the

membrane potential. The effect of a synapse also decays some time after an impulse.

There are two kinds of summation: spatial summation means that the simulta-

neous effects of several synapses on different places on the neuron are combined,

whereas temporal summation means that the effects of several impulses of a single

synapse are combined. The latter only occurs if the pause between two successive

impulses is sufficiently short, i.e. the effect of the impulse does not fully decay.

2.2.3 Sensory Reception

Neurons communicate with each other via neurotransmitters. But how is informa-

tion coming from outside the nervous system translated into a code that can be

understood there? There must be some transducers that react to physical stimuli

and accordingly excite neurons. And indeed, for every perceivable kind of physi-

cal stimulus a sensory transducer exists. Such a transducer emits neurotransmitters

when excited by a stimulus it is reactive to. This signal is then forwarded by neu-

rons called afferent neurons into the central nervous system. There are also some

transducers like for example the pain receptors that are themselves neurons that can

directly send action potentials via their axons into the central nervous system.

The information is processed in the central nervous system by intraneurons and

often results in the excitation of efferent neurons, also called motor or effector neu-

rons, which then excite the muscles they are connected to. A very simple example of

this process is the patellar reflex, which is sometimes tested by physicians. When the

patellar tendon is tapped just below the knee, the lower leg automatically, i.e. with-

out intentional control, kicks forward. The tap excites elongation receptors inside

the quadriceps muscle which is connected to the patellar tendon. Afferent neurons

connected to the elongation receptors forward this information to intraneurons in-

side the spinal cord. These excite efferent neurons controlling the quadriceps muscle

which then contracts and in this way lifts the lower leg. In the same time the effer-

ent neurons controlling the opposite muscle are inhibited to make this movement

possible.


The elongation receptors inside the muscles, also called muscle spindles, are ex-

amples of mechanoreceptors. This type of receptors respond to mechanical stress

or mechanical strain. The function of the muscle spindles is to send propriocep-

tive information, i.e. information on the relative position of neighbouring parts of

the body, to the central nervous system. Another kind of mechanoreceptors are hair

cells, which can be found in the cochlea of the inner ear and in the vestibular system.

The latter is also known as balance system and provides information on accelera-

tion and the direction of gravity. The cutaneous mechanoreceptors are responsible

for touch perception and are described in detail in Sect. 2.3.

Nocioceptors are free dendrites that send signals to the central nervous system

that cause pain in response to a possibly damaging stimulus. Different types of no-

cioceptors react to extreme heat, strong pressure or chemical substances like his-

tamine and several acids. Pain is one of the most important perceptions, because it

can result in a defensive reaction that prevents damage to the body.

Another kind of receptors are the thermoreceptors which are part of the system

that regulates the temperature of the body. These receptors are not fully understood

yet.

Chemoreceptors detect chemical stimuli in their environment. These can either

be very specialised to a certain substance or can react to a broad class of stimuli.

The olfactory and gustatory systems use chemoreceptors to detect smell and taste,

respectively. There are also chemoreceptors in the human body, that control blood

parameters like the concentration of oxygen and carbon dioxide.

Photoreceptors respond to electromagnetic radiation with wavelengths of about

400–700 nm. This is the range of light visibile to the human eye. Colour is per-

ceived by three types of photoreceptors that respond to different parts of the visual

spectrum.

2.3 Human Tactile Perception

Tactile perception is the sensation of pressure, vibration and temperature via recep-

tors of the skin. Together with proprioception, which delivers information on the

position of the joints and the tension inside of the muscles and the tendons, it be-

longs to the haptic perception.

2.3.1 Mechanoreceptive Afferents

The glabrous skin of the human hand is innervated by four types of mechanorecep-

tive afferents, which are responsible for the perception of pressure and vibration.

They are presented in Table 2.2. They can be categorised by receptor type, rate of

adaptation, size of the receptive field and by their function. In this context adap-

tation is the change over time in responsiveness of a sensory system to a constant

stimulus. The receptive field of an afferent is the area of the skin where stimuli have

an effect on its firing rate. The functions of the afferents are summarised according

to [15, 14].

2.3 Human Tactile Perception 19

Table 2.2 Characteristics of mechanoreceptive afferents

Afferent Receptor type Adaptation Receptive

field

Function

SA1 Merkel discs slow small form and texture

SA2 Ruffini’s corpuscles slow large hand conformation and forces

acting on the hand

RA Meissner corpuscles rapid small minute, low-frequency skin

motion and feedback required

for grip control

PC Pacinian corpuscles rapid large detection of distant events by

vibrations

The receptive fields of the slowly adapting type 1 (SA1) and rapidly adapting

(RA) afferents are very small in the fingertip, about 3 mm2, whereas the fields of the

slowly adapting type 2 (SA2) afferents are large and even larger for Pacinian (PC)

afferents. The latter spread over the whole finger or half of the palm.

SA1 sensors measure the indentation depth of a mechanical stimulus to the skin

and continually send impulses to the central nervous system during the whole dura-

tion of the stimulus. Having small receptive fields they can collect information on

form and texture of surfaces.

Similar to the SA1 sensors the SA2 sensors continually send impulses during the

whole duration of the stimulus. But their receptive fields are much larger and they

mainly react to shear forces inside the skin.

RA sensors react to the change of a stimulus and thus belong to the differential

receptors. Their impulse frequency depends on the velocity of the indentation.

PC sensors measure the acceleration of a pressure stimulus and thus are espe-

cially appropriate for the detection of vibration. In [15, 14] it is therefore concluded

that PC sensors are mainly responsible for the detection of remote events via vibra-

tions. They are most sensitive to vibrations between 100 and 300 Hz.

2.3.2 Perception of Form and Texture

Independent of the functionality of single receptors the sense of touch can also be

investigated with psychophysical experiments. MDS was used in [12] to find tactile

properties characterising surfaces. The sample surfaces employed in the experiment

could be represented as points in a three-dimensional metric space.

The relation between mechanical deformation of the skin and the firing rate of

the SA1 sensors was analysed in [9] by means of a finite element model of the

fingertip. Measurements on neural activity recorded on monkeys were used to match

the model. A model based on continuum mechanics for the prediction of the SA1

and RA sensors is presented in [18].


2.3.3 Perception of Fine Textures

SA1 receptors cannot be the only receptors responsible for the perception of rough-

ness, because humans are able to discriminate very fine textures with structure sizes

well below the resolution of the SA1 receptors.

Experiments described in [13, 1, 3, 2] show that these fine textures are perceived

by the PC receptors. While being rubbed over a surface the skin is oscillated in a

way characteristic of the surface’s texture, which enables the discrimination of fine

surface textures.

The importance of the fingerprints for the perception of fine textures is investi-

gated in [17]. Two sensors imitating the geometrical and mechanical properties of

the human fingertip were developed. Only one sensor surface was patterned with

ridges mimicking the fingerprints. Comparing the frequency responses of the sen-

sors, the sensor with ridges shows a reduced damping in a frequency range that

corresponds to the sensitivity range of the PC sensors at a typical exploration ve-

locity of 12–15 cm s−1. The authors conclude that the fingerprints play an important

role in the perception of fine surface textures.

2.3.4 Perceptional Model of Tactile Simulation

With the information given in this chapter a theoretical framework for the following

chapters can be defined. Figure 2.8 outlines such a model, which was inspired by

a systems theoretical model of human perception as proposed in [16]. On the left

side of the figure the steps of the information processing taking place when directly

touching a surface are depicted. The right side equals the left side in most parts, but

the surface is palpated indirectly via the combination of a tactile sensor, renderer and

display. Because of the latter, which can be seen as a filter of the signals, relevant

information is lost and the stimuli and thus also all following steps of perception

differ on both sides.

In this model direct and indirect touch can be compared on every level of human

perception:

• On the level of the stimuli an objective measurement with appropriate sensors is

possible. These sensors should correspond to the human fingertip in their geo-

metrical and mechanical properties.

• Neural activity of single afferent nerves can also be measured. This, however, is

only possible with invasive methods.

• The sensations emerging from neural activity can be investigated with appropri-

ate methods of psychophysics as explained earlier in this chapter.

• On the level of symbolic perception psychophysical experiments are also

possible.

Because of redundancies on the levels of information processing there exist subsets

of the stimuli whose elements cause the same percept. This implies a partition into

equivalence classes of the set of stimuli.

2.3 Human Tactile Perception 21

!"#$%!&#'()"*

#+ ,-*'./!%.!/0'-)

%!)&!%!&#'()"*

#!)#"0'-)#

)!$%"*".0'1'0+

#0' $*'

203

4'

.'203

"'203

#'203

&203

4&

.&203

"&203

#&203

#$%5".!/%-/!%0'!#

%!"*#$%5".!

#!)#-%

%!)&!%!%

&'#/*"+&'%!.060-$.7

!

"

!

"

8

8

9:; 9:;

#!)#-% #!)#-%

8/%-,! /%-,!

Fig. 2.8 A model of the contact mechanical interaction of the fingertip and the further pro-

cessing of the nervous system


2.4 Conclusion

In this chapter an introduction to psychophysics is given. It starts with the basic

laws of psychophysics and continues with methods for measuring psychometric dis-

tances. With a statistical analysis of the collected data and a detection model from

signal detection theory it is possible to gain a measure of subjective similarity. Given

a set of stimuli and psychometric distances for all pairs within this set, multidimen-

sional scaling represents stimuli as points of a multidimensional space. The latter

might be referred to as perceptional space of the given class of stimuli. The second

section of this chapter describes the human nervous system. This section is neces-

sary to understand the third section, which enlarges on human tactile perception. In

order to embed the research on tactile simulation into a theoretical framework a per-

ceptional model was developed within this work which is described in Sect. 2.3.4.

With the information given in this chapter the reader should be prepared to fully

understand the simulation systems and the experiments described in Chap. 5.

References

[1] Bensmaıa, S.J., Hollins, M.: The vibrations of texture. Somatosensory & Motor Re-

search 20(1), 33–43 (2003)

[2] Bensmaıa, S.J., Hollins, M.: Pacinian representations of fine surface texture. Perception

& Psychophysics 67(5), 842–854 (2005)

[3] Bensmaıa, S.J., Hollins, M., Yau, J.: Vibrotactile intensity and frequency information

in the pacinian system: A psychophysical model. Perception & Psychophysics 67(5),

828–841 (2005)

[4] Bi, J.: Sensory Discrimination Tests and Measurements: Statistical Principles, Proce-

dures, and Tables. Blackwell, Malden (2006)

[5] Blyth, C.R., Still, H.A.: Binomial confidence intervals. Journal of the American Statis-

tical Association 78(381), 108–116 (1983)

[6] Borg, I., Groenen, P.J.F.: Modern Multidimensional Scaling: Theory and Applications,

2nd edn. Springer, Heidelberg (2005)

[7] Campbell, N.A., Reece, J.B.: Biology, 8th edn. Pearson/Cummings (2008)

[8] Craven, B.J.: A table of d′ for M-alternative odd-man-out forced-choice procedures.

Perception & Psychophysics 51(4), 379–385 (1992)

[9] Dandekar, K., Raju, B.I., Srinivasan, M.A.: 3-d finite-element models of human and

monkey fingertips to investigate the mechanics of tactile sense. Journal of Biomechani-

cal Engineering 125(5), 682–691 (2003)

[10] Ekman, G.: Dimensions of color vision. Journal of Psychology 38, 467–474 (1954)

[11] Foley, J.D., van Dam, A., Feiner, S.K., Hughes, J.F.: Computer Graphics: Principles and

Practice, 2nd edn. Addison-Wesley, Reading (1990)

[12] Hollins, M., Faldowski, R., Rao, S., Young, F.: Perceptual dimensions of tactile surface

texture: A multidimensional scaling analysis. Perception & Psychophysics 54(6), 697–

705 (1993)

[13] Hollins, M., Bensmaıa, S.J., Washburn, S.: Vibrotactile adaption impairs discrimina-

tion of fine, but not coarse, textures. Somatosensory & Motor Research 18(4), 253–262

(2001)

References 23

[14] Johnson, K.O.: The roles and functions of cutaneous mechanoreceptors. Current Opin-

ion in Neurobiology 11(4), 455–461 (2001)

[15] Johnson, K.O., Yoshioka, T., Vega-Bermudez, F.: Tactile functions of mechanoreceptive

afferents innvervating the hand. Journal of Clinical Neurophysiology 17(6), 539–558

(2000)

[16] Kammermeier, P., Buss, M., Schmidt, G.: A systems theoretical model for human

perception in multimodal presence systems. IEEE/AMSE Transactions on Mechatron-

ics 6(3), 234–244 (2001)

[17] Scheibert, J., Leurent, S., Prevost, A., Debregeas, G.: The role of fingerprints in the

coding of tactile information probed with a biomimetric sensor. Science 323(5920),

1503–1506 (2009)

[18] Sripati, A.P., Bensmaıa, S.J., Johnson, K.O.: A continuum mechanical model of

mechanoreceptive afferent responses to indented spatial patterns. Journal of Neurophys-

iology 95, 3852–3864 (2006)

[19] Wickens, T.D.: Elementary Signal Detection Theory, Oxford (2002)

Chapter 3

Devices for Tactile Simulation

Nearly every office or home computer is capable of graphics or sound output in a

quality that, because of the limits of human perception, hardly needs to be improved.

The haptic impression however is limited to the contours of the keyboard and the

mouse.

In some special fields force-feedback devices are used, but these only simulate

mechanical contact with virtual objects via special tools (e.g. like a thimble) and

are therefore not applicable for more general uses. It would be better if the contact

could be simulated without such tools limiting the perceptional capabilities.

For a convicing simulation of direct contact with the skin a single force is not

sufficient. Instead the skin has to be deformed in a more realistic way by a so called

tactile display. In Sect. 3.1 existing tactile displays are evaluated according to their

usefulness for virtual reality applications.

The overview of existing tactile displays is followed by Sect. 3.2, which describes

the tactile displays used within this work in more detail.

The tactile signals calculated on a computer need to be transmitted to the tac-

tile display. An electronic circuit that receives tactile data from the computer and

converts these to analogue signals for the tactile display is presented in Sect. 3.3.

The combination of a tactile display with a force-feedback device is described

in a later chapter. Therefore, Sect. 3.4 provides a short introduction to the haptic

simulation with force-feedback devices.

3.1 Survey of Existing Devices

As there exist a lot of very different tactile displays it would not make sense to

describe all of them in this book. The main interest of this work is the applicability

of existing displays for virtual-reality simulations. This restriction already excludes

many displays that were only designed for the transmission of information, e.g. for

systems that help blind persons to navigate around obstacles (cf. [12]).



26 3 Devices for Tactile Simulation

If a user of a virtual-reality simulation wants to touch a virtual object he or she

will probably use the hand to explore the object. A restriction of the survey to tac-

tile displays for the hand is therefore only natural. As the fingertips are the most

sensitive parts of the human hand most displays are designed for that part of the

skin.

There are two completely different approaches to establish the connection be-

tween the finger and the display: the finger is either permanently connected to the

display or not. The latter allows for an active exploration of the simulated surface,

i.e. the user can move the fingers over the display to asses the properties of the

virtual surface. An active exploration with a display tied to a finger is only possi-

ble if the display follows the movements of the finger. This approach seems to be

more difficult to realise, but it is also more flexible. If the display is attached to a

force-feedback device as demonstrated, for example, in [13] it can simulate the sur-

face of curved objects in a three-dimensional space while the force-feedback device

delivers information on the elasto-mechanical properties of the object. Because of

its greater flexibility this approach is preferable for virtual-reality systems and this

survey is restricted to this kind of displays.

The most common approach in the development of tactile displays is based on

a dense array of stimulator pins. The fingertip is stimulated by the movement of

individual pins which are usually moved normally to the skin’s surface. Different

tactile stimuli are created by different patterns of pin movements.

In the following the large variety of different actuator principles is shown by pre-

senting a selection of tactile displays. However, this is not a comprehensive survey

which would go beyond the scope of this chapter.

3.1.1 Electromagnetic Displays

In [31], [26] and [18] electromotors are used to control the heights of the contact

pins of the tactile displays.

In [31] each pin is lifted by a small rotating lever. The presented prototype con-

tains 36 independently controlled pins with a maximal deflection of 2 mm. The pins

are positioned with an interspacing of 2 mm and are covered by a sheet of rubber

that functions as a spatial low-pass filter. The system is relatively large and reaches

a bandwidth of about 25 Hz.

In [26] 4096 pins are positioned on an area of 200 mm times 170 mm. The dis-

tance between adjacent pins is about 3 mm. The pins are lifted by elevating screws,

which allow for deflections of a few mm. Because of the transmission ratio of the

screws large forces can be generated. However, an update of the heights of the pins

might need as much as 15 seconds.

In [18] a linear actuator is used to adjust the heights of the display pins. In this

way contact forces as high as several Newtons at a maximal deflection of 2.5 mm

are reached. The prototype offers 400 pins on an area of 1 cm2, but it is very large

because of the large number of linear actuators.

3.1 Survey of Existing Devices 27

3.1.2 Pneumatic Displays

A pneumatic approach is pursued in [11]. The 16 pins can be separately controlled

with forces up to 2 N at a maximum height of 3.5 mm. Vibrations in the range of 20–

300 Hz are also possible. The pins are positioned with an interspacing of 1.75 mm

and can additionally be moved laterally to the skin with a pneumatic muscle.

In [23] the pressure inside 25 chambers made of silicone is controlled allowing

for a maximum extension of the chambers of 0.7 mm normal to the skin. With this

approach a bandwidth of only 5 Hz is reached and the distance between adjacent

contact points is about 2.5 mm. Because of the pneumatic actuation additional un-

wanted vibrations occur.

3.1.3 Displays with Shape Memory Alloys

Shape memory alloys change their shape depending on the temperature. In [19],

[32] and [30] wires made of these alloys are used to lift the pins of the dis-

plays. The approaches used differ in the transfer of the movement to the pins. All

presented solutions have problems with hysteresis and a high power dissipation.

The latter is mainly transformed into heat which requires active cooling for some

displays.

In [19] 24 pins are lifted by levers. In this way high amplitudes and high contact

forces can be created, but the bandwidth of about 10 Hz is relatively low.

In [30] 64 pins are pushed upwards by springs while the actuator wires pull the

pins downwards when contracting. The prototype only reached a bandwidth of about

1–3 Hz.

In [32] 10 pins are positioned in a single row. Each pin is attached to the middle

of an actuator wire. When the wire contracts it pushes the pin upwards. This display

allows a bandwidth of up to 30 Hz.

3.1.4 Piezoelectric Displays

Piezoelectric materials deform proportional to an electric voltage. This principle

is used in [15, 14] to control the vibration of 50 contact pins. The prototype only

allows a single vibration frequency of 250 Hz, the amplitude can be adjusted from 5

to 57 µm.

Amplitudes up to 0.7 mm can be generated by the display presented in [22]. The

bandwidth of the 48 contact pins is, however, limited to 20 Hz. The spatial resolution

is about 8 mm2 per pin.

A relatively high resolution of 1 mm2 per contact pin is described in [27]. The

presented display can independently control 100 pins from 20 to 400 Hz with an

amplitude of up to 50 µm.


In [24] a different approach is taken: the 64 contact pins are not moved normally

but laterally to the skin’s surface.

3.1.5 Other Actuator Mechanisms

Electroactive polymers change their form under an electric voltage. In [20] an ionic

electroactive polymer is used for a tactile display with 55 actuators. It has a band-

width of 100 Hz but only works in water.

A rather unconventional solution is described in [8]. The skin is stimulated with

an air jet which creates a perception not comparable to the other tactile displays.

The prototype only offers one contact point.

As the skin is conductive, forces act on it in an electric field. This effect is used in

[29] to create friction forces. In this approach the finger moves over the display, i.e.

the area of the simulated surface is the same as the area of the display. The system

is sensitive to changes of the skin’s moisture.

In [16, 25, 21] the forces occuring in an electric field are used for miniature

actuators. Capacitor plates are linked by an elastic isolator which is contracted when

a voltage is applied. Because of their small size the actuators can be arranged in a

grid.

Electrocutaneous displays stimulate the receptors of the skin with electric im-

pulses. These displays can be very simple and compact, e.g. they could be inte-

grated in a glove. Also the electrodes can be arranged in a dense grid. However, the

variable electric resistance of the skin requires an adjustment of the current which

makes it sometimes impossible to avoid pain. In [17] a display with 49 electrodes is

presented.

3.2 The Tactile Displays Used

The Biomedical Physics Group at the University of Exeter kindly provided several

tactile displays for the experiments described in this work. All displays are devel-

oped further from the display described in [27] and also use piezoelectric bimorphs

for the transformation of electrical signals into movements of the stimulator pins.

The displays mainly differ in their geometrical configuration.

In the colour model of computer graphics a colour consisting of a whole spec-

trum of light is substituted with the combination of only three colours creating

the same sensory impression. This model works because the human eye only pro-

vides three channels for the perception of colour. Virtually all computer screens

and televisions are based on this principle. The success of this model suggests that

a similar model for tactile perception might also be successful. The displays de-

veloped by the Biomedical Physics Group follow such a suggestion by Bernstein

et al. (cf. [9]). Like the colour model two different tactile channels of the human

skin are stimulated with sinewaves at two different frequencies. For the stimulation

3.2 The Tactile Displays Used 29

of the two types of mechanoreceptors a superposition of sinewaves at 40 and 320 Hz

was chosen.

3.2.1 The Piezoelectric Effect

If a piezoelectric crystal is deformed under mechanical pressure a voltage is induced.

This effect is known as the piezoelectric effect. Figure 3.1 shows this effect on

a schematic drawing of a quartz. The displacement of the negative and positive

ions results in a displacement of the centres of the positive and negative charges

and thus in a voltage. The inverse effect is also possible: If a voltage is applied to

the piezoelectric crystal, forces act on the positive and negative ions in opposite

directions, which result in a deformation of the crystal.

F

F

Q−

Q+

++

+

−

− −

+ +

+

Q−

Q+

−

−−

Fig. 3.1 Schematic drawing of the piezoelectric effect of a quartz

The inverse piezoelectric effect can be used to move the stimulator pins of a tac-

tile display. Piezoelectric crystals do not provide sufficient displacement at moder-

ate voltages, so either the voltages used in the system have to be increased or some

form of mechanical leverage is necessary. For the displays used in this work the

latter solution has been chosen. The piezoelectric material is used in a bimorph con-

figuration. Figure 3.2 shows a piezoelectric bimorph that is connected to a voltage

source such that one layer contracts while the other layer expands. As both layers

are bonded the whole bimorph bends when a voltage is applied. One side of the bi-

morph is clamped to make use of the deflection and to control the force at the other

end of the bimorph.

Readily available bimorphs were acquired from APC International Ltd., USA.

With these bimorph cantilevers of thickness 0.6 mm and a free length of 32 mm

displacement amplitudes of maximal 100 µm are reached.


Fig. 3.2 Schematic drawing of a piezoelectric bimorph

3.2.2 Mechanical Behaviour

Electromechanical systems like the piezoelectric bimorphs usually do not transform

electrical signals into mechanical movements independent of the signal frequen-

cies. Therefore, special care has to be taken that signals in the intended frequency

range are transformed in an unbiased way. In [28] and [7] a mathematical model

of the piezoelectric drive system and the mechanical load presented by the skin is

described. The model assumes a linear behaviour of the piezoelectric material and

the skin load, which is reasonable for the relatively small amplitudes of less than

100 µm. With this assumption the system can be represented by a simple mass-on-

a-spring model as depicted in Fig. 3.3. This model predicts the resonance and the

frequency responses for different lengths of piezoelectric drive elements as depicted

in Fig. 3.4.

Fig. 3.3 Mass-on-a-spring

model of the piezoelectric

material and the skin load

piezoelectric

drive system

finger

In Fig. 3.5 the response of the system with and without skin load measured with

a miniature accelerometer is shown. From this data the effect of the skin can be esti-

mated: Significant stiffness of about 100 N m−1 and resistance of about 0.1 N s m−1

3.2 The Tactile Displays Used 31

Fig. 3.4 Predicted fre-

quency response for dif-

ferent length of the piezo-

electric drive element from

[28]

is added to the system. The additional effective mass of about 10−5 kg is negligible.

As can be seen in the figure the resonant frequency of the system is increased by the

load of the finger while the narrowness of the resonance (the Q-factor) is reduced.

Fig. 3.5 Measured fre-

quency response of the

piezoelectric drive element

with and without the me-

chanical load presented by

the skin from [28]

As the tactile stimuli only consist of superpositions of 40 and 320 Hz sinewaves,

bimorphs have been chosen which have a principal resonance that lies between the

two operating frequencies of 40 and 320 Hz.

3.2.3 Geometrical Configurations

For the experiments conducted within this work three tactile displays mainly differ-

ing in their geometrical configuration were used. Figure 3.6 shows these displays

together with a depiction of the arrangement of their stimulator pins as seen from

the direction of the finger. Each circle represents a contactor pin. Not all pins are


2 mm

2 mm

2 mm

Fig. 3.6 The tactile displays used within this work

driven by an actuator. The moving pins are represented by filled circles. The other

circles represent stationary pins.

The first display has been developed for a previous study in which five of these

displays were used to stimulate the digits of one hand. Compared to the other dis-

plays it is relatively large.

The second display is quite similar to the first. But instead of a 5-by-5 array of

contactor pins it provides only 24 pins in a 6-by-4 configuration. The size of the

display is significantly reduced compared to the first display. To achieve this the

size of the bimorphs also had to be reduced.

The third display is intended for the integration with a force-feedback device as

described in Chap. 5. The bimorphs are positioned such that they do not lie beneath

the contactor surface. This allows thumb and fingers to be brought close together in

a system with more than one tactile display per hand. This display provides a curved

contactor surface matching the shape of the fingertip.

3.3 Drive Electronics 33

3.3 Drive Electronics

The tactile displays used within this work are operated with input signals at voltages

of up to 40 V rms. As the tactile signals are calculated on a digital computer an

electronic component converting the digital data to appropriate analogue signals had

to be developed. The analogue parts of the electronics have been designed following

suggestions by the Biomedical Physics Group at the University of Exeter, which also

provided the tactile displays used. The development and testing of some components

is presented in [10].

Nowadays the most common interface to peripheral hardware is the Universal Se-

rial Bus (USB). Nearly every modern personal computer provides at least one USB

connector. Therefore, it was decided that the drive electronics should be controlled

via USB.

Fig. 3.7 Schematic of the

drive electronics

DACOPA

Data b

us

OPA DAC

USB controller

Figure 3.7 shows a block diagram of the drive electronics. A USB controller

receives tactile data for all stimulator pins from the host computer. After some pro-

cessing of the data as described below it distributes the signals to the different chan-

nels via a data bus. The bus is designed for the unidirectional transmission of data

from the USB controller to the digital–analogue converters. To avoid noise from

ground loops the bus is galvanically isolated from the USB controller via optical


couplers. In each channel a digital–analogue converter (DAC) receives the data for

a single stimulator pin and outputs an analogue voltage ranging from −Vref to +Vref.

The reference voltage Vref for the DAC is set by a variable voltage supply to a value

from 0 to 2.5 V for all channels. In this way the global gain of the system can be set.

An operational amplifier (OPA) is used to amplify the analogue signal to voltages

appropriate for the tactile display.

Figure 3.8 shows the developed drive electronics for a tactile display with 24

pins. The electronic board on the bottom of the figure hosts the USB controller, the

optical couplers and the variable voltage supply. Each of the six boards above hosts

a DAC for four channels, four amplifications circuits and a circuit for the address

logic of the data bus. The board on the right just provides connectors for the other

boards.

Fig. 3.8 The drive electron-

ics for one finger

3.3.1 USB Controller

The USB controller communicates with the host computer. It receives, processes

and forwards data to the DACs. An integrated circuit (IC) that provides all necessary

features is the CY7C64713 produced by the Cypress Semiconductor Corporation. It

includes a USB transceiver and an Intel 8051 microprocessor with 16 kBytes of on-

chip random access memory (RAM). The software running on the microprocessor

can be uploaded via USB, which makes the IC especially attractive for development

purposes.

The design of the supporting circuit is taken from [6]. Only a few capacitors

stabilising the voltage of the power supply and an oscillator with a resonance of

24 MHz are needed to run the IC. The IC can be directly connected to the USB.

Although the IC can also be directly connected to the power supply of the USB

it was decided to include a fixed-voltage regulator reducing the 5 V of the USB to

3.3 V preferred by the IC.


Fig. 3.9 The board hosting the USB controller and the variable voltage supply

Figure 3.9 shows the board hosting the USB controller. On the left third of the

board the CY7C64713 IC, the oscillator, the USB connector and the fixed-voltage

regulator can be seen. The black connector provides access to most ports of the IC

for debugging purposes.

The software running on the USB controller is called firmware. The firmware

processes the data sent from the host computer. For the tactile simulation of fabrics

only the relevant properties have to be modeled and displayed. Human performance

in tactile perception and the reduced abilities of the tactile display build the frame in

which tactile signals make sense. The display used within this work creates vibro-

tactile stimuli consisting of a superposition of only two sinewaves (see Sect. 3.2).

Therefore, instead of transmitting the whole tactile signal via USB only the time-

varying amplitudes of the two base signals have to be transmitted. This results not

only in a reduced amount of data to be transferred via USB but also in a reduced

amount of computation time on the host computer running the tactile renderer, be-

cause the actual waveform of the signal is computed by the firmware running on the

USB controller.

It should be noted that the restriction to only two frequencies is only implemented

in the software part of the system. The designed hardware is still capable of produc-

ing more complex signals.

The implementation of the firmware is able to serve at least 24 channels with a

sample rate of at least 5 kHz. It stores a table defining the wave form of the signal.

With some optimisations of the assembler code, the firmware reached a sample rate

of 10 kHz for 24 channels.

The output signal is created from a periodic function f with period length 2π .

Let ωi be the angular frequencies of the signals with indices i = 1;2 and let an,i(t)be the time dependent amplitudes. Then the output signal for channel n is computed

as

gn (t) = an,1(t) f (ω1t)+ an,2 (t) f (ω2t) . (3.1)


The angular frequencies are defined with constants wi and a basic angular fre-

quency ω0:

ωi = wiω0 (3.2)

From the periodicity of the function f follows

f (τi) = f (wiω0t) (3.3)

τi := wiω0t −2πk = wi(ω0t −2πk

wi

) k ∈ Z, (3.4)

such that 0 ≤ τi < 2π .

For the digital processing the function f and the amplitudes an,i have to be dis-

crete in time and value. Let Δ t denote the time between two samples. Then the

function f can be represented as an array of length l. The value of f at time m is

found at position si,m = l2π τi (0 ≤ sm < l) within the array.

The output values are computed as follows:

gn,m = an,1 f [s1,m]+ an,2 f [s2,m] with m :=⌊ t

Δ t

⌋

(3.5)

Let ω0 := 2πlΔ t

be the basic angular frequency. Then

si,m+1 = (si,m + wi) (mod l) (3.6)

follows from (3.4).

The host computer transmits data to the USB controller with a well defined pro-

tocol. The first byte of a data packet determines the command to be performed.

Necessary data follow dependant on the command. The protocol is summarised in

Table 3.1.

The first and second commands are used to define the array f holding the values

of the waveform of the tactile signals. The command “create new table” destroys

a possibly already existing table and allocates memory on the USB controller for a

new table of the given length. As the table might be larger than the maximum size of

a data packet, the command “transmit part of the table” allows for the transmission

of only a part of the table.

The next command sets the number of channels, i.e. the number of stimulator

pins to be controlled by the system. This value has also an impact on the sample

rate of the generated tactile signals.

The command “transmit amplitudes” is frequently issued by the tactile renderer

running on the host computer. With this command the amplitudes an,i are set for

each channel and signal frequency.

The two following commands are used to start and stop the output of the USB

controller. With the command “start output” the increments wi for the two signal

frequencies can be configured.


Table 3.1 Protocol for the communication with the USB controller

Command Byte Value

create new table 0 0

1 length

transmit part of the table 0 1

1 offset

2 length

3– data

set number of channels 0 2

1 number of channels

transmit amplitudes 0 3

1 signal number (0 or 1)

2– amplitude data for the channels (descending order)

start output 0 4

1 reserved

2 reserved

3 increment for signal 1

4 increment for signal 2

stop output 0 5

set global gain 0 6

1 gain factor

The command “set global gain” is used to set the gain factor of the whole sys-

tem. The transmitted value is forwarded to the variable voltage supply that sets the

reference voltages −Vre f and +Vre f for the DACs.

3.3.2 Data Bus

The signals created by the USB controller are unidirectionally transmitted to DACs

via a bus system which makes the system scalable in the number of contactors sup-

ported.

The data bus consists of 8 data lines, 8 address lines and 2 control lines, namely

“write” and “load”. The availability of a valid date and address is signalled with a

high state on the control line “write” as depicted in Fig. 3.10. The data transmitted

in this way is expected to be buffered at the location specified by the address lines.

After all buffers have been updated a low state on the control line “load” signals

that the data should be used, which guarantees that all DACs update their output

simultaneously. The timing requirements of the bus are determined by the DACs

used.


data

address

write

load

D1 D2 D3 D4

A1 A2 A3 A4

Fig. 3.10 Digital timing diagram of the data bus

As there are 8 lines for the data 256 different voltages are available to define

the output signals of the DACs. The bus also provides 256 different addresses of

which the 4 highest are reserved for the variable voltage supply, which means that

the system could be extended to drive up to 252 stimulator pins.

On the upper right of Fig. 3.11 the address logic circuit can be seen. It consists of

a 6 Bit dual in-line package (DIP) switch used to configure the address range of the

board and a comparator IC. The comparator IC compares the address range defined

by the switch with the highest 6 address lines and the “write” signal with a high

state to signal the transfer of data into the buffer. The two lowest address lines are

directly connected to the address ports of the four-channel DAC.

Fig. 3.11 The board hosting the DAC and the OPAs

The data bus galvanically isolates the USB controller and thus also the host com-

puter from the rest of the system to avoid problems with noise. It also amplifies

the signals from the USB controller. Both aims are reached with the circuit depicted


in Fig. 3.12. An optical coupler galvanically isolates both parts of the system. The

6N137 IC described in [5] features the necessary characteristics. Since the optical

coupler inverts the signal and since the output currency of the CY7C64713 is not

sufficient to drive the LEDs inside the optical couplers, an inverter is inserted be-

tween the output ports of the CY7C64713 and the input of the optical couplers. The

use of optical couplers has the additional advantage that a different voltage may be

used as ground voltage as needed for the DACs. The 18 optical couplers can be seen

in the middle of Fig. 3.9.

Fig. 3.12 The circuit gal-

vanically isolating the USB

controller from the rest of

the system

inV

outV

GND1 GND2

VCC

3.3.3 Variable Voltage Supply

The variable voltage supply is used to adjust the global gain of the system by setting

the reference voltages for the DACs −Vre f and +Vre f . Its circuit is placed on the

same board as the USB controller (see the right third of Fig. 3.9). Like the DACs

for the channels it is also connected to the data bus and thus can be configured by

the USB controller. A comparator compares the address on the data bus with a fixed

address range from 252 to 255. If data is available for one of these addresses it is

transfered to a DAC. In this circuit the AD558 IC is used for the digital–analogue

conversion (see [1]). Dependent on the digital input the DAC sets its output to a

voltage from 0 V to 2.55 V. This output is inverted twice by the circuit depicted in

Fig. 3.13, which is implemented by the L272 IC (see [2]).

3.3.4 Digital–Analogue Conversion

The DAC receives data from the bus, which is kept in a buffer until the “load” sig-

nal indicates the output of the value as the new voltage. For the digital analogue

conversion the MAX5100 IC described in [3], which provides four independent

channels, was used. For the selection of the channel the two lowest address lines

of the data bus were directly connected to the IC. The channels of the MAX5100


Vin

Vref

Vref

−

10k

10k

10k

10k

Fig. 3.13 Amplification stages of the variable voltage supply

are already buffered. It provides a low-active input that initiates the transfer of the

buffered value to the analogue output. The MAX5100 can be seen in the middle of

the right half of Fig. 3.11.

Fig. 3.14 Passive, first order

low-pass RC filter

υout

υin

R

C

To reduce imaging effects caused by the discrete quantisation of the output values

of the DAC a passive, first order low-pass RC filter as depicted in Fig. 3.14 was

used. The cutoff frequency fc, i.e. by definition the frequency at which the signal is

damped by 3 dB, can be computed with

fc =1

2πRC. (3.7)

The filter used consists of a 33 kΩ resistor and a 4.7 nF capacitor, which results in

a cutoff frequency of roughly 1 kHz. The values of the resistor and capacitor were

chosen such that the sampling rate of the signals is well above and their maximum

frequency content is well below the cutoff frequency.


3.3.5 Amplification

High-voltage operational amplifiers (OPA) are used to amplify the output of the

DACs to the levels required by the bimorphs. The OPA551 described in [4] was

used as amplifier in this circuit. The implemented circuit is able to amplify the out-

put of the DACs, which have a maximum output of 2.5 V, to 60 V. To avoid high

voltages with respect to the ground voltage the 2.5 V signal is first amplified by a

non-inverting amplifier as depicted in Fig. 3.15.

Fig. 3.15 Non-inverting

amplifier with an adjustable

gain factor between 11 and

13.5

outυ

inυ

200k

50k

20k

The exact amplification factor can be adjusted with a potentiometer. The output

voltage from the first amplification stage is inverted with respect to the ground volt-

age by an inverting amplifier with a fixed amplification factor of −1 as depicted in

Fig. 3.16. Between the two resulting voltages the signal can in theory reach voltages

of up to 60 V.

Fig. 3.16 Inverting ampli-

fier with gain factor -1

υin

υout

20k

20k

Figure 3.17 shows the output of a single channel recorded with an oscilloscope.

The signal is a superposition of two sinewaves with frequencies 40 and 320 Hz.


Fig. 3.17 The output of a channel

3.4 Force-Feedback Devices

Force-feedback devices are much more common than tactile displays. While the

latter are still rather experimental, a number of force-feedback devices is already

commercially available. Figure 3.18 shows such a device.

Fig. 3.18 PHANToM Omni

haptic device by SensAble

Technologies

Force-feedback devices only allow for a haptic interaction with virtual objects

via a special tool. Such a tool could be a pen-like probe, a thimble or more special

tools like for example the scalpel of a surgeon. The simulation of direct touch is

not possible. Instead the user of the system touches the real representation of the

tool, often called end effector. The force-feedback device tracks the position of the

3.4 Force-Feedback Devices 43

end effector and exerts forces on it. The end effector also has a representation in the

virtual scene that follows the position of the real end effector.

3.4.1 The Haptic Loop

In the real world bodies never overlap. When real bodies touch, forces act on these

bodies that prevent them from penetrating each other. The magnitude of the forces

depends on the stiffness of the bodies. The same behaviour is simulated in the virtual

scene. If the virtual end effector collides with another object in the virtual scene, the

occuring forces are computed and the force acting on the end effector is sent to the

force-feedback device. This uses its motors to create a similar force on the real end

effector. This haptic loop is depicted in Fig. 3.19.

Fig. 3.19 The haptic loop

!"#$% $$&'(#)&$*+#$

#!,,$#-$&.-!/01(,.!2$"(-!"

2/34+#(5.4+105(-+!,! .*+"-0(5.4#$,$

2!4+-+!,.! .$,&%$ $#-!"

!"#$.(#-+,6.!,.$,&%$ $#-!"

As the physical simulation of the virtual scene is implemented as a computer

program, the physical state of the scene can only be updated in discrete moments in

time. This, however, poses a problem: since the user decides on the position of the

end effector between two discrete moments, the user can move it to a position that

would require extremely high forces. This is particularly a problem if the stiffnesses

of the virtual objects are high. To prevent these high forces the system has to be

updated relatively often. Usually haptic systems are updated at 1000 Hz. Compared

to computer graphics, where frame rates of 25 frames per second (fps) are usually

sufficient, the relatively higher frame rate of force-feedback systems poses higher

requirements to the computation of the physical simulation.

3.4.2 Modeling the Contact Forces

As the force acting on the end effector has to be computed in a very short time

usually a simple model of the contact situation is used. The probably most simple

and therefore widely used contact model is the spring–damper model: if the position


Fig. 3.20 The spring–

damper contact model

surface of the

virtual object

position at

the surface

position at

time t i−1

position at

time t i

of the end effector lies inside a virtual object a simple spring is attached to the end

effector and to a point on the surface of the object as depicted in Fig. 3.20.

At time frame ti−1 the end effector is still outside the object, but because of its

movement it is already inside the object at time frame ti. To counteract this physi-

cally impossible situation the spring forces the end effector out of the object. The

spring has a resting length of zero and a stiffness that is determined by the stiff-

nesses of the virtual object and the end effector. To avoid oscillation of the system a

damper is added to the spring.

3.4.3 Different Degrees of Freedom

A body in three-dimensional space has a three-dimensional position and a three-di-

mensional orientation. Altogether it has six degrees of freedom (DOF). However,

only some force-feedback devices fully provide six DOF. Many devices are able to

track only the position and not the orientation of their end effector. These devices

can only exert a three-dimensional force but no torque forces. One of these devices

is the PHANToM Premium 1.0 haptic device by SensAble Technologies that has

been used for an experiment described in Chap. 5.

There also exist some force-feedback devices that are able to track all six DOF,

but which cannot exert torque forces. Such a device was kindly provided by the

PERCRO laboratory at the Scuola Superiore Sant’Anna in Pisa for the integration

of the tactile display described in Sect. 3.2 with a force-feedback device as described

in Chap. 5.

3.5 Conclusion

A lot of different tactile displays have been created by various research groups, but

not one of these devices can create all tactile impressions necessary for a convinc-

ing tactile simulation. However, some displays follow interesting approaches that

are worth to be further investigated. The display used in this work is such a display,

References 45

because it is based on the interesting idea of substituting a whole spectrum of fre-

quencies with a signal consisting of only very few frequencies intended to cause an

equivalent sensation. Such a scheme is already successfully utilised for the render-

ing of colours and might also be promising for tactile rendering.

For the use of the displays with a tactile renderer running on a computer an elec-

tronic component had to be created to translate the digital signals into voltages suit-

able for the tactile display. This component has been designed following the tactile

two-channel model, but is flexible enough to create other waveforms as well. Four

modules of this drive electronics have been created and are in use at the MIRALab

at the University of Geneva, PERCRO at the Scuola Superiore Sant’Anna in Pisa,

the Biomedical Physics Group at the University of Exeter and the Institute of Man–

Machine Communication at the Leibniz University of Hannover.

A tactile display alone does not suffice to create a satisfactory simulation of direct

touch to the full extend, because it does not provide the forces necessary for the

creation of kinaesthetic sensory impressions. Therefore, it is sensible to combine a

tactile display with a force-feedback device as described in Chap. 5. The principle

function of these devices is described in Sect. 3.4.

References

[1] AD558. Analog Devices, Inc., One Technology Way, Norwood, MA, USA (1987)

[2] L272. SGS-THOMSON Microelectronics (1995)

[3] MAX5100. Maxim Integrated Products, 120 San Gabriel Drive, Sunnyvale, CA, USA

(1999)

[4] OPA551, OPA552. Burr-Brown Corporation, 6730 S. Tucson Blvd., Tucson, AZ 85706,

USA (1999)

[5] 6N137. Fairchild Semiconductor Corporation (2005)

[6] CY7C64713/14. Cypress Semiconductor Corporation, 198 Champion Court, San Jose,

CA, USA (2005)

[7] Allerkamp, D., Bottcher, G., Wolter, F.E., Brady, A.C., Qu, J., Summers, I.R.: A vibro-

tactile approach to tactile rendering. The Visual Computer 23(2), 97–108 (2007)

[8] Asamura, N., Tomokori, N., Shinoda, H.: A tactile feeling display based on selective

stimulation to skin receptors. In: Proceedings of the Virtual Reality Annual International

Symposium, pp. 36–42. IEEE Computer Society, Los Alamitos (1998)

[9] Bernstein, L.E., Eberhardt, S.P., Demorest, M.E.: Single-channel vibrotactile supple-

ments to visual perception of intonation and stress. The Journal of the Acoustical Soci-

ety of America 85(1), 397–405 (1989)

[10] Blume, S.: Konstruktion eines Gerates zur Simulation von Oberflachenstrukturen fur

den Einsatz in VR-Umgebungen. Student research paper, Welfenlab, Universitat Han-

nover (2006)

[11] Caldwell, D.G., Tsagarakis, N., Giesler, C.: An integrated tactile/shear feedback array

for stimulation of finger mechanoreceptor. In: Proceedings of IEEE International Con-

ference on Robotics and Automation, vol. 1, pp. 287–292 (1999)

[12] Cardin, S., Thalmann, D., Vexo, F.: Wearable obstacle detection system for visually

impaired people. In: Haptex 2005 – Workshop on Haptic and Tactile Perception of De-

formable Objects, Welfenlab, Universitat Hannover, pp. 50–55 (2005)




tactile rendering into a single VR system. In: International Conference on Cyberworlds,

pp. 277–284. IEEE Computer Society, Los Alamitos (2007)

[14] Ikei, Y., Shiratori, M.: Textureexplorer: a tactile and force display for virtual textures.

In: Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment

and Teleoperator Systems, pp. 327–334 (2002)

[15] Ikei, Y., Wakamatsu, K., Fukuda, S.: Vibratory tactile display of image-based textures.

IEEE Computer Graphics and Applications 17(6), 53–61 (1997)

[16] Jungmann, M., Schlaak, H.F. (2002) Miniaturised electrostatic tactile display with high

structural compliance. In: Proceedings of Eurohaptics (2002)

[17] Kaczmarek, K.A., Tyler, M.E., Bach-y-Rita, P.: Pattern identification on a fingertip-

scanned electrotactile display. In: Proceedings of the 19th Annual International Confer-

ence of the IEEE Engineering in Medicine and Biology Society, vol. 4, pp. 1694–1696

(1997)

[18] Killebrew, J.H., Bensmaıa, S.J., Dammann, J.F., Denchev, P., Hsiao, S.S., Craig, J.C.,

Johnson, K.O.: A dense array stimulator to generate arbitrary spatio-temporal tactile

stimuli. Journal of Neuroscience Methods 161(1), 62–74 (2007)

[19] Kontarinis, D.A., Son, J.S., Peine, W., Howe, R.D.: A tactile shape sensing and display

system for teleoperated manipulation. In: IEEE International Conference on Robotics

and Automation, vol. 1, pp. 641–646 (1995)

[20] Konyo, M., Tadokoro, S., Takamori, T.: Artificial tactile feel display using soft gel ac-

tuators. In: IEEE International Conference on Robotics and Automation, vol. 4, pp.

3416–3421 (2000)

[21] Matysek, M., Lotz, P., Schlaak, H.F.: Braille display with dielectric polymer actuator. In:

Conference proceedings: Actuator. In: 10th International Conference on New Actuators

& 4th International Exhibition on Smart Actuators and Drive Systems, pp. 997–1000

(2006)

[22] Maucher, T., Meier, K., Schemmel, J.: An interactive tactile graphics display. In: Sixth

International Symposium on Signal Processing and its Applications, vol. 1, pp. 190–193

(2001)

[23] Moy, G., Wagner, C., Fearing, R.S.: A compliant tactile display for teletaction. In: IEEE

International Conference on Robotics and Automation, vol. 4, pp. 3409–3415 (2000)

[24] Pasquero, J., Hayward, V.: STReSS: A practical tactile display system with one mil-

limeter spatial resolution and 700 Hz refresh rate. In: Proceedings of Eurohaptics 2003,

pp. 94–110 (2003)

[25] Schlaak, H.F., Jungmann, M., Matysek, M., Lotz, P.: Novel multilayer electrostatic solid

state actuators with elastic dielectric. In: Bar-Cohen, Y. (ed.) Proceedings of the SPIE.,

Presented at the Society of Photo-Optical Instrumentation Engineers (SPIE) Confer-

ence, vol. 5759, pp. 121–133 (2005)

[26] Shinohara, M., Shimizu, Y., Mochizuki, A.: Three-dimensional tactile display for the

blind. IEEE Transactions on Rehabilitation Engineering 6(3), 249–256 (1998)

[27] Summers, I.R., Chanter, C.M.: A broadband tactile array on the fingertip. The Journal

of the Acoustical Society of America 112(5), 2118–2126 (2002)

[28] Summers, I.R., Brady, A.C., Syed, M.K., Chanter, C.M.: Design of array stimulators

for synthetic tactile sensations. In: Proceedings of the First Joint Eurohaptics Confer-

ence and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator

Systems, pp. 586–587. IEEE Computer Society, Los Alamitos (2005)

References 47

[29] Tang, H., Beebe, D.J.: A microfabricated electrostatic haptic display for persons with

visual impairments. IEEE Transactions on Rehabilitation Engineering 6(3), 241–248

(1998)

[30] Taylor, P.M., Moser, A., Creed, A.: The design and control of a tactile display based on

shape memory alloys. In: IEEE International Conference on Robotics and Automation,

vol. 2, pp. 1318–1323 (1997)

[31] Wagner, C.R., Lederman, S.J., Howe, R.D.: A tactile shape display using RC servomo-

tors. In: Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environ-

ment and Teleoperator Systems, pp. 354–355. IEEE Computer Society, Los Alamitos

(2002)

[32] Wellman, P.S., Peine, W.J., Favalora, G., Howe, R.D.: Mechanical design and control

of a high-bandwidth shape memory alloy tactile display. In: Experimental Robotics V,

pp. 56–66. Springer, Heidelberg (1998)

Chapter 4

Generation of Virtual Surfaces

To achieve a realistic tactile simulation of fabrics a display, a tactile renderer and

virtual representations of the fabrics’ surface properties are needed. The two former

requisites are covered in other chapters, the latter is the scope of this chapter.

As this research aims at a realistic simulation of real fabrics a set of fabric sam-

ples has been acquired. These fabrics were used to test the methods for the genera-

tion of virtual surfaces and to conduct experiments for the evaluation of the tactile

renderers. In Sect. 4.1 these sample fabrics are described.

The first step in the generation of virtual surfaces from real surfaces is always the

measurement of the real fabrics’ properties. A standard for the measurement of fab-

rics has been proposed by Kawabata in [11]. Although this standard was orignially

intended to assess the subjective hand properties of fabrics, it was used within this

work to measure the properties relevant for the generation of virtual surfaces and is

therefore described in Sect. 4.2.

There exist many different ways to model the surface properties of fabrics. In this

work only a few methods could be tested. These are described in Sects. 4.3, 4.4 and

4.5. All these methods have in common that a fabric’s surface is represented as a

two-dimensional property map P(u,v). However, the description of these properties

depend on the model of the fabric. The generation of virtual fabrics is not a real-time

operation and may even be manually supported.

Virtual fabrics generated from real fabrics are not always suitable to test the tac-

tile renderers. Artificial surfaces having well-defined properties were used instead.

These surfaces are described in Sect. 4.6.

4.1 Sample Fabrics

The systems for tactile simulation developed in this work are not only theoretical

constructs but aim at practical applications in the future. Therefore their usefulness

has to be properly evaluated. In Chap. 2 the basis for appropriate evaluation meth-

ods is presented. This section is about the selection of fabric samples useful for

the evaluation of the whole tactile simulation system and for testing the different

algorithms.



50 4 Generation of Virtual Surfaces

Using fabrics for the evaluation is handy, because there exists a large variety of

fabrics with different surface properties. It is possible to test the simulation system

with samples having rather extreme properties, e.g. extremely rough or extremely

smooth. Nevertheless it is also possible to test the system with samples that are

not very different in order to investigate the ability of the system to simulate only

very little differences. However, fabrics are just a test case, the systems could be

generalised to a larger class of surfaces later.

4.1.1 Fibres, Yarn and Fabrics

Fabrics are usually made of yarns that are themselves made of fibres. There are also

some fabrics that are not made of yarns. These are called “non-woven” fabrics and

will be treated later in this section.

Fibres are much longer than thick, i.e. the ratio length/diameter is very high. For

example the length of a cotton fibre is about 2000 times greater than its diameter.

This property of fibres makes spinning of yarns possible, because a large number of

long, fine fibers grip each other when twisted in a thread.

Figure 4.1 shows how fibres can be categorised (cf. [19]). Natural fibres either

stem from plants like cotton, flax or jute or from animals like wool or silk. The

manufactured fibres can either be created from natural or synthetic material. While

natural fibres are always staple fibres, i.e. fibres with a length, there are some syn-

thetic fibres that are continuous filaments. The latter are of indefinite length and run

continuously throughout the length of the yarn without a break.

The choice of the fibre material already determines a large quantity of a fab-

ric’s properties. These properties can also be altered by applying special finishing

processes. Fibres differ for example in their fineness, absorption of moisture, static

electricity, tensile strength, extensibility, elastic recovery, stiffness, friction and their

visual appearance. Fabrics are often made of blends to profit from the properties of

the individual fibre types.

For the production of yarn the fibres are usually first carded and sometimes even

combed to align the fibres in one direction. After that the fibres are spun to form a

yarn, which is called “single yarn”. If two or more single yarns are twisted together

the result is called “folded yarn” and if two or more folded yarns are twisted together

they form a “cabled yarn”.

Fibres and yarns have a rather one-dimensional shape. Several mechanisms like

for example weaving, knitting or felting exist to obtain a two-dimensional fabric

from fibres or yarn, respectively. Fabrics are woven by interlacing two sets of threads

to each other. These two sets are called warp and weft. The warp threads are already

fixed to the loom while the weft threads run orthogonally to the warp threads. The

interlacing pattern, known as the weave, is obtained by lifting some of the warp

threads such that the weft thread currently interlaced runs under these warp threads

and over the others. Figure 4.2a shows such a pattern where every second warp

thread was lifted alternately. This pattern is called plain weave. It is the simplest

and most frequently used weave (cf. [19]).

4.1 Sample Fabrics 51

!"#$!%&'()$*+

,*%%#%-+* !""!# $%&' ()"*

.$-"*(/ +!!% ,&-./0*121/3&456*.*7 8-%9/03)%"-:&"*;/!./<-%;7

0(/*$!% =4>*4"!4

0!/#1!2"#$*3&'()$*+

!"#$!%&.-%45*$ ?-43!4* @!;&% A.-&3*"&"* =3*"&"* BC!3*%%

64/"7*"(2&.-%45*$ D!%C&6-;* D!%C*4"*. E%&4"&#* 111

8"7*$+ &.>!#/F->.* G%&44/F->.*

Fig. 4.1 Categorisation of different fibre types

Weaves can be described on squared paper as shown in Fig. 4.2b. Columns cor-

respond to warp threads while rows correspond to weft threads. Thus, each square

represents a junction of a warp and a weft thread. It is filled when, looking at the

fabric’s face side, the warp thread lies over the weft thread.

Knitted fabrics consist of a series of interlinked loops of yarn. Two main cate-

gories of machine knitting exist: weft knitting and warp knitting. Figure 4.3 shows

examples of these categories. A weft knitted fabric could be made from a single

yarn. Loops are formed in succession by needles with hooks at their tops. The hook

holds the loop. When the yarn arrives again it pulls the yarn through the existing

loop forming a new loop in this way. Warp knitted fabrics are produced from a set

of warp yarns. Loops are formed simultaneously by needles with hooks. To interlink

the parallel threads, the threads are moved sideways at intervals. Unlike weft knitted

fabrics warp knitted fabrics do not ladder, i.e. they cannot be unravelled.

There exist several different definitions of “non-woven” fabrics (cf. [22]). In this

work fabrics that are not produced by weaving or knitting are called “non-woven”

fabrics. Examples for non-woven fabrics are leather, which is made from animal

skin, and felt, which is directly made from fibres, not yarns.


(a) Plain weave fabric (b) Weave diagram of plain weave

Fig. 4.2 Plain weave fabric with its weave diagram

(a) Face of plain weft knitting (b) Back of warp-knitted half-tricot

Fig. 4.3 Examples of knitted fabrics

4.1.2 Selection of Sample Fabrics

A set of altogether 54 different fabric samples has been kindly provided by the Smart

Wear Lab at the Tampere University of Technology. These fabrics are presented in

[21] and the specification of all 54 samples is also tabulated in Annex A.

The samples were acquired in three subsets. The first set of 32 samples was cho-

sen such that the fabrics largely differed in many properties. It was expected to result

in a very wide range of different virtual fabrics. The samples were differently fab-

ricated (woven, knitted and non-woven), and they largely differed in their weight,

thickness and in their fibre contents (cf. [14]).

4.2 Kawabata Evaluation System for Fabrics 53

The second set consists of 10 samples. These were taken from a narrower col-

lection of men’s woollen or wool blend woven suit fabrics, weft and warp knitted

blended or synthetic fabrics containing elasthane (cf. [21]).

The third set of samples consists of 12 samples of which 6 are non-laminated

synthetic warp knits used for car seats. The other samples consist of the originally

non-laminated fabrics laminated to foam.

4.2 Kawabata Evaluation System for Fabrics

The subjective hand properties of fabrics are called “fabric hand”. These properties

are assessed when a real fabric is touched and manipulated with the hands. Several

more precise but slightly differing definitions exist (cf. [1]), but are not of greater

interest for this work. The objective measurement of fabric hand preoccupied tex-

tile researchers for decades (cf. [4]). Today several measurement standards exist.

The probably most well-known are the KES-F system (cf. [11]) developed by Sueo

Kawabata and the FAST system (cf. [15]) developed by the research organisation

CSIRO.

The KES-F system consists of standardised testers for the bending, compression,

tensile, shear, surface roughness and surface friction properties of fabrics. These

testers measure the properties in weft and warp direction and for the surface prop-

erties on the face and back sides of the fabrics. From the data recorded by each

tester characteristic values are calculated. Table 4.1 summarises these characteristic

values together with their physical units. Tensile, bending, shearing and compres-

sion are elasto-mechanical properties while surface properties, weight and thickness

are other physical properties related to the elasto-mechanical properties. The KES-F

system is described in more detail in [11].

The KES-F standard uses the unusual unit gram-force (gf) which is defined

as the magnitude of the force exerted on one gram of mass by standard gravity

(9.80665 m s−2). Thus, 1 gf is equal to 0.00980665N.

Although the KES-F system is only intended to produce the aforementioned char-

acteristic values, the testers could be used to obtain the whole measurements. To be

able to further process the recorded data, a system for the conversion of the analogue

measurements to digital data files was used as described in [14]. As the KES-F sys-

tem is not intended for such a modification special care had to be taken to avoid

problems with noise.

As the surface measurements are the most important for this work they are de-

scribed in more detail in this section. The surface properties were measured on both

sides (right and back side) of most of the 54 fabric samples. For each side there are

four measurement directions: weft and warp each in forward and backward direc-

tion. As most of the measurements were repeated there were altogether 611 mea-

surement files available for the experiments. However, some measurements had to

be discarded, because some fabrics were too difficult to measure, e.g. for some very

stretchable fabrics material accumulated in front of the sensor.


Table 4.1 The characteristic values of the KES-F properties (cf. [11])

Parameter Definition Unit

Tensile LT Linearity -

WT Tensile energy gf·cm/cm2

RT Resilience %

EM Max. extension %

Bending B Bending rigidity gf·cm2/cm

2HB Hysteresis gf·cm/cm

Shearing G Shear stiffness gf/cm·◦

2HG Hysteresis at 0.5◦ gf/cm

2HG5 Hysteresis at 5◦ gf/cm

Compression LC Linearity -

WC Compressional energy gf·cm/cm2

RC Resilience %

Surface MIU Coefficient of friction -

MMD Mean deviation of MIU -

SMD Geometrical roughness µm

Weight W Weight/unit area mg/cm2

Thickness T Thickness at 0.5 gf/cm2 mm

4.2.1 KES-F Roughness Test

The KES-F roughness tester measures a one-dimensional height profile along a seg-

ment of length 2 cm. Therefore, the fabric is moved under the sensor at 1 mm s−1

for 20 s as depicted in Fig. 4.4. The tension of the fabric is kept at 20 gf cm−1 along

the measurement direction.

Fig. 4.4 Schematic drawing of the KES-F roughness tester

4.2 Kawabata Evaluation System for Fabrics 55

(a) Photograph of the contactor

20 gf/cm

1 mm/s

10 gf

(b) Front and side view of the contactor

Fig. 4.5 The contactor of the KES-F roughness tester

The contactor of the sensor is a steel piano wire of diameter 0.5 mm. In the stan-

dard version it is bent as shown in Fig. 4.5. The sensor is pressed onto the fabric

with a contact force of 10 gf by a spring with stiffness 25 gf mm−1 and the fabric is

supported by a smooth steel plate. The data recorded by the sensor is digitised and

saved in a measurement file for further processing.

For the computation of the characteristic SMD value by the tester (see Table 4.1)

the analogue signal is first passed through a high-pass filter with transfer function

G( jω) =( jω)2

( jω)2 + 2( jω)ξ ωn + ω2n

, (4.1)

where ωn = 2π rad/s, ξ = 0.6 and ω is the angular frequency. This filter allows

signals of wave length smaller than 1 mm to pass. The SMD value is the mean

deviation of the surface thickness in µm and is calculated as

SMD =1

X

X∫

0

|T (x)− T |dx (4.2)

where X = 2 cm, x is the position of the contactor on the fabric, T (x) is the thickness

of the fabric at position x and T is the mean value of T .

The calculation of the SMD value as done by the KES-F roughness tester has

also been implemented for the digital data to test the consistency of the digital data

and the result from the tester.

4.2.2 KES-F Friction Test

The surface friction is measured with the same device as the surface roughness, but

with a different sensor. It consists of ten piano wires resulting in a larger contact area


20 gf/cm

1 mm/s

50 gf

Fig. 4.6 Front and side view of the KES-F friction contactor

as shown in Fig. 4.6. The contactor is pressed onto the fabric with the compressional

force of 50 gf by a dead weight. The speed at which the fabric is moved under the

sensor is the same as with the roughness measurement.

The MIU and MMD (see Table 4.1) values are calculated by the tester. As with

the SMD value the measured signal first passes the high-pass filter. MIU is the mean

value of the coefficient of friction and MMD is its mean deviation. The definitions

of both values are

MIU =1

X

X∫

0

µ(x)dx (4.3)

MMD =1

X

X∫

0

|µ(x)− µ|dx (4.4)

where X = 2 cm, x is the position of the contactor on the fabric, µ(x) is the frictional

force at position x divided by the compressional force and µ is the mean value of µ .

4.3 Spatial Frequency Analysis

The raw KES-F measurements are not adequate for use with the tactile renderer.

They have to be converted to a two-dimensional property map P(u,v), where the

map P depends on the requirements of the tactile rendering algorithm. One class of

these algorithms translates the spatial frequency content of the surfaces into appro-

priate temporal frequencies. The preparation of the surface measurements for these

algorithms is described in this section.

The surface model is computed in two steps as depicted in Fig. 4.7. At first the

fast Fourier transform (FFT) is used to obtain the spatial frequency content of the

Small Scale

Surface Model

Fast Fourier

Transform

Kawabata

Surface Profile

Measurements

Two−Dimensional

Composition

Fig. 4.7 The sequence of the spatial frequency analysis

4.3 Spatial Frequency Analysis 57

measurements. The resulting spectra are combined to a two-dimensional property

map.

4.3.1 Selection of Appropriate Sections

For the KES-F surface measurement the sensor is moved about 3 cm in forward di-

rection and then about the same length in backward direction. During this procedure

the height in µm for the roughness measurement and the friction force in gf divided

by 50 gf for the friction measurement, respectively, are recorded at a sample rate of

about 1000 Hz. For both measurements also the position on the fabric measured in

cm is recorded. Before any further processing can take place two sections of length

2 cm are selected. The first section is taken from the part of the measurement with

positive sensor speed (from 0.5 to 2.5 cm) and the second section is taken from the

part with negative sensor speed (from 2.5 to 0.5 cm). The selected sections are suf-

ficiently far away from the starting point at 0 cm, the turning point at 3 cm and the

end point at 0 cm to avoid unwanted side effects that might occur in the proximity of

these points, for example due to the change in velocity. Figure 4.8 shows an example

of a selection of appropriate sections.

−20

−15

−10

−5

0

5

10

15

0 10000 20000 30000 40000 50000 60000

Fig. 4.8 Selection of appropriate sections

A linear regression analysis of the position measurements showed that a length

of 2 cm corresponds to about 18,756 samples for all measurements. This result con-

flicts with the sensor speed of 1 mm s−1 specified in the KES-F and the sample rate

of about 1000 Hz. However, as the KES-F tester is regularly calibrated it was as-

sumed that it correctly measured the position and the sample rate was actually only


about 938 Hz. Therefore, a common length of 18,756 samples, corresponding to

2 cm, was used for all sections.

4.3.2 Discrete Fourier Transform

In Chap. 5 the surface models developed in this chapter are used to compute vi-

brations that occur when the finger moves over the fabric. In the approach that is

prepared in this section the spatial frequency spectrum of the surface is directly

transformed into the temporal frequency spectrum of the vibrations. As the mea-

surements are discrete signals their frequency content is computed with the discrete

Fourier transform (DFT). Here, only a very short introduction to the DFT is given.

In [6] and [3] the topic is treated more comprehensively.

The DFT detects harmonic components of a discrete signal fi with i = 0, . . . ,n−1

by comparing the signal with unit-waves of given frequency:

Fk =n−1

∑i=0

fi cos

(

2πki

n

)

− jn−1

∑i=0

fi sin

(

2πki

n

)

. (4.5)

Here j is the imaginary number and therefore the Fk with k = 0, . . . ,n − 1 are

complex numbers. The absolute magnitude |Fk| =√

ℜ(Fk)2 + ℑ(Fk)2 describes

the magnitude of the k-th frequency component while the angular measure ∠Fk =

arctan(

ℑ(Fk)ℜ(Fk)

)

describes its phase.

If the DFT was directly implemented as shown in (4.5), each computation of

the n different frequency components would involve a summation of 2n different

multiplications. So the complexity would be O(n2). However, this complexity can be

reduced: a FFT is an implementation of the DFT that has a complexity of O(n logn).The DFT assumes the signal to be periodic with a period corresponding to the

length of the signal. However, this is rarely the case and discontinuities are intro-

duced at the start and the end of the signal adding erroneous frequency content to

its spectrum. To avoid this problem “windows” are used that are multiplied with the

original signal and cause it to continuously vanish at the start and the end. Let zi

be the discrete signal and wi be a window of the same length, i.e. i = 0, . . . ,n− 1.

The resulting signal is then defined as fi = ziwi. Windows commonly in use are the

trivial rectangular window

wi = 1, (4.6)

the Blackman window

wi = 0.42−0.5cos

(

2πi

n−1

)

+ 0.08cos

(

4πi

n−1

)

(4.7)

and the Hamming window

wi = 0.54−0.46cos

(

2π i

n−1

)

. (4.8)

4.3 Spatial Frequency Analysis 59

0

0.2

0.4

0.6

0.8

1

0 100 200 300 400 500 600 0

0.2

0.4

0.6

0.8

1

0 100 200 300 400 500 600

-1

-0.5

0

0.5

1

0 100 200 300 400 500 600-1

-0.5

0

0.5

1

0 100 200 300 400 500 600

0

50

100

150

200

250

1 10 100 0

20

40

60

80

100

120

140

160

1 10 100

Fig. 4.9 A rectangular (left column) and a Hamming window (right column) and their effect

on a signal in time (second row) and frequency domain (third row)

Figure 4.9 shows the rectangular and the Hamming windows and their effect on a

signal.

In order to obtain a spatial frequency spectrum at different points on the fabric’s

surface a Hamming window of length 4096, which corresponds to 4.37 mm, was

used at different offsets of the measurement. The window was shifted by 938 sam-

ples or about 1 mm and the FFT was used to compute the spectra at the different

offsets. In this way 16 different spectra were computed for a measurement with a

length of 2 cm.

Most fabrics are created such that there is a repetition in their surface pattern

(cf. Sect. 4.1). Therefore, an ideal fabric would result in a periodic measurement

which could be described with only one discrete frequency spectrum. However, due

to irregularities in the yarns and in the weaves the measurements are usually not


perfectly periodic. As these irregularities affect the tactile properties of fabrics it is

important to include them in the surface model. This is the reason why the frequency

spectra are computed at different positions.

4.3.3 Two-Dimensional Composition

Up to now only the one-dimensional KES-F surface measurements were regarded

in this section. However, these represent only a small part of the two-dimensional

surface and are not sufficient to define the whole surface. But with some additional

assumptions a reasonable surface model can be deduced.

Most fabrics are created such that there is a repetition in their surface pattern (cf.

Sect. 4.1). But due to irregularities in the yarns and in the weaves this repetition is

not always clearly recognisable in the measurements.

However, because of the randomness of these irregularities periodicities can often

still be recognised in the measurements in weft and warp direction with lengths teand ta, respectively. It is assumed in the following that te ≤ 16mm and ta ≤ 16mm.

As there are 16 different spectra per measurement, te and ta are defined as

te := maxk∈IN, kte≤16mm

kte, (4.9)

ta := maxk∈IN, kta≤16mm

kta. (4.10)

If there is no recognisable periodicity observable at all, te and ta are both set to

16 mm.

The exact positions of the measurements on the fabric are not known. As the

measurements are orthogonal to each other it is assumed that the measurement in

weft direction lies on the u-axis of the fabric’s coordinate system whereas the warp

measurement lies on the v-axis. For points on the u-axis or v-axis, i.e. if v = 0 or

u = 0, the corresponding spectra can be found with

ie = ⌊u (mod te)⌋ (4.11)

for v = 0 and with

ia = ⌊v (mod ta)⌋ (4.12)

for u = 0, respectively, where ie and ia denote the indices of the corresponding

spectra. Here, the operator ⌊·⌋ is defined as

⌊x⌋ := maxi∈Z, i≤ x

0.001m

i. (4.13)

Figure 4.10 shows a grey-scale image of a fabric’s surface. As indicated by the green

line the surface has a pattern with a certain angle. The surface structure along the

weft and warp direction, respectively, has to be shifted according to that angle, as

depicted by the red lines, to be consistent with each other.

4.4 The Correlation–Restoration Algorithm 61

Fig. 4.10 Example of a

surface having a pattern

with a certain angle

Let α be the angle between the u-axis and the direction of the surface pattern.

Then a point (u,v) on the fabric’s surface can be projected to the axes with

u =u−v

tanα, (4.14)

v =v−u tanα (4.15)

and after substitution of u,v with u, v in (4.11) and (4.12) the indices of the spectra

can be computed with

ie =⌊(

u− v(tanα)−1)

(mod te)⌋

, (4.16)

ia =⌊(v−u tanα) (mod ta)⌋ . (4.17)

These two equations are also depicted in Fig. 4.11.

4.4 The Correlation–Restoration Algorithm

In [9, 10] an algorithm is presented, which also uses the KES-F surface measure-

ments to create a surface model. In contrast to the algorithm presented in the previ-

ous section, this algorithm constructs a two-dimensional height profile of arbitrary

size from the one-dimensional measurements. In [9, 10] friction and sound is re-

garded as well, but for this research only the algorithm for the construction of the

height profile was reimplemented (see [12]). As depicted in Fig. 4.12 the algorithm

reduces the data from the KES-F surface tester to its essentials first and then con-

structs a two-dimensional surface from the remaining data.


Fig. 4.11 A point on the

fabric’s surface is projected

into the measurements

ta

te1.

2.

2.

1.

warp

α

weft

Two−Dimensional

Composition

Small Scale

Surface Model

Kawabata

Surface Profile

Measurements

Data Reduction

Fig. 4.12 Construction of a two-dimensional height profile from the KES-F surface

measurements

4.4.1 Data Reduction

The correlation–restoration algorithm (CRA) is used to decompose a signal into

its strongest harmonic components and a rest, which is modeled as noise. It uses

autocorrelation to emphasise the strongest harmonics and is therefore more stable

in the presence of noise than the usual FFT alone. The algorithm is useful as a

lossy compression technique as the components are computed in the order of their

relevance. The latter holds paticularly for periodic functions. The remaining noise

gives information about the error of the approximation, so maximum error values

can be specified.

Usually only a FFT is used to compute the frequency content of a given signal.

The main idea of the CRA is to compute the autocorrelation of the signal before

applying the FFT. According to [9] the advantage of this procedure is a greater

stability in the presence of noise.

For a discrete signal zi, with i = 0, . . . ,n−1, the autocorrelation is defined as

Rzz(p) =1

n− p

n−p−1

∑i=0

zizi+p (4.18)

4.4 The Correlation–Restoration Algorithm 63

for p = 0, . . . ,n− 1, i.e. the autocorrelation itself is a discrete signal of the same

length. The strongest harmonic component of the original signal is found by apply-

ing a FFT to its autocorrelation.

In this way, however, only the frequency ω j of the harmonic component can be

determined. To also obtain its amplitude and phase the correlation of the original

signal with a unit wave si = cos(2πω jΔxi) with i = 0, . . . ,n−1 is computed. Here,

Δx is the position difference between two consecutive sample points. This unit wave

has the same frequency as the harmonic component found before, its amplitude is

one and its phase is zero. The correlation of two discrete signals is defined as

Rzs(p) =1

n

n−1

∑i=0

zisi+p (4.19)

for p = 0, . . . ,n− 1. It is proven in [9] that the correlation Rzs(p) is a harmonic

function with only one single frequency ω j and that the amplitude A j and the phase

φ j can be obtained with

A j = limn→∞

Rzs(p1peak)− lim

n→∞Rzs(p1

valley), (4.20)

φ j = 2πω jΔxp1peak = 2πω jΔxp1

valley + π . (4.21)

It is of course impossible to have n → ∞ in the numerical implementation of the

algorithm, but if N − p is large the error is negligible.

After amplitude, frequency and phase of a harmonic component have been de-

termined, the component is subtracted from the original signal and the procedure is

repeated until there are not any clearly recognisable harmonic components left in

the signal. The remaining signal is assumed to be noise.

4.4.2 Two-Dimensional Composition

Like before the one-dimensional KES-F surface measurements have to be composed

to create a two-dimensional surface. Similarly to Sect. 4.3.3 the angle α between

the fabric weaving pattern and the weft direction is used in [9, 10] to define a two-

dimensional height profile:

Z(x,y) =ne

∑i=1

Aei cos(2πωe

i (x + y tanα)+ φ ei )

+na

∑i=1

Aai cos(2πωa

i (y− x tanα)+ φai )

(4.22)

where −45◦ < α ≤ 45◦. The height profile defined in this way is a superposition of

cosine waves that linearly propagate in two perpendicular directions. These waves

are defined by their amplitudes Aei and Aa

i , their projected frequencies ωei and ωa

i

and their projected phases φ ei and φa

i .


A frequency component wei appears in the spectrum of the KES-F measurement in

weft direction and multiplied with tanα also in the measurement in warp direction.

Similarly wai appears in the warp measurement and multiplied with tanα also in the

weft measurement. Therefore, these pairs of harmonic components in weft and warp

direction have to be identified and modeled with only one corresponding harmonic

component in (4.22).

It is claimed in [10] that over 70% of all fabrics can be modeled with such a

simple harmonic function. This is probably true for the large-scale structure of the

fabrics but questionable for small structure sizes. However, for the indirect haptic

interaction via a stylus as investigated in [9, 10] the large-scale structure is probably

sufficient to model a fabric. This model is further investigated in Chap. 5.

4.5 Surface Reconstruction from an Optical Surface Scan

An optical surface scan of a fabric sample is easy to obtain and can be used to

reconstruct its height profile. This method called “Shape from Shading” is tested in

[18] and [12] for the application in haptic virtual-reality environments. Results are

presented in [17].

Small Scale

Surface Model

Optical Surface

Scan

Finding Tilings Shape From Shading

Fig. 4.13 Surface Reconstruction from an Optical Surface Scan

A method that reconstructs a height function from a grey-value image is de-

scribed in this section (see Fig. 4.13). The first step in this reconstruction process is

the detection of symmetry in the grey-value image. The term “symmetry” is defined

with regard to a set P ⊆ E 2 (E 2 denoting the Euclidean plane) as a bijective map

α : E 2 → E 2 with α(P) = P . Every symmetry of E 2 is a translation, rotation,

reflection or glide reflection. The symmetries of most fabrics are combinations of

two nonparallel translations. These translations define a basic element called “tile”

or “primitive” that is repeated in the translational directions (see Fig. 4.14). An

algorithm for finding such a tile was implemented based on [7] at the Institute of

Man–Machine Communication at the Leibniz University of Hannover. As the sym-

metries of real surfaces are not perfect in general, stochastic methods are used to

find an idealised tiling.

Since the tile is repeated on the grey value image the average of these repetitions

is used to create a new synthetic image of the surface, that way being cleaned from

noise. This image in turn is used as input for a shape-from-shading algorithm (see

[20]) that computes an appropriate height profile. See Fig. 4.15 for an example. The

tile can be used to describe fabrics of arbitrary size with relatively few data.

4.5 Surface Reconstruction from an Optical Surface Scan 65

Fig. 4.14 Tile with transla-

tions t1 and t2

t1

t 2

Fig. 4.15 Photograph of a fabric and the corresponding height profile

4.5.1 Symmetry Detection

In order to extrapolate the fabric sample beyond its borders the smallest tile has to

be found, which is equivalent to the detection of the isometry group. An algorithm

to obtain this tile using stochastic methods is described in [18] and in more detail in

[16].

To detect symmetrical portions of a texture, its picture has to be segmented into

repeating sections. The average of all these sections is called primitive and can provide

synthetic textures of arbitrary size. Our implementation of the symmetry detection

employs ideas originating from [7] and [13]. The first step of the identification of

symmetries within textures is the creation of the autocorrelation matrix (ACM), which

describes the translations of parts of the texture onto other parts of the same texture.

A statistical feature matrix (SFM) of a discrete grey-value image t under the

operation αα1...αn with an arbitrary feature f t is defined as


SFMf tαα1...αn

(t) = ( f t(t,αα1...αn(t)))αα1 ...αn. (4.23)

In this context an operation αα1...αn is a translation in the Euclidean plane E 2. Let

p,q,r ∈ E 2. Then a translation τ from p to q is defined as

τp,q(r) = r +(q− p). (4.24)

The translation may not leave the region of the grey-value image. Furthermore, to

achieve reasonable results each translation needs to affect at least one fourth of the

original image.

The correlation ρ was utilised as feature f t. It is defined as

ρ(t,t(s)) :=Cov(t,t(s))

√

Var(t)Var(t(s))(4.25)

where Cov() denotes an estimator for the covariance and Var() for the variance.

The ACM is an SFM with translation τp,q as operation αα1...αn and correlation ρas feature f t. From the ACM a grid of primitives can be computed with the following

steps:

1. identification of the maxima of the ACM

2. computation of the main directions of the maxima

3. computation of the distances between the lines of the grid

The correlation describes the amount of congruence of two parts of the texture be-

ing compared. Large elements of the ACM correspond to a large congruence (see

Fig. 4.16). Therefore, the maxima of the ACM are of interest.

Fig. 4.16 The grey value image of a texture (left) and its ACM (right), bright pixels in the

ACM correspond to large values

As not all maximal values are adequate for the computation of the grid, criteria

for the selection of the maxima have to be defined. Because of the large amount

of different textures an automatic determination of these criteria is not possible.

Therefore, a minimum distance R between two adjacent maxima and a threshold S

4.5 Surface Reconstruction from an Optical Surface Scan 67

for the value of the maximum have to be set. Let ACM(i, j) denote the value of the

autocorrelation matrix at position (i, j). It is a maximum if and only if

S < ACM(i, j) (4.26)

and for all ACM(p,q) with i− R2≤ p ≤ i + R

2and j − R

2≤ q ≤ j + R

2and ¬(i =

p∧ j = q)

ACM(p,q) < ACM(i, j). (4.27)

Given the maxima of the ACM the main directions of the grid can be computed.

Therefore the Hough transformation recognising straight lines and circles is used.

Straight lines in the Euclidean plane can be uniquely described by

d = xcosα + ysinα (4.28)

where α is the angle between the normal vector and the x-axis and d is the distance

to the origin.

For the computation of the main directions the angle α runs in small steps from 0

to π . The size of the steps depends on the size of the image. Each point P(x,y) in the

set of the maxima can be seen as point on a straight line with angle α . The distance

d can be computed with (4.28). Points resulting in similar distances are considered

to be on the same line.

The two angles with most matching distances are selected as main directions. We

denote them by q1 and q2. To get the length of the vector with direction q1, all max-

ima of the ACM are projected in direction q2 onto a straight line with direction q1.

The projected position of each maximum can be understood as a normal distribution

with given variance σ . The maximum of the superposition of all these distributions

is taken as optimal distance for the vector with direction q1 (see [16]). The length

of the vector with direction q2 is computed in the same way, with q1 and q2 being

interchanged.

In Fig. 4.17 the intermediate steps are depicted: First the maxima of the ACM are

extracted. Then the main directions are found (here represented by two lines). In the

last step the lengths of the spanning vectors are computed. Here, the length of the

vectors is represented by the shape of the primitive. The primitive is computed by

averaging values from the original picture. In this way the primitive is cleaned from

noise. The picture on the bottom right shows the synthetically generated image of

the texture. It is created by repetition of the primitive.

4.5.2 Shape from Shading

Shape-from-Shading (SFS) algorithms compute a surface profile of an object from

one or more two-dimensional images. Pioneer work in this field was done by Horn

in [8].


Fig. 4.17 Maxima of the ACM, the directions represented by two lines, the primitive and the

synthetically generated image (from top left to bottom right)

The algorithm used here was developed by Tsai and Shah in [20]. It extracts the

information of the height from only one picture. In [23] the algorithm of Tsai and

Shah is recommended for images with little noise and a source of light being close to

the object. These two criteria are fulfilled in our case, since the symmetry detection

cleans the image from noise and the textures were digitised with a flat bed scanner,

so that the source of light was indeed very close to the texture.

4.6 Artificial Surfaces

For testing purposes artificial surfaces were generated. Figure 4.18 shows some ex-

amples, where the grey values represent different heights. These surfaces were used

to validate the functioning of the tactile renderer. Synchrony of the visual and the

tactile display was tested with bars of variable width.

It is shown in [2] that there exists a strong correlation between the fractal di-

mension of a haptic texture and the impression of roughness perceived by tactually

exploring it with a pen-like probe. Therefore, surfaces with different fractal dimen-

sions were used for experiments.

4.6 Artificial Surfaces 69

Fig. 4.18 Artificially generated surfaces with bars (left and middle) and sinusoidal Surfaces

(right)

4.6.1 Brownian Surfaces

Brownian surfaces are randomly generated surfaces with a defined fractal dimen-

sion. Before the construction of these surfaces is described, one of the various ways

to define fractal dimension, the Hausdorff dimension, is presented at first.

Let F ⊆ IRn, F �= /0 with d(x,y) denoting the Euclidean distance in IRn. Then

diam(F) := sup{d(x,y)|x,y ∈ F} is called the diameter of F .

Let δ ∈ IR+ and {Ui}i∈IN be a countable set of subsets of IRn with diam(Ui) ≤ δ .

Then {Ui}i∈IN is called a δ -cover of F if F ⊂⋃∞

i=1 Ui.

Let s ∈ IR+. For every δ ∈ IR+ let

Hs

δ (F) := inf

{

∞

∑i=1

(diam(Ui))s|{Ui}i∈IN δ -cover of F

}

. (4.29)

Then the s-dimensional Hausdorff measure is defined as

Hs := lim

δ→0H

sδ (F). (4.30)

Hausdorff proved the existence of a number DF fulfilling

Hs(F) =

{

∞ for s < DF

0 for s > DF .(4.31)

Now the Hausdorff dimension dimH(F) can be defined as

dimH(F) := inf{s|H s(F) = 0}

=sup{s|H s(F) = ∞}.(4.32)

The Hausdorff dimension dimH(F) has some basic properties:

• If F ⊂ IRn, then dimH(F) ≤ n.

• If F ⊂ G, then dimH(F) ≤ dimH(G).• If F is countable, then dimH(F) = 0.


A Brownian surface XH : IR2 → IR with index H (0 ≤ H ≤ 1) is a stochastic process

with:

1. XH(0,0) = 0.

2. XH(x,y) is continuous.

3. Let σ ,τ ≥ 0. Then the increments XH(x + σ ,y + τ)−XH(x,y) are normally dis-

tributed with expectation 0 and variance (σ2 + τ2)H

=: ρ , i.e.

P(XH(x + σ ,y + τ)−XH(x,y) ≤ z) = (2π)−12 (ρ)−

12

z∫

−∞

exp

(

−r2

2ρ

)

dr.

XH(x,y) itself is also normally distributed with expectation 0 and variance (x2+ y2)H

.

Figure 4.19 shows some examples of Brownian surfaces with different indices.

It is shown in [5] that a Brownian surface XH(x,y) with index H has Hausdorff

and box dimension dimH XH(x,y) = dimB XH(x,y) = 3−H. In the following these

dimensions are shortly refered to as fractal dimension dimXH(x,y).

Fig. 4.19 Brownian surfaces with index 0.1 (red), 0.5 (green) and 0.9 (blue)

A Brownian surface with (x,y) ∈ [0,1]× [0,1] can be constructed with the fol-

lowing algorithm (see [18]):

1. Set XH(0,0) = 0 and randomly draw XH(0,1), XH(1,0) and XH(1,1) from a nor-

mal distribution with expectation 0 and variance 1.

2. Iterate over (i, j) and randomly draw XH(k2−i, l2− j) from a normal distribution

with expectation

4.6 Artificial Surfaces 71

1

4

[

XH

(

(k−1)2−i,(l −1)2− j)

+ XH

(

(k−1)2−i,(l + 1)2− j)

+XH

(

(k + 1)2−i,(l −1)2− j)

+ XH

(

(k + 1)2−i,(l + 1)2− j)

]

and variance

2(−i− j)H.

The variables k, l, i and j have to satisfy

0 ≤ k ≤ 2i, i ≥ 1, k odd and

0 ≤ l ≤ 2 j, j ≥ 1, l odd.

Because of the continuity of XH(x,y) the resulting surface is completely determined.

The correlation between fractal dimension and impression of roughness was in-

vestigated in [2] using a single-point contact with the virtual surface. The PHAN-

ToM device was used to simulate the feel of surfaces with different indices H. The

test subjects could use a pen-like probe to explore the surfaces. To permit compa-

rability the smoothest and the roughest surface were presented and they were given

fixed roughness values of 1 and 20. The test persons were asked to rate randomly

chosen surfaces inside this range.

Figure 4.20 shows some results of this experiment. As one can see there exists

a strong correlation between the fractal dimension of a haptic texture XH(X ,Y ),i.e. dimXH(x,y) = 3 −H and the impression of roughness received by tactually

exploring it.

0

2

4

6

8

10

12

14

16

18

20

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

test person 1test person 2test person 3test person 4test person 5

Fig. 4.20 Results of the experiment with the indices H of the Brownian surfaces on the ab-

scissa and the impression of roughness on the ordinate (with 20 corresponding to the roughest

and 1 to the smoothest surface)


4.7 Conclusion

Two different types of surfaces are investigated in this chapter: artificial surfaces

with well-defined properties described in Sect. 4.6 and real surfaces of 54 different

sample fabrics which are described in Sect. 4.1. The aim of this chapter is the gener-

ation of virtual surfaces that are useful for the tactile simulation systems described

in Chap. 5. In the case of artificial surfaces only the computation of height profiles is

regarded (see Sect. 4.6), but in the case of real surfaces the surface properties have

to be measured and digitised before any further processing can take place. Three

different sources of information are used in this chapter: information on the produc-

tion process of fabrics described in Sect. 4.1, the one-dimensional KES-F surface

measurements as described in Sect. 4.2 and a two-dimensional optical surface scan

obtained with a standard flat bed scanner. From these sources of information either

descriptions of the spatial frequency content at different positions on the surface or

two-dimensional height profiles are computed with different methods. These meth-

ods are described in Sects. 4.3, 4.4 and 4.5 and are indicated with different types of

arrows in Fig. 4.21.

!"#$"%&'()*+),-.-/,#),#

/,)01$2), $/,"%34506

2)" +()2),#

()"% +('"-)

"(#$'$-$"% +('"-)

$,'/(2"#$/,/,&'"7($-"#$/,

!(/-)

#8/01$2), $/,"%/!#$-"%

+('"-)& -",

#8/01$2), $/,"%9)$:9#&!(/'$%)

Fig. 4.21 Four different methods for the generation of surface models indicated by the differ-

ent types of arrows

References 73

The method described in Sect. 4.3 is the only one described in this chapter that

does not create a height profile. Instead it describes a surface in terms of its local

spatial frequency content. In contrast to a height profile this is a description conve-

nient for the tactile renderer, because it computes the tactile signals for the tactile

display in the frequency domain anyway as described in Chap. 5.

Both methods described in Sects. 4.3 and 4.4 analyse the frequency content of the

KES-F surface measurements. While the spectra are adopted without any changes

in Sect. 4.3, the method in Sect. 4.4 attempts to separate the true harmonic com-

ponents from the noise. The latter is necessary because a two-dimensional height

profile is constructed from the two perpendicular one-dimensional KES-F surface

measurements under the assumption that the whole surface can be modeled with

simple harmonic functions. As in Sect. 4.3 additional information on the production

process of fabrics has to be integrated for the two-dimensional composition. There-

fore, both methods can only be applied to regular fabrics or very similar surfaces.

The methods fail with more complicated fabrics like Jacquard fabrics.

The method described in Sect. 4.5 does not require specialised hardware like

the KES-F testers but works with a standard flat bed scanner. Because the mea-

surement is already two-dimensional it does not require the assumptions needed

for the two-dimensional composition carried out by the other methods. However,

it is assumed that a tiling of the regarded surface can be found in order to be able

to clean it from noise and to extrapolate the height profile beyond the borders of

the image taken by the flat bed scanner. Furthermore, the method does only work

with monochrome surfaces, because otherwise the darker colours are recognised as

shades by the shape-from-shading algorithm.

References

[1] Behery, H.M. (ed.): Effect of mechanical and physical properties on fabric hand. Wood-

head Publishing Limited (2005)

[2] Bergmann, M., Herbst, I., von Wieding, R., Wolter, F.E.: Haptical rendering of rough

surfaces using their fractal dimension. In: Proceedings of the First PHANToM Users

Research Symposium, German Cancer Research Center, Heidelberg, Germany, pp. 9–

12 (1999)

[3] Bloomfield, P.: Probability and Statistics, 2nd edn. Fourier Analysis of Time Series.

John Wiley and Sons, Chichester (2000)

[4] Boos, A.D.: Concepts and understanding of fabric hand, ch. 2, pp. 11–44. Woodhead

Publishing Limited (2005)

[5] Falconer, K.J.: Fractal geometry: mathematical foundations and applications. Wiley,

Chichester (1990)

[6] Grover, D., Deller, J.: Digital signal processing and the microcontroller. Prentice-Hall,

Englewood Cliffs (1998)

[7] Handley, C.: The analysis and reconstruction of repetitive textures. In: Computer Graph-

ics International 1998, pp. 273–276. IEEE Computer Society, Los Alamitos (1998)

[8] Horn, B.K.P.: Shape from shading: A method for obtaining the shape of a smooth

opaque object from one view. PhD thesis, Department of Electrical Engineering, Mas-

sachusetts Institute of Technology (1970)


[9] Huang, G.: Feel the “fabric” via the phantom. PhD thesis, University of Pennsylvania

(2002)

[10] Huang, G., Metaxas, D., Govindaraj, M.: Feel the “fabric”: an audio-haptic interface.

In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer

animation, Eurographics Association, pp. 52–61 (2003)

[11] Kawabata, S.: The standardization and analysis of hand evaluation. Tech. rep., The Hand

Evaluation and Standardization Commitee, The Textile Machinery Society of Japan.

Osaka Science and Technology Center Bld., 8-4, Utsubo-1-chome, Nishi-ku, OSAKA

550 Japan (1980)

[12] Kobberling, T.: Verschiedene Methoden zur Generierung virtueller taktiler

Oberflachenstrukturen. Master’s thesis, Welfenlab, Universitat Hannover (2006)

[13] Kruse, S.M.: Ein Beitrag zur Segmentierung von Bildszenen im Kontext der objektori-

entierten Bilddatencodierung. Wiss.- und Technik-Verlag, Berlin (1997)

[14] Makinen, M., Meinander, H., Luible, C., Magnenat-Thalmann, N.: Influence of phys-

ical parameters on fabric hand. In: Haptex 2005 – Workshop on Haptic and Tactile

Perception of Deformable Objects, Welfenlab, Universitat Hannover, pp. 8–16 (2005)

[15] Minazio, P.G.: Fast – fabric assurance by simple testing. International Journal of Cloth-

ing Science and Technology 7(2/3), 43–48 (1995)

[16] Peinecke, N.: Symmetriefindung in periodischen Texturen. Student research paper,

Welfenlab, Universitat Hannover (1999)

[17] Peinecke, N., Allerkamp, D., Wolter, F.E.: Generating tactile textures using periodicity

analysis. In: 2007 International Conference on Cyberworlds, pp. 308–313. IEEE Com-

puter Society, Los Alamitos (2007)




[19] Taylor, M.A.: Technology of Textile Properties, 3rd edn. Forbes Publications, London

(1990)

[20] Tsai, P.S., Shah, M.: Shape from shading using linear approximation. Image and Vision

Computing Journal 12(8), 487–498 (1994)

[21] Varheenmaa, M., Meinander, H.: Mechanical properties as a base for haptic sensing of

virtual fabrics. In: Proceedings of the Autex 2007 Conference, Tampere, Finland (2007)

[22] Vaughn, E.A.: An introduction to textile manufactoring — principles, products and pro-

cesses. Tech. rep., School of Textiles, Fiber and Polymer Science, Clemson University,

Clemson, South Carolina, USA (1996)

[23] Zhang, R., Tsai, P.S., Cryer, J.E., Shah, M.: Analysis of shape from shading techniques.

In: IEEE Computer Vision and Pattern Recognition 1994 Conference, pp. 377–384.

IEEE Computer Society, Los Alamitos (1994)

Chapter 5

Tactile Rendering

While moving a fingertip over a fine surface we experience a sensation that gives us

an idea of its properties. A satisfactory simulation of this feeling is still an unsolved

problem, although a lot of different tactile displays have been developed. In most

attempts to display real surfaces only a few features, like e.g. ridges and grooves,

are simulated. A system for the display of real surfaces is presented in [8], but

the computation of the tactile signals is only based on a photograph of the surface

texture and is not further motivated with respect to its physical correctness. In this

chapter rendering strategies based on vibrations, which play an important role in the

tactile exploration of fine surfaces, are described. To produce appropriate excitation

patterns the tactile displays described in Sect. 3.2 are used.

The tactile renderers described in this chapter are all based on the idea of using a

superposition of vibrations at 40 and 320 Hz to stimulate the SA1 and PC channel,

respectively. These frequencies have been chosen because the two tactile channels

are especially sensitive in the respective frequency regions and the fact that one fre-

quency is a multiple of the other is quite handy for the generation of the vibrations.

This approach has the advantage that only amplitude data have to be transmitted to

the electronic component driving the tactile display. But it is not immediately clear

how the amplitudes have to be computed from a given surface model. Several rea-

sonable approaches exist and only some of these could be investigated within this

work.

All renderers have in common that the position of the finger on the surface has

to be tracked, that appropriate commands have to be sent to the tactile display via

USB, that a possibility to select the fabric to be rendered has to be offered and that

also a visual feedback has to be given to the user. Therefore a common framework

was implemented which is described in Sect. 5.1.

When the first experiment was conducted a tactile display was not available yet.

In order to test the experimental methods presented in Sect. 2.1 with the methods for

the generation of virtual surfaces described in Sects. 4.4 and 4.5 a force-feedback

device was used instead. The experimental setup and the results are described in

Sect. 5.2.



76 5 Tactile Rendering

The tactile renderer presented in Sect. 5.3 extracts the two amplitudes for the

40 and 320 Hz vibrations from the frequency spectrum of the time-varying height

along the trajectory of the finger on the rendered surface. For each tactile channel

a band-pass filter is used to extract the relevant parts of the frequency spectrum. A

single amplitude is obtained via integration.

The tactile simulation system was integrated with a force-feedback system which

was developed within the HAPTEX project. The involved software components and

their integration as well as the integrated hardware are described in Sect. 5.4. The

results of a subjective evaluation of the system is also described.

5.1 Rendering Framework

A common framework was developed for the different versions of the tactile render-

ing algorithms, which is described in this section. It consists of four threads executed

in parallel as shown in Fig. 5.1. Two threads are responsible for acquiring the input

data needed by the two other threads. The integrity of the data shared between the

threads is guaranteed by using mutex objects.

!"#$%&'($!)"*

(!"&+%, ($!)"*

,)(-.!+($!)"*

("&(%/)!),*)!%, ($!)"*

-.!+'#"&)

#.'%(%.,.01("&(%/)*%'#/"2

34156 78156

#/"&)'10"9!%&'.,1-.!+'#"&)

')('1#.'%(%.,1.0("&(%/)1*%'#/"2

*!"-10"9!%&'

*!"-1("&(%/)*%'#/"2

!),*)!

!),*)!

Fig. 5.1 The four threads of the tactile rendering framework

The network thread listens on a network port for user requests and accordingly

places fabrics on a virtual workspace. In this way an experiment supervisor is able

to set up the workspace for the subject from a remote computer.

The tracking thread continuously receives positional data from a device track-

ing the position of the tactile display. For this work a standard computer mouse, a

5.1 Rendering Framework 77

graphics tablet or a force-feedback device were used as tracking device. Figure 5.2

shows the tactile display connected to a computer mouse used as tracking device.

The whole platform is supported by roller bearings that allow it to be moved on the

table in any direction.

Fig. 5.2 The tactile display connected to a computer mouse used as tracking device

The function implementing the rendering algorithm is constantly called by the

tactile rendering thread every 25 ms, which corresponds to 40 Hz. This function

computes the new amplitudes for the tactile display based on the positional data

from the tracking device and the configuration of the virtual workspace.

The graphics thread shows the workspace and the position of the tactile display

on the subject’s computer screen (see Fig. 5.3). The screen is constantly updated at

25 Hz.

5.1.1 Position Tracking

Different devices were used to track the position of the tactile display. The accuracy

of the position measurements is mainly influenced by the spatial resolution of the

device, usually specified in lines per inch (lpi), and by the sample rate at which the

position is measured.

Apart from the position the tactile rendering algorithms also need to know the

velocity of the display. A Kalman filter not only reduces noise but also can compute

the velocities corresponding to the measured positions (cf. [24]). It is based on the


Fig. 5.3 Graphical presentation of the workspace and the position of the tactile display pins

(depicted as green boxes)

idea of estimating the state x ∈ IRn of a discrete-time controlled process by predict-

ing the state of the next time step and by correcting the predicted state using the

corresponding measurement of the process as depicted in Fig. 5.4.

!"#$%&'()#*&+#'!,)!-./

0#(12+#"#.)%&'()#

*,-++#,)!-./

Fig. 5.4 Kalman filter as a predictor–corrector method

The discrete-time controlled process mentioned above is described by

xk = Axk−1 + Buk + wk−1. (5.1)

The matrix A ∈ IRn×n relates the state at time step k− 1 to the state at time step k.

For the position tracking the state is chosen to be

x =

⎛

⎝

p

v

a

⎞

⎠ (5.2)

5.1 Rendering Framework 79

where p, v and a denote the position, the velocity and the acceleration, respectively.

The used model assumes the acceleration a to be approximately constant and thus

the matrix A is chosen to be

A :=

⎛

⎝

1 Δ t 12Δ t2

0 1 Δ t

0 0 1

⎞

⎠ (5.3)

where Δ t denotes the time between two time steps. The term Buk describes a driving

force, which is not existent in the model used for the position tracking. The random

variable wk−1 represents the process noise, which is assumed to be white, with a

normal probability distribution, a mean of zero and a covariance matrix Q. The

latter is defined as

Q :=

⎛

⎝

14Δ t4 1

2Δ t3 1

2Δ t2

12Δ t3 Δ t2 Δ t

12Δ t2 Δ t 1

⎞

⎠var(a). (5.4)

In this way the influence of the user on the position of the tactile display is modeled

as noise of the acceleration. The variance of the acceleration has been experimen-

tally estimated as 90 m s−2.

A measurement zk ∈ IRm is related to the state with

zk = Hxk + vk. (5.5)

As only the position is measured H has the form

H :=(

1 0 0)

. (5.6)

The random variable vk represents the measured noise. Like the process noise it

is assumed to be white, with a normal distribution, a mean of zero and covariance

matrix R. For the position tracking R is a 1×1 matrix and has been estimated as

R = (0.001m). (5.7)

In our model (see Fig. 5.5) every contactor of the tactile array is assigned a two-

dimensional device coordinate

(

xi

yi

)

. Let

(

0u(t)0v(t)

)

denote the position of the origin

of the device coordinate system on the fabrics surface (trajectory) and φ(t) the angle

between the x-axis and the u-axis at the time t, then the position of the contactors

on the fabric’s surface can be computed with

(

ui(t)vi(t)

)

=

(

0u(t)0v(t)

)

+

(

cosφ(t) −sinφ(t)sinφ(t) cosφ(t)

)(

xi

yi

)

(5.8)

Using the property map P(u,v) from Chap. 4 we can now define the time dependent

property function Pi(t) := P(ui(t),vi(t)) along the trajectory.


Fig. 5.5 Tactile contactors

on the fabric’s surface

u

v

x

y

φ

5.2 Experiment with a Force-Feedback Device

In Chap. 4 different methods for the generation of virtual surfaces are presented. In

order to compare these methods using the evaluation methods described in Chap. 2 a

first series of experiments was conducted. The main intention of these experiments

was not to compare the virtual surface models but to test whether the evaluation

methods are useful for such a comparison. The experiments presented in this section

are also described in [9] and in [14].

When this first series of experiments was conducted a tactile display was not yet

available then. However, this was not a real drawback as the intention was to try the

evaluation methods first and not to already evaluate the tactile simulation systems.

It was decided to simulate the surface textures with a force-feedback device as there

already existed some experience in this area (cf. [16] and [13]). However, it should

be noted that the pen-like probe used for the experiments is not a natural tool to

assess the properties of fabrics. Therefore, this series of experiments is unsuitable

for the evaluation of the usefulness of the methods for the generation of virtual

surface models for tactile simulation systems.

5.2.1 Method

Two methods are compared in this section: The method described in Sect. 4.4, here

shortly refered to as the “Kawabata method”, and the method described in Sect. 4.5,

here simply called “optical method”. Both methods create a height profile which is

then used to render the information using a PHANToM Premium 1.0 haptic device

as described in Sect. 3.4.

Since we wanted to find out more about discriminability, we chose a subset of

relatively similar fabrics. Furthermore, for our experiments the haptic properties

of the fabrics had to be visible in order to be reconstructable from an optical scan.

5.2 Experiment with a Force-Feedback Device 81

These criteria resulted in a test set of the cloth samples 8, 9, 18, 22, 23, 28, 29 (see

Annex A).

To compare both methods with each other, and to compare them with real fabrics,

a setup similar to [18] was chosen. In a triangular method three objects were pre-

sented to the subject, with two of them being identical. The subject had to tell, which

one of the three objects was different from the two other ones (cf. Sect. 2.1.3). After

that, the subject had to judge, if it was “easy”, “normal” or “hard” to make a deci-

sion. Since each odd-man-out experiment incorporates two different textiles, each

subject had to answer n(n−1)/2 questions for n samples.

In order to understand which properties of the fabrics are important for the dis-

crimination task an MDS procedure was employed (see Sect. 2.1.6). To gather the

perceptual dimensions subjective distances between the fabrics had to be found. In

Sect. 2.1.5 the computation of subjective distances is explained. Here a little differ-

ent approach was used: each answer was weighted with 1, 0.5 or 0.1 if the test was

considered “easy”, “normal” or “hard”, respectively. If the answer was wrong, i.e.

the subject did not identify the odd sample, the answer received weight 0. Note that

the choice of weights is more or less arbitrary here. Obviously we had to choose

a non-zero value for the “hard” value but this could have been a lesser value than

0.1 also. Aside from that choosing other values results in a slight transformation

of the resulting spaces. But the overall qualitative distances should be kept as long

as the “normal” value is located near the middle between “hard” and “easy”. The

subjective distance is then defined as the sum of all weights. The distances between

all samples build up a complete distance graph which is then embedded into a low-

dimensional space. This projection is of course subject to an error which depends

on the dimension of the embedding space. The dimension is chosen as maximal in a

sense that higher dimensions do not reduce the error significantly (see Sect. 2.1.6).

Twelve subjects (4 female, 8 male) participated in the experiments. They were

aged between 20 and 64 years (on average 32 years). None of the subjects used

a force-feedback device or participated in a similar experiment before. During the

tests the subjects could neither see the fabric nor hear the noises generated by the

pen or the device, so that visual and acoustic impressions could not interfere with

the tactile sensation.

5.2.2 Results

In the first experiment the subjects could freely move a real pen (resembling the

interface pen of the force-feedback device) over a sample of real fabric. Figure 5.6

shows an MDS plot of the subjective distances gathered from the first experiment. In

contrast to Fig. 5.6 Fig. 5.7 shows the fabrics in an objective parameter space (with

values taken from [9] and [12]). As one can see the relative positions of the fabrics

(emphasised by triangles) are quite similar in both figures, with the exception of

Sample 29. Thus the first dimension of Fig. 5.6 probably resembles the roughness

of the fabric as also stated in [9]. The second dimension probably corresponds to the

compressional resilience.


-6

-4

-2

0

2

4

6

-6 -4 -2 0 2 4 6 8

dim

en

sio

n 2

dimension 1

8

9

18

22

23

28

29

Fig. 5.6 MDS plot of the first experiment

In the second experiment the subject could explore a virtual sample of fabric

generated from the optical method (cf. Sect. 4.5) rendered by the force-feedback

device. Just like in the first experiment the size of the virtual samples was 2cm2.

Figure 5.8 shows an MDS plot of the subjective distances gathered from the second

experiment. Note that the MDS method has reduced the data set to one dimension

here. Similar to one of the dimensions in the first experiment the one dimension that

is left after reducing gives a good approximation of the textures roughness. Just like

in the real world set up the samples 8 and 28 are very close to each other, resembling

the real impression. Since compressional resilience was not simulated by the system

this dimension does not appear as in the first experiment.

The third experiment was identical to the second, except that the haptic sensation

was generated using the Kawabata measurements (cf. Sect. 4.4). Figure 5.9 shows

an MDS plot of the subjective distances gathered from the third experiment. Just like

the optical method the Kawabata method gives a good discrimination of the fabrics’

roughness. There is only one exception concerning the samples 8 and 28, that were

very similar in the first two experiments but are placed at distant positions by the

Kawabata measurements. This could possibly be caused by interpolation problems

of the very fine grained surfaces of these fabrics.

During these odd-man-out tests the subjects were asked if they found the dis-

criminability of the samples “hard”, “normal” or “easy”. In Fig. 5.10 one can see

the accumulated results of these questions. One observes that it seems to be much

easier and more accurate to tell the different fabrics apart in the real setup. The two

5.2 Experiment with a Force-Feedback Device 83

30

40

50

60

70

80

90

100

2 4 6 8 10 12 14 16

co

mp

ressio

na

l re

sili

en

ce

roughness

8

28

18

22

9

23

29

Fig. 5.7 Roughness and compressional resilience of the fabrics

-8 -6 -4 -2 0 2 4 6

89 182223 2829

Fig. 5.8 MDS plot of the second experiment

-8 -6 -4 -2 0 2 4 6 8

89 182223 2829

Fig. 5.9 MDS plot of the third experiment

virtual methods perform relatively similar with no significant drawbacks on both

sides.

In Fig. 5.11 the errors of the subjects are shown for each fabric. It can be seen

that the optical method performs comparable to the Kawabata method with the only,

already known, exception of Sample 8. One can see in the figure that the errors for

the samples 18, 28 and 8 are very low for the experiment with real fabrics. This is

probably due to their relatively high distance in the second perceptual dimension

mentioned earlier in this section (see Fig. 5.6).


0

10

20

30

40

50

60

wrong hard normal easy

an

sw

ers

in

%real

opticalKawabata

Fig. 5.10 Classification of discriminability

0

5

10

15

20

25

30

29 23 9 22 18 28 8

err

or

in %

realoptical

Kawabata

Fig. 5.11 Errors of the subjects

5.3 Vibrotactile Rendering 85

5.3 Vibrotactile Rendering

Basically, a tactile renderer needs two ingredients to work properly: a computer

representation of a fabric and the trajectory of the fingertip on the fabric’s surface.

As described in Sect. 2.3 vibrotaction plays an important role in the perception of

fine surface textures. Therefore, we compute the vibrations occurring in the finger-

tip while moving along the trajectory. These are decomposed into only two basic

frequencies intended to directly stimulate the “Pacinian” (PC) and “non-Pacinian”

(SA1) receptors. We describe this process later in this section. The used stimulator

hardware is described in Sect. 3.2. This approach has already been presented in [1].

5.3.1 Computation of Resulting Vibrations

For each contactor we need to compute the vibrations occurring due to the move-

ment of the fingertip over the surface. This computation depends on the model gen-

erated in Chap. 4. If we represent the surface as a height function as in Sects. 4.4

and 4.5, Pi(t) describes the height under the i-th contactor as a function of time. As a

start we act on the simple assumption that the vibration at the contactor position on

the finger directly corresponds to Pi(t). We want to express a vibration as a function

F assigning to each frequency ω a corresponding amplitude F(ω), which is done

by Fourier transforming Pi(t). Since we need to render the device signals for the

next 25 ms, this time segment becomes our window for the Fourier transform. We

smoothed the boundaries of this segment with a Blackman window.

If the surface is represented as spatial frequency spectra Fe(k) and Fa(k) in weft

and warp direction respectively as computed in Sect. 4.3 a corresponding temporal

frequency spectrum F(ω) has to be computed. When the finger moves over a surface

a spatial frequency k and a temporal frequency ω are related by

ω = vk (5.9)

where v is the velocity of the finger on the surface. This results in the following

conversion

Fe(ω) = Fe

(

ω

ve

)

Fa(ω) = Fa

(

ω

va

) (5.10)

where ve and va denote the component of the velocity in weft and warp direction,

respectively. Assuming that the vibrations in the two orthogonal directions can be

simply combined by addition we obtain

F(ω) = Fe(ω)+ Fa(ω) = Fe

(

ω

ve

)

+ Fa

(

ω

va

)

. (5.11)


5.3.2 Decomposition of Vibrations into Base Frequencies

The previous steps result in a frequency spectrum F(ω), which now has to be de-

composed into amplitudes at only two base frequencies that will then be reproduced

on the tactile stimulator. We use 40 and 320 Hz as base frequencies. The function

mapping the spectrum to these two amplitudes has to ensure that the resulting tac-

tile sensation is very similar to the sensation which would result from reproducing

the whole spectrum on the tactile stimulator. This approach is analogous to the ap-

proach used for multicolour video displays which produce almost arbitrary colours

as mixture of only three fundamental colours.

Similar to the colour model of computer graphics (e.g. see [4]) we assume the

existence of two functions H40 and H320 such that the two amplitudes for 40 and

320 Hz can be computed as

a40 =

∫

H40(ω)F(ω)dω

a320 =

∫

H320(ω)F(ω)dω .(5.12)

This hypothesis still has to be verified with appropriate experiments. Nevertheless,

in the rest of this section we will show the reasonableness of this approach and we

will even derive first approximations of the functions H40 and H320.

With Ip(F) and Inp(F) we denote the stimulation intensity caused by a vibra-

tion with spectrum function F at the Pacinian and non-Pacinian channel, respec-

tively. Note that these intensity functions are subjective and thus hard to define. Our

suggestion for an approximation of these functions is

Ip(F) =

∫

1

Tp(ω)F(ω)dω

Inp(F) =

∫

1

Tnp(ω)F(ω)dω

(5.13)

Since the channels react depending on the frequency (e.g. the Pacinian channel

is most sensitive around 300 Hz while the non-Pacinian channel is most sensitive

around 50 Hz) we use 1Tp(ω)

and 1Tnp(ω)

as frequency dependent damping factors.

The assignment of a linear factor to each frequency seems to be a reasonable ap-

proximation suggested by some curves of equal sensation magnitude plotted as a

function of stimulus frequency (see [20]). Furthermore, the approach of using an

integral is supported by [19, 1], where mixtures of 40 Hz and 320 Hz sinewaves re-

sulting in a constant subjective intensity have been measured. The results of these

measurements can be explained with a linear model as taken as a basis for (5.13).

In [6] thresholds of the tactile channels as functions of stimulus frequency were

measured (see Fig. 5.12 for approximating curves). These thresholds refer to the

smallest perceivable amplitude of a stimulus with only a single frequency. Along

these measured curves the intensity is considered to be constant, making these


0.01

0.1

1

10

100

1000

10000

1 10 100

Dis

pla

cem

ent

Frequency (Hz)

T_pT_np

Fig. 5.12 The functions Tp and Tnp

functions suitable as frequency dependent damping functions Tp(ω) and Tnp(ω)that normalise the intensity to 1 at threshold level.

Let

Sω,α(ω) =

{

α for ω = ω

0 else(5.14)

denote a simple sinusoidal vibration with frequency ω and amplitude α . Our goal

is to replace a given vibration F by two vibrations S40,a40and S320,a320

such that

Ip(F) = Ip(S40,a40)+Ip(S320,a320

)

Inp(F) = Inp(S40,a40)+Inp(S320,a320

)(5.15)

These equations are equal to a two-dimensional linear equation system with (5.13):

∫

F(ω)

Tp(ω)dω =

a40

Tp(40)+

a320

Tp(320)∫

F(ω)

Tnp(ω)dω =

a40

Tnp(40)+

a320

Tnp(320)

(5.16)

The solutions of this system have the form

a40 = B11

∫

F(ω)

Tp(ω)dω + B12

∫

F(ω)

Tnp(ω)dω

a320 = B21

∫

F(ω)

Tp(ω)dω + B22

∫

F(ω)

Tnp(ω)dω

(5.17)


Setting

H40(ω) :=B11

Tp(ω)+

B12

Tnp(ω)

H320(ω) :=B21

Tp(ω)+

B22

Tnp(ω)

(5.18)

we obtain (5.12). H40 and H320 are depicted in Fig. 5.13.

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 100 200 300 400 500 600

Frequency (Hz)

H_40H_320

Fig. 5.13 The functions H40 and H320

5.3.3 First Results with Brownian Surfaces

We used Brownian surfaces (see Sect. 4.6) to test our vibrotactile rendering strat-

egy. Since the system was not yet fully operable for active exploration of a surface in

real time, we used a passive presentation, i.e. stimuli were presented to the station-

ary fingertip computed for the case of the virtual surface moving over the fingertip

at a constant speed of 8 cm s−1. As expected from the single-point-contact experi-

ment, different Brownian surfaces could be distinguished on the basis of roughness

corresponding to the fractal dimension. However, there was little or no sensation of

movement over the fingertip, presumably because individual surface features were

not sufficiently identifiable to allow them to be tracked as they moved over the array.

In the case of active exploration envisaged for the fully operable system, the sensa-

tion of movement would be available from kinaesthetic and visual cues, and so the

results would presumably be more convincing.


5.3.4 Further Evaluation

The vibrotactile simulation system was evaluated using the method already de-

scribed and tested in Sect. 5.2. In order to allow for the comparability of the results

the same set of fabric samples 8, 9, 18, 22, 23, 28, 29 (see Annex A) was used. Again

the subjects had to tell which one of three presented fabrics was different from the

two other ones (cf. Sect. 2.1.3) and had to judge whether it was “easy”, “normal” or

“hard” to make a decision. Each subject performed all comparisons twice in random

order.

Two female and six male subjects aged between 22 and 32 years participated in

the experiment. All subjects used the tactile display with the right hand although

one subject stated to be left-handed. Two subjects used the tactile display before,

the other subjects had no experience with such systems. The simulation system did

not display the visual appearance of the fabrics and subjects wore ear protectors. So

visual and acoustic impressions could not interfere with the tactile sensation.

In the prior experiment (see Sect. 5.2) each answer was weighted with 1, 0.5 or

0.1 if the test was considered “easy”, “normal” or “hard”, respectively. If the answer

was wrong, i.e. the subject did not identify the odd sample, the answer received

weight 0. The subjective distance was then defined as the sum of all weights. This

time subjects were asked to rate the weights for “normal” and “hard” where the

weight for “easy” was predefined as 1 and a weight of 0 was considered to represent

no distinguishablity. Interestingly almost all subjects chose weights very similar to

the rather arbitrarily chosen weights of the prior experiments. The mean values of

the answers are 0.51 for “normal” and 0.14 for “hard”. Therefore, for the sake of

simplicity weights were chosen as in the prior experiments.

The subjective distances between all samples build up a complete distance graph

which is then embedded into a low-dimensional space (see Sect. 2.1.6). Figure 5.14

shows such an MDS plot of the subjective distances gathered from the experiment.

As the underlying surface model of the tactile simulation system is mainly based

on the roughness of the fabrics, the KES-F geometrical roughness values in weft and

warp direction (cf. Sect. 4.2) have been chosen to position the fabrics in an objective

parameter space. Figure 5.15 shows this space. In contrast to the prior experiments

the relative positions of the fabrics are not very similar in Fig. 5.14 and Fig. 5.15.

However, some similarities are still recognisable as depicted by the connecting lines

in both figures.

The visible deformation of the MDS plot might be caused by the design of the

experiment. While the small differences between fabrics were rated by the subjects

in a relatively detailed way this was not the case for the obvious differences which

virtually always received a weight of 1. Therefore, the maximum subjective distance

between fabrics was limited and the distance between very different fabrics was

underestimated.

During these odd-man-out tests the subjects were asked if they found the dis-

criminability of the samples “hard”, “normal” or “easy”. In Fig. 5.16 one can

see the results of these questions. As can be seen the answers of the subjects are


-8

-6

-4

-2

0

2

4

6

8

-12 -10 -8 -6 -4 -2 0 2 4 6

dim

en

sio

n 2

dimension 1

8

9

18

22

23

28

29

Fig. 5.14 MDS plot of the experiment

2

4

6

8

10

12

14

16

18

0 2 4 6 8 10 12 14

rou

gh

ne

ss w

arp

roughness weft

8

9

18

22

23

28

29

Fig. 5.15 Roughness of the fabrics in weft and warp direction

5.4 Integration with a Force-Feedback Device 91

0

10

20

30

40

50

60

wrong hard normal easy

an

sw

ers

in

%

Fig. 5.16 Classification of discriminability

comparable to the results from the prior experiment with real fabrics and better than

with the force-feedback simulation of the fabrics (cf. Fig. 5.10).

In Fig. 5.17 the errors of the subjects are shown for each fabric. These are com-

parable to the errors of the prior experiments shown in Fig. 5.11.

5.4 Integration with a Force-Feedback Device

The simulation systems regarded until now only provide tactile feedback. With these

systems only planar, not curved, surfaces can be simulated. Furthermore, mechani-

cal properties like elasticity cannot be presented. In order to create a more compre-

hensive simulation system the combination of a tactile display with a force-feedback

device providing the large-scale forces is necessary.

Such an integrated system was developed within the EU-funded HAPTEX project

(cf. [15, 5, 11]). It simulates fabrics that are square shaped with a side length of

20 cm. The user can select a fabric from the property database which is then sim-

ulated hanging from a stand. The user can touch the virtual fabric with thumb and

index finger. The fabric can be squeezed, stretched, rubbed and lifted.

The simulated scene is quite challenging, because the physical simulation of the

fabric, the force-feedback renderer and the tactile renderer have to be integrated

in a single simulation system. In order to reduce complexity the above mentioned

subsystems were implemented and tested separately before they were integrated.


0

5

10

15

20

25

30

29 23 9 22 18 28 8

err

or

in %

Fig. 5.17 Errors of the subjects

amplitudes

Textile

Simulation

Thread

Local

Simulation

Thread

Force

Feedback

Thread

running at60Hz running at

1kHz

running at

1kHz

bordering geometry LocalGeometry

force generation ForceExtrapolation

Force

Feedback

Device

forces positions

MotionEstimation

movements

Visual

Display

global mesh adaption

FingertipModel

contact

formulation

deformrefine

Tactile

Rendering

Thread

Tactile

Array

contact area

force distribution, velocity

Fig. 5.18 The structure of the integrated software system from [3]

Figure 5.18 shows the structure of the integrated system. In one thread the whole

fabric is simulated by a simulation software module kindly provided by the MI-

RALab at the University of Geneva. As this simulation module is relatively accurate

it does not provide an update rate sufficiently large for the integration with a force-

feedback device. Therefore, another simulation module has been introduced as in-

termediate layer between the simulation of the whole fabric and the force-feedback


thread. This module reaches the necessary update rate but is less accurate and only

simulates the part of the fabric that surrounds the contact point between the user’s

virtual finger and the fabric. This part of the fabric is called “local geometry” in

the following. The force renderer manages the whole communication between the

physical simulation and the force-feedback devices. Incoming local geometries are

processed and resulting forces are transferred to the force-feedback devices. Geom-

etry updates are sent back to the physical simulation. The tactile renderer controls

the tactile devices. The amplitudes generated from the fingertips’ movement over

the surface are sent to the drive electronics at an update rate of 40 Hz. To enable

the simulation system to work without the tactile renderer a loose coupling between

both was chosen: the tactile renderer is implemented as a separate application, data

is exchanged via inter-process communication (IPC).

5.4.1 Physical Simulation and Haptic Rendering

The simulation software used for the integrated system is described in [22, 21, 23].

Here only a very simplified model is presented in order to explain the fundamentals

of the simulation of fabrics. Although the mass is quite continuously distributed

on a fabric its simulation model consists of a finite set of discrete points called

“particles” each having a position xi and a mass mi. These points are kept together

by internal forces acting on adjacent masses that reflect the real tensile behaviour of

the simulated fabric. Although these forces are usually non-linear they are simply

depicted as springs in Fig. 5.19. In addition to the internal forces external forces, like

for example gravity, act on the particles. Usually also damping forces are included

in the model. The sum of all these forces acting on a single particle is denoted by

Fig. 5.19 A simple model

for the simulation of fabrics1m

m

m m m

m m m

m m m m

mmmm

2 3 4

5 6 7 8

9 10 11 12

13 14 15 16


Fi(x1,x2, . . . , x1, x2, . . .). With Newton’s second law of motion we obtain the system

of differential equations

mixi = Fi(x1,x2, . . . , x1, x2, . . .). (5.19)

This system of ordinary differential equations can be numerically solved with stan-

dard algorithms.

The physical simulation used within the HAPTEX project considers stretch/

tensile, shear, bending and damping forces. Furthermore, the contact simulation

considers friction forces. As for the tactile rendering the relevant physical data have

been acquired with the KES-F system.

In Fig. 5.20 the haptic loop is depicted. During the interaction with the fabric the

position of the user’s fingers and the shape of the fabric change. Both is considered

by the contact model that computes the forces affecting the finger and the fabric.

Here a problem becomes evident: to create a convincing illusion and to avoid stabil-

ity problems the system needs to react to the user’s motion within one millisecond

which is not possible involving the slower simulation of the fabric. However, note

that the single loop depicted in Fig. 5.20 can also be seen as two loops: one between

the user and the contact model and the other between the simulation and the con-

tact model. This view led to the solution described in [3]. A dual-layer approach is

employed there to allow the two loops to run at different speeds.

Fig. 5.20 The haptic loop

with the physical simulation

forc

eposition

shape forc

e

contact model

FF Device

textile simulation

5.4.2 Integrated Interface Hardware

The GRAB device is a force-feedback device consisting of two identical and in-

dependent robotic manipulators (see Fig. 5.21), each having the base link fixed to

the desktop and the end effector (contact part) attached to the palmar surface of

the user’s thumb or index fingertip. Each manipulator measures the absolute posi-

tion and orientation of the contact part. These are used by the haptic renderer to

compute an appropriate force. The manipulator is able to generate this force on the

contact part within a workspace of 400 mm in width, 300 mm in height and 200 mm


Fig. 5.21 The final demonstrator of the HAPTEX Project

in depth. The workspaces of both manipulators overlap. Force errors are limited in

a range of about 10 g (0.1 N). The device can exert forces up to 20 N. The GRAB

device used for the HAPTEX project is described in [2]. It was kindly provided by

the PERCRO laboratory at the Scuola Superiore Sant’Anna in Pisa.

The end effectors of the GRAB device have been replaced with the tactile dis-

plays described in Sect. 3.2. These displays are designed such that the size of the

mechanism between the finger and thumb is minimised. Still, the spacing between

the digits caused by the tactile mechanisms is about 10 mm. As the maximum stiff-

ness between the finger and thumb is limited because of instabilities in the force

control loop the force-feedback renderer enforces an additional separation of 6 mm

to avoid collisions between the mechanisms (cf. [3] and [17]).

5.4.3 Software Integration

The tactile renderer needs the following contact data to work properly:

• contact area between the fingertip and the fabric (in local coordinates of the

fabric)

• velocity of the fingertip relative to the fabric (in local coordinates of the fabric)

• distribution of normal forces inside the contact area


For the computation of the force feedback the haptic renderer already requires

among others the contact point, the velocity of the fingertip relative to the fabric and

the normal force (cf. [22, 3]). Instead of a single contact point the tactile renderer

requires a contact area to decide which contactor pins have to be activated. Also

a single normal force is not sufficient for a convincing tactile simulation. Rather a

distribution of the normal force over the whole contact area is needed by the tactile

renderer.

For each contactor pin the geometry of the tactile actuator determines the point on

the fingertip where the stimulation caused by that pin occurs. These points have been

included in the model of the fingertip. Employing the contact model the position (in

local coordinates of the fabric) and the normal force at each point is computed. The

velocity is computed by applying a discrete differential operator.

The tactile renderer is called every 25 ms, i.e. it runs in a 40 Hz loop. However,

the contact data is computed by the haptic renderer every millisecond and we need

to resample that data. To avoid aliasing the data is passed through an antialiasing

filter that suppresses frequency components above 20 Hz (cf. [7]).

In the stand-alone version of the tactile renderer the fabric was assumed to be

planar. Furthermore the finger was assumed to always being in contact with the

plane of the fabric. However, in the integrated case the fabric may have folds and

wrinkles resulting in a more complex contact geometry. Due to the contact pressure

the fingertip deforms which also influences the shape of the contact area. As a con-

sequence not all pins of the tactile actuator should be activated, only those inside

the contact area. The shape of each finger’s contact area is provided by the force-

feedback renderer. Note that a contact area may not exist, i.e. in that case the finger

does not touch the fabric.

While moving a fingertip over a rough surface we experience a tactile sensation.

Its intensity also depends on the force with which the finger is pressed against the

fabric. For relatively small forces a linear dependency between the force and the

sensation seems to be reasonable. Therefore the amplitudes computed by the tactile

renderer are multiplied by kFF before they are transmitted to the tactile actuator. F

denotes the force between the finger and the fabric and kF is a constant relating the

force to the intensity. Previous to the integration there was no possibility to assess

the contact pressure between the fingertip and the fabric. Therefore a constant force

normal to the fabric was assumed implicitly by the tactile renderer.

5.4.4 Evaluation of the Integrated System

The goal of the HAPTEX project was the development of a system for the realistic

haptic simulation of real fabrics. In order to evaluate the system four subjects were

asked to rate the tensile, roughness, friction and bending properties of five fabrics.

Two subjects (A, B) rated the properties of real fabrics whereas two other subjects

(C, D) rated the same properties of the corresponding virtual fabrics simulated by the

system. For each property a single manipulation for its assessment was defined (cf.

[17] and [10]). The interaction with the fabrics was restricted to these manipulations

for the experiments. As using the simulation system is very difficult without vision


Fig. 5.22 Reference samples 1 (top row) and 5 (bottom row) for tensile (left) and bending

(right)

a coarse wireframe model of the fabric was shown as a compromise. Consequently

the tensile and bending tests were performed with vision on the real textiles as well.

The fabric samples 33, 34, 36, 37 and 39 were used for the experiment (see

Annex A). The subjects were asked to rate the fabrics’ properties in the range 1

to 5. For each property two other fabrics, called reference 1 and reference 5, were

provided to define the scale. Figure 5.22 shows the reference samples for the tensile

and bending properties. Samples 39 and 10 were used as reference 1 and reference

5 for tensile stiffness, 24 and 10 for roughness, 24 and 16 for friction, and 21 and

32 for bending. Each fabric was presented twice in a sequence of 10 samples to

subjects C and D.

The data obtained by the experiment were analysed by calculating the correla-

tion of two successive runs of the same subject (“repeatability”), the correlation of

the results of C and D (“consistency”) and the correlation of the mean ratings of A


Table 5.1 The correlation results of the evaluation (cf. [17])

repeatability consistency realism

C D

tensile 0.99 0.97 0.91 0.96

roughness 0.91 0.95 0.93 0.90

friction 0.66 0.67 0.76 0.93

bending 0.30 0.07 0.61 0.00

and D with the mean ratings of C and D (“realism”). Table 5.1 summarises the re-

sults of the evaluation. Tensile and roughness gave the best results with a correlation

coefficient of at least 0.9 for repeatability, consistency and realism. Friction had a

realism coefficient of 0.93 although its repeatability correlation coefficient was only

average with about 0.66. This relatively high realism coefficient might be caused by

the use of the mean ratings as averaging has the effect of reducing the amount of

random variation. Unfortunately bending showed no real correlation, giving coeffi-

cients as low as 0.07 for successive runs with the same subject. The reason for the

latter was that with the haptic feedback being unable to represent the small forces

resulting from bending, the user had to rely on what one could see to rate the bend-

ing property. However, rating the bending property by visual assessment was most

likely impaired by the coarseness of the mesh not giving enough hints to the user.

For complete results see [17]. The results of the evaluation are also presented in [3].

5.5 Conclusion

Only few works exist on the tactile simulation of real surface textures. The only

system intended for the rendering of arbitrary surface textures that is comparable to

the systems presented in this chapter is described in [8]. However, this system is not

aimed at a realistic simulation of the surfaces.

In this chapter several experiments are described that were conducted to explore

the problem from different points of views. Therefore, a common software frame-

work was developed which manages a virtual workspace where the virtual fabrics

are presented, which processes the date from the tracking device and which trans-

mits the amplitude data to the tactile display.

The experiment described in Sect. 5.2 was conducted as a pilot study. Its purpose

was to test the experimental methods from Sect. 2.1. Instead of a tactile display a

force-feedback device with a pen-like end effector was used to display the virtual

surfaces since there already existed some experience in the simulation of surface

textures with force-feedback devices (cf. [16] and [13]). The results of the exper-

iment suggest that subjects mainly regarded compressional resilience and surface

roughness when distinguishing real fabrics with a pen-like probe. The results fur-

ther suggest that from these two properties only the roughness of the surfaces was

References 99

properly produced by the simulation system. Although the system accurately ren-

dered the height profile of the two-dimensional surfaces, i.e. the space of all possi-

ble surfaces having a lot of dimensions, the perceptual space seems to consist of the

one-dimensional roughness only.

As the pen-like probe used for the experiments is not a natural tool to assess

the properties of fabrics it is an interesting question whether the perceptual space

has more dimensions when a tactile display is used. In Sect. 5.3 a tactile rendering

system based on vibrations is presented. It is important to note that the rendering

algorithm proposed in the section is only one of many possible rendering schemes.

However, the renderer is based on current knowledge on human tactile perception

and only a few reasonable assumptions had to be presumed. An evaluation of the

system showed that subjects were able to distinguish relatively similar fabrics when

rendered by the system. Statements of the subjects suggest that their decision was

mainly based on the roughness of the fabrics in weft and warp direction. However,

the latter assumption was not clearly supported by the results of the evaluation.

In Sect. 5.4 the integration of the tactile simulation system with a force-feedback

system is presented. In such an integrated system many different subsystems have

to be integrated: a physical simulation of the fabric, a force-feedback renderer with

an appropriate force-feedback device and a tactile renderer with a tactile display. As

shown by a subjective evaluation the integration effort resulted in a system that is

capable of not only simulating the surface roughness of fabrics but also their tensile

and friction properties.

References

[1] Allerkamp, D., Bottcher, G., Wolter, F.E., Brady, A.C., Qu, J., Summers, I.R.: A vibro-

tactile approach to tactile rendering. The Visual Computer 23(2), 97–108 (2007)

[2] Bergamasco, M., Salsedo, F., Fontana, M., Tarri, F., Avizzano, C.A., Frisoli, A., Ruf-

faldi, E., Marcheschi, S.: High performance haptic device for force rendering in textile

exploration. The Visual Computer 23(4), 247–256 (2007)

[3] Bottcher, G., Allerkamp, D., Glockner, D., Wolter, F.E.: Haptic two-finger contact with

textiles. The Visual Computer 24(10), 911–922 (2008)

[4] Foley, J.D., van Dam, A., Feiner, S.K., Hughes, J.F.: Computer Graphics: Principles and

Practice, 2nd edn. Addison-Wesley, Reading (1990)



tactile rendering into a single VR system. In: 2007 International Conference on Cyber-

worlds, pp. 277–284. IEEE Computer Society, Los Alamitos (2007)

[6] Gescheider, G.A., Bolanowski, S.J., Pope, J.V., Verillo, R.T.: A four-channel analysis

of the tactile sensitivity on the fingertip: frequency selectivity, spatial summation, and

temporal summation. Somatosensory & Motor Research 19(2), 114–124 (2002)

[7] Grover, D., Deller, J.: Digital signal processing and the microcontroller. Prentice-Hall,

Englewood Cliffs (1998)

[8] Ikei, Y., Wakamatsu, K., Fukuda, S.: Vibratory tactile display of image-based textures.

IEEE Computer Graphics and Applications 17(6), 53–61 (1997)


[9] Kobberling, T.: Verschiedene Methoden zur Generierung virtueller taktiler

Oberflachenstrukturen. Master’s thesis, Welfenlab, Universitat Hannover (2006)

[10] Luible, C., Varheenmaa, M., Magnenat-Thalmann, N., Meinander, H.: Subjective fab-

ric evaluation. In: 2007 International Conference on Cyberworlds, pp. 285–291. IEEE

Computer Society, Los Alamitos (2007)

[11] Magnenat-Thalmann, N., Volino, P., Bonanni, U., Summers, I.R., Bergamasco, M.,

Salsedo, F., Wolter, F.E.: From physics-based simulation to the touching of textiles:

The HAPTEX project. International Journal of Virtual Reality 6(3), 35–44 (2007)

[12] Makinen, M., Meinander, H., Luible, C., Magnenat-Thalmann, N.: Influence of phys-

ical parameters on fabric hand. In: Haptex 2005 – Workshop on Haptic and Tactile

Perception of Deformable Objects, Welfenlab, Universitat Hannover, pp. 8–16 (2005)

[13] Meyer-Spradow, J.: Ein mathematisches Echtzeit-Modell taktiler Merkmale von

Oberflachen. Diploma thesis, Welfenlab, Universitat Hannover (2005)

[14] Peinecke, N., Allerkamp, D., Wolter, F.E.: Generating tactile textures using periodicity

analysis. In: 2007 International Conference on Cyberworlds, pp. 308–313. IEEE Com-

puter Society, Los Alamitos (2007)

[15] Salsedo, F., Fontana, M., Tarri, F., Ruffaldi, E., Bergamasco, M., Magnenat-Thalmann,

N., Volino, P., Bonanni, U., Brady, A.C., Summers, I.R., Qu, J., Allerkamp, D., Bottcher,

G., Wolter, F.E., Makinen, M., Meinander, H.: Architectural design of the HAPTEX

system. In: Haptex 2005 – Workshop on Haptic and Tactile Perception of Deformable

Objects, Welfenlab, Universitat Hannover, pp. 1–7 (2005)




[17] Summers, I.R.: Final demonstrator and final integration report. Tech. rep., HAPTEX –

HAPtic sensing of virtual TEXtiles (2008),

http://haptex.miralab.unige.ch

[18] Summers, I.R., Irwin, R.J.: Tactile discrimination of paper. In: Haptex 2005 – Work-

shop on Haptic and Tactile Perception of Deformable Objects, Welfenlab, Universitat

Hannover, pp. 43–49 (2005)

[19] Summers, I.R., Brady, A.C., Syed, M.K., Chanter, C.M.: Design of array stimulators

for synthetic tactile sensations. In: Proceedings of the First Joint Eurohaptics Confer-

ence and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator

Systems, pp. 586–587. IEEE Computer Society, Los Alamitos (2005)

[20] Verrillo, R.T., Fraioli, A.J., Smith, R.L.: Sensation magnitude of vibrotactile stimuli.

Perception & Psychophysics 33, 379–387 (1969)

[21] Volino, P., Magnenat-Thalmann, N.: Accurate anisotropic bending stiffness on particle

grids. In: 2007 International Conference on Cyberworlds, pp. 300–307. IEEE Computer

Society, Los Alamitos (2007)

[22] Volino, P., Davy, P., Bonanni, U., Magnenat-Thalmann, N., Bottcher, G., Allerkamp,

D., Wolter, F.E.: From measured physical parameters to the haptic feeling of fabric.

In: Haptex 2005 – Workshop on Haptic and Tactile Perception of Deformable Objects,

Welfenlab, Universitat Hannover, pp. 17–29 (2005)

[23] Volino, P., Davy, P., Bonanni, U., Luible, C., Magnenat-Thalmann, N., Makinen, M.,

Meinander, H.: From measured physical parameters to the haptic feeling of fabric. The

Visual Computer 23(2), 133–142 (2007)

[24] Welch, G., Bishop, G.: An introduction to the Kalman filter. Tech. rep., University of

North Carolina, Chapel Hill (2001)

http://haptex.miralab.unige.ch

Chapter 6

Summary and Outlook

The research described in this monograph is the first attempt to develop a realistic

tactile simulation system for real existing surfaces. Although there exist many differ-

ent tactile displays developed by various research groups, most systems are designed

to convey information to visually impaired persons (e.g. [3] or [2]). Other systems

are limited to the simulation of simple geometrical surface features like ridges and

grooves (e.g. [4]). However, unlike the work at hand none of these systems aims at

a comprehensive and realistic tactile simulation of real existing surfaces.

This work originated from the EU-funded HAPTEX project and thus inherited

some of its prerequisites: the class of the surfaces to be simulated by the system

is limited to fabrics and only one type of tactile displays was available for the ex-

periments. The former is only a minor restriction as there exists a large variety of

different fabrics—certainly enough to argue that the system is in principle able to

simulate arbitrary surfaces. The latter prerequisite is a necessity since the resources

of the research project were limited and the development of a tactile display is rel-

atively cost-intensive. Therefore, it is reasonable to conduct an in-depth analysis of

only a single display. Moreover, the display used within this work follows an in-

teresting approach. Instead of simply reproducing the geometry of a surface it is

intended for the direct stimulation of two different kinds of tactile receptors with

superimposed vibrations of two different frequencies.

Before a tactile rendering system could be implemented as presented in Chap. 5

a number of preparations had to be made which are described in Chap. 2, Chap. 3

and Chap. 4. In order to systematically tackle the problem it was necessary to gain

insight into many different disciplines: psychophysics, neurophysiology, electronic

circuit design, textile engineering and digital signal processing. For each discipline

a short introduction is given in this monograph or at least a reference to an introduc-

tory book.

In Chap. 2 three important preparations are made. Firstly, it presents the current

knowledge on human tactile perception which is used later to motivate the tactile

rendering algorithms presented in this book. Therefore, it is necessary to introduce

psychophysics and neurophysiology first. Secondly, a perceptional model of tactile

simulation is developed that builds a theoretical frame for this work. And thirdly,



102 6 Summary and Outlook

psychophysical methods for the evaluation of the tactile simulation systems are pre-

sented. These are used later to analyse the developed simulation systems.

The tactile displays used within this work are presented in Chap. 3. In order to

use these displays the development of an electronic component was necessary. This

component translates the digital amplitude data from the tactile rendering algorithm

into appropriate electronic signals which are then transformed into mechanical vi-

brations by the display. The design of this electronic component is described in the

chapter. As explained above the choice of the tactile display was inherited from the

HAPTEX project. Nevertheless, the chapter still gives an overview on other existing

displays. Finally, force-feedback systems are introduced as these are also used later.

A tactile rendering system transforms measured surface data into appropriate tac-

tile signals. For a realistic simulation of active touch the system needs to react to the

movement of the user’s finger, i.e. the system needs to compute the amplitudes for the

tactile display in real-time according to the movement of the finger. However, there

is no need to use the measured data directly. The surface data can be prepared pre-

viously for the real-time part. The latter generation of virtual surfaces is the topic of

Chap. 4. Three different methods for the generation of virtual surfaces are described.

One of these uses an optical surface scan as input data whereas both other methods

rely on the KES-F surface measurements. As all these methods more or less make

use of surface properties specific to fabrics an introduction to the production of fab-

rics is also given. Furthermore, the selection and measurement of sample fabrics is

described. A comprehensive list of all sample fabrics can be found in Annex A.

After all these preparations a tactile simulation system can be developed which

is described in Chap. 5. For different tactile rendering algorithms a common frame-

work was created which tracks the position of the tactile display, i.e. the position of

the user’s finger on the virtual fabrics. The tactile rendering algorithm presented in

the chapter is based on vibrations occurring while moving a fingertip over a rough

surface. The system attempts to reproduce these vibrations with a tactile display

such that a similar sensation is created. An evaluation of the system showed that

subjects were able to distinguish relatively similar fabrics when displayed by the

tactile simulator. The tactile simulation system was combined with a force-feedback

system. An evaluation showed that subjects recognised the tensile, roughness and

friction properties of the sample fabrics.

Psychophysical experiments described in [1] suggest that the tactile properties of

surfaces build a three-dimensional perceptual space (cf. Sect. 2.3). The number of

perceptual dimensions is further reduced when the surface is rendered by a tactile

simulation system. The dimensionality of these perceptual spaces stands in contrast

to the dimensionality of the space of all possible virtual surfaces which is much

higher. This provokes the question whether the virtual surface models presented in

Chap. 4 are still too sophisticated. However, it should be noted that the experiments

conducted do not reveal the realism of the simulation. It might be possible that only

a few perceptual dimensions are sufficient to distinguish the properties of a fabric

but many more are needed for a realistic simulation.

How the realism of the simulation could be evaluated is not answered in this work

and could be the subject of further research. The experiments showed that subjects

References 103

were able to rate the roughness of the sample fabrics but this does not mean rough-

ness was realistically rendered. It is possible that the stimuli causing a real sensation

of roughness were substituted by other stimuli which gave enough information on

the roughness but did not feel realistic. However, asking subjects to rate the real-

ism of the simulation would probably rather reveal the subjects’ sympathy for the

experimenter than anything else. So this is also not a solution. In this work the real

situation and its simulation were only compared on the level of sensory impressions

(cf. Sect. 2.3.4). A comparison on the level of neural activity or of physical stimuli

could be more suitable for the evaluation of realism. Measuring the neural activity

of single afferent neurons could probably deliver interesting insight into classes of

stimuli resulting in equivalent sensations. However, the invasive methods needed in

this case raise ethical problems.

The displays used within this work follow an interesting approach. Recreating

the vibrations occuring while moving a fingertip over a rough surface might result

in an appropriate stimulation of the PC sensors, however, it is questionable whether

the same holds for the SA1 sensors. The developed system only creates vibrations

when the fingertip is moving. If the finger stops its movement the vibrations vanish.

This behaviour does not reflect reality as surface features can still be distinguished

in the real case without movement of the fingertip. Representing this static structure

of the surface by vibrations is not a good solution as the vibrations do not create

a static sensation but are always recognised as vibrations. A display recreating the

height profile under the fingertip intended for the stimulation of the SA1 sensors

together with vibrating elements for the stimulation of the PC sensors might create

a more realistic sensation. The development of such a display could be an interesting

research project for the future.

In this monograph a system is described that simulates the tactile properties of

fabrics. Although it is not perfectly realistic the sensations caused are relatively

convincing and subjects were able to distinguish different fabrics and correctly rate

their roughness. However, this is just a small step towards a completely convincing

simulation of direct touch and a lot of research needs still to be done in this area.

References

[1] Hollins, M., Faldowski, R., Rao, S., Young, F.: Perceptual dimensions of tactile surface

texture: A multidimensional scaling analysis. Perception & Psychophysics 54(6), 697–

705 (1993)

[2] Petit, G., Dufresne, A., Levesque, V., Hayward, V., Trudeau, N.: Refreshable tactile

graphics applied to schoolbook illustrations for students with visual impairment. In: Pro-

ceedings of the 10th international ACM SIGACCESS conference on Computers and ac-

cessibility, pp. 89–96. ACM, New York (2008)

[3] Vidal-Verdu, F., Hafez, M.: Graphical tactile displays for visually-impaired people. IEEE

Transactions on Neural Systems and Rehabilitation Engineering 15(1), 119–130 (2007)

[4] Wang, Q., Hayward, V.: Tactile synthesis and perceptual inverse problems seen from the

view point of contact mechanics. ACM Transactions on Applied Perception 5(2), 1–19

(2008)

Appendix A

Fabrics

In this appendix all fabrics that were available for this work are described (cf. [1, 2]).

The samples have been kindly provided by SmartWearLab at the Tampere Univer-

sity of Technology. The abbreviations of the contents are summarised in Table A.1.

An image of the fabric has been included true to scale where available. The selection

of the samples and their measurements is described in Chap. 4.

Table A.1 Abbreviations of the fabrics’ fibre contents.

AF Other fibres CV Viscose PAN Polyacryl

CA Acetat EL Elastane PES Polyester

CLY Lyocell JU Jute PU Polyurethane

CMD Modal LI Linen SE Silk

CO Cotton PA Polyamide WO Wool

106

AF

abrics

# Image Description Content Structure Weight Thickness

% g/m2 mm

1 Denim 100 CO twill 380 1.6

2 Shirt cotton 100 CO combined twill 120 0.61

3 Cord 100 CO velveteen 330 1.76

4 Linen 100 LI plain weave 250 1.09

5 Gabardine 100 WO twill 175 0.55

Continued on next page.

AF

abrics

107

Continued from previous page.


% g/m2 mm

6 Crepe 100 WO plain weave 145 0.93

7 Silk 100 SE plain weave 15 0.1

8 Natural silk

(bourette)

100 SE plain weave 150 0.8

9 Wild silk (dupion) 100 SE plain weave 80 0.44

10 Jute 100 JU plain weave 300 1.44


108

AF

abrics



% g/m2 mm

11 Flannel 80/20 WO/PES twill 290 1.53

12 Denim 62/35/3 PES/CO/EL twill 275 1.13

13 Plaid 35/35/30 PES/AF/WO twill 270 1.14

14 Tweed 66/14/10/10

AF/WO/PES/CMD

combined twill 270 3.9

15 Velvet 92/8 CO/CMD velvet 300 1.88


AF

abrics

109



% g/m2 mm

16 Lurex knit 70/30 PES/PA held stitch knit 215 2.94

17 Crepe-jersey 85/15 PES/EL rib knit 135 0.73

18 Motorcyclist. coated 72/28 PA/PU plain weave 90 0.39

19 Woven easy care 65/35 PES/CO twill 180 0.57

20 Warp knitted velour 90/10 PA/EL warp knit velour 235 1.56


110

AF

abrics



% g/m2 mm

21 Weft knitted plain 98/2 CLY/EL single jersey 172 1.21

22 Taffeta 100 CA plain weave 125 0.33

23 Crepe 100 PES plain weave 85 0.25

24 Satin 100 PES satin 125 0.3

25 Felt 100 PES nonwoven 155 1.25


AF

abrics

111



% g/m2 mm

26 Organza 100 PES plain weave 25 0.16

27 Fleece 100 PES weft knit 250 3.99

28 Woven upholstery 100 PES woven jacquard 600 2.38

29 Woven leisure 100 PES plain weave 90 0.2

30 Tulle 100 PA warp knitted tulle 10 0.3


112

AF

abrics



% g/m2 mm

31 Warp knitted tricot-

satin

100 PA warp knitted tricot-

satin

100 0.4

32 Leather 100 leather 815 1.68

33 Men’s woven suit

fabric (plain)

60/38/3 WO/PES/EL plain weave 195 0.57

34 Men’s woven suit

fabric (herringbone)

100 WO broken twill 232 0.83

35 Men’s woven over-

coat fabric

80/20 WO/PA modified plain weave 324 2.64


AF

abrics

113



% g/m2 mm

36 Men’s woven over-

coat fabric (twill)

59/25/11/5

CO/PAN/WO/PES

twill 460 3.23

37 Woven outdoor

leisurewear fabric

100 PES plain weave 98 0.26

38 Weft knitted jersey

fabric

48/48/4 CO/CMD/EL single jersey 208 1.09

39 Weft knitted terry

fabric

55/45 CV/PES weft knit terry 288 1.69

40 Warp knitted jersey-

based fabric

100 PES warp knit jersey-

based

154 0.51


114

AF

abrics



% g/m2 mm

41 Warp knitted mesh

fabric

100 PES warp knit mesh 128 0.51

42 Brushed warp knitted

fabric

100 PES brushed warp knit 215 0.98

43 warp knit for car

seats

89/11 PES/EL pile warp knit 278 1.22


seats

100 PES flattened pile warp

knit

168 1.1


seats

100 PES raised pile warp knit 147 1.6


AF

abrics

115



% g/m2 mm


seats

100 PA raised pile warp knit 98 1.55


seats lamin

Face fabric: 100 PES (sam-

ple 42) Foam: E-35 PF (35

kg/m3 polyether) Backing:

100 PES (sample 52)

laminated brushed

warp knit

386 4.09


seats lamin

Face fabric: 89 PES/ 11 EL

(sample 43) Foam: E-35 PF

(35 kg/m3 polyether) Back-

ing: 100 PA (sample 53)

laminated pile warp

knit with

438 4.19


seats lamin


ple 44) Foam: E-35 PF (35

kg/m3 polyether). Backing:

100 PES (sample 52)

laminated flattened

pile warp knit

335 4.11


seats lamin


ple 45) Foam: L30PS (30

kg/m3 polyester) Backing:

100 PES

laminated raised pile

warp knit

332 3.54


116

AF

abrics



% g/m2 mm


seats lamin

Face fabric: 100 PA (sam-

ple 46) Foam: E-25 TX (25

kg/m3 polyether)

laminated raised pile

warp knit

170 4.03

52 backing 100 PES warp knit 53 0.42

53 backing 100 PA interlock 121 0.81


seats lamin

Face fabric: 89 PES/ 11 EL

(sample 43. different colour)

Foam: L30PS (30 kg/m3

polyester)

laminated pile warp

knit

287 3.35

References 117

References

[1] Makinen, M., Meinander, H., Luible, C., Magnenat-Thalmann, N.: Influence of physical

parameters on fabric hand. In: Haptex 2005 – Workshop on Haptic and Tactile Perception

of Deformable Objects, Welfenlab, Universitat Hannover, pp. 8–16 (2005)

[2] Varheenmaa, M., Meinander, H.: Mechanical properties as a base for haptic sensing of

virtual fabrics. In: Proceedings of the Autex 2007 Conference, Tampere, Finland (2007)

Index

absolute threshold, 6

afferent neuron, 17

alternative hypothesis, 10

amplitude, 58

artificial surface, 68

autocorrelation, 62

autocorrelation matrix (ACM), 66

Bernoulli experiment, 9

bimorph, 29

binomial distribution, 9

Brownian surface, 70

confidence interval, 10

confidence level, 10

consistency, 98

correlation, 63, 66

correlation–restoration algorithm (CRA), 62

Craven’s distances, 12

data bus, 37

digital–analogue converter (DAC), 39

discrete Fourier transform (DFT), 58

discrimination methods, 8

drive electronics, 33

dual-layer approach, 94

efferent neuron, 17

end effector, 43

evaluation, 96

fabric, 50

fabric hand, 53

fabric samples, 52

false alarm rate, 11

fast Fourier transform (FFT), 58

fibre, 50

firmware, 35

force-feedback device, 42

frequency response, 30

geometrical roughness (SMD), 55

gram-force (gf), 53

HAPTEX project, 91

haptic loop, 43

haptic rendering, 94

Hausdorff dimension, 69

hit rate, 11

hypothesis testing, 10

intraneuron, 17

inverse piezoelectric effect, 29

just noticeable difference (jnd), 6

Kalman filter, 77

Kawabata evaluation system for fabrics

(KES-F), 53

KES-F friction tester, 55

KES-F roughness tester, 54

knitting, 51

low-pass RC filter, 40

maximum likelihood estimator, 9

mechanoreceptive afferent, 18

Pacinian (PC), 19

rapidly adapting (RA), 19

120 Index

slowly adapting type 1 (SA1), 19

slowly adapting type 2 (SA2), 19

mechanoreceptor, 18

model of human perception, 20

multidimensional scaling (MDS), 12

neural activity, 20

neuron, 15

non-woven, 51

null hypothesis, 10

odd-man-out forced-choice procedure, 9

operational amplifier (OPA), 41

phase, 58

physical simulation, 93

piezoelectric bimorph, 29

piezoelectric effect, 29

primitive, 64

psychometric function, 6

psychophysical scale, 7

psychophysics, 5

Q-factor, 31

realism, 98

repeatability, 98

response bias, 8

sensations, 20

shape from shading (SFS), 67

signal-detection theory, 10

spatial summation, 17

spring–damper model, 43

statistical feature matrix (SFM), 66

Stevens’ Power Law, 8

stimuli, 20

symbolic perception, 20

symmetry, 64

tactile colours, 86

tactile display, 25

tactile perception, 18

tactile rendering, 77

temporal summation, 17

tile, 64

tracking device, 77

translation, 66

triangular method, 9

USB controller, 34

variable voltage supply, 39

vibrotactile rendering, 85

virtual workspace, 76

warp, 50

weaving, 50

Weber’s law, 6

Weber-Fechner law, 7

weft, 50

window, 58

Blackman, 58

Hamming, 58

yarn, 50

Cognitive Systems Monographs

Edited by R. Dillmann, Y. Nakamura, S. Schaal and D. Vernon

Vol. 1: Arena, P.; Patanè, L. (Eds.):

Spatial Temporal Patterns for

Action-Oriented Perception

in Roving Robots

425 p. 2009 [978-3-540-88463-7]

Vol. 2: Ivancevic, T.T.; Jovanovic, B.;

Djukic, S.; Djukic, M.; Markovic, S.:

Complex Sports Biodynamics

326 p. 2009 [978-3-540-89970-9]

Vol. 3: Magnani, L.:

Abductive Cognition

534 p. 2009 [978-3-642-03630-9]

Vol. 4: Azad, P.:

Visual Perception for Manipulation

and Imitation in Humanoid Robots

270 p. 2009 [978-3-642-04228-7]

Vol. 5: de de Aguiar, E.:

Animation and Performance Capture

Using Digitized Models

168 p. 2009 [978-3-642-10315-5]

Vol. 6: Ritter, H.; Sagerer, G.;

Dillmann, R.; Buss, M.:

Human Centered Robot Systems

216 p. 2009 [978-3-642-10402-2]

Vol. 7: Levi, P.; Kernbach, S. (Eds.):

Symbiotic Multi-Robot Organisms

467 p. 2010 [978-3-642-11691-9]

Vol. 8: Christensen, H.I.;

Kruijff, G.-J.M.; Wyatt, J.L. (Eds.):

Cognitive Systems

491 p. 2010 [978-3-642-11693-3]

Vol. 9: Hamann, H.:

Space-Time Continuous Models of Swarm

Robotic Systems

147 p. 2010 [978-3-642-13376-3]

Vol. 10: Allerkamp, D.:

Tactile Perception of Textiles in a

Virtual-Reality System

120 p. 2010 [978-3-642-13973-4]

Documents

Tactile Perception of Textiles in a Virtual-Reality System (Cognitive Systems Monographs)