Haptic Interface using Sensory Illusion - Inriapeople.rennes.inria.fr/Anatole.Lecuyer/amemiya.pdf · Haptic Interface using Sensory Illusion ... , 2101-2106, Oct 2007. T. Amemiya,

Haptic Interface using Sensory Illusion

Tomohiro Amemiya

*NTT Communication Science Labs., [email protected]

NTT Communication Science Laboratories 2/39

IEEE VR 2008 Tutorial #2 © 2008 [email protected]

OutlineIntroduction

Haptic Interface using Sensory IllusionsKinesthetic force illusionVestibular illusionTactile illusions

Summary



A. Kitaoka (2006),http://www.ritsumei.ac.jp/~akitaoka/



E. Adelson (2002), http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html



Stimulation

Environment HumanPerception InformationSensation

=

Interface

Different!Measurement

Nonlinearity betweensensation and perception

==

Stimulation Perception InformationSensation

Sensory illusions



Perceived World is not Identical to Physical World.

A series of still photos Motion pictures

Information compression by subtracting color

difference information

Almost sameas original picture

Subtracting frequencies difficult for humans to hear

Physical world Psychological world

Compression algorithm in mp3 format or Mini Disk

Compression algorithm on JPEG

Basis of animation

Illusion

Almost sameas original sound



Human Interface using Sensory IllusionHuman science

To understand the distortion between the physical and psychological world

EngineeringTo utilize the distortion to build information displays

T. Amemiya, I. Kawabuchi, H. Ando, T. Maeda, "Double-Layer Slider-Crank Mechanism to Generate Pulling or Pushing Sensation without an External Ground", In Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2007), 2101-2106, Oct 2007.T. Amemiya, H. Ando, T. Maeda, ""Lead-Me Interface" for a Pulling Sensation from Hand-held Devices", ACM Transactions on Applied Perception 5(4), 2008.

Buru-Navi

[ Laval Virtual “Grand Prix” Award (2007) ]

[ Laval Virtual VR interface Award (2007) ]



???

Physical constraints∥

Both the user and device must be connected toan external ground in order to generate force

Haptics in Mobile Devices??

Relatively heavySpatially constrained



Related Work: Non-grounded Force Displays

Prior non-grounded force displays:Unable to generate constant forceUnable to generate translational force

Gyro Effect(Yano et al., 2003)

Angular Momentum(Tanaka et al., 2001)

Mechanical Brake(Ando et al., 2002)



My Approach Creating a Force Sensation

Periodic translational motion (=oscillation)

Not creating a “physical” force

Exploiting characteristics of human perception

To create a constant and translational force by non-grounded displays



Psychophysical quantity is notidentical to physical quantity.

1. Exploiting Characteristics of Human Perception

If the asymmetric oscillation is designed well, a directed force sensation can be induced.

1.0

0.0

0.2

0.4

0.8

0.6 ∫+

=Tt

td 0)( ττα

∫+

≠Tt

td 0)( τταϕ o

x = α(t): accelerationT: time cycle

y = ϕ(x): sigmoid curve of perception

Psychophysical quantity

Physical quantity



2. Slider-crank Mechanism to Generate Asymmetric Oscillation

The weaker acceleration is not noticed.The washout-effect of the weaker acceleration allows humans to be tricked into perceiving this as uni-directional force.

Stimuli

Perception

Illusion



“Buru-Navi” PrototypeCrank wheel

Swinging arm

Guide roller

Connecting rod

Weight

Motor

Swinging armMotor pinionCrank wheel(with crown gears)



http://www.dailymotion.com/video/x1t8yl_buru-navi

http://labolib3.aecl.ntt.co.jp/member_servlet_home/asx/Large/2006-08.asx

Introduction movie



Asymmetric oscillation(test stimuli)

Evaluation

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-200

-100

0

100

Time [s]

Acc

eler

atio

n [m

/s2 ] 5 cycle/sec

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-200

-100

0

100

Time [s]

Acc

eler

atio

n [m

/s2 ] 5 cycle/secSymmetric oscillation

(control stimuli)

xy

xy



Frequency [cycles/sec]

Symmetric oscillation(comparison)

0102030405060708090

100

3 5 7 9 11 13 15

Proportion of “Being pulled” to x-direction [%]

Asymmetric oscillation

Asymmetric Oscillation Generates a Pushed/Pulled Sensation

x

x



Another Approach to Generate an Asymmetric Oscillation

Spring-cam mechanismResonant motion by linear actuators

Cam Follower

Extension Spring

CamSlider Linear Guide

xy

Phase 1 Phase 2

T. Amemiya, H. Ando, T. Maeda, "Hand-held Force Display with Spring-Cam Mechanism for Generating Asymmetric Acceleration", In Proc. of World Haptics Conference 2007, pp. 572-573, 2007.



Example of Applications (1): Accessibility

Collaborative research withKyoto City Fire DepartmentKyoto Prefectural School for the Visually Impaired

Haptic direction indicatorBuru-Navi+GPS+Electric compass



Example of Applications (2): Entertainment

Force feedback “tray” for novice waiters

Buru-NaviRotation mechanism

Position and posture identification system

Video processing



Tray with Force Display

Pochette

stepper motor a belt with the belt pulley

Buru-Navi

Tray



Singapore Science Center (2008)

T. Amemiya, T. Maeda, H. Ando, "Location-free Haptic Interaction for Large-Area Social Applications", Personal and Ubiquitous Computing, 2008.

Laval, France (2007)

T. Maeda, H. Ando, T. Amemiya, N. Nagaya, M. Sugimoto, M. Inami, "Shaking The World: Galvanic Vestibular Stimulation As A Novel Sensation Interface", In Proc. of ACM SIGGRAPH 2005 Emerging Technologies, 2005.

Human Interface Using GVS



Illusion in Balance Sensation

Motion Platforms

Balance can be also controlled by thermal or galvanic vestibular stimulations.

Vestibular

semicircular canal

Otolith

auditory ossicle

eustachian tubeeardrumexternalauditorymeatus

cochlea

vestibular nerve

cochlear nerve

???Mobile/Wearable



GVS (Galvanic Vestibular Stimulation)

If it is strong enough, it not only throws you off balance but alters the course of your movement.

When a weak DC current is delivered to the mastoid behind your ear, your body responds by shifting your balance toward the anode.



Radio-controlled Humans

T. Maeda, H. Ando, T. Amemiya, N. Nagaya, M. Sugimoto, M. Inami, "Shaking The World: Galvanic Vestibular Stimulation As A Novel Sensation Interface", In Proc. of ACM SIGGRAPH 2005 Emerging Technologies, 2005.

Over 3,000 people have experienced this! (no accidents so far..)



Remote-controlled Riders

H. Ando, T. Amemiya, J. Watanabe, M. Inami, T. Maeda, "The Evaluation of Nail-Mounted Tactile Display", In Proc. of World Haptics Conference 2005, 2005.(poster presentation)

Nail-mounted Tactile Display



Conventional tactile displays

Tactile display

Human

ground

Nail-mounted tactile display

Vibration actuator

Force sensor

B/W sensor

+

+

Voice-coil motor

Finger-color sensor

Reflection sensor

Human

Tactile display

ground???



Vibration during Active Touch



J. Watanabe, S. Hayashi, H. Kajimoto, S. Tachi, S. Nishida, Tactile motion aftereffects produced by appropriate presentation for mechanoreceptors, Experimental Brain Research 180 (3), pp. 577-582, 2007.

Tactile Motion Aftereffect



Adaptation (moving stimulus) Test stimulus

Vision

Illusory motion

Tactile

Motion Aftereffect

Lerner (2002)Hollins (1994), Lerner (2002)

Previous Studies:Clear MAEs like in vision were NOT observed.

???



Adaptation

Interval

Test Stimulus

Judgment

（400-ms stimulus＋600-ms rest）x10

2 sec

One of nine ISOIs

“Upward” or “Downward” ?

Four subjectsThree adaptation conditions (No、Upward、Downward）

ISOI=100 ms

Experimental Procedure

30 Hz 200ms

Time

ISOI

ISOI



△ No Adaptation× Upward Adaptation○ Downward Adaptation

Experimental Result (one subject)



Mechanoreceptors under the skinMeissnerRA（Rapid Adaptation）TransientRespond to vibration 30 Hz

Skin surface

Epi-dermis

Dermis

Endodermis

Merkel SA（Slow Adaptation）SustainedRespond to static distortion

this reportAdapt: RA →test : RA=reproductive MAE

earlier studiesAdapt: RA→test :SA=No MAE

Hollins (1994)Lerner (2002)

Lerner (2002)

What are the differences between earlier studies and our experiments?



SummarySensory illusions can be tools for designing human interfaces.

Kinesthetic illusionsVestibular illusionsTactile illusions

In overcoming certain difficulties, sensory illusions are very effective.

Virtuous circle between human science and engineering is essential for human interfaces using sensory illusions.



Members & AcknowledgementTomohiro Amemiya, Ph.D.

(NTT CS Labs.)Taro Maeda, Ph.D.

(Osaka Univ.)

Junji Watanabe, Ph.D.(PREST, JST)

Hideyuki Ando, Ph.D.(NTT CS Labs.)



ReferenceT. Amemiya, H. Ando, T. Maeda, ""Lead-Me Interface" for a Pulling Sensation from Hand-held Devices", ACM Transactions on Applied Perception, Vol. 5, No. 4, 2008.T. Amemiya, T. Maeda, H. Ando, "Location-free Haptic Interaction for Large-Area Social Applications", Personal and Ubiquitous Computing, Springer, 2008.T. Amemiya, I. Kawabuchi, H. Ando, T. Maeda, "Double-Layer Slider-Crank Mechanism to Generate Pulling or Pushing Sensation without an External Ground", Proc. of IEEE/RSJ International Conference on Intelligent Robotsand Systems (IROS 2007), pp. 2101-2106, 2007.H. Ando, T. Amemiya, J. Watanabe, M. Inami, T. Maeda, "The Evaluation of Nail-Mounted Tactile Display", In Proc. of World Haptics Conference 2005, 2005.Maeda, H. Ando, T. Amemiya, N. Nagaya, M. Sugimoto, M. Inami, "Shaking The World: Galvanic Vestibular Stimulation As A Novel Sensation Interface", Proc. of ACM SIGGRAPH 2005 Emerging Technologies, 2005.J. Watanabe, S. Hayashi, H. Kajimoto, S. Tachi, S. Nishida, Tactile motion aftereffects produced by appropriate presentation for mechanoreceptors, Experimental Brain Research 180 (3), pp. 577-582, 2007.

Documents

Haptic Interface using Sensory Illusion - Inriapeople.rennes.inria.fr/Anatole.Lecuyer/amemiya.pdf · Haptic Interface using Sensory Illusion ... , 2101-2106, Oct 2007. T. Amemiya,