23rd October 2008, Florence, Italy

Natural InteractionHaptic and Training

Emanuele Ruffaldi

Carlo Alberto Avizzano

PERCRO Scuola Superiore Sant’Anna

Pisa, Italy

23rd October 2008, Florence, Italy

The term haptics derives

from the Greek word

haptesthai

that means

“relative to the sense of touch”

23rd October 2008, Florence, Italy

Haptics in Human Machine InteractionClassic Robot

Teleoperated System

Networked Teleoperation

Haptics in Virtual Environment

Haptic CollaborativeVirtual Environment

Augmented Reality with Haptics

23rd October 2008, Florence, Italy

Haptic Interfaces

A haptic interface is a force reflecting device which allows a user to touch, feel, manipulate, create, and/or alter simulated three-

dimensional objects in a virtual environment.

23rd October 2008, Florence, Italy

Haptic Interfaces

A haptic interface is a force reflecting device which allows a user to touch, feel, manipulate, create, and/or alter simulated three-

dimensional objects in a virtual environment.

Kinesthetic Vibrotactile

Sensable, inc.

PERCRO

23rd October 2008, Florence, Italy

Challenge: Haptic Interfaces are not so Natural…

23rd October 2008, Florence, Italy

Research in Haptics• Haptic Perception

– Study of the perception of Haptic interaction• Haptic Devices

– Design and Construction• Computer Haptics

– Haptic Rendering computes forces based on the Virtual Environment

Robotic Haptic Human

modeling

perception

23rd October 2008, Florence, Italy

Haptics and Natural Interaction?

• Major Problem– Haptic feedback

requires to generate forces

– Forces require power– Power requires

actuators• Best use of Haptic

interfaces– Tool Mediated– Exoskeleton

• What about Natural?

HAND-Exoskeleton 3+3 DOF Hand Ex

Arm-Exoskeleton Haptic PenPERCRO PERCRO

PERCROPERCRO

23rd October 2008, Florence, Italy

Encounter InterfacesAn Encounter Interface is an interface that allows to provide haptic feedback only when the hand is in direct contact with the virtual object

Video Yokokohji98

WYSIWYF – What You See is What You Feel

Yokokohji 2000, Virtual Control Panel

23rd October 2008, Florence, Italy

Encounter Interface and Large Workspace

PERCRO

• Haptic Feedback: modified 3DOF GRAB Device• Tracking: VICON Motion Capture

0 . 2 5 0 . 3 0 . 3 5 0 . 4 0 . 4 5 00 . 1

0 . 20 . 3

- 0 . 2

- 0 . 1

0

0 . 1

0 . 2

0 . 3

Y ( m )

X ( m )

Z (

m)

h a p t i c d e v i c e

u s e r h a n d

It allows the Hands Free interaction for the PureForm Project

23rd October 2008, Florence, Italy

Sample interaction

•Track user hand with external systems•Make haptic and vision space coherent (calibration)•Model object physics and Evaluate distance from hand to objects•Determine closest surface and contact orientation•Control HI End Effector to display surface location•Control HI to slide below fingers during motion contacts

23rd October 2008, Florence, Italy

How can we use such Interfaces?

23rd October 2008, Florence, Italy

SKILLS Project

The SKILLS Integrated Project deals with the acquisition, storing and transfer of human skill by means of multimodal interfaces, Robotics and Virtual Environments technologies.

http://www.skills-ip.eu• 15 partners• 2006-2010• Coordinated by PERCRO-SSSA

23rd October 2008, Florence, Italy

• Sport and Entertainment– Juggling– Rowing

• Surgery and Rehabilitation– Maxillo Facial Surgery– Minimally Invasive Surgery– Upper Limb Rehabilitation

• Industry and Maintenance– Training of Maintenance and Operations– Programming-by-Demonstration

3 Demonstration Areas

23rd October 2008, Florence, Italy

Juggling

• What are the relative influences of haptic and visual feedback on learning to juggle?

• Can “slow motion” juggling in the virtual world improve juggling in the real world?

• Can timing cues for juggling patterns improve the time to learn the pattern? In what modality should timing cues be presented?

• Is an accurate toss more important than an accurate catch?

CEA-LIST

23rd October 2008, Florence, Italy

23rd October 2008, Florence, Italy

Workspace considerations:the ball throwing experiment

23rd October 2008, Florence, Italy

Multiple Balls

Real and Virtual Trajectories

Two Robotic Arms, Two Virtual Balls, Colocated Stereo

23rd October 2008, Florence, Italy

Vibrotactile

PERCRO Vibrotactile Glove

Tactile Jacket, TNO

PERCRO Vibrotactile Rowing

23rd October 2008, Florence, Italy

Encounter Interface with Shape Perception

Precise but Not yet Hands Free

PERCRO 2006-2008

23rd October 2008, Florence, Italy

Test conducted with TSD approach• The experiment consists in series of 100 trials. In each series two sphere Sa and Sb

with different radius were randomly presented. • So on each trial the participant have the same a priori probability of 0.5 to explore a

pair of sphere with the same radius (Sa,Sa or Sb,Sb) or with different radius (Sa,Sb or Sb,Sa).

• The observer’s task was to judge on each trial if the curvature of the two surfaces was ”different” or the ”same”.

• Three series were presented to each participant, changing the value of the difference curvature = Sb − Sa between the two possible curvatures. The values are defined in the table:

23rd October 2008, Florence, Italy

Future

• Intelligent Interfaces– Exploit Computing Power– Exploit Machine Learning

• Portable– Within Limits of Actuation

23rd October 2008, Florence, Italy

Next Event – ENACTIVE08

http://www.enactive2008.org http://www.skills-ip.eu