Upload
georgiana-chandler
View
215
Download
0
Embed Size (px)
Citation preview
1/69
Manuel FerreUniv. Politécnica de Madrid March 2006
Human Interfaces for Human Interfaces for TeleoperationTeleoperation
2/69
OutlineOutline
Haptic interfacesHaptic interfaces
Guidance of telemanipulatorsGuidance of telemanipulators
Stereoscopic visual interfacesStereoscopic visual interfaces
3/69
General control schemeGeneral control scheme
Perception-reasoning-actuationPerception-reasoning-actuation
4/69
Haptic interfacesHaptic interfaces
5/69
Haptics : tactile and kinaesthesia Haptics : tactile and kinaesthesia
Touch is the first information lost in remote Touch is the first information lost in remote telemanipulationtelemanipulation
There are two components involved in manipulation, they There are two components involved in manipulation, they are defined as somatic senses because they are in the are defined as somatic senses because they are in the body:body:
TactileTactile: information is received by the skin and other : information is received by the skin and other closer tissues. Temperature, vibrations and pressure closer tissues. Temperature, vibrations and pressure are detected by these kind of receptors.are detected by these kind of receptors.
Kinaesthesia: Kinaesthesia: information is provided by internal information is provided by internal organs of the body such us muscles and joints. organs of the body such us muscles and joints. Movement and weight of our body is informed by Movement and weight of our body is informed by these organs.these organs.
Haptic perception is the combination of both impressions Haptic perception is the combination of both impressions
6/69
Haptics : tactile and kinaesthesiaHaptics : tactile and kinaesthesia
Main features of haptics are:Main features of haptics are:
TactileTactile: : Detects temperature and Detects temperature and
patterns of pressurepatterns of pressure
High frequency stimulusHigh frequency stimulus
Distributed on all skinDistributed on all skin
KinaesthesiaKinaesthesia:: Detects body positions and Detects body positions and
movementsmovements
Low frequency stimulusLow frequency stimulus
Related to muscular effort Related to muscular effort
7/69
Haptics : physiology of touchHaptics : physiology of touch
Main cutaneous receptors:Main cutaneous receptors:
8/69
Haptics : physiology of touchHaptics : physiology of touch
Minimum value able to Minimum value able to perceive:perceive:
• 80 mg in the finger tip80 mg in the finger tip• 150 mg in the palm150 mg in the palm
Higher sensitivity at 32º C, from Higher sensitivity at 32º C, from 40º C stimulus is degraded40º C stimulus is degraded
Higher surface in the contact Higher surface in the contact implies higher sensitivity when implies higher sensitivity when pressure is changedpressure is changed
9/69
Haptics : stimulus resolutionHaptics : stimulus resolution
Resolution of touch:Resolution of touch:
Spatial resolution varies between 1-2 Spatial resolution varies between 1-2 mmmm22 to 45 cmto 45 cm22 according to the location of the according to the location of the stimulus:stimulus:
• FingerFinger: minimum distance to distinguish two : minimum distance to distinguish two stimuli is 2.5 mmstimuli is 2.5 mm
• PalmPalm: minimum distance is about 10 mm: minimum distance is about 10 mm
• ThighThigh: distance of less than 67 mm is difficult to : distance of less than 67 mm is difficult to distinguishdistinguish
Temporal resolution: its minimum value is Temporal resolution: its minimum value is around 5 ms.around 5 ms.
10/69
Haptics : stimulus resolutionHaptics : stimulus resolution
Sensitivity of haptic stimuli is Sensitivity of haptic stimuli is measured by the measured by the JND (Just JND (Just Noticeable Difference)Noticeable Difference)
11/69
Haptics : graspingHaptics : graspingGraspingGrasping
Forces depends on: geometry, Forces depends on: geometry, constitution, skill.constitution, skill.
Two different kind of grasping:Two different kind of grasping:• Power:Power:
–High forces and stable configurationHigh forces and stable configuration–Palm and fingers are usedPalm and fingers are used–Maximum applied force is around 400 N Maximum applied force is around 400 N
(men) and 228 N (women)(men) and 228 N (women)• Precision:Precision:
–Low force is exertedLow force is exerted–Finger are skilfully usedFinger are skilfully used
12/69
Haptics : graspingHaptics : grasping
Classification of graspingClassification of grasping
13/69
Haptics : bandwidth Haptics : bandwidth
Bandwidth of haptic stimuli:Bandwidth of haptic stimuli:Each receptor has a specific Each receptor has a specific
range:range:• Vibration: 5-10 kHz (texture)Vibration: 5-10 kHz (texture)
• Tactile: 0-400 HzTactile: 0-400 Hz
• Kinaesthesia: 20-30 HzKinaesthesia: 20-30 Hz
The human answer to the haptic The human answer to the haptic stimuli (our actions) is around stimuli (our actions) is around 5 – 10 Hz5 – 10 Hz
14/69
Haptics : bandwidthHaptics : bandwidth
Finger bandwidthFinger bandwidth
15/69
Haptics : bandwidthHaptics : bandwidth
Finger bandwidthFinger bandwidth
16/69
Haptics : requirementsHaptics : requirements
Main requirements for haptics interfacesMain requirements for haptics interfaces
17/69
Haptics : devicesHaptics : devices
Background:Background:
1940s1940s: Tongs for manipulating hazardous : Tongs for manipulating hazardous materialsmaterials
18/69
Haptics : devicesHaptics : devices
GRIPS telemanipulator by Kraft Telerobotics:GRIPS telemanipulator by Kraft Telerobotics:Electro-HidraulycElectro-HidraulycBilateral master-slaveBilateral master-slave6 DOF + grip6 DOF + grip
19/69
Haptics : devicesHaptics : devices
TITAN by Schilling:TITAN by Schilling:
Electro-HidraulycElectro-Hidraulyc
Bilateral master-slaveBilateral master-slave
Force sensor in the wristForce sensor in the wrist
6 DOF + grip6 DOF + grip
20/69
Haptics : devicesHaptics : devices
Phantom by Phantom by SensAble Tech.:SensAble Tech.:
Force reflected in Force reflected in 3 dof3 dof
Serial Serial configuration configuration
Large bandwidth Large bandwidth (pulley (pulley transmission)transmission)
21/69
Haptics : devicesHaptics : devices
PUMA robot and PUMA robot and master interface by master interface by Oxford UniversityOxford University
Master is a 6 dof Master is a 6 dof parallel device parallel device (Steward platform)(Steward platform)
Large bandwidthLarge bandwidth
Indexed workspaceIndexed workspace
22/69
Haptics : devicesHaptics : devices
Magister parallel haptic interfaced Magister parallel haptic interfaced developed at UMH. developed at UMH.
23/69
Haptics : devicesHaptics : devices
Joysticks and similar devices:Joysticks and similar devices:
Rate controlRate control
Low workspaceLow workspace
Focused on specific developmentsFocused on specific developments
24/69
Haptics : devicesHaptics : devices
Hands developed by Hands developed by K. Salisbury at the K. Salisbury at the MIT:MIT:
16 gdl16 gdl
Simplification of Simplification of human handhuman hand
Exos Dextrous Hand Exos Dextrous Hand Master:Master:
16 gdl16 gdl
25/69
Haptics : devicesHaptics : devices
MIT hand and Exos Dextrous Hand MIT hand and Exos Dextrous Hand Master integratedMaster integrated
26/69
Haptics : devicesHaptics : devices
Artificial hand developed by CSIC-IAI:Artificial hand developed by CSIC-IAI:
3 fingers actuated and 2 fixed3 fingers actuated and 2 fixed
27/69
Haptics : devicesHaptics : devices
Exoesqueleton of Utah UniversityExoesqueleton of Utah University
28/69
Haptics : devicesHaptics : devices
Space-Mouse developed by G. Hirzinger:Space-Mouse developed by G. Hirzinger:
6 gdl6 gdl
Forces and torques are measuredForces and torques are measured
29/69
Haptics : devicesHaptics : devices
Gloves:Gloves:
Hand posture is identifiedHand posture is identified
Resolution is poorResolution is poor
30/69
GuidanceGuidance
31/69
GuidanceGuidance
Main topics:Main topics:
Time executionTime execution
TeleproprioceptionTeleproprioception
Working areaWorking area
32/69
Guidance Guidance
Operator’s movements are transformed to Operator’s movements are transformed to movement commands to remote robotsmovement commands to remote robots
33/69
GuidanceGuidance : time execution: time execution
Time execution:Time execution:
This is an experimental parameter that shows This is an experimental parameter that shows how easy is executed a task of guidancehow easy is executed a task of guidance
It permits to compare different devices and It permits to compare different devices and technologiestechnologies
Productivity is connected to time executionProductivity is connected to time execution
Learning is strongly Learning is strongly connected connected to time execution to time execution in the firsts in the firsts trialstrials
0
10
20
30
40
50
60
0 10 20 30 40 50
Task nº
Tim
e (m
in)
34/69
GuidanceGuidance : teleproprioception: teleproprioception
Propioception: Propioception: It is the ability of a human being to know the It is the ability of a human being to know the
status of his membersstatus of his membersThis information is obtained by the body This information is obtained by the body
joints, ear and other nerve impulsesjoints, ear and other nerve impulses
Telepropioception:Telepropioception:It is the ability of a human operator to know It is the ability of a human operator to know
the consequences of his actions with respect the consequences of his actions with respect to the remote environment.to the remote environment.
It implies that remote and local reference It implies that remote and local reference systems must be coherentsystems must be coherent
35/69
GuidanceGuidance : teleproprioception: teleproprioception
Disruption between stimuli and actions:Disruption between stimuli and actions:
Mechanical disruption Mechanical disruption
Geometrical disruption Geometrical disruption
C om m anded m otion
Expected m otion
O bserved m otion
Actualm otion
Com m anded m otion
Expected m otion
O bserved m otion
Actualm otion
36/69
Guidance : indexationGuidance : indexation
Working areaWorking area
It happens when slave workspace is It happens when slave workspace is bigger than master. Examples:bigger than master. Examples:
Big slave and small masterBig slave and small master
Different kinematicsDifferent kinematics
SolutionSolution: indexation (telepropioception is lost): indexation (telepropioception is lost)
37/69
Conclusions about haptics and guidance Conclusions about haptics and guidance
Bilateral devices must guarantee Bilateral devices must guarantee operator’s teleproprioceptionoperator’s teleproprioception
Kinaesthetic stimulus is predominant in Kinaesthetic stimulus is predominant in force reflectionforce reflection
Tactile stimulus is increased according Tactile stimulus is increased according to the bandwidth of the haptic interfaceto the bandwidth of the haptic interface
Low bandwidth is useful for detect Low bandwidth is useful for detect collision. Is it sufficient? …. It depends collision. Is it sufficient? …. It depends on applicationon application
38/69
Visual interfacesVisual interfaces
39/69
Visual interfaces: outlineVisual interfaces: outline
Description of visionDescription of vision
Spatial perceptionSpatial perception
DevicesDevices
40/69
Visual interfaces: visionVisual interfaces: vision
Vision: is the sense that detects light. ItVision: is the sense that detects light. It informs informs about shape, size, movements, distances and colour.about shape, size, movements, distances and colour.
This stimulus is caused by the light received throughThis stimulus is caused by the light received through the retina photoreceptors.the retina photoreceptors.
Crystalline is a lens that changes its shape to focus Crystalline is a lens that changes its shape to focus light on near or far objects.light on near or far objects.
41/69
Visual interfaces: visionVisual interfaces: visionThere are two types of light sensors in the There are two types of light sensors in the
retina:retina:
Rods: work best in dim light and allow black Rods: work best in dim light and allow black and white vision and white vision
Cones: work best in bright light and detect Cones: work best in bright light and detect color. In the human eye, cones respond to red, color. In the human eye, cones respond to red, green and blue wavelengths of light green and blue wavelengths of light (29-70).(29-70).
42/69
Visual interfaces: visionVisual interfaces: vision
Eye resolution is due to Eye resolution is due to light sensors size. light sensors size. Maximum resolution is between Maximum resolution is between 30 sec. arc. and 1 30 sec. arc. and 1 min. It depends on light conditions.min. It depends on light conditions.
Resolution decreases from fovea to periphery. Resolution decreases from fovea to periphery. Sensor resolution is a limit for pixel size Sensor resolution is a limit for pixel size
43/69
Visual interfaces: visionVisual interfaces: vision
The image frequency perceived depends on The image frequency perceived depends on illumination conditions. This frequency is close illumination conditions. This frequency is close to 20-25 images per second. Lower frequencies to 20-25 images per second. Lower frequencies produces flicking.produces flicking.
The visible spectrum is roughly from 700 The visible spectrum is roughly from 700 nm (red end) to 400 nm (violet end).nm (red end) to 400 nm (violet end).
44/69
Visual interfaces: depth visionVisual interfaces: depth vision
Field of view is the part of the observable Field of view is the part of the observable world that is seen. Humans have a world that is seen. Humans have a 180º180ºxx120º FoV.120º FoV.
Binocular vision only covers 120º degrees Binocular vision only covers 120º degrees of the vision field in humans. The of the vision field in humans. The remaining area is the peripheral vision remaining area is the peripheral vision
45/69
Visual interfaces: depth visionVisual interfaces: depth vision
Depth perception combines several types Depth perception combines several types of depth clues grouped into two categoriesof depth clues grouped into two categories
Monocular clues:Monocular clues: Motion parallaxMotion parallax
Image realism (Color vision, perspective, relative Image realism (Color vision, perspective, relative size, occlusion, texture gradient)size, occlusion, texture gradient)
Binocular clues:Binocular clues: Stereopsis Stereopsis
ConvergenceConvergence
46/69
Visual interfaces: depth perceptionVisual interfaces: depth perception
Motion parallax : Motion parallax :
The apparent relative motion of The apparent relative motion of several stationary objects against a several stationary objects against a background when the observer is background when the observer is moving. moving.
This effect can be seen clearly This effect can be seen clearly when driving a car, near things pass when driving a car, near things pass quicker, while far off objects appear quicker, while far off objects appear stationary. stationary.
This mechanism is predominant This mechanism is predominant for far objects (>1m)for far objects (>1m)
47/69
Visual interfaces: depth perceptionVisual interfaces: depth perceptionImage realism is a monocular clue that provides spatial Image realism is a monocular clue that provides spatial
information according to our experience (pictures, movies)information according to our experience (pictures, movies)
Color visionColor vision: correct interpretation of color, and especially : correct interpretation of color, and especially lighting clues, allow the beholder to determine the object lighting clues, allow the beholder to determine the object shape.shape.
PerspectivePerspective: the feature of parallel lines converging at infinity : the feature of parallel lines converging at infinity allows us to reconstruct relative distances.allows us to reconstruct relative distances.
Relative sizeRelative size: close objects to us seem larger than further : close objects to us seem larger than further ones; our visual system exploits the relative size of known ones; our visual system exploits the relative size of known objects to judge distancesobjects to judge distances
OcclusionOcclusion: close objects blocking the sight of far objects, this : close objects blocking the sight of far objects, this clue informs us about relative distances.clue informs us about relative distances.
48/69
Visual interfaces: depth perceptionVisual interfaces: depth perception
Several levels of realism:Several levels of realism:
49/69
Visual interfaces: depth perceptionVisual interfaces: depth perception
Binocular disparity:Binocular disparity:
By using two images of the same By using two images of the same object taken from slightly different object taken from slightly different points of view, it is possible to know points of view, it is possible to know the distance to an object.the distance to an object.
Binocular disparity resolution:Binocular disparity resolution: Resolution 1 min.Resolution 1 min.
Eye distance 6,5 cm.Eye distance 6,5 cm.
46 cm from object to eye implies a 46 cm from object to eye implies a resolution of 0,5 mmresolution of 0,5 mm
It is predominant in close distance It is predominant in close distance (<1m).(<1m). 6.5 cm6.5 cm
46 cm46 cm
0.5 cm0.5 cm
50/69
Visual interfaces: depth perceptionVisual interfaces: depth perception
Convergence:Convergence: it represents the angle between eyes it represents the angle between eyes when they are fixed to an object. Convergence when they are fixed to an object. Convergence angle is larger when objects are closer.angle is larger when objects are closer.
StereopsisStereopsis: : is the ability to blend images of is the ability to blend images of both eyes together, it allows us to both eyes together, it allows us to appreciate depth and judge distances.appreciate depth and judge distances.
51/69
Visual interfaces: depth perceptionVisual interfaces: depth perception
Stereopsis depends on the slight Stereopsis depends on the slight differences (disparity) in the two pictures differences (disparity) in the two pictures projected on the retinas. The brain can fuse projected on the retinas. The brain can fuse both images when disparity is into the both images when disparity is into the proper limits.proper limits.
52/69
Visual interfaces: depth perceptionVisual interfaces: depth perception
Binocular disparity is the horizontal Binocular disparity is the horizontal distance between related points in both distance between related points in both images. It depends on:images. It depends on:
Distance between object to camera, andDistance between object to camera, and
Convergence angle of camerasConvergence angle of cameras
53/69
Visual interfaces: depth perceptionVisual interfaces: depth perception
Un-crossed disparity Zero disparity Crossed disparity
Three possible Three possible values of values of disparitydisparity
54/69
Visual interfaces: depth perceptionVisual interfaces: depth perception
0 20 40 60 80 100 120 140 160 180 2000
5
10
15
20
25
30
35
40
45
50
ConvergenceConvergence 4040 6060 8080
Range (cm.)Range (cm.) 23-7023-70 40-11040-110 50-15050-150
x
dxoabsdisparidad
55/69
Visual interfaces: depth perceptionVisual interfaces: depth perception
Brain fusion is limited by the maximum Brain fusion is limited by the maximum value of disparityvalue of disparity
It is a limit to use stereoscopic images in It is a limit to use stereoscopic images in telemanipulationtelemanipulation
0
1
2
3
4
5
6
7
8
9
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160 165 170 175 180
DISTANCE BETWEEN CAMERA AND OBJECT
IMAGEDISPARITY
NO FUSED IMAGESFUSEDIMAGESS
56/69
Visual interfaces: experimentsVisual interfaces: experiments
Experiment about limits of disparityExperiment about limits of disparity
Setup:Setup:
Steroscopic images are recordedSteroscopic images are recorded3 different convergences and 10 distances 3 different convergences and 10 distances
object-cameraobject-camera
20 participants20 participants
0
1
2
3
4
5
6
7
8
9
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160 165 170 175 180
DISTANCE BETWEEN CAMERA AND OBJECT
IMAGEDISPARITY
NO FUSED IMAGESFUSEDIMAGESS
57/69
Visual interfaces: experimentsVisual interfaces: experiments
Maximum disparity around to:Maximum disparity around to: 2º 2º
0
2
4
6
8
10
12
Distancia de la cámara al objeto (cm)
Dis
pari
dad
an
gu
lar
y V
alo
ració
n
este
reo
scó
pic
a
Disparidad Angular Valoración estereoscópica
58/69
Visual interfaces: experimentsVisual interfaces: experiments
Experiment to compare stereo vs. Experiment to compare stereo vs. Monoscopic images for teleroboticsMonoscopic images for telerobotics
Experiment to compare stereo vs. Experiment to compare stereo vs. Monoscopic images for teleroboticsMonoscopic images for telerobotics
59/69
Visual interfaces: experimentsVisual interfaces: experiments
Comparison between mono and stereo Comparison between mono and stereo imagesimages
0
50
100
150
200
250
300
350
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
Operators
Exe
cuti
on T
ime
(s)
Estereo
Mono
60/69
Visual interfaces: devicesVisual interfaces: devices
61/69
Visual interfaces: devicesVisual interfaces: devices
Stereoscopic and y Stereoscopic and y autoestereoscopicautoestereoscopic
Stereoscopic Stereoscopic : they use : they use additional material like additional material like glasses or a helmet to glasses or a helmet to show a different image to show a different image to each eye.each eye.
Autostereoscopic: they Autostereoscopic: they show a different image to show a different image to each eye without each eye without additional devices.additional devices.
62/69
Visual interfaces: devicesVisual interfaces: devices
Binocular devicesBinocular devices
ParallelParallel StereoscopeStereoscope
Head Mounted Head Mounted Displays (HMD)Displays (HMD)
Polarization coding Polarization coding glasses (IMAX)glasses (IMAX)
Color coding glassesColor coding glasses
SequentialSequentialActive shutterActive shutter
63/69
Visual interfaces: devicesVisual interfaces: devicesAutostereoscopic:Autostereoscopic:
Lenticular sheets. Lenticular sheets. This device is based on an array of This device is based on an array of cylindrical lenses that generates a threedimensional image by cylindrical lenses that generates a threedimensional image by showing different vertical sections to each eye. showing different vertical sections to each eye.
The observer’s head must be placed at a predefined position The observer’s head must be placed at a predefined position in order to see the 3D image properly, otherwise the cross in order to see the 3D image properly, otherwise the cross sections will be overlaid as the head moves.sections will be overlaid as the head moves.
64/69
Visual interfaces: devicesVisual interfaces: devices
Parallax barrier.Parallax barrier. This is a vertical slit plate placed in This is a vertical slit plate placed in front of a display in order to block part of the screen front of a display in order to block part of the screen shown to each eye. The screen displays two images shown to each eye. The screen displays two images divided into vertical strips. Each eye can only see the divided into vertical strips. Each eye can only see the corresponding strips due to the slit plate performance.corresponding strips due to the slit plate performance.
The observer’s eyes must be placed at the proper The observer’s eyes must be placed at the proper position to avoid the blending of the strips.position to avoid the blending of the strips.
65/69
Visual interfaces: devicesVisual interfaces: devices
Holograms and autostereogramsHolograms and autostereograms
66/69
Visual interfaces: devicesVisual interfaces: devices
Immersive system:Immersive system:
Curve-screen theatreCurve-screen theatre
Flat-screen walls.Flat-screen walls.
67/69
Visual interfaces: examplesVisual interfaces: examples
3D interface of ROTEX3D interface of ROTEX
68/69
Visual interfaces: examplesVisual interfaces: examples
TelesurgeryTelesurgery
NASANASA DeminingDemining
69/69
Visual interfaces: conclusionsVisual interfaces: conclusions
Stereo vision is necessary for proper Stereo vision is necessary for proper guidance in order to avoid uncertainty guidance in order to avoid uncertainty
Remote workspace is limited by maximum Remote workspace is limited by maximum disparity (2º). Active convergence is a disparity (2º). Active convergence is a solutionsolution
Excellent quality of images are required Excellent quality of images are required (monocular clues). It implies specific (monocular clues). It implies specific hardwarehardware
Many solutions are in the market, none of Many solutions are in the market, none of them can reproduce human vision them can reproduce human vision perceptionperception