Haptic Interaction in Mixed Environments and Virtual Reality

  • Published on
    28-Jan-2015

  • View
    105

  • Download
    0

Embed Size (px)

DESCRIPTION

Event: "IEEE Day 2012" IEEE Student Branch-University of Patras 09.10.2012 "Haptic Interaction in Mixed Environments and Virtual Reality" by K.Moustakas Museum of Sciences & Technology of the University of Patras More info: http://ieee-upatras.gr/en/events/2012/123-ieee-day-2012

Transcript

  • 1. Haptic Interaction in Virtual and Mixed Reality EnvironmentsKonstantinos Moustakas, Assistant ProfessorElectrical and Computer Engineering Department, University of Patras

2. Outline Haptic interactionOutline What is haptic interaction? Haptic psychophysics outline Haptic interaction essential processing: processing: Collision detection Haptic rendering Applications Future research directions 3. IntroductionHaptic interaction Haptics Wikipedia, Haptics: Haptics refers to the sense oftouch (from Greek = "I fasten onto, I touch). Merriam-Webster, Haptic: relating to or based onthe sense of touch Haptics is the science of applying touch(tactile) sensation and control to interactionwith computer applications. 4. Introduction Haptic interaction Haptics Tactile information The responses of the receptors of the skin in contact with an object Kinestheticinformation Net forces applied to limbs 5. Introduction Haptic interaction Haptic interaction - Importance The sense of touch is underestimated! Loss of the sense of touch can cause impairments thatcannot be compensated by sight Hand dexterity Haptic capabilities, grasping, etc. Walking Limb position perception Eating Speaking In virtual environments the loss of the sense of touch canimpair performance and interaction efficiency 6. IntroductionHaptic interactionHaptic interaction - Importance Strike a match (with vision) Normal case Anesthesia 7.5 sec29 sec 7. Haptic PsychophysicsHaptic interaction Different types of tactile sensors 8. Haptic Psychophysics Haptic interaction Surface encoding deformation matters 9. Haptic PsychophysicsHaptic interaction Spatial resolution 10. Haptic Psychophysics Haptic interaction Temporal resolution ~1 kHz constraint while in graphics ~25 Hz constraint 11. Haptic PsychophysicsHaptic interaction What do we sense? Tactile Haptic Surface texture Mass/Weight Orientation Force Shape Stiffness Size Viscosity Pressure Thermal Curvature Cooling Motion Warming 12. Haptic Rendering Haptic interaction Haptics & Environment 13. Haptic DevicesHaptic interactionHaptic devices Haptic devices: Tactile devices Force feedback devices Exoskeletons One point of interaction devices 14. Haptic DevicesHaptic interaction Haptic devices 15. Haptic DevicesHaptic interaction Haptic devices 16. Haptic DevicesHaptic interaction Haptic devices 17. Haptic DevicesHaptic interaction Haptic devices 18. Haptic DevicesHaptic interaction Haptic devices 19. Haptic DevicesHaptic interaction Haptic devices 20. Haptic DevicesHaptic interaction Haptic devices 21. Haptic DevicesHaptic interaction Haptic devices 22. Haptic DevicesHaptic interaction Haptic devices 23. Haptic DevicesHaptic interaction Haptic devices VS Human Sensors Human sensorsHaptic devices Many sensors One sensor (x N) Direct tactile sensation Only through motion DeformationN/A Haptic devices act as a filter to the human tactile/haptic sensors. How would you see a visual scene with such a filter? 24. Haptic Devices Haptic interaction Haptic devices VS Human Sensors 25. Haptic RenderingHaptic interaction Haptic vs Graphics rendering processesApplicationGeometry Rasterizer DisplayScene View LightingProjection Texturing Traversal TransformTraversalForce Tactile Display CollisionForceForceForceHaptic Detection Calculation Smoothing MappingTexturing 26. Collision detection Haptic interactionCollision detection Collision is one of natures powerful tools: Object interaction Walking Machines are based on collisions Sound source In the real world physical laws and mattertake-over In the virtual world??? 27. Collision detection Haptic interactionCollision detection In no case a trivial issue!!! 28. Collision detectionHaptic interactionBounding volume hierarchies 29. Collision detection Haptic interaction Bounding volume hierarchies (video) 30. Collision detectionHaptic interaction Force feedbackA. Vogiannou, K. Moustakas, D. Tzovaras and M.G. Strintzis, Enhancing Bounding Volumes using SupportPlane Mappings for Collision Detection, Eurographics Computer Graphics Forum, vol. 29, no. 5, pp. 1595-1604, August 2010. 31. Haptic RenderingHaptic interaction Haptic rendering basics 32. Haptic RenderingHaptic interaction Haptic rendering: Stiffness vs StabilityVirtual wall Moving into the wall Force Calculation FVirtual wall V 1 F ( x, y , z ) = a1 a2 a3 43. Haptic renderingHaptic interaction Collision detection: Layer 1 2 2 2 12 x 2 + y 2 + z 1 < 1 F ( x, y , z ) = a1 a2 a3 44. Haptic rendering Haptic interaction Collision detection: Layer 2 45. Haptic rendering Haptic interaction Collision detection: Layer 2 PDSQdIf d>DSQ collision is detected 46. Haptic renderingHaptic interaction Collision detection: Layer 3 Similar processing to layer 2 with two add-ons Possibility to perform distance map subdivision so as to increase accuracy The Layer 3 distance map is used that projects also mesh vertices onto the superquadric surfaces 47. Haptic renderingHaptic interaction Accuracy analysis Lemma: The distance map and the superquadric can provide an exact representation of the 3D model iff function fC that maps all points of the objects surface onto the superquadric is injective. Proof in the text 48. Haptic renderingHaptic interaction Accuracy analysis Concave region of type Q2 Concave region of type Q1 49. Haptic renderingHaptic interaction Haptic rendering Force magnitude estimation 50. Haptic renderingHaptic interaction Haptic renderingForce direction estimation Can be analytically calculated 51. Haptic renderingHaptic interaction Haptic rendering Advantages: Rapid estimation of the reaction force Possibility to analytically process and handle theforce feedback Friction: Haptic texture: Gaussian noise 52. Haptic rendering Haptic interaction Comparisons 53. Haptic renderingHaptic interaction Qualitative results OBBSQ-Map, Layer 1 SQ-Map, Layer 2SQ-Map, Layer 3 54. Haptic rendering Haptic interaction Qualitative results 55. Applications Haptic interactionApplication examples Surgical simulation Telemanipulation,robot-assistedsurgery Computer aideddesign Entertainment Haptic Visualization Applications for thedisabled 56. ApplicationsHaptic interactionHaptic mapsHaptic map with embedded semantic iformation K. Moustakas, G. Nikolakis, K. Kostopoulos, D. Tzovaras and M.G. Strintzis, Haptic Rendering of Visual Data for the Visually Impaired, IEEE Multimedia, vol. 14, no. 1, pp. 62-72, January 2007. 57. ApplicationsHaptic interaction Haptic browser 58. ApplicationsHaptic interactionVirtual reality cane simulationD. Tzovaras et.al., Design and implementation of haptic virtual environments for thetraining of the visually impaired, IEEE Neural Systems and Rehab. Eng., 2004 59. Applications Haptic interaction Mixed reality cane simulation Magnetic Virtual environment sensor 1CyberGrasp VirtualcaneCane replica Magnetic sensor 2 D. Tzovaras, K. Moustakas, G. Nikolakis and M.G. Strintzis, "Interactive Mixed Reality White Cane Simulation for the Training of the Blind and the Visually Impaired", Springer Journal on Personal and Ubiquitous Computing, vol.13, no.1, pp.51-58, January 2009. 60. Applications Haptic interaction VR game for the hearing and the visually impairedK. Moustakas, L. Dybkjaer, O. Aran, D. Tzovaras and N.O. Bernsen, "Communication Between Blind and HearingImpaired People Through a Multimodal Interactive Game", IEEE Multimedia, accepted for publication. 61. ApplicationsHaptic interaction The ideal case 62. ApplicationsHaptic interaction Architecture 63. ApplicationsHaptic interaction Game steps 64. ApplicationsHaptic interaction Game video 65. Future workHaptic interaction Future directions Haptic visualization Use the haptic channel to complement visionin information visualization applications Haptic interaction with videos Highly dynamic Content Interaction Partial input of the environment, ill-posed Towards a theory of haptic rendering 66. Haptic Interaction in Virtual and Mixed Reality EnvironmentsThank you for yourattentionKonstantinos Moustakas, Assistant Professormoustakas@ece.upatras.grElectrical and Computer Engineering Department, University of Patras

Recommended

View more >