Upload
hoangkhuong
View
225
Download
0
Embed Size (px)
Citation preview
Why is it important?
Interaction is basic to VEs – We defined them as ‘interactive in real-time’
No interaction => NOT a VE Ideal interaction:
– Very low latency - i.e fast – Multi-modal – Unencumbered – Intuitive
Technology falls well short of this of course
Interaction types
Navigation – Staying on the ground? – Walking v flying
• Depends on size of model wrt display system • Degree of immersion
Interaction with objects: selection, manipulation – Depends on the object and interaction – Select, lift, rotate, throw, steer, hit
Interaction with other users – Gesture
Navigation
Move between locations: simplest form of interaction – Explore the environment; look at it
Ambiguous term: – Path integration/dead reckoning – Wayfinding – Locomotion
Path integration/dead reckoning
From known coordinates: – How far have I come?
• Distance/rotation
– So where am I now? – Continuous update cycle
• Compare with desired end coordinates
Wayfinding
What route from here to there? – Requires spatial knowledge/understanding
• Of landmarks, routes
A natural approach? – Line-of-sight; Virtual signage; virtual map
• If used for training for example
– Sound? Floating arrow? Head-up display? Follow a ‘guide’?
– Naturalism works better for direction than distance
Process
Orientation – Where am I wrt to nearby objects and the target
location?
Route Decision – Choose route that does go to destination.
Route Monitoring – Check on the correct route and going in the right
direction. Destination Recognition
– Are we there? Or at least close?
Locomotion
Teleportation Scene In Hand Eye In Hand Flying Vehicle Manipulation methods as locomotion Leaning (relative vs absolute) Speech driven Device Specific (driving)
Movement
Physical locomotion – User physically moves within tracker coverage
area – Frame of reference for interaction is own body – Real-world orientation cues
Virtual locomotion – Virtual world moves/rotates around user – Frame of reference is display device/physical
frame – Can cause disorientation quicker then physical
motion
Locomotion styles Unconstrained
– Tracker orientation determines direction of travel • Users may roll or go up/down as they move forward-left-right-
back • May disorient new users; difficult to control
– Users travel through objects;no collision detection • High-speed navigation,needs high frame rate
Constrained – Restricts locomotion to plane/direction (no roll is common)
• Sets some speed/acceleration limits (may take frame rate into account)
– Collision detection • adds computational load to application • Better orientation cues • Enhances solidity & realism of environment
Selection
Direct manipulation – Intuitive; can be tiring; or infeasible
Ray-casting – Laser pointer; less tiring – May be imprecise: small objects, distances, tracker
noise – Manipulation?
Cone-casting (spotlight) – Easier than ray-casting – May select multiple objects – Also manipulation issue
Tracking the human body
Large displays require position and orientation of viewer’s body to be tracked – tracking information fed to runtime system as input signal.
Most commonly tracked is head but sometimes also hands, arms, legs, eyes etc. – Head tracking used to update virtual
viewpoint orientations.
Body tracking needed for lifelike interaction with objects and creatures. – say user wishes to wave at another person
in the VE: their real-world motions can be tracked and replicated in the VE.
Tracking the human head
An essential basic requirement in immersive VR systems.
Imagine axes mounted on top of your head – pans, tilts and yaws of head
measured around those axes.
HMDs rotation sensors to measure these three angles.
Angles passed to run-time VR software which updates viewing angles.
HMD
Tracking devices
Many tracking devices and systems developed over the years – some aimed specifically at VR systems – others borrowed from other areas.
Some systems are portable and cheap - some require permanent installations in large rooms and are very expensive indeed. – Trackers can be magnetic, electro-magnetic, acoustic, inertial,
optical, or mechanical.
Mechanical trackers
Mechanical linkage system – arm-like structure of several joint, one end fixed, the
other free to move with the user.
Measure position and angular orientation of free end – by measuring angles at each joint and
factoring in length of each segment.
Fake Space BOOM (right)
Mechanical Tracking Advantages Simple sensors, no need for transmitter/receiver low-cost device very low latency High positional accuracy Disadvantages The user is tethered Lots of inertia Typically small working volume Mechanical parts wear out
Electro-magnetic trackers
transmitter generates electromagnetic signals
received by a receiver (or sensor). Signal strength used to determine
absolute position and orientation of receiver relative to transmitter.
Example: Polhemus FASTRAK
FASTRAK electro-magnetic sensor from Polhemus – accurately computes the position and orientation of tiny receiver
as it moves through space.
Dynamic, real time six degree-of-freedom measurement of position (X, Y, and Z) and orientation (yaw, pitch, and roll) – RS-232 signal updated at 120 records/sec.
Transmitter constantly puts out a weak magnetic field. – passive receiver generates an electric signal as it is moved
through the field.
– Polhemus' processing electronics then amplify and analyse this signal to determine the real-world position and orientation of the receiver relative to the transmitter.
Polhemus FASTRAK system
Polhemous trackers well proven and widely used since the very early 1990’s.
The FASTRAK system shown here has one receiver and one transmitter.
System expanded by adding up to three more receivers – can attach receivers to
different parts of body – log data for gait and limb
analysis or computer animation.
Electromagnetic Tracking Advantages Small receivers Reasonably cheap Line-of-sight (LOS) not required Disadvantages Accuracy diminishes with distance Not very large working volume High latency due to filtering Transmitter/receiver required
Electro-magnetic interference
Major problem of electro-magnetic trackers – magnetic fields easily affected by the surrounding environment.
Large metal objects produce eddy currents in the presence of the magnetic fields – These can interfere and distort the original signal causing
inaccurate measurements. – same effect appears near electric currents, such as in cabling – also ferromagnetic materials – Also electromagnetic sources such as computer monitors.
Ferromagnetic and/or metal surfaces cause field distortion
Ultrasonic trackers
Two main components – transmitter generating an ultrasound signal – receiver detecting the signal.
Distance is calculated by measuring time-of-flight of ultrasonic pulse. – Three transmitters and receivers
needed to calculate full 3D position and orientation.
Ultrasonic tracking used by Logitech Head Tracker (shown) and 3D mouse.
Acoustic Tracking
Advantages Well known transducers (mics), lightweight Low cost device Disadvantages Line-of-sigh (LOS) required Echoes Low accuracy (speed of sound in air varies) Transmitter/receiver required
Inertial tracking systems
Very popular (because cheap) – based on inertial gyro technology – Detects acceleration and thus can calculate velocity (since mass in
known) giving 3DoF – Newish example is the Intersense IS-300.
Can be coupled with ‘add-on’ ultrasonic system to give 6 DoF sensing – example of a hybrid technology tracker.
IS-300 can operate in metallic environments, – 6 DoF tracker operates only
in LoS of transmitter.
Other examples: Intersense Intertrax2 and the Ascension 3D-Bird.
Inertial Tracking Advantages Cheap Small size No transmitter/receiver required LOS not required Disadvantages Only 3 DOF on their own Drift Not accurate for slow movements
Optical tracking methods
Many different forms – Often use image processing and pattern recognition and matching – Much work outside of VR: numerous ideas suitable for tracking
object position and pose
For example fiducial mark detection – light sources or reflective
colour markers attached to object at important locations such as joints or extremities.
Easier for image processing algorithm to track in cluttered conditions.
Optical tracking methods
Outside-in tracker – tracking apparatus is fixed – object to be tracked (e.g. the user) is viewed from the "outside".
Inside-out systems – take tracking measurements from the object to be tracked – for instance a camera can be mounted on the HMD – images analysed to produce pose and distance estimations based on
the position of fixed patterns within the environment.
Visible images or infra-red used. Many optical systems (but not all!) are one-offs, expensive
and require careful calibration procedures.
Motion Capture (Mo-cap)
Originally developed for gait analysis – Artificial limbs
Taken up by film/animation industry – Artificial actors (Titanic, LoR, etc..)
Can also be used in real-time
Placement of markers
Depends on application – Body/face? – Occlusion issues?
Attaching markers – Body suit
• CMU database example
– Sweat bands etc
Also in processing software
Occlusion problems
Need a skeleton to find a lost marker that reappears – Included with camera sw
packages – But not always in data-
streaming mode
Optical Tracking
Advantages Can work over a large area. Inherently wireless Disadvantages LOS needed Transmitter/receiver required Expensive Requires computer vision technology
Speckled computing
Specks: small computing devices – Eventual aim: like
dust motes – Present size: a few
cm
Supports inside-out tracking – University of
Edinburgh
Natural Interaction
Set of devices using sensing for unencumbered tracking and interaction – Kinect is the best known; now multiplying
Open NI consortium – Formed Nov 2010: includes Primesense
originators of Kinect
Kinect
Developed for xbox – Sold 8 million units in 60 days from Nov 2010 launch – Almost at once used for lots of other things – E.g http://kinecthacks.net/
Spec: • Colour and depth-sensing lenses • Voice microphone array • Tilt motor for sensor adjustment
– Field of View • Horiz: 57 degrees Vert: 43 degrees • Tilt range: ± 27 degrees • Depth sensor range: 1.2m - 3.5m
What is in it
Projects an IR grid – Affected by other IR sources (e.g. sunlight)
Built-in facial and voice recognition – simultaneously tracking of up to six people, – two active players with RT feature
extraction of 20 joints per player
Interfacing with Kinect
Originally unofficial use: Open NI – http://www.openni.org/
Driver Interface: – Two buffers
• rgb-data • depth-map (10bit
resolution)
After June 2011 official Windows 7 SDK
Kinect examples
Basic functions of Kinect
With OpenNI on Linux
Gesture recognition using FUBI • https://www.informatik.uni-augsburg.de/en/
chairs/hcm/projects/fubi/
Childs paint application
Eye trackers
Eye tracking systems are examples of optical tracking devices. – viewpoint in the virtual world follows
the gaze of user’s eye.
Originally developed as a mouse replacement – simply look at object – interact through eye movement
(such as a slow blink).
Support physically impaired users. Combined eye and head tracking systems
• also exist - use in practice is complicated.
Cybergloves and similar
Inherent in the folklore and hype of VR is the cyberglove - a wearable device that monitors the the position and orientation of hand and fingers.
The name CYBERGLOVE® is registered by Virtual Technologies Inc (VTi). – uses 18 or 22 patented
angular sensors for tracking the position of fingers and hand.
Gloves Virtual Technologies
CyberGlove - 18-sensor model - 22-sensor model Variants are: - CyberTouch - CyberGrasp
Gloves Fifth Dimensions Technologies - Data Glove
Data Glove finger flexure hand orientation -roll & pitch
Gloves
Fakespace - Pinch Glove Pinch Glove gesture recognition reliable low cost electrical sensors in
each fingertip contact among any 2 or more
digits
Mouse as input device in VR
Normal 2D mouse can be used (as in Octaga for example). – Need user selectable modes to switch between
DoF’s.
More sophisticated mice provide 3 or more DoF: these include the Spaceball (shown here) and Spacemouse.
Standard games joysticks or gamepads also used to give 2 or more DoF’s.
The WiiMote
3 accelerometers – Enough for 6 DOF – But will drift – Bluetooth connection
to 10m
Optical (IR) sensor – To 5m from sensor
bar – Triangulation from
ends of bar – Allows accurate
pointing Speaker
WiiMote interaction
Head-tracking – WiiMote stationary, head-mounted IR source
Finger-tracking - touch-free interaction – IR tape on finger + fixed IR source
Gesture recognition – Using accelerometers – Feature classification – Fast movements work better; beware variable arm
orientation
Software
Free libraries – WiiGLE
• http://mm-werkstatt.informatik.uni-augsburg.de/documents/WiiGLE/doku.php
• Provides a set of classifiers
– WiiGee • http://www.wiigee.org/index.html • Java-based, one classifier
Issues with Bluetooth stacks – Flakey implementations, especially Vista – BlueSoleil seems a good driver
• http://www.bluesoleil.com/