1
March 9th 2008
Data-Driven Haptic Rendering
Matthias Harders, Raphael HoeverVirtual Reality in Medicine GroupComputer Vision Lab, ETH Zurich
2March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Analogy – Computer Graphics
Conventional CG rendering
Rendering
3March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Analogy – Computer Graphics
Image based rendering
Rendering
4March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Process Overview
Record data from real object
Process data
Synthesize feedbackin virtual environment
5March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Key Concepts
Haptic signals acquired during user interaction
No physical models employed
Forces synthesized based on recordings
6March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
(Brief) Related Work
Kuchenbecker et al., Event-based haptics
Mahvash, Hayward, Tool-tissue contact
Pai et al., ACME, WHaT, Interaction cap. & syn.
MacLean, Haptic camera
Colton, Hollerbach, Switch force playback
…
2
7March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Process Overview
Record data from real object
Process data
Synthesize feedbackin virtual environment
8March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Recording Setup
9March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Technical Specifications
Position sensing6 DOF, resolution 0.03 mmworkspace 381 x 267 x 191 mm3
Force sensing6 DOF, resolution 1.6 mN, range ±25 N, noise ±10 mN
Force display3 DOF, resolution 2.2 mN, range ±1.4 N (8.5 N short term)
10March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Signal Acquisition Artifacts
Position and force sensing not synchronized
Haptic rendering quality reduced due to jitter
Time
Position samplesForce samples
11March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Real-Time System
RTAI machine with real-time drivers (Comedi)
Positions acquired via Sensoray DAQ board
Forces acquired via National Instruments board
Sampling frequencies up to 5 kHz
Sensoray DAQModel 626
National InstrumentsPCI-6220 board
12March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Acquisition Synchronization
Acquisition jitter reduction (from ~100µs to ~1µs)
No signal interpolation required
Flexible sampling rates
Time
Position samplesForce samples
3
13March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Material Samples
Box-shaped frame (200 x 80 x 28 mm3)
Solid silicone cuboid (SmoothOn Ecoflex 0030)
Rubber membrane
siliconerubber
14March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Data Acquisition Strategy
Positions, forces measured directly
Velocities derived via central differences, smoothed with moving average filter (100 samples)
Forces at same position and velocity averaged
Data reduction by only considering forces changes > 50mN
Typical interaction, 1kHz sampling: 75% reduction
15March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Data Acquisition Strategy
Poking at different frequencies for elasticity andviscous damping
Force steps for material relaxation
16March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Process Overview
Record data from real object
Process data
Synthesize feedbackin virtual environment
17March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Data Preprocessing
Interaction trajectory not constrained
Sparse sampling of object
Resample interaction domain
Interpolation strategies requiredSimplex-based interpolationRadial basis functions
Selection of data dimensions
18March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Simplex-Based Interpolation
Fast computation of domain tessellation
Limited to convex hull of data
Slow calculation of interpolated values
No implicit smoothing
4
19March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Simplex-Based Interpolation
Select resampling density according to datadimensionality, storage requirements, andcomputation time
Example for 1D indentationΔp=0.5mm, Δv=7-40mm/s100 position and 21 velocity samples provide sufficient results
Additional dilation steps20March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Radial Basis Functions
Interpolation in the whole domain
Faster computation of interpolated values
Requires solution of linear system
Always smooth interpolants
21March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Radial Basis Functions
1D example
Interpolation condition
Orthogonality constraints
( )∑ ∑= =
+−=N
j
l
kkkjj gdwF
1 1)()( xxxx φ
NifF ii ,,1for )( …==x
ligwN
jij ,,1for 0)(1j
…==∑=
x
22March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Radial Basis Functions
Linear equation system results
Ill-conditioned due to sparse sampling
Apply truncated singular value decomposition
Polyharmonic splines as basis functions
High computational effort solving large systems
⎟⎟⎠
⎞⎜⎜⎝
⎛=⎟⎟
⎠
⎞⎜⎜⎝
⎛⎟⎟⎠
⎞⎜⎜⎝
⎛0f
dw
0TGGA
23March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Process Overview
Record data from real object
Process data
Synthesize feedbackin virtual environment
24March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
1D Experiment – Rubber
1002 samples (8324 before reduction)
Voxel grid 100x21 samples (0.21mm, 7mm/s)
Simplex interpolation 3sec, RBF 50sec
rubber
Simplexinterpolation
5
25March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
1D Experiment – Rubber
Offline validation of interpolated results
Compare to force measurements of test-trajectories
Simplexinterpolation
26March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
1D Experiment – Silicone
1461 samples (5962 before reduction)
Voxel grid 100x21 samples (0.16mm, 8mm/s)
Simplex interpolation 3sec, RBF 128sec
silicone
Simplexinterpolation
27March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
1D Experiment – Silicone
Same validation strategy
Replication of stickiness of silicone
Simplexinterpolation
28March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Analysis of Rendering Error
Mean error silicone: 86mN (simpl.), 102 mN (RBF)
Mean error rubber: 26mN (simplex), 23mN (RBF)
JND=7%
max=11.9%JND=7%max=4.8%
Silicone Rubber● Simplex ● RBF
29March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
2D Experiment – Rubber
Contact point fixed, no slip, lateral deflection
3752 samples (15709 before reduction)
Voxel grid 45x45x11x11 samples (pos_x,y0.53mm, 0.51mm, vel_x,y 40.3mm/s, 28.7mm/s)
Simplex interpolation 4h (mainly precalculation)
RBF interpolation 106min
rubber
30March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
2D Experiment – Rubber
Comparison of actual and synthesized forces
RBFinterpolation
rubber
6
31March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
2D Experiment – Silicone
Contact point fixed, no slip, lateral deflection
3320 samples (15441 before reduction)
Voxel grid 28x26x11x11 samples (pos_x,y0.49mm, 0.53mm, vel_x,y 19.7mm/s, 13.7mm/s)
Simplex interpolation 4h (mainly precalculation)
RBF interpolation 37min
silicone
32March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
2D Experiment – Silicone
Comparison of actual and synthesized forces
RBFinterpolation
silicone
33March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Analysis of Rendering Error
Silicone Rubber
JND=7%
max=23.7%
JND=7%
max=13.8%
● Simplex ● RBF
34March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Extension to Sliding Contact
Inspired by Dahl's friction model
Detect interaction point outside of convex hull
Displace interpolation field to new position
35March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Extension to Sliding Contact
Example: sliding over rubber membrane
36March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Conclusions
Framework for data-driven haptic rendering
Comparison of different interpolation techniques
Offline experiments with test-trajectories
Future work focuses on adapting data dimensions due to material properties
Examine further materials
Perform perceptional user studies
7
37March 9th 2008 Matthias Harders / Computer Vision Laboratory / [email protected]
Acknowledgements
This work was supported by the EU project Immersence IST-027141
Main work by Raphael Hoever
Presentation at Haptic Symposium
Tutorial support by EU Network ofExcellence INTUITION IST/NMP-507248-2