ISMAR 2014: Comprehensive Workspace Calibration for Visuo-Haptic Augmented Reality

University ofSouth Australia

Comprehensive Workspace Calibration for Visuo-Haptic Augmented RealityUlrich Eck1, Frieder Pankratz2, Christian Sandor3, Gudrun Klinker2, Hamid Laga1 1 PBRC, University of South Australia, Australia 2 FAR, Technische Universität München, Germany3 IMD Lab, Nara Institute of Science and Technology, Japan !10/09/2014, IEEE International Symposium on Mixed and Augmented Reality, Munich

1


Visuo-Haptic Augmented Reality (VHAR)

2

[Sandor C., TEDx Adelaide, 2010]


Contributions

3

Previous work: position calibration only Improved: with orientation calibration

Novel algorithm for orientation calibration: >40% improvement

Position calibration: ~20% improvement

Time-delay compensation between sensors: faster and better calibration procedure

University ofSouth Australia 4

Related Work

[Sandor, C. et al., IEICE 2007][Vallino, J. et al., ICMCS 1998]

Early examples of visuo-haptic co-location


PHANToM Haptic Device: Joint Angles

5


PHANToM Haptic Device: Gimbal Angles

6


PHANToM Haptic Device: Sensor Errors

7


Related Work

[Harders, M. et al., TVCG 2009][Ikits, M. et al., EG 2003]

Compensation of sensor errors


Missing: Orientation Calibration

9

Position calibration only With orientation calibration

Orientation Error in Degrees


Haptic Device Calibration

External tracker provides reference pose

Forward kinematics model calculates stylus pose from sensor measurements

Sensor errors are modelled as linear or quadratic function

Sensor errors are compensated by minimizing the distance to the reference

Complete coverage of sensor ranges required

Accurate pose correspondences required for good calibration results

10

External Tracker (ET)

ETorigin

HIPpose

HIPtarget

Haptic Device (HD)

Camera (C)

Ctarget

HDorigin


Challenge: Unsynchronized Sensor Input

Unknown delay between haptic device and external tracker

Received measurements do not match during movement

Related work

tedious point and hold process for sampling corresponding measurements

Our solution

determine time-delay between sensors first and compensate it in software to allow users to move freely during the calibration process

11


Calibration Procedure

Tooltip (Tuceryan et al., 1995)

Absolute Orientation (Horn, 1987)

Time-Delay Estimation (Huber et al., 2012)

Joint-Angles (Harders et al., 2009)

Reference Orientation

Gimbal-Angle Errors

12


ETorigin

HIPpose

HIPtarget

Haptic Device (HD)

Camera (C)

Ctarget

HDorigin








Gimbal-Angle Errors

13


ETorigin

HIPpose

HIPtarget

Haptic Device (HD)

Camera (C)

Ctarget

HDorigin


Time-Delay Estimation

14


ETorigin

HIP

Haptic Device (HD)

HDorigin


Time-delay estimation

[Huber, M et al., ISMAR 2009]


Time-aware dataflow network execution model

[Puskta, D et al., ISMAR 2007]








Gimbal-Angle Errors

17


ETorigin

HIPpose

HIPtarget

Haptic Device (HD)

Camera (C)

Ctarget

HDorigin


Orientation Reference Estimation

18


ETorigin

HIP

Haptic Device (HD)

HDorigin



x

y

z

xy

z

HIPpose

HIPtarget

REFzaxis

x

y

z

xy

z

HIPorigin

REFzaxis


Reference OrientationTracked markers travel on circular paths around the z-axis

Transform marker positions with inverse forward kinematic pose using only the first 5 angles

Fit circles to trajectories to find centers and fit a line through the centres and the HIP position

20








Gimbal-Angle Errors

21


ETorigin

HIPpose

HIPtarget

Haptic Device (HD)

Camera (C)

Ctarget

HDorigin


Gimbal-Angle Error Compensation

22


ETorigin

HIP

Haptic Device (HD)

HDorigin


Gimbal-Angle Calibration

Use reference z-axis to determine the errors of the first two gimbal sensors

Model errors as linear or quadratic function

Minimize error using Levenberg-Marquardt optimization

Exploit mechanical limits of the last gimbal sensor by finding the gaps in the circular paths to compensate the offset

23


Evaluation

Numerical Evaluation of calibration results:

Two VHAR workspace setups:

Low-fidelity (LF): PHANToM Omni, OptiTrack

High-fidelity (HF): PHANToM Premium, ART-Tracking

Visual Evaluation:

Unwrapped spherical heatmap of recuded errors

Visual overlays on haptic stylus in calibrated system

24


Numerical Evaluation

25

Reduced position errors by ~ 20%

Reduced orientation errors by > 40%

No Calib Harders Our Work No Calib Harders Our Work

No Calib Harders O u r Wo r k No Calib Harders O u r Wo r k


Improved Orientation Accuracy

26

Reduction of Orientation Errors in Degrees

ISMAR 2014 Live DemoYou can: - perform a calibration - paint on a fish We’re in Room 00.07.014

27


SummaryContributions:

Novel algorithm for orientation calibration (>40% improvement)

Improved position calibration (~20% improvement)

Compensated time-delay for faster and better calibration

28

x

y

z

xy

z

HIPpose

HIPtarget

REFzaxis

x

y

z

xy

z

HIPorigin

REFzaxis

Questions ?Thank You!!


ReferencesHarders, M. & Szekely, G., 2009. Calibration, Registration, and Synchronization for High Precision Augmented Reality Haptics. IEEE Transactions on Visualization and Computer Graphics, 15(1), pp.138–149. Fu, M.J. et al., 2011. Effect of Visuo-Haptic Co-location on 3D Fitts' Task Performance. In Proceedings of the International Conference on Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ. San Francisco, CA, USA: IEEE, pp. 3460–3467. Rhienmora, P. et al., 2010. Augmented Reality Haptics System for Dental Surgical Skills Training. In Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology. Hong Kong: ACM, pp. 97–98. Ikits, M., Hansen, C.D. & Johnson, C.R., 2003. A Comprehensive Calibration and Registration Procedure for the Visual Haptic Workbench. In the workshop. New York, NY, USA: ACM, pp. 247–254. Vallino, J. & Brown, C., 1999. Haptics in Augmented Reality. In Proceedings of the IEEE International Conference on Multimedia Computing and Systems. pp. 195–200. Horn, B.K.P., 1987. Closed-form Solution of Absolute Orientation using Unit Quaternions. Journal of the Optical Society of America, 4(4), pp.629–642. Tuceryan, M., Greer, D.S. & Whitaker, R.T., 1995. Calibration Requirements and Procedures for a Monitor-Based Augmented Reality System. Visualization and …, 1(3), pp.255–273. Pustka, D. et al., 2006. Spatial Relationship Patterns: Elements of Reusable Tracking and Calibration Systems. In Proceedings of the International Symposium for Mixed and Augmented Reality. Santa Barbara, CA, USA, pp. 88–97. Sandor, C. & Uchiyama, S., 2007. Exploring Visuo-Haptic Mixed Reality, IEICE. Huber, M., Michael, S. & Klinker, G., 2012. Temporal Calibration in Multisensor Tracking Setups. 9. Workshop der GI-Fachgruppe Virtuelle und Erweiterte Realität (VR-AR 2012), pp.201–212.

29