Beyond the Surface : Supporting 3D Interactions for Interactive Surface With An Invisible Projection

Yi-Ping Hung

Professor, Graduate Institute of Networking and Multimedia National Taiwan University

On-Surface InteractionsMulti-TouchTangible Object Detection

Beyond-Surface InteractionsInteractive 3D Viewer

Motivation

IDEA: Using an Invisible Projection

Location & Orientation: 6 degrees of freedom

Color Projector IR Projector

IR Cameras x 2

MirrorMirror

Applications

iLamp

Project high-resolution content on the surface, mimicking the use of a desk lamp.

iFlashlight

A mobile version of iLamp, facilitating information explorations and cooperative tasks.

iView

An intuitive tool to see 3D content or augmented information from different perspectives.

Implementation

Marker Split and Merge

Adapting the maker size Markers split, when camera at closeup Markers merge, when at far distance

Marker On/Off

Marker on for camera positioning Marker off for finger detection Finger-ROI detection for marker on/off

With Marker OffWithout Marker Off

ROI

IR Projector

IR Cameras

IR Camera of Mobile Device

Smoothing

Simulated Background

Observed Image Foregrounds

Tangible Objects

Finger Touches

Layout Manager

Prediction

Color Projector

ROI

Applications

KalmanFiltering

Real scene

Block Diagram

Video Demo

Thanks