http://www.mcrlab.uottawa.ca

ASM Mahfujur Rahman

Bridging Virtual to Real: A New Approach to Annotation, 2D/3D Representation and Interaction with Ambient Environment

Agendas

• Introduction• Motivation• Physical browsing• Related works• POLE framework• Milestones

Introduction

• Lack of interaction with the physical objects• New advancement in ambient intelligence and

natural interaction techniques • Making crossing the border between virtual and

real world natural and even invisible to the user• Possibilities of new applications that combines

interactions from both virtual and real world

Interaction with physical objects

• Museum experience– Lack of interaction with

the displayed object

• Edutainment – The interactive game

based learning– Instills entertainment

and assists in the learning process

]a

]b

Physical browsing (cont)

User interacting with the real object Locate the annotated information at the tangible device position and augment associated information in

the output video

+ =

Related works: Collect and Drop [Broll 2008]

Related works: P2D Mobile Interaction [Froehlich 2008]

Related Works: Pull Me [Hinske 2007]

Related works: Poster Interaction [Ruzkio 2007]

Limitations in the existing works

• Annotation process– RFID, Visual based tags: not intuitive, limited distance

and performance trade offs– Using existing maps: not usable in indoor locations and

where unavailable

• Position determination– GPS: not available in indoor locations

• Interaction mechanism– Pull-me, collect and drop, touch etc.: Lacks natural

interaction capabilities

• Content delivery– Mobile phone

Solution: POint to Learn (POLE) framework

• Annotation process– Everything could be an

information tag [rahman, vecims et al 2009]

• Position determination– IR based position determination

• Interaction mechanism– Point and touch: with hands or

mobile devices [rahman et al, acmm, 2009]

• Content delivery– Mobile phones, laptops,

computers

Research milestones

• Real to virtual interaction– Annotation scheme: almost complete– 2D position determination is working: complete– Mobile phone to access content: incomplete– Mobile phone to point to content: will work– Hand to point to physical object: almost complete– Haptic feedback: will work

• Virtual to real interaction– 3D environment construction: work in progress– Virtual interaction: will work, need more inputs– Creating link between virtual and real world services and

interactions: working

Gesture based interaction with physical objects

Gesture based interaction with ambient media

Multi modal interaction with annotated media

Applications

• In Museum to know more about civilization, animals, fossils, space ships (a mockup for each of these needed to enhance the interaction that are mostly available in the museums)

• Teaching the younger about a complex subject and gain their attention (the map example)

• Medical sciences: Learning the operations of scientific apparatus/substances (amoeba, skull etc)

• Interactive TV ???

YOUR THOUGHTS

(c) Multimedia Communications Research Laboratory (MCRLab) http://www.mcrlab.uottawa.ca

متشکرم

谢谢！ありがとう！