Interactive Modelling for AR Applications John Bastian, Ben Ward, Rhys Hill, Anton van den Hengel, Anthony Dick

Australian Centre for Visual TechnologiesSchool of Computer ScienceUniversity of Adelaide

Introduction Our method allows users to rapidly build

3D models forManipulation within AR workspaceUse as occlusion masks

Our approach is characterised byAn interactive modelling processModelling by segmentation and silhouette

carving

Overview Video an object with a hand-held webcam Track camera movement with PTAM User selects object in one frame System segments object in subsequent frames Observations are combined to construct the

3D model

Segmentation User marks the object, using a

mouse or the camera Build foreground and background

colour models from user marking Apply graph cut segmentation

Optimal segmentation given colour models and image edges

Additional marking refines the segmentation

Segmentation

Silhouette Carving Use the silhouettes

to generate 3D models

Start with a volume around the object

Remove voxels with projection outside the silhouette

Feedback Current shape estimate is used to

predict subsequent silhouettes Predicted silhouette is used to:

Define cut regionUpdate colour modelsReduce ambiguity when foreground has

similar colour to background

3D Prior Silhouette is predicted from

current shape estimate Initial mesh used until volume

is defined Volume model is used to

generate a confidence map for inclusion of each pixel in the silhouette

User Feedback User can view

the model as its shape evolves

Results

Results

Results

Future Work Register models from a

database to the images Incorporate photoconsistency Use more sophisticated

segmentation Increased automation

Questions?