Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 1
Mixed-Reality for Intuitive Photo-Realistic 3D-Model Generation Wolfgang Sepp, Tim Bodenmueller, Michael Suppa, and Gerd Hirzinger DLR, Institut für Robotik und Mechatronik
@ GI-Workshop VR/AR 2009
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 2
virtual world
real world
3D-Modeling with hand-held devices Overview
scanning process cycle over
acquire datareconstructInspect
deferred problem-recognition
non-efficient user-interface
observe
inspect reconstruct
move
scanobject
scanner
object
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 3
virtual world
real world
3D-Modeling with hand-held devices Overview
scanning process cycle over
acquire datareconstructInspect
deferred problem-recognition
non-efficient user-interface
object
scanner
object
?
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 4
virtual world
real world
3D-Modeling with hand-held devices Overview
scanning process cycle over
acquire datareconstructInspect
deferred problem-recognition
non-efficient user-interface
object
scanner
object
?
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 5
3D-Modeling with hand-held devices Aspects
Level of concurrency “which results are immediately available”
View Planning Support “which views are necessary to scan the object“
accumulated viewing positions or scan resultsnext best viewing positions
Navigation Support „how to navigate the scanner to a desired view“
link virtual object to real object relative poses between object, scanner, operator
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 6
3D-Modeling with hand-held devices State of the Art
system modalities registration view highest level of concurrency
Metris Modelmaker shape mechanical arm
N shaded points
Perceptron/Romer/PolyWorks shape mechanical arm
N shaded points
Handyscan shape & texture activeego-motion
N points
NDI Laser Scanner shape optical tracker N points
David shape & texture software 1 points
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 7
3D-Modeling with hand-held devices Requirements
spatial domaindepth measurement unit (scanner) pose measurement unit (tracker) calibration
radiometric domainsurface texture measurement unit (camera)
temporal domainsynchronisation (time coherence) latencies (online generation of 3D model)
requirements on spatial synchronisation depend on velocities of movements (expected 0.5m/sec, 45deg/sec)
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 8
The Multi-sensory 3D-Modeler Hardware
depth sensing principleslaser triangulation (rotating laser-scanner, laser-stripe profiler)stereo triangulation
pose measurementsinternal optical tracker (image-based ego-motion) external optical tracker (active/passive markers) coordinate measurement (mechanical arm)
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 9
The Multi-sensory 3D-Modeler Hardware
depth sensing principleslaser triangulation (rotating laser-scanner, laser-stripe profiler)stereo triangulation
pose measurementsinternal optical tracker (image-based ego-motion) external optical tracker (active/passive markers) coordinate measurement (mechanical arm)
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 10
The Multi-sensory 3D-Modeler Synchronisation
hardware synchronisation
avoid interferences software synchronisation
un-triggerable devices
labeling of sensor data
consistent data fusion
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 11
The Multi-sensory 3D-Modeler Data-Flow
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 12
Streaming Generation of 3D Models Data Flow
Level of concurrencyimmediate triangulationimmediate view integration
estimation of surface normal
densitylimitation
update ofnormal
selection ofvertices
localtriangulation
meshlimitation
pointmodel
wire-framemodel
mesh generation
3D pointmeasurement
visualisation
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 13
Photo-Realistic Texture Mapping
Texture-mapping modessingle-view mappingsingle-view mapping with brightness correctionmulti-view mapping
Dependenciesaccuracy of 3D modelaccuracy of pose measurementsynchronisation
Level of concurrencysequential (manual) acquisition of texture imagesimmediate mapping to surface model
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 14
Navigation Support Linking of virtual and real world objects
seamless factor
view alignment display navigation support usability
high operator pose see-through HMD, AR
operator – scanner - virtual object – real object
requires high accuracy in pose measurement
scanner pose desktop, AR scanner – virtual object – real object
scanner pose desktop, VR scanner – virtual object
measurement desktop, VR triangulation – virtual object
separate manual inspection
low mouse desktop, VR operator manual inspection
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 15
Navigation Support via sensor-pose
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 16
View Planning Support Deciding next best view(s)
seamless factor
modality of visualisation view planning support
usability
high textured surface-model texture- oriented
virtual object and real object not distinguishable in AR
shaded surface- model shape- oriented
good
wire-frame model link-oriented weak 3D impression
low raw point model measurement -oriented
missing 3D impression
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 17
View Planning Support via shaded, textured surface
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 18
Concurrent Scanning & Reconstruction with AR-support
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 19
Thank you!
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 20
Further Examples 1/2
MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009
Folie 21
Further Examples 2/2
Mixed-Reality for Intuitive Photo-Realistic 3D-Model Generation�Wolfgang Sepp, Tim Bodenmueller, Michael Suppa, and Gerd Hirzinger�DLR, Institut für Robotik und Mechatronik��@ GI-Workshop VR/AR 20093D-Modeling with hand-held devices�Overview3D-Modeling with hand-held devices�Overview3D-Modeling with hand-held devices�Overview3D-Modeling with hand-held devices�Aspects3D-Modeling with hand-held devices�State of the Art3D-Modeling with hand-held devices�RequirementsThe Multi-sensory 3D-Modeler�HardwareThe Multi-sensory 3D-Modeler�HardwareThe Multi-sensory 3D-Modeler�SynchronisationThe Multi-sensory 3D-Modeler �Data-FlowStreaming Generation of 3D Models�Data FlowPhoto-Realistic Texture MappingNavigation Support�Linking of virtual and real world objectsNavigation Support�via sensor-poseView Planning Support�Deciding next best view(s)View Planning Support�via shaded, textured surfaceConcurrent Scanning & Reconstruction�with AR-supportFoliennummer 19Further Examples 1/2Further Examples 2/2