Upload
sally
View
45
Download
0
Tags:
Embed Size (px)
DESCRIPTION
A Search-Classify Approach for Cluttered Indoor Scene Understanding. Liangliang Nan 1 , Ke Xie 1 , Andrei Sharf 2. 1 SIAT , China 2 Ben Gurion University, Israel . Digitalization of indoor scenes. Indoor scenes from Google 3D Warehouse. Acquisition of indoor scenes. Goal. - PowerPoint PPT Presentation
Citation preview
A Search-Classify Approach for Cluttered Indoor Scene Understanding
Liangliang Nan1, Ke Xie1, Andrei Sharf2
1 SIAT, China
2 Ben Gurion University, Israel
Digitalization of indoor scenes
Indoor scenes from Google 3D Warehouse
Acquisition of indoor scenes
Goal
• Scene understanding
Challenges
• Clutter– Densely populated– Arbitrary arrangements
• Partial representation– Occlusions
• Complex geometry
Classification & Segmentation
• Two interleaved problems– What are the objects?– Where are the objects?
• Chicken-egg problem– Classification needs segmentation– Segmentation needs a prior
Our solution
• Search – Propagate / accumulate patches
• Classify– Query classifier to detect object
Related Work
• Indoor scenes (This Session)– [Fisher et al. 2012] [Shao et al. 2012] [Kim et al. 2012]
• Semantic relationship– [Fisher et al. 2010, 2011]
• Recognition using depth + texture (RGB-D)– [Quigley et al.2009], [Lai and Fox 2010]
• Outdoor classification– [Golovinskiy et al. 2009]
• Semantic labeling– [Koppula et al. 2011]
Controlled region growing process
Our search-classify idea
0.6 0.8
0.92 0.94 0.94 0.94
Method overview
Training
Search-Classify
Point cloud features
– Height-size ratio of BBox– Aspect ratio of each layer– Bottom-top, mid-top size ratio– Change in COM along horizontal slabs
Bh
BdBw
Classifier
• Handle missing data– Occlusion
• Random decision forest– Efficient multi-class classifier
• Trained with both scanned and synthetic data– Manually segmented and labeled– 510 chairs – 250 tables – 110 cabinets – 40 monitors etc.
[Shotton et al. 2008, 2011]
Search-Classify
• Starts from seeds– Random patch triplets– Remove seeds with low confidence
• Accumulating neighbor patches– Highest classification confidence
• Stop condition– Steep decrease in classification confidence
0.65 0.92 0.93 0.88
Seed
• Segmented - classified objects problems– Overlap, outliers, ambiguities etc.
• Refinement – Outliers = patches with large distance
Segmentation refinement by template fitting
Template deformation
• Different styles for each class• Predefined scalable parts• Templates can deform [Xu et al. 2010]
Template deformation
• Different styles for each class• Predefined scalable parts• Templates can deform [Xu et al. 2010]
Fitting via template deformation
Confidence Fitting error Best fitting
• Best matching template– One-side Euclidean distance from points to template
Results and discussion
Results and discussion
Results and discussion
• Scalability test with varied object density
0 (25) 1 (45) 5 (60)
Results and discussion
• ComparisonLai et al. 2011
Ours
Limitation
• Upward assumption– Features– Template fitting
Future work
• Contextual information
Thank you