SA2014.SIGGRAPH.ORG SPONSORED BY Automatic Semantic Modeling of Indoor Scenes from Low-quality RGB-D Data using Contextual Information Kang Chen 1 Yu-Kun Lai 2 Yu-Xin Wu 1 Ralph Martin 2 Shi-Min Hu 1 1 Tsinghua University Beijing 2 Cardiff University SA2014.SIGGRAPH.ORG SPONSORED BY Overview Input Output Semantic Modeling SA2014.SIGGRAPH.ORG SPONSORED BY Challenges Depth information: noisy may be distorted have large gaps Interior objects: complex 3D geometry with messy surroundings variation between parts SA2014.SIGGRAPH.ORG SPONSORED BY Previous Work User interaction High-precision 3D scanners [Shao et al. 2012] [Nan et al. 2012] SA2014.SIGGRAPH.ORG SPONSORED BY Observations Objects normally have strong contextual relationships Interior objects often have an underlying structure SA2014.SIGGRAPH.ORG SPONSORED BY Pipeline SA2014.SIGGRAPH.ORG SPONSORED BY 2D Images ill-posed 3D scenes full 3d relationships easy to acquire (Trimble 3D warehouse) Why virtual scenes? SA2014.SIGGRAPH.ORG SPONSORED BY Can virtual scenes do the job Real-world scene SA2014.SIGGRAPH.ORG SPONSORED BY Training Data Generation Objects and their parts in virtual scenes are often parallel or perpendicular to each other. unlikely to occur in real world scenes. SA2014.SIGGRAPH.ORG SPONSORED BY Training Data Generation SA2014.SIGGRAPH.ORG SPONSORED BY Training Data Generation moveable objects: chair, mouse and keyboard. SA2014.SIGGRAPH.ORG SPONSORED BY Context Representation SA2014.SIGGRAPH.ORG SPONSORED BY Comparisons Without Context With Context SA2014.SIGGRAPH.ORG SPONSORED BY Context-based Optimization SA2014.SIGGRAPH.ORG SPONSORED BY How much context ? SA2014.SIGGRAPH.ORG SPONSORED BY Top-down Matching SA2014.SIGGRAPH.ORG SPONSORED BY Top-down Matching balancing weight geometric closeness color similarity SA2014.SIGGRAPH.ORG SPONSORED BY Top-down Matching --- Toy Example SA2014.SIGGRAPH.ORG SPONSORED BY Top-down Matching --- Round 1 SA2014.SIGGRAPH.ORG SPONSORED BY Top-down Matching --- Round 2 SA2014.SIGGRAPH.ORG SPONSORED BY Top-down Matching --- Round 3 SA2014.SIGGRAPH.ORG SPONSORED BY Results SA2014.SIGGRAPH.ORG SPONSORED BY Limitations Too much depth information is missing. Scenes not conforming to our context. SA2014.SIGGRAPH.ORG SPONSORED BY Thanks Anonymous reviewers. Trimble 3D Warehouse. Authors providing datasets and results. People offered me their rooms for scanning. SA2014.SIGGRAPH.ORG SPONSORED BY Thank you very much. Any Questions?