Automatic reconstruction of Building Information Models · 66 Surface extraction Chauve et al. Plane arrangement – Visible planes – Hidden plane hypotheses (called ghosts) Binary

Automatic reconstruction of Building Information Models

Alexandre Boulch

December 19th, 2014

Defense committee:

Dr. Pierre ALLIEZDr. Bruno LEVYDr. Reinhard KLEINDr. Jan BOEHMDr. Jean-Philippe PONSDr. Renaud MARLETDr. Martin DE LA GORCE

2

Building Information Models

The BIM is a digital representation of a building

Building Information Modeling (BIM): Benefits, Risks and ChallengesSalman Azhar et al. (2008)

Model interoperability in buildinginformation modeling Steel, James et al. (2010)

3

Building Information Model

BIM

4


ParticipantsCoordination

Ifc foundation

BIM

5


Louis Vuitton Fundation (CSTB)

SimulationParticipantsCoordination

Ifc foundation

BIM

6


Coriolis Building (ENPC)

Visualization



Ifc foundation

BIM

7


Coriolis Building (ENPC)

VisualizationLifecycleManagement



Ifc foundation

BIM

8

Renovation of old buildings

● Huge market(e.g., improvement of thermal performance)

● No digital model● Wrong or inexisting

blueprintshttp://www.agrconstruction.fr/

9

Data

● Photogrammetry

10

Data

● Photogrammetry● Depth sensors

11

Data

● Photogrammetry● Depth sensors● Lasers

12

Reconstruct and semantize surfaces

Building materials:● Wall: aerated concrete● Window: double glazing● Window frame: aluminium● Door: wood● ...

Heat capacity:● ...

+Geometry

Semantics

13

Pipeline

Point cloud

Digital model

Structure andinformationenrichment

14

Pipeline

15

Pipeline

Normalestimation

Point cloudPoint cloud +

Normals

Local information

16

Pipeline

Normalestimation

Primitivedetection

Point cloud +Normals

Primitives

Cluster-level information

17

Pipeline

Surfaceextraction

Normalestimation

Primitivedetection

Primitives Surface

Global geometric information

18

Pipeline

Surfaceextraction

Normalestimation

Primitivedetection

Surfacesemantization

Surface Digital model

Global semantic information

19

Pipeline

Surfaceextraction

Normalestimation

Primitivedetection


20

Pipeline

Surfaceextraction

Primitivedetection

Normalestimation

Point cloudPoint cloud +

Normals

Local information


21

Normalestimation Introduction

● Local estimation of the surface orientation

Robustness to:

noise

outliers

sharp edges

anisotropy

Speed

22



● Robustness to

– noise

outliers

sharp edges

anisotropy

Speed

23



● Robustness to

– noise

– outliers

sharp edges

anisotropy

Speed

24



● Robustness to

– noise

– outliers

– sharp edges

anisotropy

Speed

25



● Robustness to

– noise

– outliers

– sharp edges

– anisotropy

Speed

26



● Robustness to

– noise

– outliers

– sharp edges

– anisotropy● Speed

27

Normalestimation Previous work

● Regression– Planes: Hoppe et al. Surface reconstruction from unorganized

points. SIGGRAPH, 1992.

– Jets: Cazals & Pouget. Estimating Differential Quantities using Polynomial fitting of Osculating Jets. Symposium on Geometry Processing, 2003.

● Voronoï diagrams– Dey and Goswami. Provable surface reconstruction from noisy

samples. Comput. Geometry, 2006.

● RANSAC– Li et al. Robust normal estimation for point clouds with sharp

features. Computers & Graphics, 2010.

28

Normalestimation Previous work

Regression Planes

Regression Jets

Voronoï diagrams RANSAC

Noise X X X

Outliers XSharp

features X X

Anisotropy X

Speed X X X

29

Normalestimation Randomized Hough transform

Generate hypotheses:pick triplets of points

30



Spacechange

31



Spacechange

Vote in a sphericalaccumulator

32



Spacechange


Select the mostprobable bin

33



Spacechange



Generatenormal

34



Spacechange



Generatenormal

How manyhypotheses must

be generated ?

35

Normalestimation Robust lower bound

● Filled accumulator of M bins:empirical approximation of a probability distribution

● Estimate the quality of the approximation.

T*, minimum number of triplets to pick such that :

δ

δ: deviation toleranceα: confidence level

36

Normalestimation Robust lower bound

● Filled accumulator of M bins:empirical approximation of a probability distribution

● Estimate the quality of the approximation.

T*, minimum number of triplets to pick such that :

● From Hoeffding inequality:

37

Normalestimation Results

Cazal & Pouget (2003)

Li & al (2010)

Ours

10°

Color code

38

Normalestimation Speed and accuracy

● Same complexity

● As accurate and faster than[Li et al. 2010]

Ours

Time (s), logarithmic scaleNote: [Dey & Goswany 2006] does not appear on the graphic, because the method is not robust to noise

Cylinder, 0.2% noise, 200k points

39

Normalestimation Visuals results: laser scans

40

Normalestimation Visual results: photogrammetric data

41

Normalestimation Conclusion

● Robust normal estimator based on Hough Transform– accurate

– robust to noise, outliers, sharp edges and anisotropy

– fast

● Limitations– smooth normals on wide angles

42

Pipeline

Surfaceextraction

Normalestimation

Primitivedetection


43

Cluster level information

Pipeline

Surfaceextraction

Normalestimation

Primitivedetection

Point cloud +Normals

Primitives


44

Schnabel & al (2007)

Primitivedetection

Extraction

RANSAC

Region growing

Fusion

45

Primitivefusion Practical example

Over-segmentation

46


Over-segmentation

47


Over-segmentation

48

Primitivefusion Existing methods

● Tolerance on parameters– 3 for a plane (2 for orientation, 1 for distance)

– 4 for a sphere (3 for position, 1 for radius)

– …

● A contrario criterion only for planes in depth maps [Bughin, E. 2010.]

49

Primitivefusion Existing methods

● Tolerance on parameters :– 3 for a plane (2 for orientation, 1 for distance)

– 4 for a sphere (3 for position, 1 for radius)

– …

● A contrario criterion only for planes in depth maps [Bughin, E. 2010.]

Our method:– One single indicator: distribution of distances to

primitive

50

Primitivefusion Criterion for fusion● Two surfaces and , associated with point sets

and ● Distance function from point to primitives● Define two sets:

● If then ● Statistical tests

– Mann-Whitney

– Kolmogorov-Smirnov

51

Primitivefusion Results

52


Nothing changed

53


Nothing changed

● Very small difference in the distributions rejection

No user control on acceptance

54


● Very small difference in the distributions rejection

● No user control on acceptance

Nothing changed

55

Primitivefusion Primitive fusion

0.0 0.2 0.4 0.6 0.8 1.0

Distance

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Den

sity

σ = 0.01

Distribution of X

Distribution of Y

0.0 0.2 0.4 0.6 0.8 1.00.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

0.18σ = 0.05

0.0 0.2 0.4 0.6 0.8 1.00.00

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09σ = 0.1

0.0 0.2 0.4 0.6 0.8 1.00.000

0.005

0.010

0.015

0.020

0.025

0.030σ = 0.3

Example: two parallel lines at distance 0.1

Smooth distributions to ensure a controlled fusion

56


Increasing the level of smoothing

57



Smoothing level: 0.01

58




59




60

Primitivefusion Conclusion

● Criterion for primitives fusion– robustness to noise

– support for any primitive type (plane, cylinder...)

– intuitive user control on merging

● Limits– empirical relation between noise and fusion

distance

61

Pipeline

Surfaceextraction

Normalestimation

Primitivedetection


62

Pipeline

Normalestimation

Primitivedetection

Surfaceextraction

Primitives Surface

Global geometric information


63

Surfaceextraction Objective

Automatic surface reconstruction from laser scan– watertight without self intersection

– piecewise planar

– extended plausibly in hidden region

64

Surfaceextraction Objective

Automatic surface reconstruction from laser scan– watertight without self intersection

– piecewise planar

– extended plausibly in hidden regionwith support for data anisotropy

65

Surfaceextraction Previous work

● Smooth priors– e.g. Poisson reconstruction

● Voxelisation– biased, expensive

● Delaunay tetrahedralization– visible regions only

● Plane adjacency– visible near adjacency

● Manhattan world assumption– too restrictive: 3 directions only

66

Surfaceextraction Chauve et al.

● Plane arrangement– Visible planes

– Hidden plane hypotheses (called ghosts)

● Binary labelization of resulting 3D cell complex (empty or full)– Surface minimization, graph-cut optimization

● Advantages– watertight and non self-intersecting surfaces

– preservation of sharp edges

– extension of primitives far in hidden regions

– more plausible surfaces thanks to hidden plane hypotheses

A.-L. Chauve, P. Labatut, J.P. PonsRobust piecewise-planar 3D reconstruction and completion from large-scale unstructured point data. CVPR 2010

67

Surfaceextraction Chauve et al. (2)● Limitations

– wrong primitive borders with anisotropic point clouds

● Our method– determination of

borders in laser depth image

α-shape

68


– missing plane hypotheses

● Our method– generation of parallel

ghosts (for thin objects without detected thickness)

69


– Holes and truncated corners due to area minimization

● Our method– Edge length and

corner number minimization

70

Surfaceextraction Overview of the method

● Laser point cloud

Primitive extraction

Ghost generation

Volume partition using a plane arrangement

Binary partition using linear programming

Surface extraction

71


● Laser point cloud● Primitive extraction

Ghost generation



Surface extraction

72


● Laser point cloud● Primitive extraction● Ghost generation



Surface extraction

73


● Laser point cloud● Primitive extraction● Ghost generation● Volume partition using

a plane arrangement


Surface extraction

74



a plane arrangement● Binary partition using

linear programming

Surface extraction

75




linear programming● Surface extraction

76




linear programming● Surface extraction

77

Surfaceextraction Partition

● Partition the volume with a plane arrangement

● Labelization of each cell as empty or occupied

78

Surfaceextraction Energy formulation

● Energy

Data term

Penalizes a surface disagreeing with

observations

Regularization term

Penalizes a complex surface in the hidden areas

79

Surfaceextraction Data term

80


Penalize if xp+

= xp-

81


Penalize if xf+

≠ xf-

82

Surfaceextraction Edge and Corner Regularization

83


Shorter edgesSmaller corner number Smaller area

84



85



Smaller areaShorter edges

Smaller corner number

86

Surfaceextraction Regularization term

Area penalization

Edge length penalization

Corner number penalization

87

Surfaceextraction Regularization term

Area penalization

Edge length penalization

Corner number penalization

88

Surfaceextraction Edge term

EdgeNo edge

1-1-1+1=0 1-0-1+1=1 1-0-0+1=21-0-1+0=0

89

Surfaceextraction Corner term

90

Surfaceextraction Energy minimization

● Objective

● Very challenging for Markov Random Field– Edges: 4th order potential

– Corners: 8th order potential

– Tree-reweighted Belief Propagation: extremely slow

– Lazy Flipper: local minimum, extremely suboptimal

91

Surfaceextraction Linear relaxation

● Reformulation and relaxation

● This is a standard Linear Program

92

Surfaceextraction Results: meeting room

Area term onlyChauve et al.

93


Corner term onlyEdge term only

Area term onlyChauve et al.

94


Edge term only Corner term only

95


Corner only Edge + corner

96

Surfaceextraction Results: facade

97

Surfaceextraction Conclusion

● A method for piecewise planar surface reconstruction

● Compared to Chauve et al.– handling of anisotropy

– better surface hypotheses in hidden areas

– better regularization on edge length and corner number

● Limits and perspectives– scaling to entire buildings

– need for regularity discovery

98

Pipeline

Surfaceextraction

Normalestimation

Primitivedetection


99

Pipeline

Surfaceextraction

Normalestimation

Primitivedetection


Surface Digital model

Global semantic information

100

Semanticenrichment Building semantization

● Input: surface + semantic priors

● Output:semantized surface Surface Digital model

Method:– Bottom-up

– Based on grammars

101

Semanticenrichment Grammars

Sentence

Subject Verb Complement

NounAdjective

The super computer rules Discovery One

Building

Facade Roof

Window WallArticle

Facade

Expression of hierarchical decompositions

102

Semanticenrichment Constrained attribute grammars

Sentence

Subject Verb Complement

NounAdjective

The super computer rules Discovery One

Building

Facade Roof

Window WallArticle

Facade

Agreement (singular)

adj

Expression of complex relationsbetween elements

103

Semanticenrichment Stairway grammar

Stairway

Polygon

Polygon

104

Semanticenrichment Basic rule

Tread t → Polygon p

Riser r → Polygon p

Step s → Tread t, Riser r

riser

tread

105

Semanticenrichment Basic rule

Tread t → Polygon p

Riser r → Polygon p

Step s → Tread t, Riser r

Riser ?

Tread ?

riser

tread

106

Semanticenrichment Conditional rule

Tread t → Polygon p ( horizontal(p), p.length < 2 )

Riser r → Polygon p ( vertical(p), 0.05 < p.width < 0.25 )

Step s → Tread t, Riser r ( edgeAdj(t,r), above(t,r) )

Step

107

Semanticenrichment Collections




Stairway w → sequence(Step s, edgeAdj) ms

Stairway

108

Semanticenrichment Combinatorial Explosion

● Enumerating collections may lead to combinatorial explosion

109



110



111



Total: 4 + 3 + 2 + 1 = 10 possible sequences

112

Semanticenrichment Maximal collections

Stairway

In practice, useful = largest collection:

Maximal operators (maxseq, maxcycle,...)

– fast computation

– low number of instances




Stairway w → maxseq(Step s, edgeAdj) ms

113

Semanticenrichment Results

● Two grammars– Facade

– Stairway

● Experiments– synthetic data: CAD models, simulated laser scans

– real data: laser scans, photogrammetric data

114

Semanticenrichment CAD Models

Triangle soup

115


Triangle soup Polygons

116


Triangle soup PolygonsSemantized

model

117


Triangle soup PolygonsSemantized

model

Semantizedmodel

+ texture

118

Semanticenrichment Real laser scan: stairs

119

Semanticenrichment Conclusion

Grammar-based building semantization method– pure bottom-up parsing

– combinatorial explosion containment

– grammars easy to design

– grammars maintainable by non computer scientist

Limits and perspectives– need of flexibility (missing elements, merged

geometries)

– learning of rule parameters

120

Conclusion

Surfaceextraction

Normalestimation

Primitivedetection


● Pipeline for surface reconstruction and semantization frompoint cloud

● A step towards automatic BIM reconstruction

121

General perspectives

Surfaceextraction

Normalestimation

Primitivedetection


● Arbitrary order– surface useful for

semantization

semantics useful for surface reconstruction

Simultaneous or loop over semantics and surface extraction

Dealing with volumes

122


Normalestimation

Primitivedetection


semantization

– semantics useful for surface reconstruction

Simultaneous or loop over semantics and surface extraction

Dealing with volumes Surfaceextraction

Pointsemantization

123


Normalestimation

Primitivedetection


semantization


● Simultaneous or iterated semantics and surface extraction


Surfaceextraction

andsemantization

124



semantization




125



semantization



● Dealing with volumeshttp://www.bimvision.eu/

126

Thank you for your attention

Fast and Robust Normal estimation for Point Clouds with Sharp Featureswith Renaud MarletSGP 2012Computer Graphics Forum

Semantizing Complex 3D Scenes using Constrained Attribute Grammarswith Simon Houiller, Renaud Marlet and Olivier TournaireSGP 2013Computer Graphics Forum

Statistical criteria for primitive mergingwith Renaud MarletICPR 2014

Piecewise-Planar 3D Reconstruction with Edge and Corner Regularizationwith Martin de La Gorce and Renaud MarletSGP 2014Computer Graphics Forum

Documents

Automatic reconstruction of Building Information Models · 66 Surface extraction Chauve et al. Plane arrangement – Visible planes – Hidden plane hypotheses (called ghosts) Binary