Segmentation Based Environment ModelingUsing a Single Image

SeungTaek Ryoo

Department of Software, HanShin University, Korea

stryoo@hs.ac.kr

2004. 9. 29

Previous Works

3D Reconstruction Planar Objects

• Single View Metrology (Criminisi1999)

• Piecewise Planar Objects from Single Images (Sturm1999)

• Automatic 3D Modeling of Architecture (Dick2000)

• Image Based Modeling and Photo-Editing (Oh2001)

Criminis1999

Sturm1999

Oh2001 Dick2000

Objective

Environment Mapping

Desired Image

Input Image (Regular Image, OCC Map)

Segmented Image

Image Segmentation

Depth Calculation

Depth Image

Environment Model

Environment Modeling

Objective Realistic & Real-time View Rendering with Full-View

Panorama• Image-Segmentation Based Environment Modeling

The Structure of Segmentation Based Modeling

Image Segmentation

Segmentation indoor scene consists of floor and ceiling. outdoor scene consists of ground and sky. the environment map is divided into

• a floor (ground), ceiling (sky), and surrounding objects.

Depth Calculation (1/4)

Assumption View direction of the camera must be parallel to the ground The floor (ground) and the ceiling must consist of a plane

that is parallel to the XZ plane in the world coordinate. The sky must consist of a hemi-sphere with an infinite radius The surrounding objects must be perpendicular to the

ground

mo

Image plane

Z

R

WC

Depth Calculation (2/4)

Extracting the depth value from Regular Image The depth value can be obtained by testing for the

intersection points between the floor and the ray, and between the Image plane and the ray

The equation can be obtained with the law of the triangle

b

drDepthZ )(

XY CylinderXZ Cylinder

Depth Calculation (3/4)

Extracting the depth value from OCC map depth value of the ceiling (floor) by testing the relationship

b/w XZ cylinder and ray, and b/w XY cylinder and ray

b

drDepthZ )(

Y = dY = d

rb

dZ

R

r Zb

d

R

V

Uo

Depth Calculation (4/4)

How to get the depth value of the surrounding objects Set the depth value of the floor(ceiling) close to

the surrounding objects. Some other assumptions should be made.

• the relationship between the surrounding objects.

• the relationship between the floor(ceiling) and the surrounding objects.

• the relationship between the image plane and the surrounding objects.

Color Image Segmented Image Depth Image

Environment Modeling

Feature Points

TriangulationRectangular Grid

Texture mapping & Point sample rendering

Wireframe Texture Mapping Point Sample Rendering

Texture mapping : the stretching effect of the 3D model occurs by the occlusion Point Sample Rendering : holes occurs by the lack of the information

Regular Image

Segmentation

Depth Calculation

Regular Image Desired Image

EnvironmentModeling

EnvironmentMapping

Regular Image

Seokguram (Stone Cave)

Gyeongbokgung (Royal Palaces)

OCC Map (Indoor Scene)

Segmentation

Depth Calculation

Environment Map (Octahedral Map)

Environment Modeling

Orthogonal Cross Cylinder

Environment Mapping

OCC Map (Outdoor Scene)

Segmentation

Depth Calculation

Orthogonal Cross Cylinder

Segmentation Based Environment Model

Desired Image

Result (Art Gallery)

Environment Map (sidestep)

Segmented Based Environment Model (sidestep)

Conclusion & Future Works

Conclusion can be used for environment modeling by extracting the depth

value by the image segmentation• navigate through the virtual environment when the viewpoint has

been changed• easy to implement on the environment map

Future Works Multiple Resolution

• the problem of the blurring-image effect caused by the fixed resolution

Multiple Images (Registration & Integration)• the problem of the stretching effect of the 3D model caused by the

occlusion Generalization to apply any type of images

• only images whose viewpoint and the floor are perpendicular may be used.