Yizhou Yu

Texture Compression for Large Real Texture Compression for Large Real EnvironmentsEnvironments

Texture Compression for Large Real Texture Compression for Large Real EnvironmentsEnvironments

Yizhou Yu

Computer Science Division

University of California at Berkeley

Yizhou Yu

Computer Science Division

University of California at Berkeley

Yizhou Yu

Varying Scene ConfigurationsVarying Scene ConfigurationsVarying Scene ConfigurationsVarying Scene Configurations

• Segment geometry into individual objects

• Related work– [ Hoffman & Jain 87 ], [ Besl & Jain 88 ], [ Newman

Flynn & Jain 93 ], [ Leonardis, Gupta & Bajcsy 95 ]

• Segment geometry into individual objects

• Related work– [ Hoffman & Jain 87 ], [ Besl & Jain 88 ], [ Newman

Flynn & Jain 93 ], [ Leonardis, Gupta & Bajcsy 95 ]

Original Configuration Novel Configuration

Yizhou Yu

FrameworkFrameworkFrameworkFramework

• Input– Multiple range scans of a

scene

– Multiple photographs of the same scene

• Input– Multiple range scans of a

scene

– Multiple photographs of the same scene

• Output– Geometric meshes of each

object in the scene

– Registered texture maps for objects

• Output– Geometric meshes of each

object in the scene

– Registered texture maps for objects

photograph

range scan 3D meshsynthetic image

Yizhou Yu

OverviewOverviewOverviewOverview

RangeImages

RadianceImages

PointCloud

PointGroups Meshes

TextureMaps Objects

Registration Segmentation Reconstruction

PoseEstimation

Yizhou Yu

Image Segmentation as Graph PartitioningImage Segmentation as Graph PartitioningImage Segmentation as Graph PartitioningImage Segmentation as Graph PartitioningBuild a weighted graph G=(V,E) from image

V: image pixels

E: connections between pairs of nearby pixels

region

same the tobelong

j& iy that probabilit :ijW

Partition graph so that similarity within group is large and similarity between groups is small -- Normalized Cuts Approximate solution from a generalized eigenvalue problem [Shi&Malik 97]

Yizhou Yu

Segmentation ResultsSegmentation ResultsSegmentation ResultsSegmentation Results

Yizhou Yu

ImageImageImageImage

Yizhou Yu

Aligning Photographs with Laser ScansAligning Photographs with Laser ScansAligning Photographs with Laser ScansAligning Photographs with Laser Scans

• Pose estimation using calibration targets– 3 rotation and 3 translation parameters

• Combinatorial search– 4 correspondences each image

• Pose estimation using calibration targets– 3 rotation and 3 translation parameters

• Combinatorial search– 4 correspondences each image

3D Targets

Yizhou Yu

Camera Pose ResultsCamera Pose ResultsCamera Pose ResultsCamera Pose Results

• Accuracy: consistently within 2 pixels• Correctness: correct pose for 58 out of 62 images

• Accuracy: consistently within 2 pixels• Correctness: correct pose for 58 out of 62 images

Yizhou Yu

Reconstructed Mesh with Camera Poses Reconstructed Mesh with Camera Poses and Calibration Targetsand Calibration TargetsReconstructed Mesh with Camera Poses Reconstructed Mesh with Camera Poses and Calibration Targetsand Calibration Targets

The “crust” algorithm, nearest-neighbors & quadric error metric

Yizhou Yu

Models of Individual ObjectsModels of Individual ObjectsModels of Individual ObjectsModels of Individual Objects

Yizhou Yu

Texture Map Synthesis ITexture Map Synthesis ITexture Map Synthesis ITexture Map Synthesis I

• Conventional Texture-Mapping with Texture Coordinates– Create a triangular texture patch

for each triangle

– The texture patch is a weighted average of the image patches from multiple photographs

– Pixels that are close to image boundaries or viewed from a grazing angle obtain smaller weights

• Conventional Texture-Mapping with Texture Coordinates– Create a triangular texture patch

for each triangle

– The texture patch is a weighted average of the image patches from multiple photographs

– Pixels that are close to image boundaries or viewed from a grazing angle obtain smaller weights

Photograph

Texture Map

3D Triangle

Yizhou Yu

Texture Map Synthesis IITexture Map Synthesis IITexture Map Synthesis IITexture Map Synthesis II• Allocate space for texture patches from

texture maps– Generalization of memory allocation to 2D

– Quantize edge length to a power of 2

– Sort texture patches into decreasing order and use First-Fit strategy to allocate space

• Allocate space for texture patches from texture maps– Generalization of memory allocation to 2D

– Quantize edge length to a power of 2

– Sort texture patches into decreasing order and use First-Fit strategy to allocate space

First-Fit

Yizhou Yu

A Texture Map Packed with Triangular A Texture Map Packed with Triangular Texture PatchesTexture PatchesA Texture Map Packed with Triangular A Texture Map Packed with Triangular Texture PatchesTexture Patches

Yizhou Yu

Texture-Mapping and Object ManipulationTexture-Mapping and Object ManipulationTexture-Mapping and Object ManipulationTexture-Mapping and Object Manipulation

Original Configuration

Novel Configuration

Yizhou Yu

Texture Map Compression ITexture Map Compression ITexture Map Compression ITexture Map Compression I

• The size of each texture patch is determined by the amount of color variations on its corresponding triangles in photographs.

• An edge detector (the derivative of the Gaussian) is used as a metric for variations.

• The size of each texture patch is determined by the amount of color variations on its corresponding triangles in photographs.

• An edge detector (the derivative of the Gaussian) is used as a metric for variations.

Yizhou Yu

Results on Edge DetectionResults on Edge DetectionResults on Edge DetectionResults on Edge Detection

Yizhou Yu

Texture Map Compression IITexture Map Compression IITexture Map Compression IITexture Map Compression II

• Reuse texture patches– Map the same patch to

multiple 3D triangles with similar color variations

• K-mean clustering to generate texture patch representatives

• Larger penalty along triange edges to reduce Mach Band effect

• Binary search to find the number of clusters

• Reuse texture patches– Map the same patch to

multiple 3D triangles with similar color variations

• K-mean clustering to generate texture patch representatives

• Larger penalty along triange edges to reduce Mach Band effect

• Binary search to find the number of clustersTexture Map

3D Triangles

Yizhou Yu

Synthetic Images with Compressed and Synthetic Images with Compressed and Uncompressed Texture MapsUncompressed Texture MapsSynthetic Images with Compressed and Synthetic Images with Compressed and Uncompressed Texture MapsUncompressed Texture Maps

Uncompressed20 texture maps

Compressed5 texture maps

20 texture maps 5 texture maps

Yizhou Yu

VideoVideoVideoVideo