Stereo analysis of low textured regions with application towards

sea-ice reconstruction

Rohith MV, Gowri Somanath, Chandra KambhamettuVideo/Image Modeling and Synthesis(VIMS) Lab, Dept. of Computer and Information Sciences

Cathleen GeigerCenter for Climatic Research, Department. of Geography

University of Delaware, USA

Sea ice

Need for reconstruction“The feasibility of using snow

surface roughness to infer ice thickness and ice bottom roughness is promising….”

“…the goal of a circumpolar high resolution data set of Antarctic sea ice and snow thickness distributions has not yet been achieved …”

“…crucial for future validation of satellite observations, climate models, and for assimilation into forecast models…”

Ref: Workshop on Antarctic Sea Ice Thickness, 2006; Annals of Glaciology

OutlineStereo in presence of large texture-less areasEntropy based SegmentationOur Approach

Two stage estimationMRF Formulation Occlusion Model

Comparison of resultsConclusion

Sample Images

Some characteristics in images

Smoothly changing disparityNo edge Low color variation

Stereo

Left Image Hierarchical BP

Graph Cuts Our Algorithm

Previous approachesMethod Matching measure Segmentation Multiple Segmentation Hierarchical Occlusion Model

Klaus et al. ICPR 2006

Matching Pixels +Voting for plane + BP

Adaptive measure with SAD and gradient Mean Shift

No(Only one image) No No

Hong et al. CVPR 2004 Matching Pixels + Graph cut SAD, Fixed Mean Shift

No(Only one image) No Yes

Wang et al. CVPR 2008

Adaptive correlation window + Voting for plane + BP Adaptive correlation Mean Shift

No(Only one image) No Yes

Nister et al. CVPR 2006

Matching Pixels + Hierarchical BP Color weighted correlation Rectangular Grid

Yes (Both images) Yes Yes

Trinh BMVC 2008 Matching Segments + BP SSD + gradient Mean shift

Yes (Only one image) Yes No

Felzenswalb et al. CVPR 2004 Matching Pixels + BP None - Yes Yes

Our Method Matching Segments + 2 level BP SADEntropy filtering + graph

based segmentation No 2 levels Yes

Entropy based segmentation

argmax

Entropy based segmentation1. Convert the image to grayscale and

calculate the histogram.2. Estimate the brightness threshold as the

gray value that maximizes the entropy of the segmented image.

3. Partition the histogram based on that threshold into two parts. Equalize the two histograms. For each histogram repeat steps 2 and 3.

Comparison with mean shift

Left Image

Entropy based segments

Entropy based segmentation

Mean Shift segments

Our approachTwo stage solution

S2 S3

S1

S2 S3

S1

S2 S3

S1

Segment disparity• Single disparity per segment• Fewer disparity levels• Segment neighborhood

Pixel disparity• Disparity per pixel• Full range of disparities• Pixel neighborhood• Occlusion Detection

Example

MRF FormulationSegment Level DisparityΕሺ𝑓ሻ= 𝐷𝑝𝑝∈𝑆 ൫𝑓𝑝൯+ 𝑉൫𝑓𝑝,𝑓𝑞൯

ሺ𝑝,𝑞ሻ∈𝑁

Where

𝑓 is the disparity assignment

𝐷𝑝൫𝑓𝑝൯ is the SAD error of the segment 𝑝 under the disparity 𝑓𝑝

𝑉൫𝑓𝑝,𝑓𝑞൯ is the penalty for assigning different disparities to adjacent segments.

𝑉൫𝑓𝑝,𝑓𝑞൯= 𝜂 ห𝑓𝑝 − 𝑓𝑞ห.

MRF ModelPixel Level Disparity𝑬′ሺ𝒇′ሻ= σ 𝑫𝒖′𝒖∈𝑰 ሺ𝒇𝒖′ ሻ+ σ 𝑽′ሺ𝒇𝒖′ ,𝒇𝒗′ ሻሺ𝒖,𝒗ሻ∈𝑵′ (2)

𝑫𝒖′ ሺ𝒇𝒖′ ሻ= ൜𝜔∗𝑒−𝜇 if 𝒇𝒖′ is OCCLUDED𝛿∗𝑆𝐴𝐷+ 𝛽∗ȁ#𝒇𝒖′ − 𝑓𝑠ȁ# otherwise

𝑉′ሺ𝒇𝒖′ ,𝒇𝒗′ ሻ= ൜0 if either 𝒇𝒖′ or 𝒇𝒗′ is OCCLUDED 𝜂′ȁ#𝑓′𝑢 − 𝑓′𝑣ȁ# otherwise

where 𝑠 is the segment to which pixel 𝑢 belongs 𝜇 is the minimum SAD error for pixel 𝑢 𝑓𝑠 is the labeling from the segment disparity for segment s. 𝜔∗𝑒−𝜇 penalizes occlusion

Occlusion Model

Rohith MV, Gowri Somanath, Chandra Kambhamettu, Cathleen Geiger Towards estimation of dense disparities from stereo images containing large textureless regions. 19th International Conference on Pattern Recognition(ICPR), 2008

Results

ResultsResults

ResultsResults

Middlebury dataset

Tsukuba Venus Teddy Cones nonocc all disc nonocc all disc nonocc all disc nonocc all disc

Our Method 1.49 3.4 7.87 0.33 0.6 3.57 3.73 7.13 9.75 3.25 9.13 8.80 Hierarchical

BP [4] 2.13 4.29 11.4 1.4 2.38 16.5 17.3 25.2 31.0 12.5 20.6 22.0

Multiscale BP [10]

4.85 7.03 19.0 7.4 8.94 28.1 14.2 22.8 30.9 10.6 20.5 22.9

ConclusionsEntropy based segmentation to handle large

texture-less regionsTwo step MRF formulation Solution using belief propagationCan handle large disparity ranges

Future workExplore combination of segmentations based

on region characteristicsUse priors over segmentation and disparity

calculation in sequence of images

AcknowledgementsThis work was made possible by National

Science Foundation (NSF) Office of Polar Program grants, ANT0636726 and ARC0612105.