Contextual high-resolution image classification by markovian data fusion.pdf

Department of Biophysicaland Electronic Engineering

University of Genoa

IGARSSIGARSS--20112011Vancouver, Canada, July 24Vancouver, Canada, July 24--29, 201129, 2011

Contextual HighContextual High--Resolution Image Resolution Image

Classification by Markovian Data Fusion, Classification by Markovian Data Fusion,

Adaptive Texture Extraction, and Adaptive Texture Extraction, and

Multiscale SegmentationMultiscale Segmentation

Gabriele MoserGabriele MoserSebastiano B. SerpicoSebastiano B. Serpico

22


University of Genoa

OutlineOutline

• Introduction– Contextual very high-resolution image classification

• The proposed method– Key ideas and overview of the method– Adaptive semivariogram extraction– Region-based multiscale MRF– Segmentation, estimation, and optimization

• Experimental results– Data set and experimental set-up– Results evaluation and comparisons

• Conclusion

33


University of Genoa

OutlineOutline

• Introduction– Contextual high-resolution image classification



• Conclusion

44


University of Genoa

IntroductionIntroduction

• Very high-resolution (VHR) optical remote-sensing images:

– Very interesting in land-use / land-cover mapping, especially in urban and built-up area analysis.

– 0.5 ÷ 5-m resolution available thanks to current (e.g., IKONOS, QuickBird, WorldView-2, GeoEye-1) and forthcoming (e.g., Pleiades) missions.

– Increased need to model spatial information due to limited spectral information (few spectral channels)

• A novel contextual classification method is proposed for HR optical images, based on:

– Adaptive texture extraction by semivariogram;

– Multiscale segmentation;

– Markov random fields for spatial information fusion.QuickBird, panchromatic, 1 m

55


University of Genoa

OutlineOutline




• Conclusion

66


University of Genoa

The Proposed ApproachThe Proposed Approach

How to incorporate spatial information?– Region-based approaches: usually effective for classes

with geometrical structures (e.g., urban).

– Texture analysis: effective for natural and artificial textured classes, especially for images with few spectral channels;

– Texture analysis: often introduce artifacts at the object borders (due to moving-window processing).

Key-ideas– Integrating segmentation and texture information by

incorporating semivariogram features into a previous multiscale region-based MRF model.

– Applying spatially adaptive texture extraction to prevent border artifacts.

77


University of Genoa

Iterative phase

Initialization phase

Overview of the Proposed MethodOverview of the Proposed Method

convergence?no

STOP

yes

Generate a preliminary classification map L0 by applying a previous region-based MRF classifier [5] to the input image X.

Extract a set Ft of texture features by applying to X the proposed adaptive semivariogram method, based on the class

borders in the current map Lt.

Stack together X and Ft and generate a set St of Qsegmentation maps, each related to a different spatial scale, by

applying a scale-dependent segmentation method to (X, Ft).

Generate the updated map Lt + 1 by applying a previous region-based MRF classifier [5] to the multiscale segmentation St.

t = t + 1

88


University of Genoa

OutlineOutline




• Conclusion

99


University of Genoa

Adaptive Semivariogram ExtractionAdaptive Semivariogram Extraction

Semivariogram– Local 2nd order statistics γi(h)

for a single-channel image.

– Multispectral extension by (possibly weighted) Euclidean distance.

– Usually estimated with a w × w moving window.

Proposed adaptive estimation– Use, for each pixel i, the pixels

that both belong to the related w × w moving window and share the same label as i in the current map.

– 1-norm on the pixel grid for convenience.Current map Lt: colors denote class labels; yellow

borders denote pixels used to estimate semivariogram

i

w × wwindow

{ }

∈

∈

∞

= − − = ≥

−

=

= − = − <

∑

∑

ℓ ℓ

ℓ ℓ

2

2

2

2

1

1( ) ( ) ( 0)

2

( , )1ˆ ( | , )2 ( , )

: , 2

ihw

ihw

i i j

t ti j i j

j Rti t t

i jj R

ihw

γ h E x x i j h h

δ x x

γ h w Lδ

wR j i j h i j

1010


University of Genoa

OutlineOutline




• Conclusion

1111


University of Genoa

Markov Random FieldsMarkov Random Fields

• MRF model for the spatial context– Representation of the statistical interactions between the pixel

labels in an image by using only local relationships:

• MRF-based classification– Minimization of a (posterior) energy function U(·), thanks to

the Hammersley-Clifford theorem. Here:

( ) ( )≠ =ℓ ℓ ℓ ℓ ∼, ,i j i jP j i P j i

=

= − −∑∑ ∑∼

ℓ ℓ ℓ01

( | ) ln ( | ) ( , )Q

t tq iq i i j

i q i j

U L S α P s α δ

Pixelwise probability mass function (PMF) of the segment labels in the segmentation map at each scale and each iteration, conditioned to each class

Labels in the neighborhood (here, 3 × 3) i

1212


University of Genoa

OutlineOutline




• Conclusion

1313


University of Genoa

Segmentation and PMF EstimationSegmentation and PMF Estimation

• Felzenszwalb & Huttenlocherm segmentation method– Graph-based region-growing method depending on a scale

parameter.– Segmentation at different scales by varying the scale

parameter.

• Class-conditional PMF estimation – Extension of a previous method that computes relative-

frequency estimate [5], based, at each t-th iteration, on a preliminary intermediate map Mt obtained classifying (X, Ft).

– To generate Mt from the HR stacked image (X, Ft), a non-parametric contextual method is desirable.

– Here, a recent (non-region-based) method that combines MRFs and support vector machines (SVMs) is used [9].

1414


University of Genoa

Parameter EstimationParameter Estimation

and Energy Minimizationand Energy Minimization

• Weight parameters α in the MRF– Extension of a recent method based on the Ho-Kashyap

algorithm.

• Energy minimization: iterated conditional mode (ICM)– Initialized with the intermediate preliminary map Mt.– Converges to a local energy minimum.– Usually good tradeoff between accuracy and processing time.

1515


University of Genoa

OutlineOutline




• Conclusion

1616


University of Genoa

Data Set and Experimental SetData Set and Experimental Set--upup

• Data set– Itaipu (Brazil/Paraguay), IKONOS, 3

channels, 1999 × 1500 pixels

• Set-up– Q = 5 scales, 7 × 7 window (w = 7).– Preliminary experiments suggested

limited sensitivty of the accuracy to (w, Q) for 5 ≤ w ≤ 31 e 2 ≤ Q ≤ 5.

– SVM applied with Gaussian kernel.– Kernel and regularization parameters

in the SVM optimized by a recent method based on the numerical minimization of the span bound.

RGB false color

Training map Test map

urbanherbaceous rangeland

schrub and brush rangelandforest landbarren land

built-up (non-urban)water

1717


University of Genoa

OutlineOutline




• Conclusion

1818


University of Genoa

Classification AccuraciesClassification Accuracies

– Very high test-set accuracies by the proposed method.

– Very similar test-set accuracies also by the previous method in [5] (multiscale segmentation and MRFs, no textures) and by an SVM applied to spectral and standard (non-adaptive) semivariogram features.

– Much lower test-set accuracies for an SVM applied only to the spectral channels (expected result: no spatial information used).

– But... test samples located only inside homogeneous areas and not at the class borders (usual in remote sensing).

1919


University of Genoa

Classification Maps: Previous MethodsClassification Maps: Previous Methods

– Relevant visual differences between the benchmark considered methods.

– Errors for “herbaceous” (textured class; e.g., white circle), but no border artifactsby the method in [5].

– Correct classification of “herbaceous,” but irregular behavior at the class borders by SVM with standard semivariogram.

RGB false color Method in [5]

SVM , spectral + semivariogram

2020


University of Genoa

Classification Maps: Proposed MethodClassification Maps: Proposed Method

– Correct classification of “herbaceous”

– no border artifacts by the proposed method.

– This suggests:• effectiveness of the proposed adaptive

semivariogram

• capability of the proposed classifier to fuse multiscale segmentation and texture

Proposed method Method in [5]

SVM , spectral + semivariogram

2121


University of Genoa

Classification Maps: Further CommentsClassification Maps: Further Comments

– Visually noisy map by the SVM applied only to the spectral bands (as expected).

– Spatially regular result, but no appreciable oversmoothing by the proposed method.

– Time < 50 minutes for all considered methods on a 2.33-GHz, 4-GB RAM pc (usually acceptable time for land-cover mapping).

RGB false color Proposed method

SVM , only spectral

2222


University of Genoa

OutlineOutline




• Conclusion

2323


University of Genoa

ConclusionConclusion

• Novel MRF-based VHR image classifier combining the multiscale segmentation and texture to model spatial information.

– Very accurate results for both textured and geometrically-structured classes.

– No border artifacts, thanks to adaptive semivariogram.– Improvement in class discrimination and/or border precision,

compared to previous methods.

• Possible future generalizations– Integrating edge information (e.g., line processes).– Approaching global energy minimization (e.g., graph-cuts).– Comparisons with other methods for VHR image classification– Experiments with other VHR data sets.

2424


University of Genoa

ReferencesReferences

1. S. Li, Markov random field modeling in image analysis, Springer, 2009.

2. X. Descombes and J. Zerubia, “Marked point process in image analysis,” IEEE Signal Processing Magazine, vol. 19, no. 5, pp. 77–84, 2002.

3. Q. Chen and P. Gong, “Automatic variogram parameter extraction for textural classification of the panchromatic ikonos imagery,” IEEE Trans. Geosci. Remote Sensing, vol. 42, no. 4, pp. 1106–1115, 2004.

4. M. De Martino, F. Causa, and S. B. Serpico, “Classification of optical high-resolution images in urban environment using spectral and textural information,” in Proc. of IGARSS-2003, Toulouse, France, 2003, vol. 1, pp. 467–469.

5. G. Moser and S. B. Serpico, “Classification of high-resolution images based on MRF fusion and multiscale segmentation,” in Proc. of IGARSS-2008, Boston, USA, 2008, vol. II, pp. 277–280.

6. A. H. S. Solberg, T. Taxt, and A. K. Jain, “A Markov random field model for classification of multisource satellite imagery,” IEEE Trans. Geosci. Remote Sensing, vol. 34, no. 1, pp. 100–113, 1996.

7. P. Li, T. Cheng, G. Moser, S. B. Serpico, and D. Ma, “Multitemporal change detection by spectral and multivariate texture information,” in Proc. of IGARSS-2007, Barcelona (Spain), 23-28 July 2007, 2007, pp. 1922–1925.

8. P. F. Felzenszwalb and D. Huttenlocherm, “Efficient graph-based image segmentation,” Int. J. Comp. Vis., vol. 59, pp. 167–181, 2004.

9. G. Moser and S. B. Serpico, “Contextual remote-sensing image classification by support vector machines and markov random fields,” in Proc. of IGARSS-2010, Honolulu (USA), 25-30 July 2010, 2010, pp. 3728–3731.

10. S. B. Serpico and G. Moser, “Weight parameter optimization by the Ho-Kashyap algorithm in MRF models for supervised image classification,” IEEE Trans. Geosci. Remote Sensing, vol. 44, pp. 3695–3705, 2006.

Documents

Contextual high-resolution image classification by markovian data fusion.pdf