Embed Size (px)
Citation preview
Towards better use of global land cover reference datasets and the related context of CGLOPS
validation Nandika Tsendbazar, Martin Herold, Brice Mora, Steffen Fritz, Myroslava Lesiv, Bruno Smets
and Ruben Van De Kerchoeve
The Hague, November 3, 2016
Outline
• Background
• Re-using existing global land cover (GLC) reference datasets • Suitability of re-use
• Example of re-using for GLC map comparison
• Example of re-using for GLC map integration
• Lessons learnt for designing a flexible reference dataset
• Following up with validation of the CGLOPS products
Background
In the last 5 years, at least 14 GLC
maps have been produced which is
more than the half of the currently
available GLC maps.
GLC mapping is progressing towards
higher resolution.
More sample sites have been used for
assessing map accuracies.
Although not directly comparable,
there is a slight increase in the
reported map accuracies. 50
60
70
80
90
1990 1995 2000 2005 2010 2015
Rep
ort
ed a
ccu
racy
(%
)
Published year
Global land cover maps Integrated global land cover maps
1
10
100
1000
10000
100000
1000000
1990 1995 2000 2005 2010 2015
Sp
ati
al
res
olu
tio
n
(m)
1
10
100
1000
10000
100000
1000000
1990 1995 2000 2005 2010 2015
Sa
mp
le s
ize
Current reference datasets
Dataset name Temporal frame Classification
scheme and class
Number of sample
Sampling scheme Sample unit size Labelled by Reference
IGBP-DISCover 1990-1993 IGBP 16 379 Stratified random 1x1km Regional experts (Scepan et al. 1999)
GLC2000 1999-2002 LCCS 22 1265 2 stage stratified cluster 3x3km Regional experts (Mayaux et al. 2006)
Globcover LCCS 22
Stratified random
International experts
2005 circa 2005
4258 1.5x1.5km (Bicheron et al. 2008)
2009 circa 2009 4164 0.9x0.9 km (Bontemps et al. 2011a)
GLCNMO
Regional experts
Version 1 validation circa 2000 LCCS 20 600 Stratified random 1x1km
(Tateishi et al. 2011) Version 1
training 2000-2003 LCCS 14 1607 No sampling > 3x3km
Version 2 validation circa 2008 LCCS 20 904 Stratified random 500 x500 m
(Tateishi et al. 2014)
Version 2 training circa 2008 LCCS 14 2080 No sampling > 500x500m
MODIS/STEP training till 2014 IGBP 17 2762 No sampling 1 to 376 pixels Regional experts (Friedl et al. 2010)
LC-CCI 2000, 2005, 2010 LCCS 22 13000 2 stage stratified cluster SSU ~1x1km Automated pre-labelling
and experts (Achard et al. 2011)
Boston U. /GOFC-GOLD circa 2010 LCCS 12 500 Stratified random 5x5 km Automated
classification (Olofsson et al. 2012)
VIIRS Circa 2010 IGBP 17 circa 5000 Stratified and cluster 1 x1km International experts (GOFC-GOLD 2014)
FROM-GLC
Circa 2010 10
250x250m International experts (Gong et al. 2013) Training 91433 No sampling
Validation 38664 Cluster
Globeland30 2010 9 150000 2 stage stratified cluster 30x30m International experts (Chen et al. 2015)
GLC-Share - 11 1000 Stratified random experts (Latham et al. 2014)
DCP 6 4200 Systematic 1 x1km Volunteers and experts (Iwao et al. 2006)
GEO-WIKI Circa 2010 10 circa 18600 similar to FROM-GLC 1 x1km Volunteers and experts
(Fritz et al. 2009; Schepaschenko et al.
2015)
VIEW-IT 2000-2010 7 46207 Stratified random and
random 250x 250m Volunteers (Clark and Aide 2011b)
FAO-FRA 1990, 2000, 2005,
2010 9 13689 Systematic 10 x 10 km Automated pre-labelling (Potapov et al. 2011)
Dataset Intended application
(map accuracy
assessment)
Other application Pre-processing Estimates Source
IGBP-DIS IGBP map FAO Global forest cover Translated into 4 general classes Overall, class specific
accuracy, standard error,
also for continental level
(FAO, 2001)
GLC2000 GLC2000 MAP Validating IGBP, GLC2000,
MODIS maps and their synergy
Translated into 5 and 11 generic
classes; quality and consistency were
checked
Overall accuracy, class
specific accuracy
(Göhmann et al., 2009)
GlobCov5 GlobCover map 2005 Some samples fed to the validation
datasets for GlobCover 2009
Re-interpretation Overall accuracy, class
specific accuracy
(Bontemps et al., 2011a)
GlobCov9 GlobCover map 2009
GLCNMO-val GLCNMO map
GLCNMO-tr GLCNMO training
MODIS-tr MODIS GLC map MODIS: Global Urban Area
mapping
Revised for training (Schneider et al., 2009)
FAO-FRA Forest resources
assessment
(Potapov, et al., 2011)
LC-CCI
GOFC-GOLD Validating MODIS IGBP map in
Europe
Re-stratification Overall and class specific
accuracies
(Stehman et al., 2012)
GEO-WIKI African hybrid cropland map;
Biofuel land availability map
The percentage of cropland within a
1 km pixel; extent of human impact
and abandoned land as well as land
cover type were recorded with
confidence levels
Overall accuracy, error of
omission and commission
for cropland delineation
of 5 GLC maps
(Fritz et al., 2011c; Perger et
al., 2012)
VIEW-IT Land change of Latin
America and the
Caribbean
Land change of Bolivia;
deforestation and reforestation of
Latin America and the Caribbean;
forest change of Guatemala; land
use, land cover map of Uruguay
Generalized into 5 classes Overall and class specific
accuracies
(Aide et al., 2012; Clark and
Aide, 2011a; López-Carr et
al., 2011; Redo et al., 2012)
What is the current use of existing GLC reference datasets?
Open accessibility is key factor in reference data re-use.
Tsendbazar, N.E, S de Bruin, and M Herold."Assessing global land cover reference datasets for different user communities." Review of. ISPRS Journal of Photogrammetry and Remote Sensing 103:93-114.
Global Land Cover maps
Globcover 2005
LC-CCI 2005
MODIS 2005
Reference Data
2. Re-using reference datasets for map comparison
Globcover-2005 reference dataset • Accessible • LCCS based classifiers • Stratified sampling • Sample unit area suitable for medium
resolution maps
Tsendbazar, N.E., de Bruin, S., Mora, B., Schouten, L., & Herold, M. (2016). Comparative assessment of thematic accuracy of GLC maps for specific applications using existing reference data. International Journal of Applied Earth Observation and Geoinformation, 44, 124-135
2. Translation and re-interpretation of the Globcover-2005 reference dataset
Tsendbazar, N.E., de Bruin, S., Mora, B., Schouten, L., & Herold, M. (2016). Comparative assessment of thematic accuracy of GLC maps for specific applications using existing reference data. International Journal of Applied Earth Observation and Geoinformation, 44, 124-135
53%
9%
11%
2%
7%
18%
Globcover
54%
10%
10%
1%
7%
18%
LC-CCI
48%
6%
3%
22%
6%
15%
MODIS-IGBP No issue
Missing tree related classifiers
(cover, phenology, leaftype)Missing other classifiers
Differences in class definitions
Mixed unit translation issue
Problem due to class proportions
• About half of the 3857 sites could be directly translated
into the different legends.
3. Re-using reference datasets for spatial accuracy assessment
Reference datasets: GOFC-GOLD reference data portal, Geo-Wiki
and GLCNMO.
1. Legend harmonization 2. The centroid points of the sample site areas
Tsendbazar, N.E., de Bruin, S., Fritz, S., & Herold, M. (2015). Spatial Accuracy Assessment and Integration of Global Land Cover Datasets. Remote Sensing, 7, 15804
Information on spatial variability in map accuracy is useful to identify (un)certain areas in the mapping and can be further used for map improvements.
Spatial variation of map accuracy can be modelled using indicator kriging or other methods e.g., GWR.
Spatial correspondence of maps with reference datasets across Africa.
3. Re-using reference datasets for spatial accuracy assessment
Tsendbazar, N.E., de Bruin, S., Fritz, S., & Herold, M. (2015). Spatial Accuracy Assessment and Integration of Global Land Cover Datasets. Remote Sensing, 7, 15804
3. Re-using reference datasets for map integration
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Africa Eurasia Australia
and
Oceania
North
America
South
America
Corr
esp
on
den
ce
Globcover
LC-CCI
MODIS
Globeland30
Integrated map
Tsendbazar, Nandin-Erdene, Sytze de Bruin, and Martin Herold. "Integrating global land cover datasets for deriving user-specific maps." International Journal of Digital Earth:1-19
• Sharing or making reference datasets accessible is beneficial for GLC mapping/validation community as well as the map users.
A flexible reference dataset: 1. Sampling design • Stratified random sampling with strata independent of any GLC maps (Olofsson et al
2012). • This allows sample augmentation and post stratification (Stehman et al 2012,
Stehman 2014). 2. Flexible thematic detail • Information on LC primitives / elements rather than classes. • Classifiers 3. Flexible spatial support • Sample unit area that can be reduced depending on map resolutions • Allows quantitative calculation of class proportions
Lessons learnt: designing a flexible reference dataset
Following up with the validation of the CGLOPS products
Objectives:
To validate the new land cover map of Africa providing statistical estimates of overall
and class specific accuracies as well as area estimates according to CEOS WGCV stage
3 and 4 validation requirements.
The validation strategy also aims to be flexible for future evolutions.
● Allow upscaling to validation at global scale
● Validating updated version of global land cover maps and user-specific maps
when service continues over time
● Adjustable to higher resolution maps (10-20m)
● Suitable for spatial accuracy assessment of land cover maps to identify
uncertain regions.
Following up with the validation of the CGLOPS products
Sampling design
1. Sample size Initial size for the map validation of Africa was set to 1000 (under discussion)
• Sample size estimation based on Foody 2009, Olofsson et al. 2014 and Congalton and Green (calculation based on binomial and multinomial distribution)
• Target accuracy 80%; Precision (5%); Confidence level 95%
• 12 land cover types
• Expected size at global scale is 4 to 5 times larger
• With yearly updates, the size will further increase
2. Sample units
• 1 pixel of Proba-V 100m data
Following up with the validation of the CGLOPS products
Sampling design
3. Sample stratification • The global stratification of Olofsson et al 2012: • addressing rare and problematic classes such as land cover classes in heterogeneous areas
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
Sa
mp
le p
rop
ort
ion
fo
r ea
ch s
tra
tum
Sample allocation
Based on stratum area Allocation sample proportion
Sample distribution
Following up with the validation of the CGLOPS products
• Reference sample collection based on Geo-Wiki platform • Sample interpretation based on points in 10x10 blocks within the sample unit area
Response design
An example sample interpretation
Following up with the validation of the CGLOPS products
Response design
Name Country Region Affiliation
1 Landing Mane DRC Central Africa OSFAC, DRC
2 Ifo Suspence Republic of Congo Central Africa Marien Ngouabi University, Brazzaville, République du Congo.
3 Muluberhan
Biedemariam Tassew Ethiopia Eastern Africa HoLiN Training and Consultancy Services PLC
4 Natasha Ribeiro Mozambique Miombo Network Universidade Eduardo Mondlane and MIOMBO and GOFC-GOLD network
5 Matthias Herkt Germany Southern and
Eastern Africa Institute of Experimental Ecology, University of Ulm, Germany
6 Ralph Adewoye Nigeria Western Africa Department of Remote Sensing, Friedrich Schiller University, Jena,
Germany
7 Emmanuel Amoah
Boakye Ghana Western Africa WASCAL, Accra, Ghana
• Involve regional experts for reference land cover interpretation
Reference data collection
• Allows objective way of quantifying the class proportions within the sample unit area in case of
heterogeneous areas.
• Land cover elements such as trees, shrubs, grass and cropland building etc., will be recorded. This allows
the collected data to be used for different legends flexibly.
• Interpretation at smaller box/points is suitable for validating/calibrating high resolution land cover maps.
For higher resolution maps the point density could be increased.
• With the paint function in the interface, sample interpretation is relatively fast.
• Sample sites to be interpreted by two experts. Agreement will be checked and disagreeing interpretations will be assessed.
Following up with the validation of the CGLOPS products
1. Overall and class specific accuracies 2. Land cover area estimation 3. Map accuracy from user perspectives weighted by the error importance
The similarity matrices (weights) used for weighted accuracy assessments,
for carbon (left) and biodiversity (right) applications.
Accuracy estimation
4. Spatial accuracy assessment
Acknowledgements
Reference data providers: GOFC-GOLD, Boston University, Université
Catolique de Louvain, IIASA and GLCNMO
CGLOPS validation design contribution: VITO, IIASA and JRC
THANK YOU [email protected]
