A Framework of Ontology-Based Knowledge Information Processing for Change Detection in Remote Sensing

Data

Shutaro Hashimoto1, Takeo Tadono1,2, Masahiko Onosato1, Masahiro Hori1,2, and Takashi Moriyama1,2

1Graduate School of Information Science and Technology, Hokkaido University2Earth Observation Research Center, Japan Aerospace Exploration Agency

July 28, 2011 IGARSS 2011 TH4.T09.2 1

Background

July 28, 2011 IGARSS 2011 TH4.T09.2 2

• Needs for automatic image interpretation– especially change detection– to handle large amount of data

Mudslides

Floods

?

• Humanlike interpretation requires:– high cognitive

ability– versatility

July 28, 2011 IGARSS 2011 TH4.T09.2 3

Solution

• Emulating manual interpretation using knowledge information processing

• We propose a framework for change detection – using ontology-based knowledge to recognize

and understand targets– input data: optical multispectral data

Knowledge

Mudslide

July 28, 2011 IGARSS 2011 TH4.T09.2 4

Framework for Change Detection

Day 1

Satellite Data

Inference Results

Day 2

Auxiliary Datae.g. DSM

Bayesian NetworkQuery for Targete.g. “mudslide”

Information ExtractionInformation Extraction

InferenceInference

Analysis of Target

Analysis of Target

Pixel-Based/Object-Based

Image Analysis

KnowledgeBased on Ontology

BayesianInference

Evidences

Synthesis of KnowledgeSynthesis of Knowledge

July 28, 2011 IGARSS 2011 TH4.T09.2 5

Requirements for Knowledge Representation

“Vegetation has high NDVI values”“Roads are long and narrow”

“Buildings are usually located along Road”“Artificial Forests are often located along River”

“Mountains are often covered by Forest”

Knowledge representation requires:•uncertainty•modularity and scalability •implicit structural definition of concepts

KnowledgeBased on Ontology

Remote Sensing Ontology

July 28, 2011 IGARSS 2011 TH4.T09.2 6

Heavyweightontology inremote sensing

– 420 concepts

Definition Structures•Inheritance (B is-a A)

• Slot (B part-of / attribute-of A)

A B

soilsoil

p/o 1..

leafleaf

chlorophyllcomponent

substancesubstance

Any

waterwater

chlorophyllchlorophyll

a/o 1

p/o 1..

density density

clustercluster

Anycomponentriverriver

fieldfield

slopeslope

continuant

entityp/o 1..

a/o 1

geographical objectgeographical object

component

structure structural attr.

Any

contextual changecontextual change

p/o 1.. subevent change event

p/o 0..

p/o 0.. Anybefore

Anyafter

superficial changesuperficial change

p/o 1.. component Any

geographical featuregeographical feature

p/o 1..

soil layersoil layer

soilcomponent

p/o 1..

water layerwater layer

watercomponent

woodwood

p/o 1..

trunktrunk

woodcomponent

p/o 1..

p/o 1..

treetree

leafcomponent

trunkcomponent

seasea

mountainmountainoccurrent

p/o 1

soil appearancesoil appearance

soil layerafter

water appearancewater appearance

p/o 1 water layerafter

a/o 1

p/o 1

location slope

mudslidemudslide

subevent soil appearance

substrate

p/o 1..

forestforest

treecomponent

change event

Main Categories

p/o 1 Slot 1 B

Slot 2 Ca/o 1

A

July 28, 2011 IGARSS 2011 TH4.T09.2 7

Knowledge Based on Ontology

• Describing relations among some concepts

• Using Bayesian probability to express uncertainty

(1) Concept-Slot Relation

(2) Concept-Evidence Relation

(3) Co-Occurrence2 Concepts 3 Concepts

BC

A

(2)

July 28, 2011 IGARSS 2011 TH4.T09.2 8

Analysis of Target & Synthesis of Knowledge

p/o 0

p/o 1 soil appearance

mudslide

soil layerbefore

a/o 1 location slope

p/o 1

p/o 1soil layerafter

p/o 1

subevent

soilcomponent

soilcomponent

Ontology

slope angle

slope

soil layer

soilsoil layer

soil appearance

soil layer

soil appearance

mudslide

slope

hue

soil

saturation

soil

value

soil

NDVI

soil

Knowledge

Bayesian Network

(1)

Day 2soil appearance

mudslide

slope angle

slope

Day 1 Auxiliary Data

soil layer

soil

huevalue

NDVI

saturation

soil layer

soil

huevalue

NDVI

saturation

(3)

July 28, 2011 IGARSS 2011 TH4.T09.2 9

Change Detection

Soil LayerImage ObjectSoil

HueValueSaturationNDVI

Satellite Image

Day 2soil appearance

mudslide

slope angle

slope

Day 1 Auxiliary Datasoil layer

soil

huevalue

NDVI

saturation

soil layer

soil

huevalue

NDVI

saturationSoil Layer

Soil Appearance

Day 2

Day 1

Calculate posterior probability of target using Bayesian network

Inference of Substance Inference of Object

Inference of Change

July 28, 2011 IGARSS 2011 TH4.T09.2 10

Experiment

To validate cognitive ability & versatility

Applying to three cases of practical change detection without tuning

Bi-temporal data• observed by AVNIR-2 onboard ALOS

3 visible + 1 near-infrared 10 m spatial resolution

• applied image registration with geometric errors of less than 0.5 pixel

July 28, 2011 IGARSS 2011 TH4.T09.2 11

Case 1: Detection of Mudslides in Yamaguchi City, Japan

Day 1 (14 June, 2009) Day 2 (30 July, 2009)

©JAXA ©JAXA

Mudslides caused by heavy rain in 19-26 July, 2009

July 28, 2011 IGARSS 2011 TH4.T09.2 12

Case 1: Detection of Mudslides in Yamaguchi City, Japan

- Inference Results -

Day 1 (14 June, 2009)

Day 2 (30 July, 2009)

©JAXA

©JAXA

soil on day 1

soil appearance mudslide

slope

Definition of mudslide

soil on day 2

p/o 0

p/o 1soil appearance

mudslide

soil layerbefore

a/o 1 location slope

p/o 1

p/o 1soil layerafter

p/o 1

subevent

soilcomponent

soilcomponent

July 28, 2011 IGARSS 2011 TH4.T09.2 13

Case 1: Detection of Mudslides in Yamaguchi City, Japan

- Comparison with Human’s Result -

small changes=> more sensitive than human’s result

changes in the flat area=> our definition of mudslide doesn’t include changes in flat area

July 28, 2011 IGARSS 2011 TH4.T09.2 14

Case 1: Mudslide Detection in Yamaguchi City, Japan

- Comparison with Survey Data -

Mudslides in Our ResultCollapsed SlopesMudflow TracesDebris

in Survey Data(investigated by Yamaguchi Pref.)

July 28, 2011 IGARSS 2011 TH4.T09.2 15

Case 2: Detection of Flooded Areas in Myanmar

©JAXA

©JAXA

Day 1 (4 May, 2008)

Day 2 (19 June, 2008)

Water (Day 1)

Water DisappearanceWater (Day 2)

Floods caused by Cyclone in 2-3 May, 2008

July 28, 2011 IGARSS 2011 TH4.T09.2 16

Case 2: Detection of Flooded Areas in Myanmar- Comparison with Human’s Result -

Our result not correctly detected due to the existence of clouds

Human’s result misdetected edges of clouds

July 28, 2011 IGARSS 2011 TH4.T09.2 17

Case 3: Detection of Flooded Areas in Pakistan

©JAXA

©JAXA

Day 1 (14 Oct., 2009)

Day 2 (1 Sept., 2010)

Water (Day 1)

Water AppearanceWater (Day 2)

Floods caused by heavy rain since late July, 2010

July 28, 2011 IGARSS 2011 TH4.T09.2 18

Discussion

• About 90% accuracy in mudslide detection• Our results were better than human’s results

due to using knowledge specialized on targets

• Fairly good results in all cases without tuning due to analyzing essential characteristics of

each targets using heavyweight ontology

• Possible to understand and recognize targets as humans do using rich ontology-based knowledge

July 28, 2011 IGARSS 2011 TH4.T09.2 19

Conclusions

• We proposed a framework for change detection – using ontology-based knowledge to recognize

and understand targets

• The experiment showed:– accuracy was about 90 % in mudslide detection– results were better than human’s results without

tuning

• More improvements are ongoing– to extract various information from data, such

as spatial information– to describe more expressive knowledge

July 28, 2011 IGARSS 2011 TH4.T09.2 20

Thank you!