Spatial and Temporal Information Spatial and Temporal Information Fusion Based on Laplace Pyramid Fusion Based on Laplace Pyramid

Peng, Zhimin

Advisor: Prof. Huang, Bo (CUHK)

Dr. Meng, Deyu (XJTU)

Outline:Outline:

� Problem

� Existing Method

� Framework

Results� Results

� Future work

Viewing the entire Earth's surface every 1 to 2 days (high temporal resolution)

Obtain image with low spatial resolution

(250m)

TERRA

MODIS

(250m)

Viewing the entire Earth's surface every 16 days, (low temporalresolution)

Obtain image with high spatial resolution

(30m)

LANDSAT

ETM

(30m)

Crop

High Spatial and High Temporal Crop

Growing Desertification UrbanizationHigh Temporal

resolution remote sensing data

Problem:Problem:

A B

A’ B’

Existing MethodsExisting Methods

� STARFM (Gao, Masek, et al. 06)

FrameworkFramework

Step 1:Step 1:

Laplace Pyramid Laplace Pyramid Laplace Pyramid Laplace Pyramid DecompositionDecomposition

Laplace Pyramid(LP)Laplace Pyramid(LP)

� Introduced by Burt and Adelson in 1983

� Function:

Each level represents a different level band of spatial frequenciesband of spatial frequencies

� Basic idea:

Step 1: Build a Gaussian Pyramid

Step 2: Take the difference between one

Gaussian Pyramid level and the next

Gaussian PyramidGaussian Pyramid

Laplace PyramidLaplace Pyramid

Step 2: Step 2:

Match FunctionMatch FunctionMatch FunctionMatch Function

Match FunctionMatch Function

Level i

iM'

iM

Image Analogies , Aaron Hertzmann, etc, 2002

Level i

iE'

iE

),,( ''

iiii MEEionMatchFunctE =

Match FunctionMatch Function

� Scheme 1：

� — Block by Block Regression

� Scheme 2：� Scheme 2：

� — Markov Network

Block by Block RegressionBlock by Block Regression

� Assumption:

� Patches of the same location satisfy linear relationship

� i.e.

Combine the above two equations

Markov NetworkMarkov Network

iM

iE

'M

Dictionary Preparation:

'

iM

'

iE Candidates

Markov NetworkMarkov Network

Belief Propagation

Step 3: Step 3:

ReconstructionReconstructionReconstructionReconstruction

ReconstructionReconstruction

� Usual Reconstruction :

Simply added back the different frequency part

from coarser to finer

Sensitive to noiseSensitive to noise

� New Reconstruction:

Frame Theory (Minh N. Do 2003)

Robust

Usual ReconstructionUsual Reconstruction

� Decomposition

� Reconstruction

Noise Pyramid: y y e= +ɶ

New ReconstructionNew Reconstruction

2( ) ( )

TG H

GH GH

=

=

† 1( )T TS A A A A−

= =

Computational Expensive

TA A I=( ) ( )GH GH=

†ˆ ( )x A y G c Hd d= = − +

ComparisonComparison

Landscape change monitoringLandscape change monitoring

MO

DIS N

ov 1

2, 2

00

0

MO

DIS N

ov 1

2, 2

00

2

MO

DIS N

ov 1

2, 2

00

0

MO

DIS N

ov 1

2, 2

00

2

ET

M N

ov 1

2, 2

00

0

ET

M N

ov 1

2, 2

00

0

Real ETM STARFM

Markov Network Regression

Scatter graphScatter graph

Our Method STARFM

Seasonal ChangeSeasonal Change

MO

DIS M

ay 20

th, 20

01

MO

DIS A

ug 2

0th, 2

00

1

, 20

01

ET

M M

ay 20

th, 20

01

ET

M A

ug 2

0th, 2

00

1

Original Our Method STARFM

Future workFuture work

� Improve prediction accuracy

� Retain detail information

— Compressive Sensing

— Matrix Completion— Matrix Completion

� Explore information contained in other band

Thank you !Thank you !Thank you !Thank you !