Welcome to the Gap Filling Comparison Workshop September 18-20, 2006

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

1

Welcometo the

Gap Filling ComparisonWorkshop

September 18-20, 2006

Antje Moffat





2

Welcome

• 12 of the 14 members from the gap filling comparison

• Over 30 participants - from all over Europe (Germany, Italy, Netherlands, Switzerland), US, Canada, Russia, and Australia





3

Goals of the Workshop

• Review of Gap Filling Techniques

• Completion of the Gap Filling Comparison– Discussion of the Results– Review of Paper– Evaluation of the Techniques

• Work Sessions and Plenary Debates to Exchange our Experiences and Expertise

• Generate Ideas for Further Improvement of the Gap Filling

• New Insights into the Eddy Flux Data





4

Outline

Workshop Program

GFC Analysis

Performance of Techniques

Error on Annual Sum





5

Program: This Morning (Monday)

9:00 Registration of the ParticipantsIf you need any kind of help, please contact Ulli or Silvana!

9:30 Setting the Stage of the Workshop• Antje Moffat: “Welcome” • Martin Heimann: “Biogeochemistry Research at the MPI in Jena”• Dario Papale: “The CarboeuropeIP Ecosystem Component Database: data

processing and availability”• Markus Reichstein: "Gap filling: Why and how?”

11:00 COFFEE BREAK (Foyer)

11:30 Review of Gap Filling Techniques: SPM and ANNs • Vanessa Stauch: “Semi-parametric models”• Dario Papale: “Gap filling of eddy fluxes with artificial neural networks”• Rob Braswell: "Gap filling by iterative regression using a regularized neural

network”

12:30 LUNCH (Campus Cafeteria)





6

This Afternoon

14:00 Review of Gap Filling Techniques (cont.): NLRs and UKF• Ankur Desai: “Towards a robust, generalizable non-linear regression gap filling

algorithm”• Asko Noormets: “NLR_AM - AQRTa-Model” (10 min recording)• Andrew Richardson: “Maximum likelihood non-linear regression model”• David Hollinger: “Data assimilation for eddy flux filling: The unscented Kalman

filter”

15:30 COFFEE BREAK

16:00 Review of Gap Filling Techniques (cont.): Models and comparison• Zisheng Xing: “A gap-filling model for tower-based NEP measurements”• Jens Kattge: “Model parameter inversion against Eddy Covariance Data using

a Monte Carlo Technique”• Bart Kruijt: “Comparing gap filling using neural networks and the CarboEurope

tool, for Fluxes and Meteo data”

19:30 Dinner Suggestion: Restaurant Papiermuehle (Please sign up!)





7

Tuesday Morning

9:00 Eddy and Component Flux Measurements• Corinna Rebmann: “Eddy covariance measurements and their shortcomings for

the determination of NEE”• David Hollinger: “Uncertainty in eddy flux data and its relevance to gap filling”• Eva van Gorsel: “Nocturnal Carbon Efflux: Can eddy covariance and chamber

measurements be reconciled?”• Pasi Kolari: “Gapfilling submodel selection based on measured component

fluxes” 10:30 COFFEE BREAK

11:00 Gap Filling of Grassland and Agricultural Sites• Christof Ammann: “Gap-Filling of CO2 Fluxes of Frequently Cut Grassland”• Mauro Colavincenzo: “A gap filling methodology used at a agricultural site in

Southern Italy”• Irene Lehner: “Carbon balance of a maize canopy: comparison of different gap

filling strategies”

12:00 LUNCHEND of OPEN SESSIONS!





8

Tuesday Afternoon

13:00-15:00 Parallel Work Sessions Part 1• Group 1: “Analysis of the partitioned GPP/ER comparison results” (Ankur

Desai)• Group 2: “Gap filling of meteorological data and water and energy fluxes”

(Dario Papale)

15:00 COFFEE BREAK

15:30-17:30 Parallel Work Sessions Part 2• Group 3: "Gap filling of sites with non-steady time series, e.g. cut grassland,

cropland" (Christof Ammann)• Group 4: "Using gap-filling techniques for estimating random errors in eddy

covariance data" (Andrew Richardson)

Please sign up for the work sessions!

19:30 WORKSHOP DINNER (Restaurant: Weinbauernhaus “Im Sack”)





9

Wednesday

Roundtable on the Gap Filling Comparison

8:30 Review of Gap Filling Comparison Paper, Antje Moffat- Interpretation of the comparison results- Derivation of key findings- Evaluation of techniques

12:30 LUNCH13:30 Minutes from the four Work Sessions14:30 COFFEE BREAK

15:00 Plenary Debates- Site dependency of gap filling technique performances- Filling of long gaps using previous years- Conception of a public domain code library with filling routines- Extended gap filling comparison for urban and crop- Workshop resume and outlook

17:00 End of Workshop





10

Handling of Presentations

• 20 min presentations: 15 min talk plus 5 min discussion

• Please transfer your presentation onto common laptop during coffee or lunch break (Ulli or Silvana)

• Publicized on GFC webpage after workshop

Questions?





11

Gap Filling Comparison

Analysis





12

Keyfile: Artificial Gap Flags

Golden File - fragmented:

Superimposition

Comparison of Observed NEP and Predicted NEP:

Basic Principle





13

Statistical Metrics

• Bias Error

• Root Mean Square Error

• Correlation Coefficient

€

BE =1

Npi − oi( )∑

€

RMSE =1

Npi − oi( )

2∑

€

R2 =( pi − p∑ )(oi − o)( )

2

(pi − p∑ )2 (oi − o∑ )2

•p - predicted NEP•o - observed NEP





14

Analysis

• Half-hourly basis

• Daily sum basis for full day artificial gaps •Daytime/Nighttime data

•Weighted ALL data

€

DSumALL =xd * hhd

48+xn * hhn

48

€

DSumd = xd =xi∑

Nd

Predicted versus Observed





15

Challenge of the Analysis

5 artificial gap length scenarios (single hh - 12 days)* 10 permutations* 3 subsets: day, night, all* 12 golden sites* 19 submissions* 15 statistical metrics: RMS, R2, Bias, Daily Sum,

normalized, benchmarked, …

513,000 comparison results! (see selection on posters in foyer)





16





17

Some Comparison Findings





18

RMSE and R2:Half-hourly Basis

Performance of gap filling techniques from bottom• MIM, MDV, UKF_LM, NLR & Others• 3 ANNs leading

Cor

rela

tion

Coe

ffici

ent R

2

Root Mean Square Error (gCm-2)

Daytime

Nighttime





19


Daily sum basisDaytime• R2: 0.8 - 0.95• RMSE: 1.0 - 1.8 gCm-2

Nighttime• R2: 0.75 - 0.9• RMSE: 0.5 - 1.0 gCm-2

Very good filling performance for daytime and nighttime data

Techniques: • MIM, Others, ANNs leading

Half-hourly basisDaytime • R2: 0.6 - 0.8• RMSE: 2.5 - 4.0 gCm-2

Nighttime• R2: 0.2 - 0.4• RMSE: 1.5 - 2.5 gCm-2

Good filling performance for daytime but not for nighttime

Techniques:• MIM, MDV, UKF_LM, NLR & Others, 3 ANNs

leading

RMSE and R2: Half-hourly & Daily Sums

Co

rre

latio

n C

oe

ffic

ien

t R

2

Daytime

Nighttime

Nighttime

Daytime






20

DailySum Bias per Site Year:Medium Gaplength, ALL

Bia

s

Techniques

Medium gap length (1.5 days):Bias of <0.07 gCm-2 per filled day





21

DailySum Bias per Site Year:Long Gaplength, ALL

Bia

s

Techniques

Long gap length (12 days):Bias of <0.2 gCm-2 per filled day





22

Annual Sum Error Estimate

Assumption:• representative choice of golden sites • good technique (red stars) Error estimate on the annual sum

Annual Sum Error• Small to med gaps: <0.07 gCm-2 per filled day equivalent• Periods of longer gaps: <0.2 gCm-2 per filled day equivalent Quality of long gap filling critical





23





Calculation Example

Example for average file with 35% gaps: • 18% small to medium gaps• 18% periods of longer gaps of 5-10 days 18% = 66 filled days

Error estimation• 66 x 0.07 gCm-2: 5 gCm-2

• 66 x 0.2 gCm-2: 13 gCm-2

Total error induced by filling of the gaps on the annual sum:

±18 gCm-2

^





24

Test using Real Gap Filling Results

• Standard deviation between techniques of filling the real dataset with 35% gaps ≤ 16 gCm-2

1) no soil temperature2) 30 day system failure

Site year NLR_F_MOD NLR_FM_OLS ANN_BR ANN_PS MDS SPM Mean SDeviationbe1_2000 -345.33 -334.88 -345.11 -332.59 -344.85 -330.72 -338.91 6.90be1_2001 -514.90 -521.12 -516.60 -512.07 -510.05 -514.68 -514.90 3.82de3_2000 -516.83 -529.61 -533.83 -506.75 -518.76 -516.25 -520.34 9.84de3_2001 -505.44 -478.84 -507.30 -501.43 -498.62 -507.29 -499.82 10.84fi1_2001 -175.21 -170.23 -177.87 -173.09 -166.84 nan -172.65 4.29fi1_2002 -219.32 -215.26 -224.59 -210.88 -219.46 -193.83 -213.89 10.85fr1_2001 -565.38 -558.17 -548.90 -560.20 -553.04 -542.40 -554.68 8.29fr1_2002 -547.27 -550.18 -545.86 -553.62 -542.51 -542.03 -546.91 4.48fr4_2002 -297.55 -304.54 -289.67 -323.76 -322.55 -328.88 -311.16 16.08 1)

il1_2002 -170.64 -172.56 -118.64 -147.36 -160.02 nan -153.84 22.09 1)2)

it3_2002 -25.94 -23.37 -35.39 -29.67 -35.55 -21.84 -28.63 5.93

Are 18gCm-2 an appropriate estimate of the error on the annual sum prediction?





25





Questions?





26



Let’s fill our “knowledge gaps”

and have a fun and productive

workshop!

Let’s fill our “knowledge gaps”

and have a fun and productive

workshop!





27

Separation of Daytime and Nighttime Data

Keyfile: 10% artificial gaps Fragmented Golden File: 80% daytime NEP data, 35% nighttime

NEP data

• Real gap filling: 20% real day gaps, 65% real night gaps

1:3• Artificial gap filling:

8% artificial day gaps, 3.5% artificial night gaps

2:1

Important to consider daytime and nighttime data separately





28

Bias on daily sumsDaytime data

• Distribution of bias error of the individual daily sums

Bia

s E

rror

(gC

m-2)

Daytime dataDaily Bias Error:

- up to 4 gCm-2

ANNs leading

ANN_BR





29

Bias on daily sums:Nighttime data

• Distribution of bias error of the individual daily sums

Bia

s E

rror

(gC

m-2)

Daytime data

Daily Bias Error:

- up to 2 gCm-2

ANN_PS leading

Documents

Welcome to the Gap Filling Comparison Workshop September 18-20, 2006