Model-data fusion for the coupled carbon-water system Cathy Trudinger, Michael Raupach, Peter Briggs...
22
Model-data fusion Model-data fusion for the for the coupled carbon-water coupled carbon-water system system Cathy Trudinger, Michael Raupach, Peter Cathy Trudinger, Michael Raupach, Peter Briggs Briggs CSIRO Marine and Atmospheric Research, CSIRO Marine and Atmospheric Research, Australia Australia and and Peter Rayner Peter Rayner LSCE, France LSCE, France Email: [email protected]Email: [email protected]
Model-data fusion for the coupled carbon-water system Cathy Trudinger, Michael Raupach, Peter Briggs CSIRO Marine and Atmospheric Research, Australia and
Model-data fusion for the coupled carbon-water system Cathy
Trudinger, Michael Raupach, Peter Briggs CSIRO Marine and
Atmospheric Research, Australia and Peter Rayner LSCE, France
Email: [email protected]
Slide 2
Outline Model-data fusion (= data assimilation + parameter
estimation) Model-data fusion (= data assimilation + parameter
estimation) Parameter estimation with the Kalman filter Parameter
estimation with the Kalman filter Australian Water Availability
Project Australian Water Availability Project OptIC project
Optimisation Intercomparison OptIC project Optimisation
Intercomparison
Slide 3
Model-data fusion Model: - Process representation - Subjective,
incomplete - Capable of interpolation & forecast Observations:
- Real world representation - Incomplete, patchy - No forecast
capability Fusion: Optimal combination (involves model-obs mismatch
& strategy to minimise) Analysis: - Best of both worlds -
Identify model weaknesses - Forecast capability - Confidence
limits
Slide 4
Choices in model-data fusion Target variables what model
quantities to vary to match observations e.g. initial conditions,
model parameters, time-varying model quantities, forcing Target
variables what model quantities to vary to match observations e.g.
initial conditions, model parameters, time-varying model
quantities, forcing Cost function measure of misfit between
observations and corresponding model quantities e.g. J(targets) =
(H(targets) - obs) 2 + (targets - priors) 2 Cost function measure
of misfit between observations and corresponding model quantities
e.g. J(targets) = (H(targets) - obs) 2 + (targets - priors) 2
Fusion method - search strategy Fusion method - search strategy
Batch (non-sequential) e.g. down-gradient, global search Batch
(non-sequential) e.g. down-gradient, global search Sequential e.g.
Kalman filter Sequential e.g. Kalman filter Approach and issues
will differ to some extent between disciplines e.g. numerical
weather prediction vs terrestrial carbon cycle
Slide 5
The Ensemble Kalman filter Ensemble Kalman filter (EnKF)
sequential method that uses Monte Carlo techniques; error
statistics are represented using an ensemble of model states.
Ensemble Kalman filter (EnKF) sequential method that uses Monte
Carlo techniques; error statistics are represented using an
ensemble of model states. Initial ensemble Update using measurement
t0t0 t1t1 t2t2 Time: Two steps: Two steps: 1. Model used to predict
from one time to next 2. Update using observation Model
predicts
Slide 6
Augmented state vector to be estimated contains Augmented state
vector to be estimated contains Time-dependent model variables
Time-dependent model variables Time-independent model parameters
Time-independent model parameters State vector estimate at any time
is due to observations up to that time State vector estimate at any
time is due to observations up to that time Parameter estimation
with the Ensemble Kalman filter
Slide 7
Our component of Australian Water Availability project: develop
a Hydrological and Terrestrial Biosphere Data Assimilation System
for Australia OBSERVATIONS NDVI NDVI Monthly river flows Monthly
river flows Weather: rainfall, solar radiation, temperature
Weather: rainfall, solar radiation, temperatureMODEL Soil moisture
Soil moisture Leaf carbon Leaf carbon Water fluxes Water fluxes
Carbon fluxes Carbon fluxes MODEL-DATA FUSION Ensemble Kalman
Filter Ensemble Kalman Filter Down-gradient method (LM)
Down-gradient method (LM) Analysis of past, present and future
water and carbon budgets Analysis of past, present and future water
and carbon budgets Maps of soil moisture, vegetation growth Maps of
soil moisture, vegetation growth Process understanding Process
understanding Drought assessments, national water balance Drought
assessments, national water balance PRIOR INFORMATION Initial
parameter estimates Initial parameter estimates Soil, vegetation
types Soil, vegetation types
Slide 8
AWAP- Dynamic Model and Observation Model State variables (x)
and dynamic model State variables (x) and dynamic model Dynamic
model is of general form dx/dt = F (x, u, p) Dynamic model is of
general form dx/dt = F (x, u, p) All fluxes (F) are functions F (x,
u, p) = F (state vector, met forcing, params) All fluxes (F) are
functions F (x, u, p) = F (state vector, met forcing, params)
Governing equations for state vector x = (W, C L ): Governing
equations for state vector x = (W, C L ): Soil water W: Leaf carbon
C L : Observations (z) and observation model Observations (z) and
observation model NDVI = func(C L ) NDVI = func(C L ) Catchment
discharge = average of F WR + F WD [- extraction - river loss]
Catchment discharge = average of F WR + F WD [- extraction - river
loss] State vector in EnKF: x = [W, C L, NDVI, Dis, params] State
vector in EnKF: x = [W, C L, NDVI, Dis, params] Timestep = 1 day
Spatial resolution = 5x5 km
JFMAMJJASOND 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97
98 99 00 01 02 03 04 05 Murrumbidgee Relative Soil Moisture (0 to
1) (Forward run with priors, no assimilation)
Slide 11
Predicted and observed discharge 11 unimpaired catchments in
Murrumbidgee basin 25-year time series: Jan 1981 to December 2005
(Forward run with priors, no assimilation)
Slide 12
Model-data synthesis approach: - State and parameter estimation
with the EnKF - Assimilate NDVI and monthly catchment discharge Why
Kalman filter? - Can account for model error (stochastic component)
- Consistent statistics (uncertainty analysis) - Forecast
capability (with uncertainty) Issues: - Time-averaged observations
in EnKF (e.g. monthly catchment discharge) - Specifying statistical
model (model and observation errors) - KF (sequential) vs batch
parameter estimation methods? (using Levenberg-Marquardt method;
also OptIC project)
Slide 13
Estimated parameters Monthly mean discharge/runoff Preliminary
results: Adelong Creek Blue = Ensemble Kalman filter (sequential)
Red = Levenberg-Marquardt (PEST) (batch)
Slide 14
OptIC project Optimisation method intercomparison International
intercomparison of parameter estimation methods in biogeochemistry
International intercomparison of parameter estimation methods in
biogeochemistry Simple test model, noisy pseudo-data Simple test
model, noisy pseudo-data 9 participants submitted results 9
participants submitted results Methods used: Methods used:
Down-gradient (Levenberg-Marquardt, adjoint), Down-gradient
(Levenberg-Marquardt, adjoint), Sequential (extended Kalman filter,
ensemble Kalman filter) Sequential (extended Kalman filter,
ensemble Kalman filter) Global search (Metropolis, Metropolis MCMC,
Metropolis- Hastings MCMC). Global search (Metropolis, Metropolis
MCMC, Metropolis- Hastings MCMC).
Slide 15
where F(t) forcing (log-Markovian i.e. log of forcing is
Markovian) x 1 fast store x 2 slow store p 1, p 2 scales for effect
of x 1 and x 2 limitation of production k 1, k 2 decay rates for
pools s 0 seed production (constant value to prevent collapse)
OptIC model Estimate parameters p 1, p 2, k 1, k 2
Slide 16
Noisy pseudo-observations T1: Gaussian (G) T4: Gaussian but
noise in x 2 correlated with noise in x 1 (GC) T2: Log-normal (L)
T6: Gaussian with 99% of x 2 data missing (GM) T3: Gaussian +
temporally correlated (Markov) (GT) T5: Gaussian + drifts (GD)
Slide 17
Estimates divided by true parameters p1 p2 k1 k2
Slide 18
Cost function Some participants used cost functions with
weights, w i (t), that depended on each noisy observation z i
(t)
Slide 19
CodeMethodWeights LM1 Monte Carlo then Levenberg-Marquardt f(z
i (t)) LM1Rob As LM1, but ignore 2% highest summands in cost fn f(z
i (t)) LM2Levenberg-Marquardt0.01 LM3Levenberg-Marquardt Adj1
Down-gradient search using model adjoint 1.0 Adj2 sd(x) EKF
Extended Kalman filter (with parameters in state vector) sd(resids)
EnKF Ensemble Kalman filter (with parameters in state vector)
sd(resids) MetMetropolissd(resids) MetRob As Met but absolute
deviations not least squares sd(resids) MetMCMC Metropolis Markov
Chain Monte Carlo 1.0 MetMCMCq As MetMCMC but quadratic weights f(z
i 2 (t)) MH_MCMC Metropolis-Hastings Markov Chain Monte Carlo 1.0
Down-gradient Global-search KF w i (t) = f(z i (t)) less successful
than constant weights
Slide 20
Choice of cost function Evans (2003) review of parameter
estimation in biogeochemical models - it was hard to find two
groups of workers who made the same choice for the form of the
misfit function, with most of the differences being in the form of
the weights. Evans (2003) review of parameter estimation in
biogeochemical models - it was hard to find two groups of workers
who made the same choice for the form of the misfit function, with
most of the differences being in the form of the weights. Evans
(2003) and the OptIC project emphasise that the choice of cost
function matters, and should be made deliberately not by accident
or default. Evans (2003) and the OptIC project emphasise that the
choice of cost function matters, and should be made deliberately
not by accident or default. (Evans 2003, J. Marine Systems)
Slide 21
Optic project results Choice of cost function had large impact
on results Choice of cost function had large impact on results Most
troublesome noise types:- temporally correlated noise Most
troublesome noise types:- temporally correlated noise The Kalman
filter did as well as the batch methods The Kalman filter did as
well as the batch methods For more information on OptIc:
http://www.globalcarbonproject.org/ACTIVITIES/OptIC.htm For more
information on OptIc:
http://www.globalcarbonproject.org/ACTIVITIES/OptIC.htm