Assessment of flood predictive

Department of Earth and Geo-Environmental SciencesUniversity of Bologna

Assessment of flood predictive uncertainty in using meteorological ensembles to hydrological

forecasting

Prof. Ezio Todini

ISSAOS 2005, ISSAOS 2005, L’AquilaL’Aquila, August 29 , August 29 –– September 2, 2005 September 2, 2005


Flood emergency managers require simple decision rules to be applied in real time.

Unfortunately these rules are strongly affected by the uncertainty on the future evolution of events.

This implies the necessity of using forecasting tools with the aim of reducing their uncertainty on future events.

The basic problem


A Flood Forecasting System is a tool aimed at reducing uncertainty on

the future evolution of a flood event..!!!

Do we all agree on what isa Flood Forecasting System?

It is the reduction of uncertainty thatallows for more reliable decisions.


Expected value= Forecast

Probability of overflow

Leve

l

Costs

Cross section

An Example:The Flood Warning Problem


When using such forecasting tools, as pointed out by Krzysztofowicz (1999), it is essential to asses the uncertainty of the future state of the quantity of interest (level, discharge, etc.) conditional to its forecasted value in order to improve decisions.

This approach is a much simpler alternative to theoverwhelming computational effort required by a fullunconditional uncertainty assessment, which would imply marginalisation of the forecasting density with respect to all possible forecasting models, model parameters, initial states, input measurement and forecasting errors.

The big question: are we actually describing it?


( ) ( ) XdddIdMdXXMIXyfyf ˆ,,,,ˆ, ϑϑ∫∫ ∫ ∫ ∫=

( ) ( ) ( ) ( ) ( ) ( ) XdddIdMdXXfXfMfXMIfXMIfXMIXyf ˆˆ,,,,,,,ˆ ϑϑϑ∫∫ ∫ ∫ ∫=

The full unconditional uncertainty assessment

the quantity of interest, the predictand

the input data

the forecasted inputs

the model

the model parameters

the initial status====

==

I

MX

Xy

ϑ

ˆ

with


Different levels of conditional uncertainty 1/5

( )XMIXyf ,,,,ˆ ϑ

Although incorrect, it is quite common to estimate the model parameter values and to use theconditional density to represent

our predictive uncertainty.

The full conditional density

expresses our uncertainty conditional to the forecastprovided by a given model with given parameter values,given initial status, given inputs and given input forecasts.

ϑ( )XMIXyf ,,,ˆ,ˆ ϑ



( ) ( ) ( )∫= ϑϑϑ dXMIYfXMIXyfXMIYXyf ,,,",,,,ˆ,,,,ˆ

A more appropriate way is to derive a “posteriordensity” of the parameters using Bayesian Inference

using a set of past observations and tomarginalise with respect to the parameters, to obtain:

( )XMIYf ,,," ϑ Y



If a density representing the uncertainty of the forecasted inputs is available,then one can obtain:

( ) ( ) ( )[ ] ( ) ϑϑϑ dXMIYfXdXfXMIXyfXMIYyf ,,,"ˆˆ,,,,ˆ,,, ∫ ∫=


NOTE that, the previous slides show that no action is generally taken to eliminate the conditioning

- on the assumed model structure,

- on the initial value of the state variables,

- on the input measurement errors.



Different levels of conditional uncertainty 5/5As a matter of fact the model is generally taken as ourprior knowledge, concentrating all the uncertainty in itsparameter values.

The initial value of the state variables heavily affectsthe predictions, but its effect can be reduced using continuous-time models and starting at the end of dry periods.

Input measurement errors will certainly affect parameterestimation, but if they are not eccessive, they will not strongly affect the predictive uncertainty.


ASSESSING PARAMETER ESTIMATION ERRORIn order to overcome the problem of formulating “formal” likelihoods, correctly representing the statistical properties of the error terms Beven and Binley, (1992) introduced the Generalised Likelihood Uncertainty Estimation (GLUE), which follows in principle the Bayesian inference scheme, but uses “less formal likelihoods”as defined by the authors.


The introduction of less formal likelihood functions overcomes the need for formulating precise distribution functions for the observable variablesand/or for the errors in complex situations originated by the presence of many sources of errors, the complexity of the explicative models considered and the high number of parameters with which to build the learning process. Unfortunately, as it was shown by Mantovan and Todini (paper in preparation), the use of lessformal likelihood functions leads to paradoxical inferential results.


( )( ) ( )

( )⎪⎩

⎪⎨

⎧

>

≤⎥⎦

⎤⎢⎣

⎡−

=22

222

2

1

0

1

nn

nn

N

n

n

n

ss

sss

sL

ϑ

ϑϑϑ

( ) ( )[ ] 122 ≥=− NsL N

nn ϑϑ

( ) ( )[ ] 123 ≥−= NNsExpL nn ϑϑ

Simulation results using three well knownGLUE “less formal likelihoods”

Nash-Suttcliffe

Inverse error variance

Exponential

( ) ( )[ ]nnnn yyVars X,ˆ2 ϑϑ −=

[ ]nn yVars =2

where


A two dimensional first order autoregressive process based on two parameters, which known true value was set to ( ) ( )1,1, *

2*1 =ϑϑ

was generated with the addition of a non central Student-t distributed autoregressive noise. The samples were of length n=m=336.The results using the three non formal likelihoodsare compared with the ones obtained using the Gaussian (wrong, but in this case relatively robust)assumption

Synthetic data testing of GLUE


Bayesian Inference: Gaussian Likelihood


GLUE: Nash-Suttcliffe Likelihood


GLUE: Inverse Error Variance Likelihood


GLUE: Exponential Likelihood


Bayesian Inference: true likelihood(Posterior to Prior distance)


Less formal likelihoods (Posterior to Prior distance)

0 2 4 6 8 10 12 14 16 18 200

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04Figure 5. Distance between posterior and prior p.f.: Nash-Sutcliffe l.f.l..

Sample size (weeks)

Frob

eniu

s no

rm

5-th percentile 50-th percentile 95-th percentile


Bayesian Inference: true likelihood(Posterior expected square error loss)

0 2 4 6 8 10 12 14 16 18 200

0.02

0.04

0.06

0.08

0.1

0.12Figure 8. Posterior expected squared error loss: Exact likelihood.

Sample size (weeks)

Frob

eniu

s no

rm

5-th percentile 50-th percentile 95-th percentile


Less formal likelihoods (Posterior expected square error loss)


Similar results were found in the Po river case study (37,000 km2 ) by comparing:

- GLUE using a less formal likelihood based on Nash-Sutcliffe statistics

- Bayesian inference assuming a χ2

density for the sum of squared errors


GLUE: Nash-Sutcliffe


GLUE: Nash-Sutcliffe


Bayesian Inference: χ2


Bayesian Inference: χ2


GLUE: Nash-Sutcliffe (4 x 400)






Bayesian Inference: χ2 (4 x 400)






Another example using GLUE on a Chinese 10,000 km2 catchment


The Bayesian inference using the Normal Quantile Transform on the same Chinese catchment


If we are able of determining a relativelypeaky posterior density for the parameterswe can resonably question ourselves whetherthe use of a set of parameters can stronglydistort the estimation of the model predictive density. In other words shouldwe use the conditionalor the unconcitional density?

Inequifinality as opposed to equifinality

( )XMIXyf ,,,ˆ,ˆ ϑ

( ) ( ) ( )∫= ϑϑϑ dXMIYfXMIXyfXMIYXyf ,,,",,,,ˆ,,,,ˆ


Unconditional

Conditional

Predictive uncertainty in hindcast model(A Chinese catchment)

As one can see from this slide and from the following one, the difference between unconditional and conditional densities canbe relatively small if the parameter posterior density is dense around the modal value.


Predictive uncertainty in hindcast mode(A Chinese catchment)

Solid: UnconditionalDashed: Conditional


In order to provide the probability densityof the future values of our predictand (stage,discharge, etc.) conditional on our forecasts,we need to answer the following questions:

Which are the means we have to express the input (mostly future rainfall) predictive uncertainty?

How do we operate in practice ?

Predictive uncertainty in forecast mode


Presently, we can hardly make direct use of the members of the Ensemble Forecasts provided by the Numerical Weather Prediction models.

For instance, the followingexample on the Po river in Italy shows that while the“deteministic” run (dashed line)provides a relatively goodrainfall forecast, the “spaghetti”ensemble produces extremelybiased estimates (possibly dueto the coarser model mesh).Similar results obtained in project EFFS using different NWP models confirm this situation.


At NOAA a post-processor for rainfall productsgenerated by NWP models has been developed using the NQT and the ensemble mean(Slide: John Schaake).

Observed0

Y

X

Archived data

Noassumptionof normalityfor observed

& forecastdistributions

ZX

Joint distribution

ZY

zX0

zY0

Normal Space

NQTObserved

Forecast

For a given forecast

Normal Space

P(ZX zX0 | ZY = zY)

ObservedzX0 ZX

0

1

Conditional Distribution

PQPFgiven a

QPFInverse

NQT

NQT = Normal Quantile Transform


Ensemble Member Climatologies

0

0,2

0,4

0,6

0,8

1

0 10 20 30 40

Daily Precipitation (mm)

Prob

abili

ty ObsRawSyn

Which allows to correctly reproduce the future precipitation distribution (Slide: John Schaake).


John Shaake has shown that using the NQT and the ensemble mean it is possible to derive a probability density of future rainfall conditional to the ensemble mean, namely

In real time, one can then derive one conditional density for each member of the ensemble andmarginalise out the uncertainty caused by the precipitation forecasts.

( )XXf ˆ

( )XXf ˆ


NormalQuantileTransform

Gaussian Space

Original Space

The Normal QuantileTransform (NQT)

Both measured andmodeled values aretransformed into a Standard Normal spaceby quantile matching.In the Normal space the derivation of the conditional densities isrelatively simple. Theconditional densities are Then reconverted back into the original space.

Conversion of data in the Gaussian space


Conversion of data in the Gaussian space

The NQT fully preserves the rank correlation. Moreoverin the Gaussian space, the joint probability distributionof the observed and modelled variables is a multi-dimensional Gaussian distribution which densitycan be easily estimated.

In hindcast mode one can directly derive the predictive density, while in forecasting mode the predictive density can be derived following the work of Krzysztofowicz (1999).


y = 0,7403xR2 = 0,4958

-4

-3

-2

-1

0

1

2

3

4

5

-4 -3 -2 -1 0 1 2 3 4

y

y

The conditional density in hindcast mode

( )ikiy x,ϑˆ

( )iif ηη ˆ

η

η


If we accept the conditionality on the chosen model, onthe initial state and on the measurement errors,the problem of predictive uncertainty, can besolved by marginalising the joint density with respectto the parameters and to the forecasted input.Therefore, in practice the original equation

( ) ( ) ( )[ ] ( ) ϑϑϑ dXMIYfXdXfXMIXyfXMIYyf ,,,"ˆˆ,,,,ˆ,,, ∫ ∫=

must be discretised to be used with an ensemble ofparameter values drawn from the parameter space and an ensemble of rainfall forecasts produced bythe NWP models.


One possible approach can thus be summarised as follows:

1) Assume a multi-uniform prior on the parameters;

2) Generate, a large number of parameter setsby means of a Monte Carlo approach;

3) Estimate the posterior density of the parametersusing the Bayesian Inference process

on historical records;

4) Estimate, for each generated parameter set, itsposterior probability of occurrence ;

si Ni ,1=∀ϑ si Ni ,1=∀ϑ

si Ni ,1=∀ϑsN

( )XMIYf ,,," ϑ

( )ϑ'f

( )XMIYf i ,,," ϑ


5) For each ensemble member and each parameter setestimate the predictive density ;

6) Marginalise out the ensemble uncertainty by uniformlyweighting the ensembles , with

the number of ensemble member used;

7) Marginalise out the parameter uncertainty:

( )XMIXyf ij ,,,,ˆ ϑ

( ) ( )∑=

=eN

jij

ei XMIXyf

NXMIyf

1,,,,ˆ1,,, ϑϑ

eN

eN

( ) ( ) ( )∑=

=sN

iii XMIYfXMIyfXMIYyf

1,,,",,,,,, ϑϑ


One has to realise that the required computationaleffort can be quite substantial given that Ne ,the numberof ensemble members is generally around 50 and Ns the number of parameter sets is generally of the order of several thousands (t10000).

Therefore, given that one must repeat the procedurefor each future value, two alternatives can be conceived,that can reduce the computational effort as well as therequired computation time.

Possible alternatives


The first alternative, similarly to what is done in meteorology, is to cluster the parameter sets in groups (possibly into equiprobable groups) and use a representative set per each group. The number of these groups could be of the order of 100, thusreducing by two orders of magnitude the computational burden.

Alternative 1


If one can prove the substantial coincidence of the predictive density marginalised with respect to the parameters and of the one conditional to an estimatedparameter set value (for instance a ML estimated value)as in the Chinese catchment case, it is possible to furtherreduce the computational effort by repeting steps 5 and 6as follows.

Alternative 2


5) For each ensemble member estimate the predictivedensity ;

6) Marginalise out the ensemble uncertainty by uniformlyweighting the ensembles , with

the number of ensemble member used;

eN

eN

( )XMIXyf j ,,,ˆ,ˆ ∗ϑ

( ) ( )∑=

∗∗ =eN

jj

e

XMIXyfN

XMIyf1

,,,ˆ,ˆ1,,,ˆ ϑϑ

Alternative 2


CONCLUDING REMARKSThe objective of this lecture was not to propose solutions, rather to set forth the perception of the main unresolvedproblems in the assessment of Real Time FloodForecasting Uncertainty and to propose and discuss a number of possible alternative approaches.

It must be clear that these are ideas and research linesfollowing which we hope to find the appropriate solutionswithin the frame of HEPEX and of a number of Europeanfunded research projects.

Documents

Assessment of flood predictive