Transcript
Page 1: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

TheNa'onalCenterforAtmosphericResearchissponsoredbytheNa'onalScienceFounda'on.Anyopinions,findingsandconclusionsorrecommenda'onsexpressedinthispublica'onarethoseoftheauthor(s)anddonotnecessarilyreflecttheviewsoftheNa'onalScienceFounda'on.

©UCAR

DARTTutorialSec'on12:Adap'veInfla'on

Page 2: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

1.Forobservedvariable,havees'mateofprior-observedinconsistency.2.Expected(prior_mean–observa'on)=

Assumesthatpriorandobserva'onaresupposedtobeunbiased.Isitmodelerrororrandomchance?

−4 −2 0 2 40

0.2

0.4

0.6

0.8

Prob

abilit

y

Prior PDF

S.D.

Obs. Likelihood

S.D.Expected Separation

Actual 4.714 SDs

σprior2 +σobs

2

DARTTutorialSec'on12:Slide2

Page 3: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

1.Forobservedvariable,havees'mateofprior-observedinconsistency.2.Expected(prior_mean–observa'on)=3.Infla'ngincreasesexpectedsepara'on

Increases‘apparent’consistencybetweenpriorandobserva'on.

−4 −2 0 2 40

0.2

0.4

0.6

0.8

Prob

abilit

y

Prior PDF Obs. Likelihood

S.D.Inflated S.D. Expected Separation

Actual 3.698 SDs

σprior2 +σobs

2

DARTTutorialSec'on12:Slide3

Page 4: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

DistanceDfrompriormeanytoobsisProby0isobservedgivenλ:

−4 −2 0 2 40

0.2

0.4

0.6

0.8

Prob

abilit

y

Prior PDF Obs. Likelihood

S.D.Inflated S.D. Expected Separation

Actual 3.698 SDs

N 0, λσprior

2 +σobs2( )= N 0,θ( )

p yo |λ( )= 2πθ2( )−1 2 exp −D2 2θ2( )

DARTTutorialSec'on12:Slide4

Page 5: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

0 1 2 3 4 5 60

1

2

Obs. Space Inflation Factor: λ

Prior λ PDF

−1 0 1 2 3 40

0.2

0.4

0.6

Observation: y

Obs. LikelihoodPrior PDF

UseBayesiansta's'cstogetes'mateofinfla'onfactorλ.

AssumepriorisGaussian: p λ,tk |Ytk−1( )= N λp ,σλ,p2( )

DARTTutorialSec'on12:Slide5

Page 6: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

0 1 2 3 4 5 60

1

2

Obs. Space Inflation Factor: λ

Prior λ PDF

−1 0 1 2 3 40

0.2

0.4

0.6

Observation: y

Obs. LikelihoodPrior PDF

UseBayesiansta's'cstogetes'mateofinfla'onfactorλ.

We’veassumedaGaussianforprior.RecallthatcanbeevaluatedFromnormalPDF.

p λ,tk |Ytk−1( )

p yk |λ( )

p λ,tk |Ytk( )= p yk |λ( )p λ,tk |Ytk−1( ) / normalizationDARTTutorialSec'on12:Slide6

Page 7: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

0 1 2 3 4 5 60

1

2

Obs. Space Inflation Factor: λ

Prior λ PDF

−1 0 1 2 3 40

0.2

0.4

0.6

Observation: y

Obs. LikelihoodInflated Prior λ = 0.75

UseBayesiansta's'cstogetes'mateofinfla'onfactorλ.

GetfromnormalPDF.Mul'plybytoget

p λ,tk |Ytk( )= p yk |λ( )p λ,tk |Ytk−1( ) / normalization

p yk |λ= 0.75( )

p λ= 0.75,tk |Ytk−1( )

p λ= 0.75,tk |Ytk( )

DARTTutorialSec'on12:Slide7

Page 8: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

0 1 2 3 4 5 60

1

2

Obs. Space Inflation Factor: λ

Prior λ PDF

−1 0 1 2 3 40

0.2

0.4

0.6

Observation: y

Obs. LikelihoodInflated Prior λ = 1.5

UseBayesiansta's'cstogetes'mateofinfla'onfactorλ.

GetfromnormalPDF.Mul'plybytoget

p λ,tk |Ytk( )= p yk |λ( )p λ,tk |Ytk−1( ) / normalization

p yk |λ=1.50( )

p λ=1.50,tk |Ytk−1( )

p λ=1.50,tk |Ytk( )

DARTTutorialSec'on12:Slide8

Page 9: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

0 1 2 3 4 5 60

1

2

Obs. Space Inflation Factor: λ

Prior λ PDF

−1 0 1 2 3 40

0.2

0.4

0.6

Observation: y

Obs. LikelihoodInflated Prior λ = 2.25

UseBayesiansta's'cstogetes'mateofinfla'onfactorλ.

GetfromnormalPDF.Mul'plybytoget

p λ,tk |Ytk( )= p yk |λ( )p λ,tk |Ytk−1( ) / normalization

p yk |λ= 2.25( )

p λ= 2.25,tk |Ytk−1( )

p λ= 2.25,tk |Ytk( )

DARTTutorialSec'on12:Slide9

Page 10: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

0 1 2 3 4 5 60

1

2

Obs. Space Inflation Factor: λ

Prior λ PDF

Likelihood y observed given λPosterior

−1 0 1 2 3 40

0.2

0.4

0.6

Observation: y

Obs. Likelihood

UseBayesiansta's'cstogetes'mateofinfla'onfactorλ.

Repeatforarangeofvaluesofλ.Nowmustgetposteriorinsameformasprior(Gaussian).

p λ,tk |Ytk( )= p yk |λ( )p λ,tk |Ytk−1( ) / normalizationDARTTutorialSec'on12:Slide10

Page 11: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

1 20

1

2

Obs. Space Inflation Factor: λ

Prior λ PDF

Posterior

−1 0 1 2 3 40

0.2

0.4

0.6

Observation: y

Obs. Likelihood

UseBayesiansta's'cstogetes'mateofinfla'onfactorλ.

Verylicleinforma'onaboutλinasingleobserva'on.Posteriorandpriorareverysimilar.Normalizedposteriorindis'nguishablefromprior.

p λ,tk |Ytk( )= p yk |λ( )p λ,tk |Ytk−1( ) / normalizationDARTTutorialSec'on12:Slide11

Page 12: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

1 2

−0.01

0

0.01

Obs. Space Inflation Factor: λ

λ: Posterior − Prior

Max density shifted to right

−1 0 1 2 3 40

0.2

0.4

0.6

Observation: y

Obs. Likelihood

UseBayesiansta's'cstogetes'mateofinfla'onfactorλ.

Verylicleinforma'onaboutλinasingleobserva'on.Posteriorandpriorareverysimilar.Differenceshowsslightshiftolargervaluesofλ.

p λ,tk |Ytk( )= p yk |λ( )p λ,tk |Ytk−1( ) / normalizationDARTTutorialSec'on12:Slide12

Page 13: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

1 20

1

2

Obs. Space Inflation Factor: λ

Prior λ PDFFind Max by search

Max is new λ mean

−1 0 1 2 3 40

0.2

0.4

0.6

Observation: y

Obs. Likelihood

UseBayesiansta's'cstogetes'mateofinfla'onfactorλ.

Oneop'onistouseGaussianpriorforλ.Selectmax(mode)ofposteriorasmeanofupdatedGaussian.Doafitforupdatedstandarddevia'on.

p λ,tk |Ytk( )= p yk |λ( )p λ,tk |Ytk−1( ) / normalizationDARTTutorialSec'on12:Slide13

Page 14: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

A.  Compu'ngupdatedinfla'onmean,.

Modeofcanbefoundanaly'cally!Solvingleadsto6thorderpolyinθ.

Thiscanbereducedtoacubicequa'onandsolvedtogivemode.Newissettothemode.Thisisrela'velycheapcomparedtocompu'ngregressions.

λu

p yk |λ( )p λ,tk |Ytk−1( )

∂ p yk |λ( )p λ,tk |Ytk−1( )⎡⎣⎢

⎤⎦⎥ /∂y= 0

λu

DARTTutorialSec'on12:Slide14

Page 15: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Varianceinfla'onforobserva'ons:Anadap'veerrortolerantfilter

B.Compu'ngupdatedinfla'onvariance,.

1.Evaluatenumeratoratmeanandsecondpoint,e.g.

2. Findsogoesthroughand

3. Computeaswhere

σλ,u2

λu λu +σλ,p

σλ,u2

N λu ,σλ,u2( )

p λu( ) p λu +σλ,p( )

σλ,u2 =−σλ,p

2 / 2 ln r r= p λu +σλ,p( ) / p λu( )

DARTTutorialSec'on12:Slide15

Page 16: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Observa'onSpaceComputa'onswithAdap'veErrorCorrec'on

−4 −2 0 2 40

0.2

0.4

0.6

Prob

abilit

y

Prior PDFObs. Likelihood

1.  Computeupdatedinfla'ondistribu'on,.

p λ,tk |Ytk( )

DARTTutorialSec'on12:Slide16

Page 17: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Observa'onSpaceComputa'onswithAdap'veErrorCorrec'on

−4 −2 0 2 40

0.2

0.4

0.6

Prob

abilit

y Obs. Likelihood

1.  Computeupdatedinfla'ondistribu'on,.2.  Inflateensembleusingmeanofupdatedλdistribu'on.

DARTTutorialSec'on12:Slide17

p λ,tk |Ytk( )

Page 18: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Observa'onSpaceComputa'onswithAdap'veErrorCorrec'on

−4 −2 0 2 40

0.2

0.4

0.6

Prob

abilit

y Obs. Likelihood

1.  Computeupdatedinfla'ondistribu'on,.2.  Inflateensembleusingmeanofupdatedλdistribu'on.3.  Computeposteriorforyusinginflatedprior.

DARTTutorialSec'on12:Slide18

p λ,tk |Ytk( )

Page 19: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Observa'onSpaceComputa'onswithAdap'veErrorCorrec'on

−4 −2 0 2 40

0.2

0.4

0.6

Prob

abilit

y Obs. Likelihood

1.  Computeupdatedinfla'ondistribu'on,.2.  Inflateensembleusingmeanofupdatedλdistribu'on.3.  Computeposteriorforyusinginflatedprior.4.  ComputeincrementsfromORIGINALpriorensemble.

DARTTutorialSec'on12:Slide19

p λ,tk |Ytk( )

Page 20: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Adap'veObserva'onSpaceInfla'oninDART

Observa'onspaceadap'veinfla'onisnotsupportedinDARTManhacanrelease

DARTTutorialSec'on12:Slide20

Page 21: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Adap'veObserva'onSpaceInfla'oninDART

TrythisinLorenz96(verifyotheraspectsofinput.nml).Use40memberensemble.(setens_size=40infilter_nml).Setredvaluesasaboveforadap'veobserva'onspaceinfla'on.

inf_flavor =1, 0,inf_ini'al_from_restart =.false., .false.,inf_sd_ini'al_from_restart =.false., .false.,inf_determinis'c =.true., .true.,inf_diag_file_name ='prior_inflate_diag', 'post_inflate_diag',inf_ini'al =1.00, 1.0,inf_sd_ini'al =0.2, 0.0,inf_damping =1.0, 1.0,inf_lower_bound =1.0, 1.0,inf_upper_bound =1000000.0, 1000000.0,inf_sd_lower_bound =0.0, 0.0,

BeforeAssimila'on

AferAssimila'on

Flavor: 0=>NONE 1=>notsupported 3=>physicalspace

Ini'alinfla'onvalueIni'alstandarddevia'on

Lowerboundons.d.

Observa'onspaceadap'veinfla'onisnotsupportedinDARTManhacanrelease

DARTTutorialSec'on12:Slide21

priorinfla'on

posteriorinfla'on

Page 22: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Adap'veObserva'onSpaceInfla'oninDART

Runthefilter.Examineperformancewithplot_total_errinMatlab.Timeseriesofinfla'onandstandarddevia'onareinprior_inflate_diag.Infla'onadjustswith'me.Standarddevia'onisnon-increasing.

DARTTutorialSec'on12:Slide22

Page 23: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

AlgorithmicVariants

1.Increaseprioryvariancebyaddingrandomgaussiannoise.Asopposedto‘determinis'c’linearinfla'ng.Setinf_determinis9cinfirstcolumnto.false.Changeitbackto.true.afercheckingthisout.

2.Justhaveafixedvalueforobs.spaceλ.

Cheap,handlesblowupofstatevarsunconstrainedbyobs.Wealreadytriedthisinsec'on9.

DARTTutorialSec'on12:Slide23

Page 24: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

AlgorithmicVariants

3.Fixvalueofλstandarddevia'on,.Reducescost,computa'onofcansome'mesbetricky.Avoidsgepngsmall(errormodelfilterdivergence,Yikes!).Havetohavesomeintui'onaboutthevaluefor.Thisappearstobemostviableop'onforlargemodels.Valuesof=0.05to0.10workforverybroadrangeofproblems.Thisisasamplingerrorclosureproblem(akintoturbulence).

Tofix:

Setinflate_sd_ini9altofixedvalue,forinstance0.10,Setinflate_sd_lower_boundtosamevalue. (s.d.can’tgetanysmaller).

Trythisinlorenz_96.Lookathowtheinfla'onvaries.

σλ σλ

σλ σλ

σλ

σλ

DARTTutorialSec'on12:Slide24

Page 25: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Poten'alproblemswithobserva'onspaceadap'veinfla'on

1.Veryheuris'c.

2.Errormodelfilterdivergence(precyhardtothinkabout).3.Equilibra'onproblems,oscilla'onsinλwith'me.4.Notclearthatsingledistribu'onforallobserva'onsisright.5.Amplifyingunwantedmodelresonances(gravitywaves)

DARTTutorialSec'on12:Slide25

Page 26: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Poten'alproblemswithobserva'onspaceadap'veinfla'on

1.Veryheuris'c.

2.Errormodelfilterdivergence(precyhardtothinkabout).3.Equilibra'onproblems,oscilla'onsinλwith'me.4.Notclearthatsingledistribu'onforallobserva'onsisright.5.Amplifyingunwantedmodelresonances(gravitywaves)

Tryturningthisonin9varmodel.Fixed0.05forinf_sd_ini9al,sd_lower_bound.

DARTTutorialSec'on12:Slide26

Page 27: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Adap'veStateSpaceInfla'onAlgorithm

Supposewewantaglobalstatespaceinfla'on,λs,instead.Makesameleastsquaresassump'onthatisusedinensemblefilter.Infla'onofλsforstatevariablesinflatesobs.priorsbysameamount.Getsamelikelihoodasbefore:Computeupdateddistribu'onforλsexactlyasforobserva'onspace.

p yo |λ( )= 2πθ2( )−1 2 exp −D2 2θ2( )

θ= λsσprior2 +σobs

2

DARTTutorialSec'on12:Slide27

Page 28: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Implementa'onofAdap'veStateSpaceInfla'onAlgorithm

1.Applyinfla'ontostatevariableswithmeanofλsdistribu'on.2.Dofollowingforobserva'onsatgiven'mesequen'ally:

a.Computeforwardoperatortogetpriorensemble.b.Computeupdatedes'mateforλsmeanandvariance.c.Computeincrementsforpriorensemble.d.Regressincrementsontostatevariables.

DARTTutorialSec'on12:Slide28

Page 29: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

inf_flavor =3, 0,inf_ini'al_from_restart =.false., .false.,inf_sd_ini'al_from_restart =.false., .false.,inf_determinis'c =.true., .true.,inf_ini'al =1.00, 1.0,inf_sd_ini'al =0.2, 0.0,inf_damping =1.0, 1.0,inf_lower_bound =1.0, 1.0,inf_upper_bound =1000000.0, 1000000.0,inf_sd_lower_bound =0.0, 0.0,

BeforeAssimila'on

AferAssimila'on

Flavor: 0=>NONE 2=>varyingstatespace 3=>constantstatespace

Ini'alinfla'onvalueIni'alstandarddevia'on

TrythisinLorenz96(verifyotheraspectsofinput.nml).Use40memberensemble.(setens_size=40in&filter_nml).Setredvaluesasaboveforadap'vespa'ally-constantstatespaceinfla'on.

Lowerboundons.d.

Experimen'ngwithspa'ally-constantstatespaceinfla'on

DARTTutorialSec'on12:Slide29

priorinfla'on

posteriorinfla'on

Page 30: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Experimen'ngwithspa'ally-constantstatespaceinfla'on

Runthefilter:Examineperformancewithplot_total_errinMatlab.

Timeseriesofinfla'onmeanandstandarddevia'onareinpreassim.ncfile:•  Thiscanbeviewedwithncview(moreonthislater).•  Infla'onadjustswith'me.•  Infla'onstandarddevia'onisnon-increasingwith'me.•  Thisfilealsohas'meseriesoftheensemble.Finalvaluesofinfla'onforrestartareinfilter_output.ncfile.

DARTTutorialSec'on12:Slide30

Page 31: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Adap'veInfla'onAlgorithmicVariants

1.IncreasepriorstatevariancebyaddingrandomGaussiannoise.Asopposedto‘determinis'c’linearinfla'ng.Setinf_determinis9cinfirstcolumnto.false.Changeitbackto.true. afercheckingthisout.

2.Justhaveafixedvalueforstatespaceλ.

Constantinspaceand'me.Cheap,handlesblowupofstatevarsunconstrainedbyobs.Wealreadytriedthisinsec'on9.Setinflate_sd_ini9alandinf_sd_lower_boundto0.Setinf_ini9altodesiredinfla'onvalue.

DARTTutorialSec'on12:Slide31

Page 32: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Adap'veInfla'onAlgorithmicVariants

3.Fixvalueofλstandarddevia'on,.Reducescost,computa'onofcansome'mesbetricky.Avoidsgepngsmall(errormodelfilterdivergence,Yikes!).Havetohavesomeintui'onaboutthevaluefor.Thisappearstobemostviableop'onforlargemodels.Valuesof=0.10to0.60workforverybroadrangeofproblems.Thisisasamplingerrorclosureproblem(akintoturbulence).

Tofix:

Setinflate_sd_ini9altofixedvalue,forinstance0.20,Setinflate_sd_lower_boundtosamevalue. (s.d.can’tgetanysmaller).

TrythisinLorenz96.Lookathowtheinfla'onvaries.

σλ σλ

σλ σλ

σλ

σλ

DARTTutorialSec'on12:Slide32

Page 33: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Adap'veInfla'onAlgorithmicVariants

4.Infla'ondampingInfla'onmeandampedtowards1.0everyassimila'on'me.Setbynamelistentryinf_damping.inf_damping=0.9:90%oftheinfla'ondifferencefrom1.0isretained.

Canbeusefulinmodelswithheterogeneousobserva'onsin'me.Forinstance,awell-observedhurricanecrossesamodeldomain.Adap'veinfla'onincreasesalonghurricanetrace.Aferhurricane,fewerobserva'ons,nolongerneedsomuchinfla'on.Forlargeearthsystemmodels,followingvaluesmaywork:

inf_sd_ini9al =0.6,inf_damping =0.9,inf_sd_lower_bound =0.6.

DARTTutorialSec'on12:Slide33

Page 34: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Simula'ngModelErrorin40-VariableLorenz96Model

Infla'oncandealwithallsortsoferrors,includingmodelerror.CansimulatemodelerrorinLorenz96bychangingforcing.Synthe'cobserva'onsarefrommodelwithforcing=8.0.Useforcinginmodel_nmltointroducemodelerror.

Tryforcingvaluesof7,6,5,3withandwithoutadap'veinfla'on.TheF=3modelisperiodic,looksveryliclelikeF=8.

DARTTutorialSec'on12:Slide34

Page 35: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Simula'ngModelErrorin40-VariableLorenz96Model

40statevariables:X1,X2,...,XN.dXi/dt=(Xi+1-Xi-2)Xi-1-Xi+F;i=1,...,40withcyclicindices.UseF=8.0,4th-orderRunge-Kucawithdt=0.05.

TimeseriesofstatevariablefromfreeLorenz96integra'on

Anderson: Ensemble Tutorial 26 12/4/07

Simulating Model Error in 40-Variable Lorenz-96 Model

40 state variables: X1, X2,..., XNdXi / dt = (Xi+1 - Xi-2)Xi-1 - Xi + F;i = 1,..., 40 with cyclic indicesUse F = 8.0, 4th-order Runge-Kutta with dt=0.05

0 50 100 150 200 250 300 350 400−8

−6

−4

−2

0

2

4

6

8

10

12

TIME

X(1)

Time series ofstate variablefrom free L96integration

DARTTutorialSec'on12:Slide35

Page 36: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Experimentaldesign:Lorenz96ModelErrorSimula'on

Truthandobserva'onscomesfromlongrunwithF=8

200randomlylocated(fixedin'me)‘observingloca'ons’

Independent1.0observa'onerrorvariance

Observa'onseveryhour

σλis0.05,meanofλadjustsbutvarianceisfixed

4groupsof20memberseach(80ensemblememberstotal)

Resultsfrom10daysafer40dayspin-up

Varyassimila'ngmodelforcing:F=8,6,3,0

Simulatesincreasingmodelerror

DARTTutorialSec'on12:Slide36

Page 37: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Assimila'ngF=8TruthwithF=8Ensemble

Anderson: Ensemble Tutorial 28 12/4/07

Assimilating F=8 Truth with F=8 EnsembleModel time series Mean value of λ

Assimilation Results

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 0 5 101

1.005

1.01

model time (pseudo−days)

F=8

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Anderson: Ensemble Tutorial 28 12/4/07

Assimilating F=8 Truth with F=8 EnsembleModel time series Mean value of λ

Assimilation Results

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 0 5 101

1.005

1.01

model time (pseudo−days)

F=8

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Anderson: Ensemble Tutorial 28 12/4/07

Assimilating F=8 Truth with F=8 EnsembleModel time series Mean value of λ

Assimilation Results

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 0 5 101

1.005

1.01

model time (pseudo−days)

F=8

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Assimila'onResults

model'me(pseudo−days)

model'me(pseudo−days)

Model'meseries Meanvalueofλ

DARTTutorialSec'on12:Slide37

Page 38: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Anderson: Ensemble Tutorial 29 12/4/07

Assimilating F=8 Truth with F=6 EnsembleModel time series Mean value of λ

Assimilation Results

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 F=6 0 5 101

1.5

model time (pseudo−days)

F=6 F=8

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Anderson: Ensemble Tutorial 29 12/4/07

Assimilating F=8 Truth with F=6 EnsembleModel time series Mean value of λ

Assimilation Results

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 F=6 0 5 101

1.5

model time (pseudo−days)

F=6 F=8

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Anderson: Ensemble Tutorial 29 12/4/07

Assimilating F=8 Truth with F=6 EnsembleModel time series Mean value of λ

Assimilation Results

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 F=6 0 5 101

1.5

model time (pseudo−days)

F=6 F=8

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Assimila'ngF=8TruthwithF=6Ensemble

Assimila'onResults

model'me(pseudo−days)

model'me(pseudo−days)

Model'meseries Meanvalueofλ

DARTTutorialSec'on12:Slide38

Page 39: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Anderson: Ensemble Tutorial 30 12/4/07

Assimilating F=8 Truth with F=3 EnsembleModel time series Mean value of λ

Assimilation Results

0 5 101

1.5

model time (pseudo−days)

F=3 F=6

F=8

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 F=6 F=3

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Anderson: Ensemble Tutorial 30 12/4/07

Assimilating F=8 Truth with F=3 EnsembleModel time series Mean value of λ

Assimilation Results

0 5 101

1.5

model time (pseudo−days)

F=3 F=6

F=8

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 F=6 F=3

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Anderson: Ensemble Tutorial 30 12/4/07

Assimilating F=8 Truth with F=3 EnsembleModel time series Mean value of λ

Assimilation Results

0 5 101

1.5

model time (pseudo−days)

F=3 F=6

F=8

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 F=6 F=3

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Assimila'ngF=8TruthwithF=3Ensemble

Assimila'onResults

model'me(pseudo−days)

model'me(pseudo−days)

Model'meseries Meanvalueofλ

DARTTutorialSec'on12:Slide39

Page 40: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Anderson: Ensemble Tutorial 31 12/4/07

Assimilating F=8 Truth with F=0 EnsembleModel time series Mean value of λ

Assimilation Results

Prior RMS Error, Spread, and λ Grow as Model Error Grows

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 F=6 F=3 F=0 0 5 101

1.5

model time (pseudo−days)

F=0 F=3

F=6 F=8

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Anderson: Ensemble Tutorial 31 12/4/07

Assimilating F=8 Truth with F=0 EnsembleModel time series Mean value of λ

Assimilation Results

Prior RMS Error, Spread, and λ Grow as Model Error Grows

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 F=6 F=3 F=0 0 5 101

1.5

model time (pseudo−days)

F=0 F=3

F=6 F=8

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Anderson: Ensemble Tutorial 31 12/4/07

Assimilating F=8 Truth with F=0 EnsembleModel time series Mean value of λ

Assimilation Results

Prior RMS Error, Spread, and λ Grow as Model Error Grows

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 F=6 F=3 F=0 0 5 101

1.5

model time (pseudo−days)

F=0 F=3

F=6 F=8

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Assimila'ngF=8TruthwithF=0Ensemble

Assimila'onResults

model'me(pseudo−days)

model'me(pseudo−days)

Model'meseries Meanvalueofλ

DARTTutorialSec'on12:Slide40

Page 41: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Anderson: Ensemble Tutorial 31 12/4/07

Assimilating F=8 Truth with F=0 EnsembleModel time series Mean value of λ

Assimilation Results

Prior RMS Error, Spread, and λ Grow as Model Error Grows

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 F=6 F=3 F=0 0 5 101

1.5

model time (pseudo−days)

F=0 F=3

F=6 F=8

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Anderson: Ensemble Tutorial 31 12/4/07

Assimilating F=8 Truth with F=0 EnsembleModel time series Mean value of λ

Assimilation Results

Prior RMS Error, Spread, and λ Grow as Model Error Grows

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 F=6 F=3 F=0 0 5 101

1.5

model time (pseudo−days)

F=0 F=3

F=6 F=8

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Anderson: Ensemble Tutorial 31 12/4/07

Assimilating F=8 Truth with F=0 EnsembleModel time series Mean value of λ

Assimilation Results

Prior RMS Error, Spread, and λ Grow as Model Error Grows

0 5 10−5

0

5

10

model time (pseudo−days)

F=8 F=6 F=3 F=0 0 5 101

1.5

model time (pseudo−days)

F=0 F=3

F=6 F=8

0 5 10−10

0

10

True State Ensemble Mean Ensemble Members

Assimila'ngF=8TruthwithF=0Ensemble

Assimila'onResults

model'me(pseudo−days)

model'me(pseudo−days)

Model'meseries Meanvalueofλ

PriorRMSError,Spread,andλGrowasModelErrorGrows

DARTTutorialSec'on12:Slide41

Page 42: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Anderson: Ensemble Tutorial 32 12/4/07

Base case: 200 randomly located observations per time

Assimilating Model Forcing, F Assim. Forcing, F(Error saturation is approximately 30.0)

Prior RMS Error, Spread, and λ Grow as Model Error Grows

0 101

1.2

1.4

1.6

1.8

0 5 10 150

2

4

6RMS ErrorSpread

Basecase:200randomlylocatedobserva'onsper'me

Assimila'ngModelForcing,F Assim.Forcing,F

(Errorsatura'onisapproximately30.0)PriorRMSError,Spread,andλGrowasModelErrorGrows

Infla'on

DARTTutorialSec'on12:Slide42

Page 43: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Anderson: Ensemble Tutorial 33 12/4/07

Less well observed case, 40 randomly located observations per time

Assimilating Model Forcing, F Assim. Forcing, F0 5 10 151

1.5

2

2.5

3

3.540 Obs.200 Obs.

0 5 10 150

5

10

15

20 RMS ErrorSpreadRMS 40 ObsSpread 40 Obs

Lesswellobservedcase,40randomlylocatedobsper'me

Assimila'ngModelForcing,F Assim.Forcing,F

DARTTutorialSec'on12:Slide43

Page 44: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Spa'allyvaryingadap'veinfla'onalgorithm

Haveadistribu'onforλforeachstatevariable,λs,i.Usepriorcorrela'onfromensembletodetermineimpactofλs,ionpriorvarianceforgivenobserva'on.Ifγiscorrela'onbetweenstatevariableiandobserva'onthenEqua'onforfindingmodeofposteriorisnowfull12thorder:

Analy'csolu'onappearsunlikely.CandoTaylorexpansionofθaroundλs,i.Retaininglineartermisnormallyquiteaccurate.Thereisananaly'csolu'ontofindmodeofproductinthiscase!

θ= 1+γ λs,i −1( )⎡

⎣⎢⎤⎦⎥2σprior2 +σobs

2

DARTTutorialSec'on12:Slide44

Page 45: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

inf_flavor =2, 0,inf_ini'al_from_restart =.false., .false.,inf_sd_ini'al_from_restart =.false., .false.,inf_determinis'c =.true., .true.,inf_ini'al =1.00, 1.0,inf_sd_ini'al =0.2, 0.0,inf_damping =1.0, 1.0,inf_lower_bound =1.0, 1.0,inf_upper_bound =1000000.0, 1000000.0,inf_sd_lower_bound =0.0, 0.0,

BeforeAssimila'on

AferAssimila'on

Flavor: 2=>varyingstatespace 3=>constantstatespace 0=>NONE

Ini'alinfla'onvalueIni'alstandarddevia'on

TrythisinLorenz96(verifyotheraspectsofinput.nml).Use40memberensemble.(setens_size=40infilter_nml).Setredvaluesasaboveforadap'vespa'ally-varyingstatespaceinfla'on.

Lowerboundons.d.

Experimen'ngwithspa'ally-varyingstatespaceinfla'on

DARTTutorialSec'on12:Slide45

models/lorenz_96/work/

Page 46: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Experimen'ngwithspa'ally-varyingstatespaceinfla'on

Cantrythiswiththeotheralgorithmicvariants.Spa'ally-varyingadap'veinfla'onisthemostcommonchoiceinDARTforlargeearthsystemmodels.

DARTTutorialSec'on12:Slide46

Page 47: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

PosteriorInfla'on

Sofar,we’vealwaysusedthefirstcolumnoftheinfla'onnamelist.Infla'onisperformedafermodeladvancesbutbeforeassimila'on.Canalsodoposteriorinfla'onusingsecondcolumn.Thisdoesinfla'onaferassimila'onbutbeforemodeladvance.Helpstoincreasevarianceinforecasts.Canalsodobothpriorandposteriorinfla'on(usebothcolumns).Diagnos'csareinsamefileswith‘post’insteadof‘prior’.

DARTTutorialSec'on12:Slide47

Page 48: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

Combinedmodelandobserva'onalerrorvarianceadap'vealgorithm

Isthisreallypossible.Yes,incertainsitua'ons...Isthereenoughinforma'onavailable?Spa'ally-varyinfla'onforstate.Infla'onfactorfordifferentsetsofobserva'ons(allradiosondeT’s).Differentλ’sseedifferentobserva'ons.TestsinL96withmodelerrorANDincorrectobs.errorvariancecancorrectforboth.

θ= 1+γ λs,i −1( )⎡

⎣⎢⎤⎦⎥2σprior2 +λoσobs

2

DARTTutorialSec'on12:Slide48

Page 49: DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

1.   FilteringForaOneVariableSystem2.   TheDARTDirectoryTree3.   DARTRunGmeControlandDocumentaGon4.   HowshouldobservaGonsofastatevariableimpactanunobservedstatevariable?

MulGvariateassimilaGon.5.   ComprehensiveFilteringTheory:Non-IdenGtyObservaGonsandtheJointPhaseSpace6.   OtherUpdatesforAnObservedVariable7.   SomeAddiGonalLow-OrderModels8.   DealingwithSamplingError9.   MoreonDealingwithError;InflaGon10.   RegressionandNonlinearEffects11.   CreaGngDARTExecutables12.   AdapGveInflaGon13.   HierarchicalGroupFiltersandLocalizaGon14.   QualityControl15.   DARTExperiments:ControlandDesign16.   DiagnosGcOutput17.   CreaGngObservaGonSequences18.   LostinPhaseSpace:TheChallengeofNotKnowingtheTruth19.   DART-CompliantModelsandMakingModelsCompliant20.   ModelParameterEsGmaGon21.   ObservaGonTypesandObservingSystemDesign22.   ParallelAlgorithmImplementaGon23.  Loca'onmoduledesign(notavailable)24.  Fixedlagsmoother(notavailable)25.   Asimple1DadvecGonmodel:TracerDataAssimilaGon

DARTTutorialIndextoSec'ons

DARTTutorialSec'on12:Slide49


Recommended