View
9
Download
0
Category
Preview:
Citation preview
TheEnsembleKalmanfilter
PartI:Theory
AlisonFowlerandSanitaVetra-Carvalho
Data-assimilationtrainingcourse.5th-8thMarch2019,UniversityofReading1
Recapofproblemwewishtosolve
• Givenpriorknowledgeofthestateofasystemandasetofobservations,wewishtoestimatethestateofthesystematagiventime.Thisisknownastheposteriororanalysis.
• Bayes’theoremstates
Figure:1DexampleofBayes’theorem.
Moderaterain HeavyrainNorain
Forexamplethiscouldberainfallamountinagivengridbox.A-prioriweareunsureiftherewillbemoderateorheavyrainfall.Theobservationonlygivesprobabilitytotherainfallbeingmoderate.ApplyingBayes’theoremwecannowbecertainthattherainfallwasmoderateandtheuncertaintyisreducedcomparedtoboththeobservationsandoura-prioriestimate.
2
Recapof4DVar
**
*
*
*
xaxb
*Observationsbackgrounduncertainty,characterisedbyBobservationuncertainty,characterisedbyR
timeAssimilationwindow
• 4DVaraimstofindthemostlikelystateattimet0,givenaninitialestimate,xb,andawindowofobservations.
t0
M(xa)analysisuncertainty.
forecast
3
Recapof4DVar
4
• J(thecostfunction)isderivedassumingGaussianerrordistributionsandaperfectmodel.
Recapof4DVar:whydoanydifferent?
Advantages• MetOfficeandECMWFbothusemethodsbasedon4DVarfortheiratmospheric
assimilation.• Gaussianandnear-linearassumptionmakesthisanefficientalgorithm.• Minimisationofthecostfunctionisawellposedproblem(theB-matrixisdesignedto
befullrank).• Analysisisconsistentwiththemodel(balanced).• Lotsoftheoryandtechniquestomodifythebasicalgorithmtomakeitapragmatic
methodforvariousapplications,e.g.incremental4DVar,preconditioning,controlvariabletransforms,weakconstraint4DVar...
Disadvantages• Gaussianassumptionisnotalwaysvalid.• ReliesonthevalidityofTLandperfectmodelassumption.Thistendstorestrictthe
lengthoftheassimilationwindow.• DevelopmentofTLmodel,M,andadjoint,MT,isverytimeconsuminganddifficultto
updateasthenon-linearmodelisdeveloped.• B-matrixispredominatelystatic.
Thismotivatesadifferentapproach…
5
SequentialDA
• Insteadofassimilatingallobservationsatonetime,assimilatethemsequentiallyintime.
• Thiscanbeshowntobeequivalenttothevariationalproblem,assumingalinearmodelandallerrorcovariancesaretreatedconsistently.
**
*
*
*
time
xb
Assimilationwindow
*ObservationsM(xb)M(xa)forecasta-prioriuncertainty,characterisedbyPfobservationuncertainty,characterisedbyRanalysisuncertainty.
6
SequentialDA-TheKalmanequations
• Ateachobservationtimektheprioruncertainty,p(xk),isupdatedtofindtheposterior,p(xk|yk).
• RecallBayes’theorem:
• LetusassumethedistributionsareGaussian, Notethechangeinnotation!Seefinalslideforalistofallnotationused
• ThisimpliesthattheposteriorisalsoGaussian• Themeanof isisgivenby
where• Andthecovarianceisgivenby
7
• Needtobeabletoevolvetheuncertaintyinthestatefromoneobservationtimetothenext.
• Thatistogofromp(xk-1|yk-1)top(xk),or
• TheExtendedKalmanfilter(EKF,GrewalandAndrews(2008))doesthisusingthenon-linearmodelanditsTLandadjoint.
TheKalmanfilter
8
Predictionstep• Evolvemeanstatefromtimen-1totimeofobservation,n.
where• Evolvecovariancetotimeofobservationallowingformodelerror,
Observationupdatestep• Updatemeanstategivenobservation
where• Updateerrorcovariancegivenobservation
TheKalmanfilteralgorithm
9
MotivationfortheensembleKalmanfilter(EnKF)
• TheEKFstillneedstheTLandadjointmodeltopropagatethecovariancematrix.
• Duetothesizeofthismatrixformostenvironmentalapplications,theEKFisnotfeasibleinpractice.
• Analternativeapproachtoexplicitlyevolvingthefullcovariancematrixistoinsteadestimateitusingasampleofevolvedstates(knownastheensemble).
10
ExtendedKalmanfilterapproachExplicitlyevolvethemeanandcovariancesforwardintimeusingM,MandMT.
EnsembleKalmanfilterapproachSamplefromtheinitialtimeuncertainty,evolveeachstateforwardintimeusingM,thenestimatethemeanandcovariancefromtheevolvedsample.
Time1 Time2
11
w
www
w
w
w
ww
w
w
w
**
*
*
*
EnKFalgorithms• TheEnKF(Envensen1994)mergesKFtheorywithMonteCarloestimation
methods.• TherearemanymanydifferentflavoursofEnKF.• EnKFalgorithmscanbegeneralisedintotwomaincategories:
– Stochasticalgorithms(e.g.theperturbedobservationKalmanfilter)– Deterministicalgorithms(e.g.theensembletransformKalmanfilter)
• AllEnKFmethodscanberepresentedbythesamebasicschematic:
time
xb
Assimilationwindow12
Predictionstep– Evolveeachensemblememberforwardusingthenon-linearmodelwith
addednoise.
– Reconstructtheensemblemean
– Anditscovariance
– Notethereisnoneedtoeverexplicitlycompute ,justtheperturbationmatrix ,,whichisgenerallyofasmallerdimension.
TheperturbedobservationKalmanFilter
13
Filteringstep– Updatetheensembleusingperturbedobservations
Where and
TheperturbedobservationKalmanFilter
14
Derivedfromtheensemble
TheperturbedobservationKalmanFilter
• TheadvantagesoftheperturbedobservationKFisthatitisverysimpletoimplementandunderstandfortoymodels.However…– Itisnecessarytoperturbtheobservationsinorderforthevarianceof
theensembleaftertheupdatesteptocorrectlyrepresenttheuncertaintyintheanalysis.
– Thisintroducesadditionalsamplingnoise.
• Thismotivatesthedevelopmentofsquare-rootordeterministicformsoftheEnKFthatdonotneedtoperturbtheobservations.
15
• TheideaofESRFistocreateanupdatedensemblewithcovarianceconsistentwith
• Recallthattheensemblecovariancematrixisgivenby
• Wecanwriteasimilarexpressionfortheanalysiserrorcovariancematrixintermsoftheanalysisperturbations
• Insteadofupdatingeachensemblememberseparately,asintheperturbedobservationKF,theESRFgeneratesthenewensemblesimultaneouslybyupdatingXfinsteadofx(i),f
EnsembleSquareRootFilter
16
Predictionstep– ThisisthesameasfortheperturbedobservationensembleKF.– Computetheevolvedensemble,,itsmean,,andits
perturbationmatrix,.– Computetheforecast-observationensemble
• Transformtheforecastensembletoobservationspace
• fromthiscancomputethemean
• andperturbationmatrix
EnsembleSquareRootFilter
17
Filteringstep• Updateensemblemean
– where
• Updateperturbationmatrix
– NeedtodefinethematrixT.
EnsembleSquareRootFilter
18
EnsembleSquareRootFilterTheTmatrix• ThematrixTischosensuchthat
• ThisdoesnotuniquelydefineTwhichiswhytherearesomanydifferentvariantsoftheESRF,e.g.theEnsembleAdjustmentKalmanFilter(Anderson(2001),andtheEnsembleTransformKalmanFilterBishopetal.(2001))
• Tippetetal.(2003)reviewseveralsquarerootfiltersandcomparetheirnumericalefficiency.Showthatalthoughtheyleadtodifferentensemblestheyallspanthesamesubspace.
19
EnsembleSquareRootFilterAnexpressionforTcanbefoundbyrearranging using
,and= .
20
TheEnsembleTransformKalmanFilter
• FirstintroducedbyBishopetal.(2001),laterrevisedbyWangetal.(2004).
• TiscomputedusingtheMorrison-WoodburyidentitytorewritethepreviousexpressionforTTT.
• TherevisionbyWangetal.highlightedthatanyTwhichsatisfiestheestimateoftheanalysiserrorcovariancedoesnotnecessarilyleadtoanunbiasedanalysisensemble,seeLivingsetal.(2008)forconditionsthatTmustsatisfyfortheanalysisensembletobecentredonthemean.
21
Modelerror• TheensembleKalmanfilterallowsforanimperfectmodelbyaddingnoise
ateachtimestepofthemodelevolution.
• ThematrixQisnotexplicitlyneededinthealgorithm,onlytheeffectofthemodelerrorintheevolutionofthestate.
• Therehavebeenmanydifferentstrategiestoincludingmodelerrorintheensemble,basedonwhereyouthinkthesourceoftheerrorlies.Afewexamplesare– Multiphysics-differentphysicalmodelsareusedineachensemblemember– Stochastickineticenergybackscatter-replacesupscalekineticenergyloss
duetounresolvedprocessesandnumericalintegration.– Stochasticallyperturbedphysicaltendencies– Perturbedparameters– Orcombinationsoftheabove
22
SummaryoftheEnsembleKalmanFilter
Advantages– Thea-prioriuncertaintyisflow-dependent.– Thecodecanbedevelopedseparatelyfromthedynamicalmodele.g.PDAF
systemwhichallowsforanymodeltoassimilateobservationsusingensembletechniques(seehttp://www.met.reading.ac.uk/~darc/empire).
– Noneedtolinearisethemodel,onlylinearassumptionisthatstatisticsremainclosetoGaussian.
– Easytoaccountformodelerror.– Easytoparrallelise.
Disadvantages– Sensitivetoensemblesize.Undersamplingcanleadtofilterdivergence.Ideas
tomitigatethisincludelocalisationandinflation(seenextEnKFlecture).– AssumesGaussianstatistics,forhighlynon-linearmodelsthismaynotbea
validassumption(seeFriday’slecturesonparticlefilters)– Theupdatedensemblemeanmaynotbeconsistentwiththemodel
equations.23
SummaryoftheEnsembleKalmanFilter
ThedifferentEnKFalgorithms– Manydifferentalgorithmsexist.– Stochasticmethodsupdateeachensemblememberseparatelyand
thenestimatethefirsttwosamplemomentstogivetheensemblemeanandcovariance.
– Deterministicmethodsupdatetheensemblesimultaneouslybasedonlinear/Gaussiantheory.Mayallowforasmallerensemblethanthestochasticmethodsasavoidssomeofthesamplingerror.
EnKFvs4DVar
– Eachmethodhasitsownadvantagesanddisadvantages-thereisnoclearwinner.
– Hybridmethodsaimtocombinethebestbitsofboth(seetomorrow'slecturesonhybridmethods).
24
FurtherreadingKalmanFilter:�GrewalandAndrews(2008)KalmanFiltering:TheoryandPracticeusingMATLAB.Wiley,NewJersey.�Kalman(1960)Anewapproachtolinearfilteringandpredictionproblems.J.BasicEngineering,82,32-45.
StochasticEnsembleKalmanFilter:�Evensen(1994)Sequentialdataassimilationwithanonlinearquasi-geostrophicmodelusingMonteCarlomethodstoforecasterrorstatistics.J.Geophys.Res.,99(C5),10143-10162.
DetermanisticEnsembleKalmanfilter:�Anderson(2001)Anensembleadjustmentfilterfordataassimilation.Mon.WeatherRev.,129,2884-2903.�Bishopetal.(2001)AdaptivesamplingwiththeensembletransformKalmanfilter.Mon.Wea.Rev.,126,1719-1724.�Tippetetal.(2003)EnsembleSquareRootFilters.Mon.Wea.Rev.,131,1485-1490.�Livingsetal.(2008)Unbiasedensemblesquarerootfilters.PhysicaD.237,1021-1028.�Wangetal.(2004)WhichIsBetter,anEnsembleofPositive–NegativePairsoraCenteredSphericalSimplexEnsemble?Mon.Wea.Rev.,132,1590-1605
Modelerror:�Berneretal.(2011)Modeluncertaintyinamesoscaleensemblepredictionsystem:Stochasticversusmultiphysicsrepresentations,Mon.WeatherRev.,139,1972–1995.
Reviews:�Bannister(2017)Areviewofoperationalmethodsofvariationalandensemble-variationaldataassimilation.Q.J.R.Meteorol.Soc.,143:607–633.�HoutekamerandZhang(2016)ReviewoftheEnsembleKalmanFilterforAtmosphericDataAssimilation.Mon.Wea.Rev.,144,4489–4532.�Vetra-Carvalhoetal.(2018)State-of-the-artstochasticdataassimilationmethodsforhigh-dimensionalnon-Gaussianproblems.TellusA,https://doi.org/10.1080/16000870.2018.1445364
25
26
Recommended