Data Assimilation for High Impact Weather Forecast Yuanfu Xie NOAA/OAR/ESRL OAR/ESRL/GSD/Forecast Applications Branch

Data Assimilation for High Impact Weather Forecast

Yuanfu XieNOAA/OAR/ESRL

OAR/ESRL/GSD/Forecast Applications Branch

Outline

• Review data assimilation techniques;• Improving variational data assimilation;• Limitations of ensemble Kalman filter (EnKF);• Highly nonlinear and non-Gaussian data

assimilation;• Some case studies of hurricane and tornado;• Summary.


History and modern techniques

• Objective analysis in the 70s;• Variational analysis in the 90s;• Kalman filter including EnKF, EnSRF, became

hot topics in the late 90s til now;• Hybrid of variational analysis (3DVAR/4DVAR)

with EnKF covariance matrix now;• What is next?


Improving variational analysis

Major reason for the recent interests in EnKF is due to the unsatisfactory variational analysis even it is so successful at ECMWF.

Of course, the easy implementation of EnKF or any Monte Carlo methods is another reason.

Variational analysis, particularly, 4DVAR, is too complicated and difficult to maintain.


Issues for the unsatisfactory story

• It is known that the error covariance is difficult to compute as it is flow, location and time dependent;

• Adjoint system is complicated, 4DVAR;

• Control variables, e.g. wind;• Constraints for 3DVAR and 4DVAR;• Forward operators.


Control Variable Issues

Xie et al 2002 has studied the analysis impact by selecting different control variables for wind, ψ-χ, u-v, or ζ-δ VARS.

Conclusions:• ζ-δ is preferable (long waves on OBS; short on

background);• u-v is neutral; • ψ-χ is not preferred: short on OBS; long on

background;


Non-physical error by ψ-χ VARS

• Ψ and χ are some type of integral of u-v. A correction to the background toward the obs changes the

integral of u. The background term adds opposite increment to the analysis for keeping the same integral as the background.


Responses of three type of VARS

For a single obs of u, e.g., ψ-χ, u-v, or ζ-δ VARS have different responses as shown here. A ψ-χ VAR may not show this clearly as many filters applied.


Limitations of Ensemble DA

Ensemble DA is relatively easy to implement and It has been considered to improve VARs.

Limitations:• Major challenge is the limited ensemble members

for any applications (more needed, Julier et al 1997);• It may not be appropriate for nonlinear and non-

Gaussian applications (e,g., severe weather);• For some nonlinear observation data, it may have

problems.


Hybrid technique

• Ensemble filter to provide an error covariance matrix for variational techniques;

At ESRL, STMAS uses a multigrid technique to combine EnKF or particle filter with a sequence of variational analysis; at coarse grids, it can use any estimated error covariance matrix, e.g. EnKF…


Multigrid Technique

Using the number of gridpoints to control the base functions. For irresolvable scales, EnKF or particle filter is used.

Nonlinear and Non-Gaussian DA

• Particle filter has been investigated for nonlinear and non-Gaussian DA. It requires more ensemble members;

• STMAS considers a DA in two steps, variational retrieval and data assimilation. With modern observation networks, more accurate information is available for retrieving. STMAS uses a variational retrieval to mimic objective analysis with simultaneous dynamic constraints.


A Lognormal 1DVAR

€

Pa (u)∝ Po(u − uo)Pb (u − ub )

Po(u − uo)∝ e−

ln(u−uo )−τ o( )

2

2σ o2

, Pb (u − ub )∝ e−

ln(u−ub )−τ b( )

2

2σ b2

minln(u − uo) − τ o

( )2

2σ o2

+ln(u − ub ) − τ b

( )2

2σ b2

PDF:

Cost function:

Recent development

• STMAS has been developed and under testing;• Its adjoint system is under development;• Radar radial wind and reflectivity DA has been

added into STMAS;• STMAS uses a u-v as control variable in its

multigrid analysis;• NCAR WRF DA group is working with ESRL for

testing ζ-δ WRF DA as seen some advantages in cloud scale analysis.


Storm Boundary Detection

• ESRL and MIT have been using STMAS surface analysis of high resolution (2-5km) and high temporal (5-15 minutes);

• It uses high frequency observation data, e.g., 5 minute ASOS (1 minute ASOS planned);

• Multi-variate analysis with certain dynamic constraints is under development;

• STMAS technique will be applied to fire weather, wind energy and other applications.


Status of STMAS Surface Analysisfor Storm/Gust Front Detection

Space and Time Multiscale Analysis System (STMAS) is a 4-DVAR generalization of LAPS and modified LAPS is running at terminal scale for FAA for wind analysis.

STMAS real time runs with 15 minute latency due to the observation data:

• STMAS surface analysis is running in real time over CONUS with 5-km resolution and 15-minute analysis cycle. It is so efficient that it runs on a single processor desktop;

• Real time 5-minute cycle run of STMAS assimilating 5-minute ASOS data also on single processor.

• Note: Modified LAPS is running at 40+ sites at terminal scales for FAA in real time.

STMAS surface analysis with 5 minute latency with (MPI/SMS) and targeting at

2-km resolution; 5-minute cycle; over CONUS domain; assimilating 1-minute ASOS; using HRRR as background.

CONUS15-mincycle

5-minASOS,5-mincycle


FAA/MIT Boundary Detection Application

STMAS for frontal boundary detection

STMAS vs. HPC

Hurricane Forecast Improvement Project (HFIP/NOAA)

NOAA HFIP is a major effort for improving hurricane forecast (both tracks and intensity);

NOAA 5 year plan:• Accuracy of hurricane intensity forecast is

improved by 50%;• Track lead time is up to 5 days with 50%

accuracy improvement.


21

Hurricane: costly natural disaster

Overview of 2008 hurricane season:

8 hurricanes (5 major)Fatalities: 855Damage: $42 billions

22

Increased National Needs

23

— 48-Hour Hurricane Track Forecast Error

Hurricane Forecast Enhancements

24

— 48 hr Hurricane Intensity Forecast Error Trend

Hurricane Forecast Enhancements

ESRL in HFIP• Global modeling: FIM (finite volume Icosahedra

grid modeling);• Statistical post-processing;• OSSE etc;DA:1.Regional EnSRF (square root EnKF). 2.Testing STMAS for hurricanes and typhoon

(supported by CWB/Taiwan) comparing to LAPS


Katrina

Pre-balanced 950-mb wind speed and height.

Balanced 950-mb wind speed and height.

LAPS analysis

KATRINA LAPS / HRS(2005)

Track: WRF 20km Katrina forecast by STMAS

Best track:every 6 hours

WRF-ARW 72 hour fcst w/Ferrier physics:every 3 hours

Intensity: WRF Katrina forecast by STMAS

Wind Barb, Windspeed image,Pressure contour at 950mb Surface pressure

j

Testing Cycling Scheme for STMAS

• The previous analysis is done by STMAS using GFS 1 degree forecast as background. The analysis does not contain detailed small scales;

• The observation data for Katrina at Aug. 27, 2005 at 18Z is sparse even though there is some dropsonde data into the hurricane eyes.

• Cycling scheme could provide detailed information through assimilation of early hours before STMAS 4DVAR is used.


Cycling Impact on STMAS analysis

STMAS-WRF ARW cycling Impact


Rapid Intensification and Rapid Weakening (RIRW)

• Cycling certainly helps with the rapid intensification;

• For the Kartina case, the WRF ARW keeps the intensification for a longer period of time, which is not realistic;

• Further study will be on the rapid weakening.


Severe Storms and Tornado

NOAA long term goal (20 years) is on tornado forecasts, for longer lead time forecasts (1-3 hour forecast)

ESRL/GSD/FAB is investigating the possibility of tornado forecast, what is needed and how to improve.

Radar data assimilation is critical to severe storms and tornado forecasts.

Examples of LAPS/STMAS analysis impact

LAPS Forecast: Jun. 16, 2002

LAPS hot-start & WRF-ARW Forecast (2km with Lin Micro-physics ): IHOP cases

LAPS Forecast: Jun. 13, 2002

(Steve Albers and Isidora will talk these cases in more details at their presentations

A well-balanced initial condition is the key for improving very short rangeforecast for tornados.

Windsor tornado case, 22 May 2008

• Tornado touched down at Windsor, Colorado around 17:40 UTC, 22 May 2008

• STMAS initialization 1.67 km 301 x 313 background model: RUC 13km, 17 UTC hot start (cloud analysis)• Boundary conditions: RUC 13km, 3-h RUC forecast (initialized at 15 UTC)• WRF-ARW 1.67 km, 1-h forecast Thompson microphysics

• Postprocessing: Reflectivity

00-01hr 800mb wind initialized at 17 UTC 22 May 2005, STMAS analysis vs. WRF forecast (STMAS)

00-01hr wind cross-section initialized at 17 UTC 22 May 2005, STMAS analysis vs. WRF forecast (STMAS)

00-01hr 800mb reflectivity initialized at 17 UTC 22 May 2005, mosiac radar vs. WRF forecast (STMAS)

00-01hr reflectivity cross-section initialized at 17 UTC 22 May 2005, mosiac radar vs. WRF forecast (STMAS)

00-01hr reflectivity cross-section initialized at 17 UTC 22 May 2005, WRF forecast, RUC vs. STMAS

Future Study

• Investigating the initialization of STMAS for WRF forecast model;

• Assimilating radar reflectivity data into the well-balance analysis of STMAS;

• STMAS rapid update cycle;• Satellite radiance and GPS impact on severe

storm and tornado forecasts;• Possible OSSE system for severe storms.

Summary

• Variational data assimilation requires further improvement, particularly the adjoint systems;

• Effectively combining variational data assimilation techniques with EnKF or particle filter for nonlinear and non-Gaussian data assimilation for severe weather forecasts;

• Improving radar, satellite data assimilation, forward operators and efficiency.


45

First Assumption in a 3DVAR

• In most cases, DA has two information sources: Background and observation;

• A standard 3DVAR treats the background (xb) and observation (xo) as random representations of the true atmosphere (xt);

• Assumption I: xb-xt and xo-xt are random variables.

46

The Bayesian Theorem

A simplified version:Pa(x) P(x=xt | x=xo) = P(x=xo | x=xt) P(x=xt);

P(x=xo | x=xt) = P(xo=xt) = Po(x-xo), i.e., observational error probability.

Assumption II. P(x=xt) = Pb(x-xb), as this prior probability can depend our knowledge of the background on the true atmosphere, see Lorenc 86.

Then we have: Pa(x)= Po(x-xo) Pb(x-xb).

47

Derivation of a 3DVAR IAssumption III. The background and observation error probabilities

follow a Gaussian distribution:Po(x-xo) exp[-(x-xo)TO-1(x-xo)]

Pb(x-xb) exp[-(x-xb)TB-1(x-xb)]

Assumption IV.Covariances O and B are known!

48

To maximize the probability for estimation of the To best approximate the true atmosphere xt , we maximize

exp[-(x-xb)TB-1(x-xb)]+exp[-(x-xo)TO-1(x-xo)].This is equivalent to minimize:

(x-xo)TB-1(x-xo)+(x-xo)TO-1(x-xo),the standard 3DVAR cost function.

Optimality = maximized probability

Derivation of a 3DVAR II

It is derived from a statistical analysis assuming the error’s distribution is Gaussian. It solves a variational problem:

subject to a model constraint for 4DVAR

Single 3-4DVAR approach

€

min1

2u − ub

( )T

B−1 u − ub( ) +

1

2Hu − y( )

TO−1 Hu − y( )

南京大学

Background

Observation Longer wave

Background

Observation Longer wave

Resolvable Information for a Given Observation Network

Difference on longer wave Difference on shorter wave

南京大学

STMAS

STMAS is implemented in two steps. 1. It retrieves the resolvable observation

information. 2. After the resolvable information retrieved,

STMAS is reduced to a standard statistical variational analysis

With long waves retrieved, STMAS deals with a localized error covariance, a banded matrix, at its last phase of analysis.

南京大学

Documents

Data Assimilation for High Impact Weather Forecast Yuanfu Xie NOAA/OAR/ESRL OAR/ESRL/GSD/Forecast Applications Branch