View
220
Download
1
Tags:
Embed Size (px)
Citation preview
Ensemble Post-Processing and it’s Potential Benefits
for the Operational Forecaster
Michael Erickson and Brian A. Colle
School of Marine and Atmospheric Sciences, Stony Brook University, Stony Brook, NY
High Ensemble Variability: Hanna 9/6/08 00Z Run 18-42 Hour Acc. Precip
NCEP SREF Probability of > 25 mph
NCEP SREF Mean SLP and Spread NCEP SREF Mean 500 hPa Height & Vorticity
Ensemble Forecasting Tools: What’s Out ThereExample from 09 UTC 10/25/2010 – 48 Hr Forecast
NCEP SREF Probability > 1” Precipitation
NCEP SREF Low Pressure Positions NCEP SREF 500 hPa 5460 m Contour
NCEF SREF Probability of > 25 mph NCEP SREF Probability of > 35 mph NCEP SREF Mean 12-hr Snow
Sources: http://www.spc.noaa.govexper/sref & http://www.meteo.psu.edu/~gadomski/ewallsref.html/
Goals
- Evaluate the deterministic and probabilistic biases within the Stony Brook
University (SBU) and Short Range Ensemble Forecast (SREF) ensembles.
- Apply bias correction and a post-processing technique known as Bayesian
Model Averaging (BMA) to the ensembles.
- Show how post-processing can be used by operational forecasters and its
potential application to river flood forecasting.
- Ensembles have biases which can affect
both the ensemble mean and probabilistic
results.
- Can post-processing model data improve
both the biases and probabilities derived
from the ensemble?
Motivation NCEP SREF Temperature Bias > 24oC
No Bias
2X Bias
- Analyzed the 00 UTC 13-member Stony Brook University
(SBU) and the 21 UTC 21-member Short Range
Ensemble Forecast (SREF) system run at NCEP for
temperature and precipitation.
- Observations consist of the Automated Surface Observing
System (ASOS) for temperature and Stage IV rain data for
precipitation.
- Stage IV data is a blend of rain gauge observations and
radar derived rain estimates.
- Results are for the 2007-2009 warm seasons (4/1-9/31).
Methods and Data
Verification Domain
Accumulated Stage IV Rain Data
Region of Study
•10 ETA members at 32 km grid spacing.
•5 RSM members at 45 km grid spacing.
•3 WRF-NMM members at 40 km grid spacing.
•3 WRF-ARW members at 45 km grid spacing.
•IC's are perturbed using a breeding technique.
NCEP SREF 21 Member Ensemble
The SBU/SREF Ensemble
SBU 13 Member Ensemble
•7 MM5 and 6 WRF members run at 12-km grid spacing nested within a 36-km domain.
•Ensemble uses a variety of initial conditions (GFS, NAM, NOGAPS,and CMC), two cloud microphysical, three convective, and three planetary boundary layer schemes.
Region of Study
12-km Model Domain
Verification Domain
Model Biases 2007-2009 - Temperature Model Bias by Member > 24oC
Diurnal Mean ErrorRaw Bias > 24oC for MYJ WRF Member
Bias Correction: Cumulative Distribution Function (Hamill and Whitaker 2006)
•A 50-day training period was used to calculate the cumulative distribution function (CDF)
of each model and the observation.
•The model CDF was then adjusted to the observation over the calibration and validation
period value by value.
•To correct for spatial bias associated with terrain, the bias for each elevation was calculated
and removed using a binning approach.
CDF For Model and Observation CDF Bias Correction Example
Diurnal Root Mean Squared Error
Bias Correction 2007-2009 - Temperature Model Bias by Member > 24oC
Diurnal Mean Error Bias Corrected Diurnal RMSEBias Corrected > 24oC for MYJ WRF Member
Model Biases 2007-2009 - Precipitation Model Bias by Member > 0.1”
Model Bias by Member > 1” Raw Bias > 0.5” for MYJ WRF Member
Bias Correction 2007-2009 - Precipitation Model Bias by Member > 0.1”
Model Bias by Member > 1” Bias Corrected > 0.5” for MYJ WRF Member
- Although biases have
been largely corrected,
the ensemble is still
underdispersed and has
unreliable probabilistic
forecasts.
- Additional post-
processing is necessary
so that more accurate
probabilistic forecasts
can be obtained.
Reliability for Temp > 24oC Reliability for Precip > 0.5”
Ensemble Underdispersion and Reliability
Temp Rank Histogram Precip Rank Histogram
Bayesian Model Averaging (BMA)•Bayesian Model Averaging (BMA, Raftery et al. 2005) is designed to improve ensemble forecasts by estimating two things:
• The weights for each ensemble member (i.e. a “better” member will have more
influence on the forecast.
• The uncertainty associated with each forecast (i.e. a forecast should not be thought
of as a point, but as a distribution).
•Although BMA has been shown to improve ensemble mean forecasts, its main advantage is
with probabilistic forecasts.
The coldest member is given the greatest weight
The second coldest member is given significantly less weight
The warmer members have varying weights
The BMA derived distribution
From Raftery et al. 2005
BMA Weights – 2007-2009 Precipitation
SREF Member WeightsSBU Member Weights
Impact of BMA on Reliability after Bias Correction for Warm Season Surface Temperature (2007-2009)
Bias Corrected Rank Histogram Reliability > 20oC
BMA Rank Histogram Brier Skill Scores
BMA CorrectedBias Corrected
-5 0 5 10 15 20 (C)
Bias Corrected Rank Histogram Reliability > 0.5”
BMA Rank Histogram Brier Skill Scores
BMA CorrectedBias Corrected
Impact of BMA on Reliability after Bias Correction for Warm Season Surface Temperature (2007-2009)
12 – 36 Hr Accumulated PrecipitationPost-Processing Application - 5/17/10 21z NCEP SREF
Raw Ensemble Probability > 1.5” Bias Cor. Ensemble Probability > 1.5”
BMA Ensemble Probability > 1.5” Stage IV Rain DataBMA Ensemble Probability > 1.5”
6 – 36 hr Accumulated PrecipitationHanna - 9/5/08 21z NCEP SREF
Raw Ensemble Probability > 1.5” Bias Cor. Ensemble Probability > 1.5”
BMA Ensemble Probability > 1.5” Stage IV Rain Data
Tropical Hanna Case: Hydrological Test Case9/6/08 00z Run: Saddle River: Lodi, NJ
QPF from Ensemble Modeled Response: NWS River Forecast System
12 cm
9 cm
6 cm
3 cm
0 cm
-33% of members predict major flooding
-42% of members predict moderate flooding
-58% of members predict flooding
Observed Flood Stage
~2.3 m 3.5m
3.0m
2.5m
2.0m
1.5m
1.0m
•Future work will investigate the potential benefits of BMA for streamflow and flood risk assessment.
Conclusions
● Ensemble members suffer from large biases for surface parameters, which can vary temporally, spatially, diurnally and between members.
● The bias correction and BMA improves the probabilistic skill, reliability and dispersion of the Stony Brook + NCEP SREF ensemble.
● Since post-processing improves the ensemble performance spatially, forecasters/users could use BMA for gridded forecast products.
● Although post-processing can remove some systematic biases, it can not correct fundamental problems within the model. For instance, BMA can not correct for large position errors in precipitation forecasts.
● Further development with BMA is needed for extreme weather events such as high QPF forecasts and river flood forecasts given the smaller sample size.