QPF ISSUES IN QPF ISSUES IN NWPNWP

William A. Gallus, Jr.

Dept. of Geological & Atmospheric Science

Iowa State University

ETAMM5

24HR PRECIPITATION 4/00-5/00

OBS PREC:3/12-4/12 OBS PREC:4/12-5/12

ETA MM5

48 HR PRECIPITATION (4/00-6/00)

OBS:3/12-4/12 OBS:4/12-5/12 OBS:5/12-6/12

What is “TRUTH” for QPF verification?

•Increased computer resources have allowed better parameterization schemes and model resolution

•2-day precipitation forecast today is now as accurate as 1-day forecast in 1974

•Each resolution improvement in NCEP Eta model improves skill scores

GOOD NEWS: QPF is GOOD NEWS: QPF is improving!!improving!!

MRF has some skill compared to persistence, even out to 7-8 days:

Roads et al. 1991 (WAF)

This skill is even more apparent for heavy rainfall cases

BAD NEWS: Problems aboundBAD NEWS: Problems abound

•Most improvement in QPF scores occurs during cold season - little improvement in warm season

•Flash flooding kills more people than any other convective-related event

•QPF problems have several potential sources

•Skill scores themselves may be misleading or of little “real” value

Roads and Maisel 1991 WAF:

MRF has regional biases in precipitation over long periods

Example of human improvements on numerical QPF (Olson et al. 1995, WAF)

NGM Manual OBS

Slow improvement in skill for human forecasters, but less skill for heavier amounts (Olson et al. 1995, WAF)

Annual bias has also improved slowly, but interestingly, is better for Day 2 than Day 1 (Olson et al. 1995, WAF)

QPF skill is better is winter than in summer, even when forecasters adjust the NWP guidance

What are sources of What are sources of QPF error?QPF error?

• Resolution inadequacies

• Parameterization errors

• Initialization deficiencies

• Observational errors in verification

If vertical motion is directly constrained by horizontal

resolution…..

•Shouldn’t forecasts for heavy rain events be greatly improved with finer resolution?

•Is there a “magic” resolution where model QPF will approach observed peak values

Gallus 1999 found QPF-horizontal resolution dependence is case-dependent and varies with convective parameterization

6/16/96 6/14/98 7/28/97

7/17/96 5/27/97

BMJ -shaded

KF - clear

Mx obs: 225 Mx obs: 330 Mx obs: 250

Mx obs: 300 Mx obs: 102

Extreme example of unexpected results and Conv. Param. Impacts: 7/17/96 00UTC

surface conditions

00 UTC 17 JUL 1996 - OMAHA

Betts-Miller- Janjic Reference T, Td profiles shown

Large MCS drops up to 300 mm of rain, causing record river crests and severe flash flooding in far eastern NE and western IA.

7/17/96 BMJ simulations with 78,39,22 and 12 km horizontal resolution

NOTE: actual reduction in peak QPF amounts as resolution improves

MX: 46 MX: 45

MX: 32 MX: 32

7/17/96 KF simulations:

NOTE: very strong QPF sensitivity to horizontal resolution. Precipitation area shifted much farther north than in BMJ runs, or observations

MX: 11 MX: 70

MX: 135MX: 186

Daytime precipitation (12-00 UTC 7/16-17/96)

BMJ produces much larger area and amounts

BMJ KF

Convective scheme influences cold pool strength, which in turn, affects evolution of events outside initial rain region

Impacts of convective schemes may be felt outside region of precipitation.

Here, stronger downdrafts in KF scheme result in greater northward transport of instability into Minnesota - leading to more intense subsequent development.

BMJ KF

Another case: Iowa flood of Another case: Iowa flood of June 1996June 1996

Large-scale region looked favorable for excessive rains

Heaviest rains (225 mm) fell in small area in warm sector

Impacts of horizontal resolution changes strongly depend on convective scheme used

Tropical-like soundings with very deep moisture

Td at 850 mb = 18 C

Td at 700 mb = 8 C

BMJ simulations:

Almost no horizontal resolution-QPF dependence

No hint of C IA maximum

12 UTC 6/16 cold pool affecting Iowa

12 UTC 6/16 Eta model 00 hr - initialization

NOTE: cold pool is missing: winds are southerly, without E component

21 UTC 6/16

Observed Surface Moisture Convergence

Flood-producing storms would form on C IA enhancement

Simulated Moisture Convergence -21 UTC - BMJ run with 12 km resolution

Despite poor initial wind field, model does show enhancement in W IA

BMJ simulation:

No general clearing into Iowa by 1 pm -

Less destabilization than actually occurred

KF simulations:

Strong horizontal resolution-QPF dependence

Some evidence of C IA enhancement with 22 and 12 km resolution

KF 6 hr forecast:

Some clearing into SW Iowa

more agreement with obs.

June case shows:June case shows:

• Moist low-mid troposphere allows BMJ scheme to be aggressive

• Even high resolution may not improve simulation of small QPF maxima if other simulated parameters are incorrect

• Generation of QPF upstream due to resolution changes may affect QPF downstream

Changes within a specific convective parameterization

can also have a very pronounced effect on QPF

Spencer and Stensrud (1998) show this using MM4 with KF scheme

Spencer & Stensrud variations in KF scheme

•Permit Precipitation Efficiency to remain at maximum (90%) instead of varying from 10-90%

•Neglect convective downdrafts

•Delay convective downdrafts

Max. Prec for 4 tests

Case OBS nomod mpe ndd DddAug86 170 53 61 107 66Jul87 254 48 53 188 79Sep89 150 76 94 170 97Jun90 127 48 36 142 52Nov92 236 51 61 132 80AVG 196 67 74 150 83

Maximum QPF in 4 KF MM4 runs

From Spencer and Stensrud 1998 - MWR

MicrophysicalMicrophysical schemes may be the schemes may be the next challenge - once resolution next challenge - once resolution

improves so that convective improves so that convective parameterization is no longer parameterization is no longer

necessarynecessary• Colle and Mass examine resolution-

orographic precipitation (1999) dependence

• Microphysical schemes influence results

OBS PRECIP IN PACIFIC NORTHWEST FLOOD EVENT (1996)

from Colle and Mass (1999; MWR)

Pronounced orographic effects

4 km MM5 run does well at crest but underestimates lee precipitation

Horizontal resolution affects precipitation patterns near mountain due to resolution of mountain wave effects. Model QPF performance in lee of mountain fluctuates - low bias is best in coarsest run, but heaviest precipitation just to lee of crest occurs with highest resolution

1.33

4

12

36

Although precipitation forecasts generally improved as resolution was refined from 36 to 4km, little additional improvement occurred with 1.3 km resolution (Colle & Mass)

Model QPF in relation to resolution of topography

Microphysical schemes may have significant influences at high resolution.

Colle and Mass (1999; MWR) found that lee-side precipitation was too small in high-res MM5 simulations, partly because snow fallspeeds were too large.

Best results may not occur with most sophisticated microphysical scheme

Microphysical scheme differences affect QPF in different areas

Mesoscale initialization may be Mesoscale initialization may be poor and affect QPFpoor and affect QPF

Stensrud and Fritsch (1994) have shown the impacts of improved cold

pool initialization

Typical initialization Typical initialization deficienciesdeficiencies

• Low-level jet characteristics

• Cloud boundaries

• Fronts and drylines

• Convective outflows

• Surface characteristics

Stensrud and Fritsch 1994 MWR:

Initialization of NE KS mesoscale boundary has important impact on QPF

MM4 -25KM

How do we verify QPF?How do we verify QPF?

• Bias scores (how many grid points have X amount of rain compared to observations)

• Threat Scores (area correct/(area forecast+ area observed - area correct))

• Probability of Detection

PRIMARY VERIFICATION PRIMARY VERIFICATION TOOLS TODAY USED BY TOOLS TODAY USED BY

NCEP FOR QPF ARE:NCEP FOR QPF ARE:

BIAS: Number of grid points having simulated rain of X amount divided by

number of observed points with X amount

EQUITABLE THREAT SCORE: Ratio of correct forecasts (hits) to total forecasts +

observations - hits (with correction for chance hits)

BIASBIAS•B=F/O

•Can vary from 0 to >> 1

•Bias > 1 means the model is generous with areal coverage of precipitation

•Bias < 1 means the model doesn’t generate enough areas with precipitation

•Many operational models have B>1 for small precipitation amounts, and B<1 for large amounts

ETSETS

•ETS=(H-C)/[F+O-(H+C)]

•0<ETS<1

•Similar to a Threat Score but takes into account that even “chance” forecasts will be correct some of the time (Schaefer 1990; Gandin and Murphy 1992)

1995-1997 ETS AT NCEP (Mesinger 1998)

.34 .36 .35 .30 .26 .22 .17 .12

.33 .355 .35 .30 .26 .22 .175 .12

.31 .34 .32 .27 .23 .20 .16 .10

.30 .315 .29 .23 .19 .16 .10 .08

.01 .10 .25 .50 .75 1.00 1.50 2.0029ETA

48ETA

MRF

NGM

How valuable are these How valuable are these verification methods?verification methods?

• Model A covers your state with 1inch of rain

•Model B simply produces 5 inches in the one county to your east

•A lone supercell drops 5 inches on your county

Which model had the Which model had the better forecast?better forecast?

What are our Bias and ETS?What are our Bias and ETS?

For measurable precip (or any category less than 1 inch):

Assume one grid point per county with 100 counties in state

Bias in model A: 100/1 = 100.0

Bias in model B: 1/1 = 1.00

ETS in A: 1/(100+1-1) = 0.01

ETS in B: 0/(1+1-0)= 0.0

Objective scores may not Objective scores may not agree with your answer!agree with your answer!

Improved mesoscale QPF verification may involve a phase shift of the simulated

precipitation field. Kalnay and others (1999) are studying such an approach

Concluding ThoughtsConcluding Thoughts

•QPF is probably the most difficult aspect of NWP - the hardest one to envision being solved in 25 years

•If convective parameterizations are used, behavior of these schemes exerts powerful impact (primary differences between different models are probably related to the Cu scheme)

•Thus, forecasters can benefit by understanding the specifics of how the schemes behave

Concluding Thoughts (Cont.)Concluding Thoughts (Cont.)

• At very high resolutions, microphysics will likewise complicate the picture

• Forecasters need to be aware of small-scale boundaries of importance, which will most likely be poorly depicted in initialization

• New methods of evaluating what is a “good” QPF will be needed