European Storm Forecast Experiment
Verification of Dichotomous Lightning Forecasts at the
European Storm Forecast Experiment (ESTOFEX)
Pieter GroenemeijerOscar van der Velde
Helge TuschyChristoph Gatzen
Johannes DahlNicholas Verge
Institute for Meteorology and Climate Research, Research Centre / University KarlsruheEuropean Severe Storms Laboratory
Laboratoire d'Aérologie, Université Paul Sabatier, ToulouseInstitute for Atmospheric Physics, German Aerospace Center (DLR)
European Storm Forecast Experiment
Overview
• About ESTOFEX
• ESTOFEX forecasts
• Verification of lightning forecasts
• Verification of threat level forecasts
European Storm Forecast Experiment
about ESTOFEX
• the name of a web-based platform for the exchange of knowledge about convective storm forecasting
• the name of a the forecast experiment that is carried out by some of its active members
• goal of the experiment is... “to find out how convective storms can best be
forecast in Europe”
European Storm Forecast Experiment
about ESTOFEXESTOFEX issues “convective
forecasts” for the next day
These forecasts consist of two parts:
• forecast discussion
• forecast map
Storm ForecastValid: Sat 02 Jun 2007 06:00 to Sun 03 Jun 2007 06:00 UTCIssued: Sat 02 Jun 2007 09:22Forecaster: VAN DER VELDESYNOPSIS
An mid/upper cut-off low dwells slowly southward with its center just west of Italy. The associated surface pressure field weakens during the period. High pressure...
European Storm Forecast Experiment
ESTOFEX is not...
• a new weather service in Europe(no such centre without an initiative from NMS’s)
• funded
• the start of a commercial company
European Storm Forecast Experiment
thunder area
level 1
level 2
level 3
ESTOFEX forecasts
The forecast maps show lines that represent
level 1, level 2, level 3 are threat levels for convective wind gusts, tornadoes and large hail
European Storm Forecast Experiment
ESTOFEX forecastsAfter the forecast hasbeen issued, verificationdata appears on themap.
Hail reportsTornado reportsWind gust reportsHeavy rain reports(source: ESSL)
Lightning strikessource: UK. Met Officearrival-time-difference system
European Storm Forecast Experiment
Verification of lightning forecasts
• Predictand
the occurrence of lightning* within 40 km of a point
• Dichotomous forecast event occurred?
yes no
event forecast?
yes hit false alarm
no misscorrect
negative
* = one or more strike detected
European Storm Forecast Experiment
Forecast lightning (1 Sep. 2006 – 31 Aug. 2007)
• number of forecast periods
(of 346 forecasts) in which
lightning was forecast to occur
within 40 km of each point
European Storm Forecast Experiment
Detected lightning (1 Sep. 2006 – 31 Aug. 2007)
• number of forecast periods
(of 346 forecasts) in which
lightning occurred within
40 km of a point
More lightning is forecast than
occurs in reality.
The forecasts have a positive bias.
European Storm Forecast Experiment
Forecast skill indicators
• False Alarm Rate =
false alarms / (hits + false alarms)
• Probability Of Detection =
hits / (hits + misses)
• Heidke Skill Score
European Storm Forecast Experiment
Probability Of Detection(1 Sep. 2006 – 31 Aug. 2007)
High POD over...
• the western and
central European continent• the central and eastern
Mediterranean Sea
Low POD over...
• the Atlantic Ocean• Scandinavia• Eastern Europe• Northern Africa
POD =
hits / (hits + misses)
European Storm Forecast Experiment
False Alarm Rate(1 Sep. 2006 – 31 Aug. 2007)
Low FAR over...
• central and eastern parts of
the European continent
High FAR over...
• western and northern Europe• parts of the Mediterranean• the Black Sea
FAR =
false / (hits + false)
European Storm Forecast Experiment
Heidke Skill Score(1 Sep. 2006 – 31 Aug. 2007)
HSS = 0 means that the forecast
is no better than chance
HSS = 1 means that the forecast
is perfect
HSS assumes that a miss is just as bad
as a false alarm.
HSS indicates the forecasts are best
over the central and eastern
European continent
and the southern Mediterranean
Forecasts are bad over western and
northern Europe
European Storm Forecast Experiment
Individual forecaster performance2006 and 2007 summer periods compared
Probability of Detection False Alarm Ratio
European Storm Forecast Experiment
Individual forecaster performance2006 and 2007 summer periods compared
Probability of Detection False Alarm Ratio
European Storm Forecast Experiment
Future plans
event occurred ?
Yes No
event forecast?
> 60 %
30-60%
15-30%
5 -15%
< 5%
• Polychotomous lightning forecasts
European Storm Forecast Experiment
Future plans
• switch to probabilistic forecasting of lightning
• start rigorous verification of severe weather(hail, tornadoes, severe winds)
• switch to probabilistic forecasting of severe weather
• test the skill of model-derived parameters for both lightning and hail, tornadoes winds – and compare with human forecasters (interesting!)
European Storm Forecast Experiment
Final remarks
• a discussion of individual storm events is presented on the poster lead-authored by Helge Tuschy and in Christoph Gatzen’s talk
• all verification data is available through the forecast archive at www.estofex.org
• I am happy to answer your questions on ESTOFEX and the verification of its forecasts