SREPS Priority Project: final report

SREPS Priority Project:final report

C. Marsigli, A. Montani, T. Paccagnella

ARPA-SIMC - HydroMeteorological Service of Emilia-Romagna, Bologna, Italy

F. Gofa, P. Louka

HNMS – Hellenic National Meteorological Service, Athens, Greece

Last FTEs of Chiara were used for this

Outline

COSMO-SREPS methodology

system set-up

analysis of the results on MAP D-PHASE DOP (JJA 2007+SON 2007)

role of different kind of perturbations

error vs spread

boundaries from mm vs different physics

ranking of different driving models and of different physics

conclusions

analysis of the results on MAP D-PHASE DOP (JJA 2007+SON 2007)

error vs spread

boundaries from mm vs different physics

ranking of different driving models and of different physics

conclusions

System set-up

16 COSMO runs 10 km

hor. res.40 vertical

levels

COSMO at 25 km on IFS

IFS – ECMWF globalb

COSMO at 25 km on GME

GME – DWD global

COSMO at 25 km on UM

UM – UKMO global

COSMO at 25 km on GFS

GFS – NCEP global

P1: control (ope)

P2: conv. scheme (KF)

P3: tur_len=1000

P4: pat_len=10000

00 UTCJJA=53 SON=54

SON=54 runsJJA=53 runs

COSMO observations

intra-group distance

Z500 COSMO analysis

JJA 2007 - 50 days

SON 2007 - 49 days

Same driving model

Different model

parameters

Same model parameters

Different

driving model

intra-group distanceJJA 2007 - 50 days

COSMO analysis

COSMO-SREPS

intra-group distanceSON 2007 - 49 days

COSMO I7 analysis

COSMO-SREPS

2mT Northern Italy

JJA 2007 - 50 days

SON 2007 - 49 days

SYNOP over D-PHASE area - Nearest grid point

JJA07 SON07

2m T - relationship between error and spread

COSMO I7 analysis

error vs spread Underdispersive

Synop stations on the Alpine area

218 stations

spread/skill relationship

2mT Alpine area (synop stations)

+12h +24h +36h

+48h +60h +72h

2mT Alpine area (synop stations)

+12h +24h +36h

+48h +60h +72h

Underdispersive

t850 relationship between EM error and

spread

COSMO-I7 interpolated on SYNOP stations over the Alpine area

t850 relationship between error and

spread

t850 Alpine area (COSMO analyses)

+12h +24h +36h

+48h +60h +72h

t850 Alpine area (COSMO analyses)

+12h +24h +36h

+48h +60h +72h

Z500 relationship between error and

spread

COSMO-I7 interpolated on Northern Italy stations and SYNOP stations over the Alpine area

Z500 relationship between error and

spread

TP 24h – ave 0.5x0.5 JJA07

noss 1400 700 400 150 50

IT + CH

Same driving models

Same perturbation

Big impact of multi model BCs!!!!!!!

Looking at the right column it is evident that even with few members the skill does not decrease too much when the driving models are different.

Same perturbation

TP 24h – ave 0.5x0.5

noss 800 500 300 100 50

IT + CH

Same driving model

Same perturbation

JJA07 +30h

IT + CH

Same driving model

Same perturbation

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

forecast range (h)

2m temperature - BIAS

Synop observations over Alpine area

Nearest grid point – lsm + altitude correction

ecmwfgmencepukmo

2m temperature - MAE

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

forecast range (h)

ecmwfgmencepukmo

2m temperature - BIAS

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

forecast range (h)

m16ecmwfgmencepukmo

2m temperature - MAE

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

forecast range (h)

ecmwfgmencepukmo

Deterministic scores – ave 0.5 x 0.5 IT

1mm/24h

5mm/24h

10mm/24h

father

1mm/24h

5mm/24h

10mm/24h

father

1mm/24h

5mm/24h

10mm/24h

1mm/24h

5mm/24h

10mm/24h

Test of more parameter perturbations (same father)

16 LM runs at 10 km

P1: control (ope)

P3: parameter 1

P4: parameter 2

P5: …

IFS – ECMWF global

SON 07

T2m deterministic scores – npo IT

0 3 6 9 12 15 18 21 24

forecast range (h)

0 3 6 9 12 15 18 21 24

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

0 3 6 9 12 15 18 21 24

forecast range (h)

0 3 6 9 12 15 18 21 24

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200-2.40

0 3 6 9 12 15 18 21 24

c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

Preliminary ConclusionsPerturbations -Multi Model ICs/BCs & Perturbations on

Ph. Params:

the use of different driving models seems to dominate with respect to physics parameter perturbations as regards the contribution to the spread; these contributions are different in the two seasons (2mT)

the selected parameters produce a detectable spread among members with the same father (driving model)

spread-skill relationship:

a correlation between error and spread exists, but the

system is under-dispersive -> a better representation of model error is needed

the different driving models contribute differently to the ensemble

skill, but there is a strong dependence on forecast range, season, verification area

the different perturbations can contribute differently to the ensemble skill as well

On-going activities and future plans

continue the analysis over the DOP MAP D-PHASE :

statistical analysis of the system

comparison with the other available mesoscale ensemble systems

verification carried out by HNMS

introduce the new parameter perturbations

analyse the impact of adding soil perturbations

COSMO-SREPS methodology

i.c. and b.c. perturbations -> INM multi-model multi-boundary ensemble (SREPS)

LAM perturbations -> physics parameter perturbations

LAM perturbations - smaller scale errors

driving model perturbations (ics and bcs) - larger scale errors

System set-up

16 COSMO runs 10 km

hor. res.40 vertical

levels

COSMO at 25 km on IFS

IFS – ECMWF globalb

COSMO at 25 km on GME

GME – DWD global

COSMO at 25 km on UM

UM – UKMO global

COSMO at 25 km on GFS

GFS – NCEP global

P1: control (ope)

P3: tur_len=1000

P4: pat_len=10000

MAP D-PHASE DOP testing period

COSMO-SREPS was running during the DOP, at 00 UTC

107 runs out of 183 days, 53 in JJA and 54 in SON

ensemble verification

intra-group distanceJJA 2007 - 50 days

Northern Italy

COSMO-SREPS

Same driving model

Different model parameters

Different driving model

intra-group distanceJJA 2007 – 50 days

Northern Italy

Same driving model

Mid Term comments

• Mid-upper troposphere: MULTI MODEL IC/BCs give the bigger contribution to the spread

• Surface/lower troposphere: model physics perturbations “gain ground”.

score evaluation

• +30h:

• ROC

• UKMO and GME the best, then ECMWF and NCEP

• P2 (KF) the best, then P4, P3, P1 (similar)

• BSS

• UKMO the best, ECMWF and GME the worst; NCEP improves with threshold

• P are similar, P2 (KF) slightly better

JJA07tp24I

score evaluation

• +54 :

• ROC

• similar for low thresholds, NCEP the best for high thresholds, then ECMWF

• P similar, P3 slightly better

• BSS

• ECMWF the best, GME the worst; NCEP improves with threshold

• P2 (KF) the worst, P3 the best but similar to P1 and P4

JJA07tp24I

1 5 10 20 30

EC +30 ROC

+54 ROC

MO +30 ROC

+54 ROC

GME +30 ROC

+54 ROC

AVN +30 ROC

+54 ROC

1 5 10 20 30

EC +30 ROC **** ** ** ** *

+54 ROC

MO +30 ROC ** **** **** **** ****

+54 ROC

GME +30 ROC * *** **** *** ****

+54 ROC

AVN +30 ROC ** * * * ***

+54 ROC

TP 24h – ave 0.5x0.5 JJA07

noss 700 350 200 60 20

father

TP 24h – ave 0.5x0.5 JJA07

father

noss 700 350 200 60 20

Grosso impatto del MULTI MODEL ai boundaries!!!!!!! Looking at the right column it is evident that even with few members the skill does not decrease too much when the driving models are different.

score evaluation

• +30h:

• ROC

• UKMO > GME > ECMWF > NCEP

• P2 (KF) the best, the others are similar

• BSS

• NCEP better for high threshold; UKMO > ECMWF > GME

• P2 (KF) the worst, the others are similar

JJA07tp24IT+CH

score evaluation

• +54 :

• ROC

• similar for low thresholds, NCEP and GME best for high thresholds

• P similar; P2 (KF) slightly better for high thresholds

• BSS

• ECMWF the best, NCEP the worst

• P similar; P2 (KF) slightly worse

JJA07tp24IT+CH

TP 24h – ave 0.5x0.5 JJA07

noss 1400 700 400 150 50

IT + CH

father

TP 24h – ave 0.5x0.5 JJA07

noss 1400 700 400 150 50

IT + CH

father

The performances reverse eith the leading time.

Bad skill adding switzerlad.

Deterministic scores – ave 0.5 x 0.5 IT+CH

1mm/24h

5mm/24h

10mm/24h

father

1mm/24h

5mm/24h

10mm/24h

father

1mm/24h

5mm/24h

10mm/24h

1mm/24h

5mm/24h

10mm/24h

1mm/24h

5mm/24h

10mm/24h

fathernoss 250 100 50

1mm/24h

5mm/24h

10mm/24h

1mm/24h

5mm/24h

10mm/24h

noss 250 100 50

1mm/24h

5mm/24h

10mm/24h

pertnoss 250 100 50

6h accumulated precipitationrelationship between error and

spread

obs Northern Italy

Relationship between error and spread

Northern Italy observations

Nearest grid point

applicata correzione per la quota LAPSE=0.7

eliminati dati minori di –10 e maggiori di +42

SYNOP over MAP area

Nearest grid point

all (16 members)

SYNOP over MAP area

Nearest grid point

GME only (4 members)

SYNOP over MAP area

Nearest grid point

ECMWF only (4 members)

SYNOP over MAP area

Nearest grid point

NCEP only (4 members)

SYNOP over MAP area

Nearest grid point

UKMO only (4 members)

SYNOP over MAP area

Nearest grid point

2m temperature relationship between error and

spread

COSMO-I7 interpolated on Northern Italy stations and SYNOP stations over the Alpine area

6h precipitation - BSS

Average over 0.5 x 0.5 deg boxes

6h precipitation – ROC area

Daily precipitation - BSS

Northern Italy + Switzerland observations

Daily precipitationreliability and resolution

Daily precipitation - ROC area

Nearest grid point eliminati dati minori di –10 e maggiori di +42

Nearest grid point

eliminati dati minori di –10 e maggiori di +42

SYNOP over MAP area

Nearest grid point

2m temperature relationship between error and

spread

Northern Italy

COSMO-SREPS

Same driving model

Northern Italy

COSMO-SREPS

score evaluation

• +30h:

• ROC

• crossing; UKMO and ECMWF slightly better, GME worse

• BSS

• ECMWF the best, GME the worst

• P are similar, P2 (KF) slightly better

score evaluation

• +54 :

• ROC

• NCEP the best, GME the worst

• BSS

• similar

TP 24h – ave 0.5x0.5 SON07

father

noss 550 300 200 80 50

TP 24h – ave 0.5x0.5 SON07

father

noss 550 300 200 80 50

score evaluation

• +30h:

• ROC

• similar; for high thresholds ECMWF is the best and GME the worst

• similar; P2 (KF) slightly better

• BSS

• ECMWF the best and GME the worst, , especially with increasing threshold

• P are similar, P4 (tur_len=1000) better for the last thresholds

tp24IT+CH

score evaluation

• +54 :

• ROC

• NCEP the best, GME the worst

• similar, P2 (KF) slightly better

• BSS

• ECMWF the best, UKMO the worst

• similar

tp24IT+CH

TP 24h – ave 0.5x0.5 SON07

IT + CH

father

noss 800 500 300 100 50

TP 24h – ave 0.5x0.5 SON07+30h

IT + CH

father

noss 800 500 300 100 50

1mm/24h

5mm/24h

10mm/24h

father

1mm/24h

5mm/24h

10mm/24h

father

1mm/24h

5mm/24h

10mm/24h

1mm/24h

5mm/24h

10mm/24h

father

1mm/24h

5mm/24h

10mm/24h

1mm/24h

5mm/24h

10mm/24h

father

1mm/24h

5mm/24h

10mm/24h

1mm/24h

5mm/24h

10mm/24h

1mm/24h

5mm/24h

10mm/24h

1mm/24h

5mm/24h

10mm/24h

1mm/24h

5mm/24h

10mm/24h

noss 250 100 50

1mm/24h

5mm/24h

10mm/24h

pertnoss 250 100 50

Test of more parameter perturbations (same father)

16 LM runs at 10 km

P1: control (ope)

P3: parameter 1

P4: parameter 2

P5: …

IFS – ECMWF global

SON 07

run nr.parameter name parameter description range default used1 ctrl ope2 lconv convection scheme T or KF T KF3 tur_len maximal turbulent length scale [100,1000] m 500 1504 tur_len maximal turbulent length scale [100,1000] m 500 10005 pat_len length scale of thermal surface patterns [0,10000] m 500 100006 rat_sea ratio of laminar scaling factors for heat over sea [1,100] 20 17 rat_sea ratio of laminar scaling factors for heat over sea [1,100] 20 608 qc0 cloud water threshold for autoconversion [0.,0.001] 0 0.0019101112

13 c_lnd

surface area density of the roughness elements over land [1,10] 2 1

14 c_lnd

surface area density of the roughness elements over land [1,10] 2 10

15 rlam_heat scaling factor of the laminar layer depth [0.1,10] 1 0.116 rlam_heat scaling factor of the laminar layer depth [0.1,10] 1 10

crsmincrsminc_soilc_soil

Minimal stomata resistance 15015011 2

50200Minimal stomata resistance

Surface area index of the evaporating soilSurface area index of the evaporating soil

[50,200] s/m[50,200] s/m

]0,c_lnd[

0 3 6 9 12 15 18 21 24

forecast range (h)

0 3 6 9 12 15 18 21 24

forecast range (h)

0 3 6 9 12 15 18 21 24

T2m deterministic scores – npo ITpia mon

0 3 6 9 12 15 18 21 24

forecast range (h)

0 3 6 9 12 15 18 21 24

forecast range (h)

0 3 6 9 12 15 18 21 24

forecast range (h)

0 3 6 9 12 15 18 21 24

forecast range (h)

Td2m deterministic scores – npo IT

0 3 6 9 12 15 18 21 24

forecast range (h)

0 3 6 9 12 15 18 21 24

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200

0 3 6 9 12 15 18 21 24

forecast range (h)

0 3 6 9 12 15 18 21 24

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200

0 3 6 9 12 15 18 21 24

forecast range (h)

0 3 6 9 12 15 18 21 24

forecast range (h)

pia mon

0 3 6 9 12 15 18 21 24

forecast range (h)

0 3 6 9 12 15 18 21 24

forecast range (h)

6 12 18 24

forecast range (h)

TP deterministic scores – ave 0.5 x 0.5 IT

1mm/6h

0 3 6 9 12 15 18 21 24

noss 452 479 470 447

6 12 18 24

forecast range (h)

1mm/6h

0 3 6 9 12 15 18 21 24

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10 noss 452 479 470 447

6 12 18 24

forecast range (h)

eTP deterministic scores – ave 0.5 x 0.5

IT1mm/6h

0 3 6 9 12 15 18 21 24

noss 452 479 470 447

6 12 18 24

forecast range (h)

10mm/6h

0 3 6 9 12 15 18 21 24

noss 73 81 69 70

6 12 18 24

forecast range (h)

10mm/6h

0 3 6 9 12 15 18 21 24

noss 73 81 69 70

6 12 18 24

forecast range (h)

eTP deterministic scores – ave 0.5 x 0.5

IT10mm/6h

0 3 6 9 12 15 18 21 24

noss 73 81 69 70

t BIA t MAE td BIA td MAE tp1 BS

tp1 TS

tp1 FA tp10 BS

tp10 TS

tp10 FA

KF = = > = = < > <> > > < >

tur_len=150 = < = = > = = <> <> < = < =

tur_len=1000 = > = < = = = <> <> <> => <24

pat_len=10000

> <> = < = > = > > = < > > <24

rat_sea=1 = > = < > = > > <> <> > <> = >24

rat_sea=60 = < = > < = > < <> = < < = < = >

qc=0.001 = = = = = = < > = > <24

crsmin=50 = < = = > = < = = < = > = = =

crsmin=200 = = = = = = = < = = =

c_soil=0 > < < > < > < = < = <>

c_soil=2 < > > > > = < > = = <>

c_lnd=1 = < > > > = = = = = =

c_lnd=10 > < < > = < = = < = < = =

rlam_heat=0.1

<> = > > = > > <> > = > = =

rlam_heat=10

<> = < < <> < <> < = < = =

- -- +++

lconv=KF tur_len=150 tur_len=1000

pat_len=10000 rat_sea=1 rat_sea=60 qc0=0.001

crsmin=50 crsmin=200 c_soil=0 c_soil=2

c_lnd=1 c_lnd=10 rlam_heat=0.1 rlam_heat=10

CSPERT fc-ctrl differences +12h04/05/07 06-12 UTC

CSPERT fc-obs differences +12hlconv=KF tur_len=150 tur_len=1000

pat_len=10000 rat_sea=1 rat_sea=60 qc0=0.001

crsmin=50 crsmin=200 c_soil=0 c_soil=2

c_lnd=1 c_lnd=10 rlam_heat=0.1 rlam_heat=10

-5 55 20

-5 -20

-20 -50

CSPERT forecasts +12h

06-12 UTC 04/05/07

Comparisons

SYNOP over MAP area - Nearest grid point

JJA07 SON07

2m T - relationship between error and spread

COSMO-I7 analyses

2mT Northern Italy

JJA 2007 - 50 days

SON 2007 - 49 days

tp6h northern Italy

JJA 2007 - 50 days

SON 2007 - 49 days

t850 COSMO analysis

JJA 2007 - 50 days

SON 2007 - 49 days

Z500 COSMO analysis

JJA 2007 - 50 days

SON 2007 - 49 days

JJA07 SON07

father

IT + CH

JJA07 SON07

father

IT + CH

father

IT + CH

score evaluation

• 2m t:

• BIA > 0

GME the largest (+), then UKMO, ECMWF, NCEP mixed

P4 (tur_len=1000) the largest (+), for any father

• MAE

GME the smallest, then UKMO, ECMWF, NCEP mixed

P4 (tur_len=1000) the largest, especially for GME at night

score evaluation

• 2m t:

• BIA < 0

ECMWF the largest (-), then GME, NCEP and UKMO

P4 (tur_len=1000) the more positive, for any father; P1, P2, P3 similar

• MAE

ECMWF the largest, then GME, NCEP and UKMO

P4 (tur_len=1000) the smallest; P1, P2, P3 similar

SREPS Priority Project: final report

Documents

EFFECTIVENESS OF BUS SIGNAL PRIORITY FINAL REPORT · Effectiveness of Bus Signal Priority Final Report 6. Performing Organization Code 7. Author(s) Chada, Shireen and Newland, Robert

FFY2011 FINAL PRIORITY SYSTEM, INTENDED USE PLAN, PROJECT …

Priority Services Register Review – Final Proposals

Final 2017 CIR Priority List with Ingredient Groupings Final Priorities.pdf · Final 2017 CIR Priority List with Ingredient Groupings (CIR will choose from these groupings to create

Comprehensive Evaluation of Transit Signal Priority … · Final Technical Report . Agreement T2695, Task 60 SS-RTSP Evaluation . Comprehensive Evaluation of Transit Signal Priority

SREPS Priority Project: final report C. Marsigli, A. Montani, T. Paccagnella ARPA-SIMC - HydroMeteorological Service of Emilia- Romagna, Bologna, Italy

15 IMPLEMENTATION OF PRIORITY PROJECTSChapter15: Implementation of Priority Projects 163 Final Report: Volume II Table 15.1.2 Implementation Schedule of Priority Projects (1/2) PROJECTS

March 1, 2017 WORKING GROUP 8 Priority Services Final Report

COSMO-SREPS Priority Project

FINAL VERSION_NSCT_Research Priority Areas for Inclusion in the NRDA_March 2012

Table 7: Final List of Priority Corridors (By Corridor Name) · 2018-12-12 · Table 7: Final List of Priority Corridors (By Corridor Name) ... 3 Canaveral Groves Boulevard Pine Street

5 Priority Preemption and QoS-Final

Final 2017 CIR Priority List with Ingredient Groupings Final... · 2020-02-29 · Final 2017 CIR Priority List with Ingredient Groupings (CIR will choose from these groupings to create

Priority Data Exchanges LocalCommCtrs FINAL

Joey Yap]FengShuiThatWorks1.pdf · feng shui that works s sreps scou0 to increase work opportunities, ossuœs

Form A: Owner-Occupied Repair Rehabilitation Priority List ... IHCDA CDBG OOR Appendices.pdf · during the final inspection. REHABILITATION PRIORITY LIST . Priority #1: Health and

Health and Wellness Strategic Priority Taskforce Final Recommendations October 2006

Bus Lane with Intermittent Priority (BLIMP) Concept Simulation … · 2017. 5. 17. · Bus Lane with Intermittent Priority (BLIMP) Concept Simulation Analysis . Final Report: November

Priority Queues Two kinds of priority queues: Min priority queue. Max priority queue

prioritization report Final - HumanitarianResponse...Third Priority 2 Not priority 1 6 Population size CSA 2015 5% >=150,000 4 >=100,000 - =50,000 - < 100,000