IPWG Validation current status and future directions Chris Kidd Beth Ebert John Janowiak The University of Birmingham, Birmingham, UK Bureau of Meteorology,

IPWG Validationcurrent status and future directions

Chris Kidd

Beth Ebert

John Janowiak

The University of Birmingham, Birmingham, UK

Bureau of Meteorology, Melbourne, Australia

CPC/NCEP/NWS/NOAA, USA

4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China

IPWG validation sites

Addresses IPWG objectives:

#2) Establish standards for validation and independent verification of precipitation measurements

#3) Foster the exchange of data on intercomparisons of operational precipitation measurements from satellites

Main regions (initiated 2002) are:

Australia, United States, Europe and South America,

with Japan, South Africa and others for selected periods

Common graphical, descriptive and statistical analysis of daily, 0.25° resolution products.


GV-IPWG synergies

Criteria GV program IPWG

Type of validation Priority on physical, also statistical

Has focused on descriptive and statistical

Source of validation data Arranged for and collected by principle investigators

Doesn't request. IPWG participants free to contribute

Source of observational data Specific satellite-based products

participants provide products directly to validation groups

Types of Validation data Gauge, radars and specialist instrumentation, diverse in specific locations

Conventional gauge and/or radar networks, usually part of a national network

Types of observational data single-sensor, instantaneous, full-resolution datasets

Blended satellite sensor products, time/area averaged.

After Turk & Arkin, BAMS 2008

Both approaches are complementary


GV=Ground Validation

Global precipitation data sets


IPWG Intercomparison regionsNear real-time intercomparison of model & satellite estimates vs radar/gauge



Processing systemInitial setup:Setting of datesCleaning out old/decayed data

Acquiring data:Searching existing dataListing missing dataCreation of .netrc fileftp data sources

Remapping of data:… to regional grid or5 km PSG projection…

Results generation:Statistical analysisGraphical output

Web pages:Generate HTML filesCopying to server

Processing checksforeach day (d0-d0-31)

dn=dn+1

set d0=today

foreach day (d0-d0-31)

foreach product (p1-pn)

if (product for day) !exist

add to .netrc file

foreach datasource (s0-sn)

ftp datasource (4k)

N

Y

foreach product & day

remap to PSG using LUTs & standardise format

standardise filename


Generate statistics

Generate plots


generate HTML files

Processing checksforeach day (d0-d0-31)

dn=dn+1

set d0=today

foreach day (d0-d0-31)

foreach product (p1-pn)

if (product for day) !exist

add to .netrc file

foreach datasource (s0-sn)

ftp datasource (4k)

YN


remap to PSG using LUTs & standardise format

standardise filename


Generate statistics

Generate plots


generate HTML files

Set up list of past dates/days

Usually okay: sometimes needs

tweaking

Checks for a products results:

Okay if no results, but not if bad data

Prepares products into

common formatUsually okay…

Generates outputs:

Okay if there is rain…

Generates raw HTML:

Occasional issues with server

FTP runs several times:

4K buffer limit on macros

Automated systems they are NOT!

Validation productsBoth gauge and radar are used – depending upon the

real-time accessibility to these data sets

Australia: gauge analysis

United States: gauge and radar analysis

Europe: radar analysis

South America: Gauge analysis


European radar data: European composite with different radar and calibration and QC strategies; nominal 5km 15-minute product; available within ~1 hour; errors include range effects and anaprop (shipping, etc)

Gauge data: 09-09Z data (dated at end of period); ~6000 daily gauges with mean 10km spacing; available 1 month in arrears

Radar vs gauge data


Rad

ar (

daily

inte

grat

ed)

Gau

ge d

ata

Overall results


• There is a clear seasonal dependence of satellite product performance (poorer in winter)

• Models overestimate rain area/extent

• Satellite products underestimate rain area/extent

• Significant day-to-day variations

• Surface contamination of satellite products noticeable in winter

Performance dependent upon precipitation extent and intensity, spatial and temporal resolution of product – implications for high-resolution precipitation products

The future of IPWG validation regions


What are the key requirements in the future to make best of our own limited resources?

What are the requirements of the user community?

What are the requirements of the algorithm/product developers?

What new sources of data are available?

Should we go beyond daily regional comparisons (local-global, instantaneous-seasonal)?


New sites:• BALTEX region – limited number of algorithms (and

need to read HDF5!); also Canada (limited coverage)• High-latitude oceanic location (Iceland; Shetlands?)• Broaden existing regions (e.g. Europe)

New comparison techniques:• high-resolution products will need to rely upon

descriptive statistics• implementation of fuzzy-logic statistical techniques

New performance results:• Seasonal and global comparisons


Internationalframework

Current European radarnetwork – used in IPWG

EUMETNET radarcoverage



Latitude profiles: rainfall occurrence

Monthly and seasonal validationMonthly and seasonal diagnostic validation summaries



Suggested analysis levels:

• real time analysis: 'correctness' of near real-time products: should these be easily available (cics?)

• 'historical' analysis: 'authorised' release of quality-controlled products

• high-resolution analysis: similar set-up to daily/0.25 degree products (including instantaneous)

• 'training' data sets: example data sets from a comprehensive range of precipitation products – common naming convention and format


Personal recommendations

Evaluation of combined techniques:• individual component products for combined algorithm• actual combination techniques

Standardisation of precipitation product format• simple 2D arrays with accompanying text files• common format –variable size• quantitative resolution – 0.1 mm/day (WMO standard)• error fields can be accompanying files• Naming conventions standardised


The IPWG gauntlet: …. to generate global 1km, 1 minute rainfall estimates….

Anagnostou & Hossain: Satellite error propagation in flood predictionNorthern Italy: Basin scales from 116 to 1200 km2, storm induced flash flood event (Nov 1996)Satellite products: Kidd, 1h/4km, and 3B42, 3h/25km: reference rainfall: radar at 1h/1kmEnsembles generated based on SREM2D; Hydrological model simulations based on tRIBS distributed rainfall-runoff model

0 20 40 60 80 100 120 140 1600

100

200

300

400

500

600

700

Time (hrs)

Dis

char

ge (m

3/s

)

radar 1km

SREM2D KIDD 4km

0 20 40 60 80 100 120 140 1600

100

200

300

400

500

600

700

Time (hrs)

Dis

char

ge (m

3/s

)

radar 1km

SREM2D 3B42

Bac

chig

lio

ne

(1

200

km2)

0 20 40 60 80 100 120 140 1600

50

100

150

200

250

Time (hrs)

Dis

char

ge (m

3/s

)

radar 1km

SREM2D KIDD 4km

0 20 40 60 80 100 120 140 1600

50

100

150

200

250

Time (hrs)

Dis

char

ge (m

3/s

)

radar 1km

SREM2D 3B42

Po

sin

a (1

16

km

2)

PMIR: 4km/30min 3B42RT: 1deg/3hr

High resolution products do work!

Recommendations

To establish guidelines for satellite precipitation product data: establish common criteria that can and will be used to promote long-term data sets

Investigate statistical tests that are more relevant to the data sets we are dealing with, and that be applied over a range of spatial and temporal scales

To promote near real-time high resolution (sub-daily, <0.25 degree) precipitation inter-comparisons


To widen our validation regions to provide globally representative (inter)comparisons: at daily scales, but also monthly and seasonal scales.

Documents

IPWG Validation current status and future directions Chris Kidd Beth Ebert John Janowiak The University of Birmingham, Birmingham, UK Bureau of Meteorology,