MPE data validation

ByHOJJAT SEYYEDI

GEM 2008January 14, 2010

Comparing satellite derived rainfall with ground based radar for Northwestern Europe

Under the supervision of

Dr. Ben MaathuisDr. Chris Mannaerts

Contents

Introduction Background Significance of research Research objective Study area Research questions and results Conclusions Recommendations

3

Introduction

- Satellite precipitation estimates are widely used to measure global rainfall on near realtime and monthly timescales for climate studies, numerical weather prediction (NWP) data assimilation, now-casting and flash flood warning, tropical rainfall potential, and water resources monitoring

- The primary reason of implementing meteorological satellites is avoid the coverage and time gap of conventional ground-based rainfall data for a number of applications, above all hydrology and weather forecasting

- Use of global and local data have significantly limited because of lack of precipitation measurements over the oceans as well as uneven distribution of rain gauges and weather radars

- Similar to any observational data, investigating their accuracy and limitations is crucial. This is done by verifying the satellite estimates against independent data from rain gauges and radars (Levizzani, V., et al. 2007).

Background

- Heinemann and Kerényi, 2003; Grose et. al., 2002 and Ebert et. al. 1998 using geostationary satellite IR data based on cloud top temperature(Infra Red Methods).

Advantages: high temporal and spatial resolution for near real estimation of rainfall in areas with a sparse radar network (e.g. Central Africa) as well as for the long-term monitoring.

Disadvantage: There is no specific separation point between rain and no rain based on cloud top radiometric temperature.

4(Source: Heinemann and Kerényi, 2003)

Background

- Elbert et al. (1998), Smith et al. (1998) and Turk et al. (2000) implemented microwave (MW) radiometers for estimating precipitation. (Passive Microwave Methods)

Advantages: The MW can penetrate the cloud so it can contact directly to the hydrometeor. more accurate result than IR data

Disadvantages: poor spatial and temporal resolution of microwave sensors leads to significant sampling error in the estimation of accumulated precipitation. MW radiometers are not available on geostationary satellites.

5

(Source: Turk et al. 2000)

Background

The combination of both kinds of data, microwave (MW) data from polar orbiting satellites and IR (10.7 μm) data from geostationary systems (Vicente and Anderson 1993, Turk et al. 1998, Huffman et al., 2006). (Blended Global Products) The blended method mainly consist of three general steps i.e.: collocation, integration and transformation (Maathuis 2006).

Advantages: Higher spatial and temporal resolution, Higher accuracy, Global coverage

6 (Source: Turk et al. 2000)

Background

Multi-sensor Precipitation Estimate (MPE) newly developed algorithm by EUMETSAT. Blending passive microwave data from the SSM/I instrument on the US-DMSP satellites and images in the Meteosat IR channel for the estimation of instantaneous rain rates and daily rainfall averages on the resolution of METEOSAT (Heinemann and Kerényi, 2003).

MPE algorithm uses the passive MW rain rate measurements as calibration values therefore adjusts it geographically and temporarily. (Heinemann and Kerényi, 2003).

7

Geostationary stellite1(meteosat 8) High temporal-spatial resolution Cloud infrared images

Polar MW- hourly rainfall measurement

Combination

Algorithm

Multi-sensor Precipitation Estimate (MPE)

Background

Radar data as reference data because of high temporal (5-10 min) and spatial (1 km) resolution, representative of timescale and areal value estimated by satellites

The C-band Doppler weather radar employs scattering of radio-frequency waves (5.6 GHz for C-band) to measure precipitation and other particles in the atmosphere (Rinehart, 2004)

Radar images are themselves indirect measurements of rainfall and are prone to errors of various kinds, the sources of error affecting quantitative weather radar observations and quantitative precipitation estimation (QPE) at mid-latitudes, are non-uniform vertical profile of reflectivity (VPR), variability of the drop size distribution (DSD), and attenuation due to strong precipitation intensity (Michelson et al. 2005, Holleman and Beekhuis, 2004; Huuskonen and Holleman, 2006)

8 (source: Holleman, 2006)

Significance of the research

- The primary reason of implementing meteorological satellites is avoid the coverage and time gap of conventional ground-based rainfall data for a number of applications, above all hydrology and weather forecasting.

- Similar to any observational data, investigating their accuracy and limitations is crucial. This is done by verifying the satellite estimates against independent data from rain gauges and radars (Levizzani, et al. 2007).

- No study has compared the potential of METEOSAT 8 and METEOSAT 9 MPE products with ground based radar data and/or gauge in estimating rainfall in high spatial and temporal resolution.

Research Objective

General Objective

Evaluating EUMETSAT MPE products in comparison with ground based rain radar data

Sub Objectives

Implementing categorical verification statistics for assessing how the spatial distribution of the MPE products from METEOSAT8 and METEOSAT9 differ from the reference data.

Implementing continuous verification statistics for assessing how the values of the MPE products from METEOSAT8 and METEOSAT9 differ from the reference data.

Source data and study area (1)

-The study area is Northwestern Europe with focus on the Netherlands and some parts of Belgium.-Due to the Earth’s curvature, the distance over which weather radars observe the entire cloud is limited a maximum range of 200 km.- Water bodies are masked out the weather radar observations to exclude unrealistically high rain rate values caused by sea clutter.

Gauge adjusted ground based radar data

(mm/h)(dBZ)

Z - R

Z = 200R1.6

Source data and study area (2)

(mm/h)

(mm/h)

Geostationary satellite (MET8 and MET9)

High temporal-spatial resolution

Cloud infrared images

Polar-SSM/I MW- hourly rainfall measurement

Blending Algorithm

Multi-sensor Precipitation Estimate (MPE)

EUMETSAT MPE products and blending algorithm

Rainfall events

1st event 2 nd event

(20090514, 19:00 UTC to 20090517, 23:45 UTC) (20090525, 0000 UTC to 20090527, 23:45 UTC)

CLAI

- Heavy rain with flooding in this period

-Heavy showers complexes over a large part of the

Netherland- Showers were accompanied

by exceptionally active thunderstorms

Visual comparison: Met8-Radar-Met9

mm/h

mm/hmm/h

1st event (20090514, 19:00 UTC to 20090517, 23:45 UTC)_UTM zone 32_WGS 84,3×3 km grid size

Inst

anta

neou

s1

Hou

r ac

cum

ulat

ed3

Hou

rs

accu

mul

ated

mm/h mm/hmm/h

mm/3h mm/3hmm/3h

Visual comparison : Met8-Radar-Met9

mm/hmm/hmm/h

2nd event (20090525, 0000 UTC to 20090527, 23:45 UTC)_ UTM zone 32_WGS 84,3×3 km

Inst

anta

neou

s1

Hou

r ac

cum

ulat

ed3

Hou

rs

accu

mul

ated

mm/3hmm/3hmm/3h

mm/h mm/hmm/h

Research question and method (1)

What are the differences in spatial distribution of EUMETSAT MPE products from METEOSAT 8 (5 min temporal resolution) and METEOSAT 9 (15 min temporal resolution) in comparison with reference data?

METEOSAT8_MPE5 min temporal

resolution(mm/h)


resolution(mm/h)

Radar rainfall data5 min temporal

resolution(mm/h)

Creating sub-map for study area

Data accumulation:- 1 hour- 3 hours

Reclassification to “Rain” and “ No Rain”

3Hours accumulated:

- Radar-Meteosat8- Radar-Meteosat9

1Hour accumulated:


Instantaneous:- Radar-Meteosat8- Radar-Meteosat9


Cros

s tab

leD

ata

sets

Cate

goric

al

statis

tics

anal

ysis



Results

Hypothesis testing

Categorical comparison


What are the differences in spatial distribution of EUMETSAT MPE products from METEOSAT 8 (5 min temporal resolution) and METEOSAT 9 (15 min temporal resolution) in comparison with reference data?


resolution(mm/h)


resolution(mm/h)


resolution(mm/h)



Reclassification to “Rain” and “ No Rain”

3Hours accumulated:


1Hour accumulated:




Cros

s tab

leD

ata

sets

Cate

goric

al

statis

tics

anal

ysis



Results

Hypothesis testing

Statistical ScoreRadar _METEOSAT 9 Radar _METEOSAT 8(RSS)

Event1 Event2 Event1 Event2

POD 0.46 0.28 0.41 0.33

FAR 0.17 0.30 0.20 0.34

CSI 0.42 0.25 0.37 0.28

Accuracy 0.74 0.72 0.71 0.72

Bias 0.55 0.40 0.52 0.51

ETS 0.27 0.15 0.22 0.17



POD 0.53 0.31 0.51 0.34

FAR 0.13 0.27 0.16 0.31

CSI 0.49 0.28 0.46 0.30

Accuracy 0.73 0.69 0.71 0.68

Bias 0.61 0.43 0.60 0.50

ETS 0.30 0.15 0.26 0.15



POD 0.57 0.35 0.59 0.36

FAR 0.07 0.15 0.10 0.17

CSI 0.54 0.33 0.55 0.34

Accuracy 0.70 0.58 0.69 0.58

Bias 0.61 0.41 0.65 0.44

ETS 0.28 0.13 0.26 0.13

Categorical statistical results obtained

Instantaneous

One hour accumulated

Three hours accumulated


What are the differences in estimated values by EUMETSAT MPE products from METEOSAT 8 (5 min temporal resolution) and METEOSAT 9 (15 min temporal resolution) in comparison with reference data?


resolution(mm/h)


resolution(mm/h)


resolution(mm/h)



Creating Histogram

3Hours accumulated:


Dat

a se

tsCo

ntin

uous

sta

tistic

s an

alys

is 1Houraccumulated:



Results

Hypothesis testing

Export to DBF

Continuous comparison


What are the differences in estimated values by EUMETSAT MPE products from METEOSAT 8 (5 min temporal resolution) and METEOSAT 9 (15 min temporal resolution) in comparison with reference data?


resolution(mm/h)


resolution(mm/h)


resolution(mm/h)



Creating Histogram

3Hours accumulated:


Dat

a se

tsCo

ntin

uous

sta

tistic

s an

alys

is 1Houraccumulated:



Results

Hypothesis testing

Export to DBF

Results of continuous comparisonIn

stan

tane

ous

1 H

our

accu

mul

ated

3 H

ours

ac

cum

ulat

ed

First event Second event

Inst

anta

neou

s1

Hou

r ac

cum

ulat

ed3

Hou

rs

accu

mul

ated


Results of continuous comparison

Inst

anta

neou

s1

Hou

r ac

cum

ulat

ed3

Hou

rs

accu

mul

ated



Inst

anta

neou

s1

Hou

r ac

cum

ulat

ed3

Hou

rs

accu

mul

ated



Conclusions

The null hypothesis (H0): “no difference in spatial accuracy of EUMETSAT MPE products from METEOSAT8 and METEOSAT9” is rejected.

For the normal rainfall events analyzed using MPE product from METEOSAT9 seems to be more reliable.

The null hypothesis (H0): “no difference in estimated values by EUMETSAT MPE products from METEOSAT8 and METEOSAT9” is rejected.

MPE products is over estimating the severe event analyzed.

Recommendations The spatial accuracy assessment shows the METEOSAT 9 MPE product has higher statistical scores for north western Europe. However, it would be better to do a similar high temporal and spatial

comparison for the other parts of the world.

The study was conducted on a short time scale only, so further work on longer series can help to improve the understanding of the accuracy of MPE products, e.g. on a seasonal basis.

Applying diagnostic verification methods such as fuzzy verification, which yield more in depth information about the nature of the errors.

Using Cloud Analysis Image (CLAI) in conjunction with MPE products to study the relationship between cloud type and rainfall estimation accuracy by METEOSAT 8 and METEOSAT 9.

Applying multi categorical analysis for a variety of rain thresholds to see how the performance depends on the rain intensity.

Thank you

2*2 contingency table

Ground based RADAR

Yes No

RSS_METEOSAT8YES Hits False alarms Estimated Yes

NO MissesCorrect

negativesEstimated No

Observed Yes Observed No N=Total

total

negativecorrecthitsAccuracy

misseshits

alarmsfalsehitsBIAS

misseshits

hitsPOD

alarmsfalsehits

alarmsfalseFAR

alarmsfalsemisseshits

hitsCSITS

random

random

hitsalarmsfalsemisseshits

hitshitsETS

total

alarmsfalsehitsmisseshitshitsrandom

))((

Scatter graph for MPE products, instantaneous comparison, second event

The diagonal dotted line in the graph shows the ideal 1:1 relationship between reference values and estimated values by METEOSAT 8 and METEOSAT 9

Regressions for mean rainfall values in MPE products, =0.05α

Instantaneous

One hour accumulated

Three hours accumulated

Radar _METEOSAT 9 Radar _METEOSAT 8(RSS)


Observations (n) 288 308 288 308

Slope coefficient,

p-value

7.35, 3.56E-122 1.84, 3.2E-64 7.96, 1.51E-127 1.21, 1.42E-58

Intercept coefficient,

p-value

-0.11, 0.047 -0.03, 0.27 -0.16, .004 0.01, 0.50

r² 0.85 0.61 0.87 0.57

Significance F 3.57E-122 3.2E-64 1.51E-127 1.42E-58




Slope coefficient,

p-value

7.60, 6.13E-35 1.84, 9.72E-17 8.23, 2.59E-36 1.20, 3.89E-15


p-value

-0.13, 0.18 -0.03, 0.58 -0.18, .06 0.01, 0.71

r² 0.89 0.60 0.90 0.56





Slope coefficient,

p-value

7.82, 1.79E-14 1.98, 6.61E-07 8.43, 2.61E-15 1.26, 7.46E-06


p-value

-0.50, 0.19 -0.13, 0.59 -0.65, 0.09 0.02, 0.92

r² 0.93 0.67 0.94 0.59


Cloud types

Source :Dr. Maathuis presentation on METEOSAT-MSG

Illustration of the IR signal from differentcloud types

Source :Dr. Maathuis presentation on METEOSAT-MSG

METEOSAT8 METEOSAT9

0º

70 º N

12 min

15º N5min

70 º S

52º N

140/12=11.65º/min

11min to the Netherlands from start point

3.5min to the Netherlands from start point

Time stamp on images are based on the end of scan time

In case of RSS, 1.5min difference between real scan time and time stamp on image

In case of MET9, 4min difference between real scan time and time stamp on image

Technology

MPE data validation