Method for Object-based Diagnostic Evaluation (MODE)

Fake forecasts

Test case results for MODE

geometric cases mode quantity A quantity B

perturbed fake cases mode quantity A quantity B percentile of intensity median of max

MET – mode_analysis results centroid distance for matched pairs percentile intensity for forecast & observed objects

Geometric

1

2 3

4 5

THE WIN

NERarea ratio

1 1

2 1

3 0.29

4 1

5 0.15

THE WIN

NER

THE WIN

NER

Geometric

1

2 3

4 5

THE WIN

NERAngle difference

1 0

2 0

3 90

4 90

5 90

THE WIN

NER

Geometric

1

2 3

4 5

THE WIN

NERTotal interest

1 0.83

2 0.35

3 0.69

4 0.59

5 0.72

Perturbed fake cases

1. 3 pts right, 5 pts down

2. 6 pts right, 10 pts down

3. 12 pts right, 20 pts down

4. 24 pts right, 40 pts down

5. 48 pts right, 80 pts down

6. 12 pts right, 20 pts down, times 1.5

7. 12 pts right, 20 pts down, minus 0.05”

MODE objects

MODE objects

MODE objects

MODE objects

MODE objects

MODE objects

MODE objects

Perturbed Fake

case Position error and bias

Max interest for observed objects 1-8

1 (+ 3, - 5) 1 .96 1 1 1 1 1 1

2 (+ 6, -10)

3 (+12, -20)

4 (+24, -40)

5 (+48, -80)

6 (+12,-20) x1.5

7 (+12,-20) -0.05”

Perturbed Fake

case Position error and bias

Max interest for observed objects 1-8

1 (+ 3, - 5) 1 .96 1 1 1 1 1 1

2 (+ 6, -10) .99 .92 .99 .99 .96 .93 .99 .92

3 (+12, -20) .96 .88 .94 .88 .81 .88 .92 .88

4 (+24, -40) .87 .76 .81 .81 .72 .78 .81 .80

5 (+48, -80) .70 .52 .64 .73 .43 .50 .50 .59

6 (+12,-20) x1.5 .96 .66 .97 .88 .88 .86 .93 .79

7 (+12,-20) -0.05” .87 .60 .93 .88 .69 .72 .88 .81

Perturbed Fake cases

case Position error/bias

Median of max interest for observed objects 1-8

1 (+ 3, - 5) 1

2 (+ 6, -10) .975

3 (+12, -20) .88

4 (+24, -40) .805

5 (+48, -80) .555

6 (+12,-20) x1.5 .88

7 (+12,-20) -0.05” .84

Perturbed Fake cases

case Position error/bias

Mean centroid distance for (un)matched objects

1 (+ 3, - 5) (92) 17 grid pts

2 (+ 6, -10) (96) 22

3 (+12, -20) (102) 28

4 (+24, -40) (110) 39

5 (+48, -80) (107) 60

6 (+12,-20) x1.5 (91) 24

7 (+12,-20) -0.05” (104) 30

would like to have…

rainfall total for each objectarea ratio defined as fcst area/obs area

not just smaller area/larger area