Brief evaluation of REMAD and CMAQ for east Texas tracers

Mark Green, DRIModel results from Christian Seigneur and Betty Pun- AER (CMAQ and REMSAD),

and Mike Barna- CIRA (REMSAD)

Methodology

• Evaluation used 6-hour tracer data and model predictions for two 10-day periods where tracer data was sequestered: August 15-25, 1999; October 5-15, 1999

1) Average observation (paired data only) = OONi

N

i

1

1

2) Average prediction (paired data only) = PPNi

N

i

1

1

3) Bias of mean OP

4) Standard deviation of predictions and observations= p , o 5) Coefficient of variation predictions and observations=

P

p , O

o

6) RMS Error = 2

1

)(1ii

N

i

OPN

7) RMS error normalized by mean observation = O

OPN

ii

N

i

2

1

)(1

8) Correlation coefficient between predicted and observed values 9) Regression equation between predicted and observed

10) Frequency distribution of observations and predictions plot 11) Scatterplots 12) Time series 13) Spatial pattern comparison

BRAVO Study Tracer Release & Monitoring Locations

bsp and estimated SO4 scattering

0102030405060708090

7/9 7/19 7/29 8/8 8/18 8/28 9/7 9/17 9/27 10/7 10/17 10/27

b sp o

r sul

fate

sca

tterin

g (M

m-1

)

24 hr SO4 scat 24 hr bsp

CMAQ CMAQ

-0.05

0.00

0.05

0.10

0.15

0.20

8/14 8/15 8/16 8/17 8/18 8/19 8/20 8/21 8/22 8/23 8/24 8/25 8/26

BNL

i-PPC

H (fL

/L)

BIBE

FTST

MARA

MONA

PRSG

SNVI

0

1000

2000

3000

4000

5000

6000

8/14 8/15 8/16 8/17 8/18 8/19 8/20 8/21 8/22 8/23 8/24 8/25 8/26

part

icul

ate

S (n

g/m

3)

BIBE

FTST

MARA

MONA

PRSG

SNVI

August- Particulate S and observed iPPCH

Northeast Texas tracer and Observed S

Northeast Texas tracer

Observed S

12km CMAQ and 36km REMSAD NE Texas tracer obs/preds time series for sequestered period

-0.05

0.00

0.05

0.10

0.15

0.20

8/14 8/15 8/16 8/17 8/18 8/19 8/20 8/21 8/22 8/23 8/24 8/25 8/26

BNL

i-PPC

H (fL

/L)

BIBE

FTST

MARA

MONA

PRSG

SNVI

0.00

0.05

0.10

0.15

0.20

0.25

0.30

8/14 8/15 8/16 8/17 8/18 8/19 8/20 8/21 8/22 8/23 8/24 8/25 8/26

CMAQ

i-PP

CH (f

L/L)

BIBE

FTST

MARA

MONA

PRSG

SNVI

August- CMAQ and observed iPPCH

Look at CMAQ first… Obs

Preds

12km CMAQ and 36km REMSAD NE Texas tracer obs/preds time series for sequestered period

-0.05

0.00

0.05

0.10

0.15

0.20

8/14 8/15 8/16 8/17 8/18 8/19 8/20 8/21 8/22 8/23 8/24 8/25 8/26

BN

L iP

PCH

(fL/

L)

BIBE

FTST

MARA

MONA

PRSG

SNVI

0.000.02

0.040.060.080.10

0.120.140.16

0.180.20

8/14 8/15 8/16 8/17 8/18 8/19 8/20 8/21 8/22 8/23 8/24 8/25 8/26

REM

SAD

iPPC

H (f

L/L)

BIBE

FTST

MARA

MONA

PRSG

SNVI

August- REMSAD and observed iPPCH

Now REMSAD… Obs

Preds

Tracer Release from Big Brown, 22-Aug-1999 00:00 GMT

CMAQ-12km REMSAD-36km

A u g u s t P P C H ( B i g B r o w n t r a c e r )o b s e r v e d a n d C M A Q 3 6 k m

- 0 . 0 5

0 . 0 0

0 . 0 5

0 . 1 0

0 . 1 5

0 . 2 0

8 / 1 4 8 / 1 5 8 / 1 6 8 / 1 7 8 / 1 8 8 / 1 9 8 / 2 0 8 / 2 1 8 / 2 2 8 / 2 3 8 / 2 4 8 / 2 5 8 / 2 6

BNL i-P

PCH (fL

/L)

B I B E

F T S T

M A R A

M O N A

P R S G

S N V I

- 0 . 1

0

0 . 1

0 . 2

0 . 3

0 . 4

0 . 5

0 . 6

8 / 1 4 8 / 1 6 8 / 1 8 8 / 2 0 8 / 2 2 8 / 2 4 8 / 2 6

PP

CH

(fL

/L)

B I B EF T S TM A R AM O N AP R S GS N V I

CMAQ 36 km

Obs

PPCH Plume Movement (36 km and 4 km)22 August 7, 13, 19 GMT

From AER

August iPPCHBNL CMAQ REMSAD Bret w=div

mean 0.029 0.017 0.036 0.019std 0.044 0.051 0.052 0.046COV 1.51 3.10 1.47 2.46bias of mean 0.56 1.22 0.63r 0.08 0.67 0.46r squared 0.01 0.45 0.21RMSE 0.066 0.040 0.048Norm RMSE 2.26 1.38 1.63

Model performance summary statistics –Northeast Texas tracer, August episode

August ocPDCHBNL CMAQ REMSAD Bret w=div

mean 0.52 0.62 0.57 0.66std 0.71 0.76 0.54 0.99COV 1.37 1.22 0.94 1.50bias of mean 1.19 1.10 1.27r 0.25 0.39 0.35r squared 0.06 0.15 0.12RMSE 0.91 0.70 1.01Norm RMSE 1.74 1.35 1.94

Model performance summary statistics – Eagle Pass tracer, August episode

-0.20

-0.10

0.00

0.10

0.20

0.30

0.40

0.50

10/4 10/5 10/6 10/7 10/8 10/9 10/10 10/11 10/12 10/13 10/14 10/15 10/16

BNL

PTCH

(fL/

L) BIBE

FTST

MARA

MONA

PRSG

SNVI

0

500

1000

1500

2000

2500

3000

3500

4000

4500

10/4 10/5 10/6 10/7 10/8 10/9 10/10 10/11 10/12 10/13 10/14 10/15 10/16

Parti

cula

teS

(ng/

m3)

BIBE

FTST

MARA

MONA

PRSG

SNVI

October- Particulate S and observed PTCH

Houston tracer and observed S

Houston tracer

Observed S

12km CMAQ and 36km REMSAD Houston tracer obs/preds time series for sequestered period

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

10/4 10/5 10/6 10/7 10/8 10/9 10/10 10/11 10/12 10/13 10/14 10/15 10/16

AER

PTCH

(fL/

L) BIBE

FTST

MARA

MONA

PRSG

SNVI

-0.20

-0.10

0.00

0.10

0.20

0.30

0.40

0.50

10/4 10/5 10/6 10/7 10/8 10/9 10/10 10/11 10/12 10/13 10/14 10/15 10/16

BNL

PTCH

(fL/

L)

BIBEFTST

MARAMONA

PRSGSNVI

October- CMAQ and observed PTCH

Look at CMAQ first…

Obs

Preds

0.00

0.05

0.10

0.15

0.20

0.25

10/4 10/5 10/6 10/7 10/8 10/9 10/10 10/11 10/12 10/13 10/14 10/15 10/16

REM

SAD

PTC

H

BIBE

FTST

MARA

MONA

PRSG

SNVI

-0.20

-0.10

0.00

0.10

0.20

0.30

0.40

0.50

10/4 10/5 10/6 10/7 10/8 10/9 10/10 10/11 10/12 10/13 10/14 10/15 10/16

BNL

PTCH

(fL/

L) BIBE

FTST

MARA

MONA

PRSG

SNVI

October- REMSAD and observed PTCH

12km CMAQ and 36km REMSAD Houston tracer obs/preds time series for sequestered period

Now REMSAD…

Obs

Preds

CMAQ 12 and 4 km simulation PTCH (Houston Tracer) October 12

6-hour average starting 19 GMT22 GMT

REMSAD PTCH

36 km

12 km

REMSAD vs. CMAQ36 km vs. 12 km

PTCH (Houston Tracer) October 12

- 0 . 2 0

- 0 . 1 0

0 . 0 0

0 . 1 0

0 . 2 0

0 . 3 0

0 . 4 0

0 . 5 0

1 0 / 4 1 0 / 5 1 0 / 6 1 0 / 7 1 0 / 8 1 0 / 9 1 0 / 1 0 1 0 / 1 1 1 0 / 1 2 1 0 / 1 3 1 0 / 1 4 1 0 / 1 5 1 0 / 1 6

BNL PTCH (fL/L)

B I B E

F T S T

M A R A

M O N A

P R S G

S N V I

O c t o b e r P T C H ( H o u s t o n t r a c e r ) o b s e r v e d a n d C M A Q 3 6 k m

- 0 . 1

0

0 . 1

0 . 2

0 . 3

0 . 4

0 . 5

1 0 / 4 1 0 / 6 1 0 / 8 1 0 / 1 0 1 0 / 1 2 1 0 / 1 4 1 0 / 1 6

PT

CH

(fL

/L)

B I B EF T S TM A R AM O N AP R S GS N V I

CMAQ 36 km

Obs

PTCH Plume Movement (36 km and 4 km)12 October 19 GMT; 13 October 1, 7 GMT

0.15

0.15

0.05

0.05

0.15

0.15

0.05

0.05

From AER

AER summary of CMAQ tracer transport

• In the 36 km simulations, tracer plumes still miss monitoring sites at and near Big Bend on some days

• In both the 36 and 4 km simulations for August, the PPCH plumes are too far East

• In both the 36 and 4 km simulations for October, the PTCH plumes are too far South

• 36 km simulations bring plume a lot closer to Big Bend than 4 km simulations

October PTCHBNL CMAQ REMSAD Bret w=div

mean 0.14 0.03 0.05 0.08std 0.10 0.07 0.04 0.14COV 0.71 2.49 0.77 1.73bias of mean 0.19 0.38 0.57r 0.00 0.18 0.10r squared 0.00 0.03 0.01RMSE 0.17 0.13 0.17Norm RMSE 1.17 0.94 1.22

Model performance summary statistics –northeast Texas tracer, August episode

Model performance summary statistics – Eagle Pass tracer, October episode

October ocPDCHBNL CMAQ REMSAD Bret w=div

mean 0.36 0.29 0.29 0.53std 0.71 0.69 0.40 1.79norm std 1.97 2.36 1.39 3.39bias of mean 0.81 0.80 1.46r 0.43 0.25 0.05r squared 0.19 0.06 0.00RMSE 0.75 0.73 1.89Norm RMSE 2.07 2.01 5.24

SO4 at K-BAR, July-October 1999

Total Sulfur at K-BAR, July-October 1999

October Episode, 4-km SimulationRegression Analysis

y = 0.3872x + 2.1778R2 = 0.6851

0

2

4

6

8

0 2 4 6 8Obs ervat ions

Sim

ulat

ed V

alue

s

Sulfate (g/m3) at Big Bend From AER