Michael S. Fox-Rabinovitz, ESSIC (Earth System Science Interdisciplinary Center, University of Maryland, and NASA/GSFC/DAO (Data Assimilation Office) foxrab@atmos.umd.edu or foxrab@dao.gsfc.nasa.gov

Acknowledgments: L. L. Takacs, R. C. Govindaraju

Variable-Resolution SG-GCMs

4th International Workshop on Next Generation Climate Models, for Advanced High Performance Computing Facilities 12-14 March, 2002, Boulder, CO

• INTRODUCTION: the variable-resolution stretched-grid (SG) approach to regional climate modeling

2. SG-GCM ANNUAL (Nov.-97 – Dec.98) simulation with the stretched grid with MULTIPLE (four) areas of interest

3. A 12-YEAR (Oct.-1986 – Dec.1998) AMIP-type SG-GCM simulation

4. Limited ENSEMBLE SG-GCM integrations

5. Development of the NEW SG-fvGCMs with the NCAR CAM PHYSICS, and advanced parallel computing techniques

6. Conclusions

7. PROPOSAL on the International Stretched-Grid Model Intercomparison Project (SGMIP)

 

Contents

Questions/hypotheses:- What are the advantages of the SG-approach vs. nested-grid approach? -What are the computational problems for model dynamics to be controlled when using the SG-approach? - What are the limitations/restrictions associated with the SG-approach for regional climate modeling, in terms of necessity to control the SG-parameters?- What is an optimal way for treating/calculating model physics and stationary surface boundary forcing (orography and land-seadifferences): on a stretched or uniform intermediate resolution grid?What is the potential of the SG-approach for obtaining: * high quality/integrity of global circulation * consistent interactions between global/large - and regional/meso- scales * efficient regional downscaling to mesoscales * computational and parallel performance efficiency-Could variable resolution be used for future global climate models?

A STRETCHED GRID WITHONE AREA OFINTEREST

An example of a stretched grid with one area of interest (AOI) with enhanced 60 km regional resolution over the U.S.

A blow-up:

An example of a stretched grid with four areas of interest over:1.the U.S./Western Atlantic ocean; 2. El Nino area/Brazil-Argentina; 3. India-China; nd 4. Eastern Indian Ocean/Australia

Q: What are optimal/necessary requirements/controls for a

SG-design to be imposed for obtaining successful computationally efficient regional climate simulations that would provide:- An efficient downscaling over the area(s) of interest;

- Consistent/realistic interactions between global/large-

and regional/meso-scales;

- A sufficient quality/skill of global circulation.

1. Fine uniform resolution over the area(s) of interest.

2. Uniform stretching with constant local stretching factors.

3. Moderate local stretching factors, within ~5%

4. Limited maximum grid intervals, within 2-3 degrees, the resolution of typical GCMs(to control the accuracy glob.appr.)

SG-design

Nested grid LAM driven by a GCM Variable Resolution GCM (SG-GCM)

• Problems with one-way interactions

• Problems with long-waves/global scales

• Abrupt change of resolution generates

noise and spurious phenomena

• Need a buffer zone along the lateral

boundaries to relax the driving and the

nested variables

• Non-autonomous. Depends on

driving data for the LBCs

• Spatial interpolations required

• Vertical and temporal resolutions can

be different

• Climate solution depends on the

domain size and resolution jump

• Proved to be a workable solution

• Consistent interactions between

long-wave/global and regional/meso-

scales

• High qualityof global scales

• No abrupt change of resolution, but

stretching factor must be moderate

• No buffer zone needed

• Autonomous. No dependence on

driving data for the LBCs

• No spatial interpolation required

• Vertical and temporal resolutions

fixed for the whole globe

• Climate solution depends on

stretching

• Proved to be a workable solution

(e.g. Déqué 1995,1998; Fox-Rabinovitz

et al. 1997,2000,2001; McGregor 1998)

Issues regarding two types of downscaling strategies

===

==

Variable-Resolution GCMs1. U.S.: Finite-difference GEOS SG-GCM (Fox-Rabinovitz et al 1997, 2000,

2001, Fox-Rabinovitz 2000)

2. U.S.: Finite-volume SG-GCM (M. Fox-Rabinovitz and Kaosan Yeh, using the Lin-Rood GEOS fvGCM)

3. Canadian RPN/CMC, GEM (Global Environmental Multiscale) model (Cote et al. 1998, 2000)

4. Meteo-France, ARPEGE (Deque and Piedelievre 1995, Deque et al. 1998 Deque 2000)

5. Australian CSIRO Model (McGregor 1996, McGregor and Dix 1997, 1998);

The new SG-GCM (under development):

6. Spectral-element SG-GCM (a companion study with F. Baer, J. Tribbia, A. Fournier)

Other variable-resolution models:

7. University of Utah (UU) model (Paegle 1989, Paegle et al. 1996))

8. 2-D spectral model (Hardiker 1997, the FSU group)

9. NCSU non-hydrostatic model (the Semazzi’s NCSU group)

GEOS (Goddard Earth Observing System) GCM (the basic model)

- Dynamics: finite-differences (Arakawa and Lamb 1981, Suarez and Takacs 1995, Fox-Rabinovitz et al. 1991)• Filters: high latitude Fourier; Shapiro (1970);• Resolution: 2x2.5 or 1x1 degree, 70 or 48 layers;• Radiation: Chou and Suarez (1994);• Turbulence/PBL: Helfand and Labraga (1989);• Convection: RAS (Relaxed Arakawa-Schubert) Moorthi and Suarez 1992; • Land-surface model: Koster and Suarez 1994;• Participation in AMIP

GEOS SG-GCM(Fox-Rabinovitz et al. 1997, 2000, 2001, 2002; Fox-Rabinovitz 2000)

- Same dynamics and physics as in GEOS GCM

- Model dynamics and orography calculated on the stretched grid

- Model physics calculated on an intermediate uniform resolution grid (Lander and Hoskins (1997))

- The integration history resides effectively on the stretched grid

- Modified for variable-resolution polar (Takacs et al. 1999) and local filters

- Regional resolution for SGs (obtained by redistribution of global grid points of the 2x2.5 or 1x1 degree uniform grids):

a) 60, 40, or 25 km with one large area of interest (like the U.S.)

b) 50 or 40 km with four large areas of interest

c) 10-15 km with one subregion of interest (like a river basin)

Development of the new SG-fvGCM (L-R dynamics, NCAR physics) (K. Yeh, M. Fox-Rabinovitz, S.J. Lin) * The SG-fv-dynamical core (dycore) has been developed and is being tested with the generalized Held-Suarez forcing * The generalized SG-fv-dycore is consistent with the original Lin-Rood (L-R) uniform-grid (UG) dycore of the NASA fvGCM * The dynamics and polar filter are modifies for variable resolution * The similarity of global fields is obtained for SG & UG dycores * Enhanced regional resolution provides efficient downscaling to mesoscales, i.e. it results in producing stronger gradients, stronger jets; and more pronounced mesoscale features over the region(s) of interest * The SG-dynamics (including orography) is combined with the intermediate resolution physics. Implementation of the SG-physics is being investigated. * Target resolution: for 4 or more areas of interest: 15-25 km; and for 1 to 4 areas of interest: 5-10 km In other words: at least doubling the resolution of any uniform grid

Parallelization techniques for the SG-GCMs

1. GEOS SG-GCM: - the MPI mode

2. SG- FV-GCM: - the MPI mode for distributed memory

- the SMP mode for shared memory

- the hybrid mode with multi-level parallelism, to take advantage of CPUs with shared memory Note: the overhead due to variable resolution is limited due to grid variability in the horizontal only

Variable-resolution grids used and planned to use: 1. Used with finite-difference approximation: 

- Stretched grid with 1 area of interest

- Stretched grid with multiple (four) areas of interest

2. Will be used with the new finite-volume SG-GCM - Enhanced resolution over the model global orography - Exploring new stretching strategies

3. Future tests: Using variable-resolution grids as an alternative to

uniform grids, for global simulation purposes:

- Stretched grids with multiple (6-16) areas of interest

- Global anisotropic monotonic grids:

* flow-and-surface-boundary-forcing dependent

* population-and-pollution distribution dependent

* stationary adaptive grids, with gradually changing intervals

* dynamic adaptive grids slowly changing in time, i.e. same as above but with added time adaptive capabilities

Simulation and data assimilation of the 1998 anomalous regional climate events with the GEOS SG-GCM and SG-DAS with

multiple (four) areas of interest1. Quality/integrity of global

fields2. Reproducing regional

anomalies

500 hPa KE spectra for the SG-GCM, SG-DAS, and ECMWF reanalyses for AMJ 1998

AMJ 1998 ZONAL U

SG-GCM(50km)

vs. SG-DAS(50km) analysis

and

ECMWF (1x1) reanalysis

GLOBAL PRECIP AMJ 1998

SG-GCM(50km)

vs.

GPCP(1x1) DATA

U.S. AMJ 1998 H-500

SG-GCM(50km)

SG-DAS(50km)

BIAS or (a) – (b)

U.S. AMJ 1998 H-500 ANOMALY

SG-GCM(50KM)

SG-DAS(50KM)

ECMWF(1X1)

U.S. AMJ 1998 U-500 ANOMALY

SG-GCM(50KM)

SG-DAS(50KM)

ECMWF(1x1)

U.S. AMJ 1998 PRECIP

SG-GCM(50km)

GPCP(1x1)

NCEP(0.25x0.25)GAUGE DATA

Mexico MAM 1998 Precipitation

SG-GCM(50km)

SG-DAS(50km)

Xie-Arkin(2x2)GAUGE and SAT. DATA

China July 1998 Precipitation

SG-GCM(50km)

SG-DAS(50km)

Xie-Arkin(2.5x2.5) GAUGE DATA

China July 1998Precip Anomaly

SG-GCM(50km)

vs.

SG-DAS(50km)

Xie-Arkin(2.5x2.5)GAUGE DATA

The12-year (1987-1998) SG-GCM simulation and limited

ensemble calculations

12-YEAR SIMULATION: 9-YEAR MEAN(1988-1996) JUNE PREC. SG-GCM(60KM)

vs.

U of DE GAUGE DATA

and

NCEP(Xie-Arkin) DATA

12-YEAR SIMULATION9-YEAR MEAN(1988-1996)JUNE PRECIP(for the SE U.S. subregion)

SG-GCM(60KM) vs.

U of DE GAUGE DATA

and

NCEP(Xie-Arkin) DATA

H-500 “Spaghetti” diagrams12-year SG-GCM ensemble

January 1987

January 1988

January 1989

H-500 Spaghetti diagrams12-year SG-GCM ensemble

July 1987

July 1989

Applications of SG-GCM and SG-DAS products:

* Hurricanes, typhoons, and severe storms: - 25 km regional resolution - all the 1998 events, for all four global quadrants

* Atmospheric chemistry transport experiments: - TOPSE - TRACE-P

* Hydrologic model experiments for the major U.S. river basins (under development)

* New satellite data impact studies (future plans)

Hurricane Floyd, 45-h forecast, 9/13/1999, SG-GCM(0.25X0.25) (low-level clouds; SLP) (wind-speed; RH)

Hurricane Floyd, 84-h forecast, 9/15/1999, SG-GCM(0.5X0.5) (SLP(lat 30N); wind-speed; omega) (U-wind; V-wind; SLP)

TOPSE Mission (FM-2000) CURTAIN PLOTS(FLIGHT TRACK: DASHED LINE)TRACE GAS: 7Be UNIFORM GRID (2.5x2.5)

STRETCHED GRID (60KM)

(the courtesy of D. Allen and K. Pickering)

1. Fox-Rabinovitz, M. S., G.L. Stenchikov, M.J. Suarez, and L.L. Takacs, 1997: A Finite-Difference GCM Dynamical Core with a Variable Resolution Stretched Grid. Mon. Wea. Rev.,vol. 125, No. 11, 2943-2968.

2. Qian, J.-H., F. Giorgi, and M. S. Fox-Rabinovitz, 1999: Regional Stretched Grid Generation and its Application to the NCAR RegCM. JGR, v. 104, NO. D6, 6501-6513.

3.  Fox-Rabinovitz, M. S., G.L. Stenchikov, M.J. Suarez, L.L. Takacs, and R.C. Govindaraju, 2000: A Variable Resolution GCM Dynamical Core with Real Orography. Mon. Wea. Rev., Vol. 128, No. 6, 1883-1898.

4.  Fox-Rabinovitz, M. S., 2000: Regional climate simulation of the anomalous U.S. summer events using a variable-resolution stretched-grid GCM. J.Geophys Res.(the special issue on Advances in Regional Climate Modeling), Vol. 105 , No. D24 , pp. 29,635-29,646.

5. Fox-Rabinovitz, M. S., L.L. Takacs, and M.J. Suarez, 2001: A Variable Resolution Stretched Grid GCM: Regional Climate Simulation. Mon. Wea. Rev., Vol. 129, No. 3, pp. 453-469.

Recent Journal Publications on the GEOS SG-GCM and SG-DAS

6. Takacs, L. L., W. Sawyer, M. J. Suarez, and M. S. Fox-Rabinovitz, 1999: Filtering Techniques on a Stretched Grid GCM, NASA/TM-1999-104606, Vol. 16, 50 pp.

7. Berbery E. H., and M.S. Fox-Rabinovitz, 2002: Simulation of the North American Monsoon System and its Link to Midwest Precipitation using a variable resolution stretched grid GCM, submitted.

8. Fox-Rabinovitz, M.S., L.L. Takacs, and R.C. Govindaraju, 2002: A variable-resolution SG-GCM and SG-DAS with multiple areas of interest: Studying the anomalous regional climate events of 1998, submitted.

9. Fox-Rabinovitz, M.S., L.L. Takacs, and R.C. Govindaraju, 2002: A 12-year AMIP-type SG-GCM simulation of the U.S. climate, to be submitted.

10. Yeh, K.-S., M.S. Fox-Rabinovitz, and S.J. Lin, 2002: A variable-resolution finite-volume dynamical core, to be submitted.

11. R.J. Park, K.E. Pickering, D.J. Allen, G.L. Stenchikov, and Fox-Rabinovitz, M.S.,: 2002: Simulation of tropospheric ozone using the University of Maryland chemical transport model. Part II: Regional simulation using a stretched-grid formulation, to be submitted.

Recent Journal Publications on the GEOS SG-GCM and SG-DAS (cont.)

1. The variable-resolution stretched-grid GCMs with enhanced resolution over the area(s) of interest are viable tools for regional climate simulation, with an advantage of providing efficient regional downscaling to mesoscales while preserving the high quality of global circulation.

2. The successful annual SG-GCM simulation of the anomalous regional climate events of 1998 with 50 km regional resolution are performed for FOUR areas of interest (one at each global quadrant): 1. the U.S. and Mexico; 2. India- China; 3. Brazil- Argentina; and 4. Eastern Indian Ocean and Australia.

3. The successful 12-year SG-GCM AMIP-type simulation is performed for the U.S. The applicability of the SG-approach to long-term multiyear regional climate modeling is confirmed.4. The limited SG-GCM ensemble integrations are found useful

for reducing the uncertainty of regional climate simulation.

Conclusions

Conclusions (cont.)5. Development of the new GEOS SG-fvGCM employing finite-

volume (Lin-Rood) dynamics and NCAR CCM3/4 physics. - The new stretching strategies are being investigated. - Using anisotropic monotonic (gradually and slowly) adaptive

(stationary and dynamic) grids as an alternative to uniform grids will be addressed in the future.

6. Continuation of the collaborative studies on application of the SG-GCM simulated products for driving tropospheric chemistry transport experiments

7. The SG-modeling has matured enough for initiating the

international SGMIP with participation of major centers and groups from the U.S., Canada, France, and Australia. The first stage is planned for 2002-03. The scope of the project (including ensembles) depends on the computer time available. SGMIP will be connected to AMIP-II at the next stage.

Future Research Areas

• SGMIP (Stretched-Grid Model Intercomparison Project) with

participation of the major centers/groups from the U.S. (NASA/GSFC/DAO and UMD), Canada (RPN/CMS), France (Meteo-France), and Australia (SCIRO) (started).

• Performing ensemble integrations (started).

• Developing extensive applications: providing forcing products for high resolution atmospheric chemistry, hydrologic, and other models (started or planned).

• Developing more flexible/sophisticated SG-generators (planned).

• Developing model physics for mesoscale resolution (a long-term effort).

• Developing new GCMs with flow-dependent anisotropic grids for both regional and global and climate simulations (a long-term effort).

Proposal on the Stretched-Grid Model Intercomparison Project (SGMIP)

1. There are several SG-GCMs with different dynamics and physics used for regional climate simulation and applications. The obtained results

have shown the feasibility of the SG-modeling approach. 

2. There are several computational topics that will be addressed by the intercomparison: 

• Different stretching strategies

• Different approximations of model dynamics, and filters

• Different model physics including treatment of physics on stretched grids

• Ensemble calculations

• Optimal performance on parallel supercomputers 

The variable-resolution SG-GCMs have matured enough to justify

the intercomparison effort

Proposal on SGMIP (cont.)

3. The 12-year simulation period (1987-1999) including the ENSO cycles is proposed for: 

• Studying the impact of resolution on efficient/realistic downscaling to mesoscales

• Studying the U.S. anomalous regional climate events (floods, droughts, etc.) and major monsoonal circulations in the context of ENSO cycles at mesoscale resolution

• Studying the impact of surface boundary condition forcing

• Studying the U.S. water and energy cycles

• Studying the impact of a multi-model ensemble

Proposal on SGMIP (cont.)

4. The representatives of major centers/groups have expressed their commitment in participating in the project.

• The participating centers/groups are: NASA/GSFC (Fox-Rabinovitz);

Meteo-France (Deque); RPN/CMS, Canada (Cote and Dugas); CSIRO Australia (McGregor); UMD (Baer); NCAR (Tribbia and Fournier).

• Five SG-GCMs will be included into the first stage of the project

(2002-2003). Two-four other SG-GCMs will join at the next stage.

• The AMIP representatives (L. Gates and P. Gleckler) have expressed their interest in this effort to become in the future a special regional project within AMIP.

• The proposal was discussed at the meeting held during the Workshop on Regional Climate Research, April 2001, Boulder, CO with all the participants. The consensus was that we have to go ahead with the project.

• The details and logistics have been discussed and finalized. The requirements and support for the intercomparison project have been specified in June, 2001. The computing resources for the U.S. participants are being identified.