1 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

2 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

3 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

Institute for Meteorology and Climate Research (IMK)

Fundamental research on climate, water cycle and trace substances

Turbulence, Convection, Cloud formation, Aerosol physics, Precipitation formation, Exchange processes on Earth’s surface

Aerosols, Trace Gases and Climate Processes (head of group: B. Vogel)

Representation of aerosols and gas phase chemistry on the regional scale (model system COSMO-ART)

Particular interest on the interaction of aerosols and clouds in the atmospheric system

4 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

Ext. Parameters

= operational weather forecast model (DWD)

COSMO–ART (ART = Aerosols and Reactive Trace Gases)

Vogel et al., 2009, ACP

5 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

PrecipitationRadiation

Aerosol

Gas phase Dynamics Humidity

CloudsTemperature

PrecipitationRadiation

Aerosol

Gas phase Dynamics Humidity

CloudsTemperature

Feedback processes realized in COSMO-ART

Aerosol cloud interaction

6 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

Future Applications:

Decadal forecasts of the regional climate of Europe

Interaction of natural and anthropogenic aerosols with climate

Model runs with CCLM-ART (= climate version of COSMO-ART)

Computational demand:

Based on AIX (16 processors) without speed-up of COSMO-ART:

10 year model run = 112 CPU years

10-20 scenario runs

Transported 3-D variables ~60 (gases) + ~ 80 (aerosols)

250x250x40 gridpoints (future applications due to increased resolution ~factor 4)

7 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

Profiling test case

Run of 24 h initialized at 22nd June 2007 at 00:00 UTC

193x113x40 grid points, dt=40s, resolution of 0.25°

Machine: IBM AIX cluster at KIT

2 nodes with 8 cores each

Profiling done with Scalasca

8 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

Total run time: 268.4 CPU hours (2 nodes x 8 cores)

cradm2 (31,48%)

ISOROPIA (14,72%)

ART org (22,72%)

9 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

Further Metrics

Metric Value ART

Computational Imbalance

17,33 CPU hours (6,46 % of total)

85% of imbalance due to ART

Synchronizations 56,63 CPU hours (21,10 % of total)

47 % ART (anth. Emissions)52 % organize_data (data output)

Communciations 9,75e7 89 % in dynamics and leapfrog

Bytes transferred 5,75e12 B 94 % in dynamics and leapfrog

Important imbalance of the code

Large need of communications due to more than 150 additional 3D prognotstic fields

10 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

Way forward

Model chemistry and physics:

Use KPP chemical solver (GPU “version”)

Use a simplified version of ISOROPIA

Call chemistry and aerosols not at every time-step

Try to reduce waiting time for anthropogenic emissions ( call only every 6 hours)

Numerics:

Code is partly vectorized -> Compiler options for non-vectorized code

Lump messages together, to reduce number of communications

(but increase their size)

Optimize utility routines (unit conversion, string comparision)

11 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

12 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

14 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

17 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

generate animated gif out of visualized states

Parallel Visualization

HPC-systemDesktop-PC

Data-Server

Visualization -Cluster

convert *.grb into *.vts

parallel visualization of output data

analyze forecast

18 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

Visualization Temperature+Rain

19 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

20 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale

21 The Interactions of Aerosols, Clouds, and Radiation on the Regional Scale