Overview of NEMS infrastructre

Mark Iredell

NEMS-NMMB tutorial

February 18, 2014

11

• NEMS stands for:NOAA Environmental Modeling System

• A shared, portable, high performance software superstructure and infrastructure

• For use in operational prediction models at the National Centers for Environmental Prediction (NCEP)

• Eventual support to community through the Developmental Test Center (DTC)

What is NEMS?

22

Motivation for NEMS

– Develop a common superstructure that can be shared by NCEP models.

– Modularize large pieces of the models with ESMF components and interfaces. http://www.earthsystemmodeling.org/

– Isolate history/restart output in a common Write component.

– No intrusion into the science code and parallelization in the respective models.

33

NEMS Developersin-house

Nicole McKee documentation, web, testing

Ratko Vasic upgrades, regression, atmos coupling

Jun Wang IO, post, configuration, WAM

Weiyu Yang ensemble, earth coupling, ESMF

Patrick Tripp NUOPC physics, software support

44

NEMS DevelopersESMF group

Gerhard Theurich NUOPC layer, HYCOM coupling

Fei Liu NUOPC layer, MOM5 coupling

Tony Craig NUOPC layer, Mediator

55

NEMS Developersmodel-specific

Tom BlackDusan JovicJim AbelesJohn Michalakes

NAM / NMM-B

S MoorthiHenry JuangEugene Mirvis

GFS

Jun WangMisha Rancic

WAM

Sarah LuArlindo da Silva

NGAC

Tom HendersonJim Rosinski

FIM

Xingren Wu CFS66

NEMS Structure

MAIN

EARTH(1:NM)

Ocean Atm Ice

NMM GSM FIM

Solver

Domains(1:ND)

Wrt Dyn Phy Wrt

NEMS

EarthEnsemble Mediator

Chem

All boxes represent ESMF components.

Atm-OcnMediator

NEMS Write component

• Could have multiple instantiations for different forecast hours.

• Each instantiation runs on the union of model processors and its own quilt processors.

• Import state is model data on model processors, decomposed on latitude.

• No export state.

NEMS Write component steps

1. Send model data to quilt processors, independently decomposed on latitude.

2. Return from model processors so model can resume running.

3. Run full UPP post-processor and write GRIB master files (if requested).

4. Write model history files using NEMSIO.

NAM/NMM-B12 km gridpoint spacing

Static nests:

- 4 km CONUS- 6 km Alaska- 3 km Hawaii- 3 km Puerto Rico

Relocatable Fire Wx nest: - 1.33 km inside CONUS - 1.5 km inside Alaska

► One-way

► Boundaries received every parent timestep

Atmosphere

Dynamics PhysicsDyn-PhyCoupler

Spectral

Color Key

Generic Component

Generic Coupler

Completed Instance

GFS Phy

unified atmosphereIncluding digital filter

GOCART

• Dynamics, physics and chemistry run on the same grid in the same decomposition • GOCART does not own aerosol tracers (i.e, do not allocate aerosol tracer fields)• PHY2CHEM coupler component transfers/converts data from physics export state to GOCART import state

– Convert units (e.g., precip rate, surface roughness)– Calculations (e.g., soil wetness, tropopause pressure, relative humidity, air density, geopotential height)– Flip the vertical index for 3D fields from bottom-up to top-down

• CHEM2PHY coupler component transfers data from GOCART export state to physics export state– Flip vertical index back to bottom-up– Update 2d aerosol diagnostic fields

Phy-ChemCoupler

NEMS GFS Aerosol Component

1111

LSM LSM Prep/PostPrep/Post

Pre-PhysicsPre-PhysicsInterfaceInterface

DynamicsDynamics

t, t, u, v, w, T, u, v, w, T, , p, z, q, p, z, qxx, c, cxx, a, axx

staggered staggered

t, u, v, w, T, t, u, v, w, T, , , p, z, qp, z, qxx, c, cxx, a, ax x

destaggereddestaggered

Post-PhysicsPost-PhysicsInterfaceInterface

TendenciesTendenciesand Updatesand Updates

TendenciesTendenciesand Updatesand Updates

Vertical flippingVertical flippingDestaggeringDestaggering

(direction check)(direction check)

Vertical flippingVertical flippingRestaggeringRestaggering

(direction check)(direction check)

Ocean Ocean Prep/PostPrep/Post

Wave Wave Prep/PostPrep/Post

LSM, Ocean, Wave, Chemistry, Aerosol, Ice DriversLSM, Ocean, Wave, Chemistry, Aerosol, Ice Drivers

fluxes, SST,fluxes, SST,coefficients, etc. coefficients, etc.

InitInitModeMode

Dynamical equations, advection, horizontal mixing, diffusion.Dynamical equations, advection, horizontal mixing, diffusion.

RadiationRadiation Deep and Deep and Shallow Shallow CumulusCumulus

Surface Surface LayerLayer

PBL PBL and and

Vertical Vertical MixingMixing

Micro-Micro-physicsphysics

Sat. adjustSat. adjust

Modified Kalnay Rules LayerModified Kalnay Rules Layer

NUOPC Physics Driver SchematicNUOPC Physics Driver Schematic

NUOPC LayerNUOPC Layer

fluxes, fluxes, precipitationprecipitation

radiation, stability, radiation, stability, exchange coefficientsexchange coefficients

qqx x (microphysics species)(microphysics species)

ccxx (chemistry species) (chemistry species)

aax x (aerosol species)(aerosol species)

OutputOutputDiagnosticDiagnostic

ss•fieldsfields•ratesrates•budgetsbudgets•othersothers

KeyKeyModel StandardModel StandardModel SpecificModel Specific

Atmospheric Physics DriverAtmospheric Physics Driver(init, run, finalize modes)(init, run, finalize modes)

InitializeInitializePhysics Physics

Tables and Tables and DatabasesDatabases

FinalizeFinalizeMode.Mode.

1212

Summary

The NOAA Environmental Modeling System is being built to unifyoperational systems under a single framework in order to more easilyshare common structures/components and to expedite interoperability.

The first two systems under NEMS have been implemented into NCEPoperations with others to follow in the next few years.

The NUOPC layer will be used to make collaboration with other groups less difficult when building/coupling modeling systems.

Incorporation of a NUOPC physics driver can help standardize theoften complex connections to physics packages thereby enhancingtheir portability.

●

●

●

●