ITPA NAKA, October 2007. P. Strand Plans and status of the European Task Force Integrated Tokamak Modelling Presented by: Pär Strand TF Leader : P. Strand,

ITPA NAKA, October 2007. P. Strand

Plans and status of the European Task Force Integrated Tokamak Modelling

Presented by: Pär Strand

TF Leader : P. Strand, Deputies: L-G. Eriksson, M. Romanelli

EFDA CSU Contact Person: K. Thomsen

ITPA CDBM MEETING NAKA, OCTOBER 2007


The aim of the task force is to:• co-ordinate the development of a coherent set of validated simulation tools • Benchmark these tools on existing tokamak experiments

The ultimate aim is providing a comprehensive simulation package for ITER plasmas.

The remit of the Task Force would extend to the development of the necessary standardized software tools for interfacing code modules and for accessing experimental data.

In the medium term, this task force’s work would support the development of ITER-relevant scenarios in current experiments, while in the long term it would aim to provide a validated set of European modelling tools for ITER exploitation

ITM Scope

EFDA(03)-21/4.9.2 (June 24th, 2003) Executive summary

ITM-Milestones in EFDA workplan


2008 - Start of the Gateway for IM and deployment of code platform

– Implies a complete set of data structures and associated tools– A fully operational portal/workflow configuration– Major code releases from all of Integrated Modelling Projects

2009 - Extended set of platform tools forming a predictive core physics capacity for ITER

– Production activities – local clusters and grid

2010/11 - Whole device modelling capability including comprehensive core-edge coupling and first principles elements

– Aiming towards Broader Approach IFERC level computations


Project Leadership 2007-

NB: All tasks remain open for participation!

Projects Name Leadership

ISIP Infrastructure and Software Integration Project B. Guillerminet (CEA)G. Manduchi (RFX)

IMP#1 Equilibrium Reconstruction and Linear MHD Stability

G. Huysmans (CEA)C. Konz (IPP)

IMP#2 Non-linear MHD and Disruptions M. Ottaviani (CEA)S. Sharapov (UKAEA)

IMP#3 Transport Code and Discharge evolution(renewed for 2008 WP)

D. Coster (IPP)D. Kalupin (FZJ), V. Basiuk (CEA), G. Pereverzev (IPP)

IMP#4 Transport Processes and Micro Stability(renewed for 2008 WP)

B. Scott (IPP)Vacant

IMP#5 Heating Current Drive and Fast particles(Renewed for 2008 WP)

T. Hellsten (VR)Y. Peysson (CEA)F. Zonca (ENEA)

Tasks

ITER Predictive ITER modelling and scenario development (review in November 2008 WP)

V. Parail (UKAEA)


Three Tiered Approach

Portal more important in defining access to resources (certificates)

Data access & application scheduling &

Resource allocation through platform

Visualization, monitoring & steering through platform

PortalWorkflow/

Code Platform

Grid

HPC

“Gateway”

LocalCluster

LocalData

Ext.Data

USER

Portal + Platform + Resources

ITM tools: standardized data structures, data access, interface definitions, Gateway (2008), Workflow (Kepler)


Shared Storage Data Area: resources to store large amounts of data generated by simulation codes and originating from experimental data of various fusion devices

Servers and infrastructure: Master node, code platform server, fileserver, etc

Computing resources: a farm of worker nodes to provide gateway elements. The software include operating environment (sys.op. distribute filesystem, resource management system, authentication system, backup). Access to Cresco HPC center.

Technologies layout (schematic)

LinuxWADFS/PFS

Auth SSORMSSAN

SWITCHesGE-FC-IBStorage

Raid Array(Raid 6)

Data servers32/64bit DualCpu/DualCore

(rackmount/Blade)(GE-FC)

Worker Nodes32/64bit DualCpu/DualCore

Xeon/Opteron(rackmount/Blade)

(GE-FC-IB)

GE: GigaEthernetFC: Fibre ChannelIB: InfiniBand

Shared Storage Data Area Computing resources

100TB 16-32 nodes 64-128 CPU cores

The Gateway - general layoutUser entry point

Provided by ENEA start January 2008

• Abstracted data structures– Description of a “Complete” set of data for describing plasma operation

and simulations• Abstracted through XML schemas• Unambiguous description with agreed sign and other conventions• SI-units (with eV)• Consistent Physical Objects CPO

– Groupings of related data - basis for code interfaces

– Serializations• Transformations providing

– Language specific implementations of CPOs(f90, C++,..) – Database structures

• Access and Storage– Universal Access Layer providing invariant API based on CPO– Plug-in backend

• MDS+ currently being implemented - implications on data structures• HDF5 considered as next step

– Used in workflow tool to connect modules.• Workflow orchestration

– Kepler tools– Integration Tools (ISE, JNI editor, ….)

Components - Infrastructure



• Full description of a tokamak : physics quantities + subsystems characteristics + diagnostics measurements

“Object based” data structure :• High degree of organisation : several subtrees corresponding to « Consistent

Physical Objects » (avoid flat structures with long list of parameter names).

– Subsystem : (e.g. a heating system, or a diagnostic) : will contain structured information on the hardware setup and the measured data by / related to this object.

– Code results (e.g. a given plasma equilibrium, or the various source terms and fast particle distribution function from an RF code) : will contain structured information on the code parameters and the physics results.

• Codes communicate by exchanging CPOs only ( data consistency)

• Programming Language flexibility : use of recent software technologies : database structure is defined using XML schemas

data structures -consistent physical objects

• First versions available for JET, ASDEX, Tore Supra– How to verify correctness? (none is officially stamped)– Part of the Data structures


Machine Descriptions

ITM will seek formal collaborations to expand description (ITER partners) and to fill with data (Int. Experiments - ITPA role?)

Working with ITM data structure

• The physics code developer does NOT need to know anything about XML

• The physics code developer must only make his code comply with

– The logic of the CPOs

– The hierarchy of data inside CPOs

– Provide accurate version release and documentation of his code

– Provide the list of arguments (CPOin, CPOout, number of time slices needed for each of them) and code-specific parameters, in a format to be defined

• ISIP support then wraps the code into the framework and links it to the access library (Universal Access Layer - UAL)

• No call to UAL inside the physics code


• Code must have I/O as :

Subroutine Physics_module(CPOin1,….,CPOinN, CPOout1,…, CPOoutM)

use euitm_schemas ! contains the type definitions of all CPOs

• Declaration of variables (CPO) must be as :

type (type_equilibrium) :: my_equilibrium ! for one time slice

type (type_equilibrium), pointer :: my_equilibrium(:) ! if the code handles several

! time slices at once

(type_equilibrium is a derived type defined in euitm_schemas.f90, which is generated dynamically from the schemas … but the physics user may ignore this completely)

Example of an ITM code (Fortran) : I/O



Code adaptation• Data access through the UAL:

– Independent of the storage– The framework does not know our data

• Principles:– data transfer through a data server (in-

memory for the simulations)– Advantages:

• Solve the problem of languages mixing

• Allow // computations

Small changes:• I/O => UAL

• Read euitmget() • Write euitmput()

Structure:ISIP Wrapper:Call euitmget(“CPO

name”,CPOin)

Call user_code(CPOin,CPOout)Use ITM structureData access is separated from the computational part

Call euitmput(“CPO name”,CPOout)

codesEFIT, …

In-memory

codesHELENA, …

codesMISHKA, …

UAL


Interfaces to Physics modules

Data exchange between different codes/modules will be based on the Universal Access Layer


in-memoryData server

SOLOVIEV

HELENA

MISHKA

Plot 2DPlot 3D


Current Status and 2008 Activities

user

Lab.

login

user

Lab.

login

user

Lab.

login

portal

Simulation editor

Workflow editor

Workflow engine

Codes:Transport, heating,…

UAL:C/C++, F95, Java, …

Gateway

Data servers:MDS+, Pub-Sub, HDF5

Clusters:JET, IPP, …

GRID EGEE

HPC

applications

resources

Private data:XML files, …

Data mining toolsCode repositories:

Mercurial, …

Post-processing:Scilab,Visit,Root,…

Catalogues

Data structures almost complete: IMP#1 (ready), IMP#3 (some components missing – essentially ready), IMP#5 (being finalized)IMP#2 and IMP#4 code dependent additions needed

“Components” - Physics

• Physics activities– Organized in 5 different Integrated Modelling Projects or IMP’s

• Equilibrium Reconstruction and Linear MHD Stability• Non-linear MHD and Disruptions• Transport Code and Discharge evolution• Transport Processes and Micro Stability• Heating, Current Drive and Fast Particles

– IMP’s are in charge of coordinating the needed physics development and to implement the tools into the ITM framework

– The “Transport code” project acts in addition to its own direct development needs also as an integrator towards the whole device modelling task.

– IMP’s should deliver standalone (but fully embedded) state of the art tools for physics exploration on current devices as well as for ITER

– In addition, IMP’s should deliver validated modules for the integration projects providing a hierarchy of different physics fidelity modelling capacity to the ITM.



Project relations

Experimental data base

Modeling data base ISIP

ISIP

IMP5IMP4IMP2IMP1

Equilibrium and LinearMHD Stability

Non linear MHD phenomena

Transport Processesand Micro-stability

Heating, Current driveand fast particles

ISIP

ISIP

ISIP

ISIP

ISIP

ISIP

IMP3

Transport code and discharge evolution

ISIP

ISIP

ISIP

ISIP

a b

MHDequilibriumand stabilitymodules

Non-linearmodules(saw-teeth,ELMs, NTMs)

c d

Transportmodels

Sourcesand sinks

e f

Interfaces toboundaries

Whole devicemodeling

Associations/Expertise

User tools


Integrated Modelling Project 1Leader: G. Huysmans

Deputy: C Konz• Objective:– To provide an integrated suite of self-consistent codes (modules) for

equilibrium reconstruction and linear MHD stability analysis

Experimental Equilibrium reconstructionCEDRES, CLISTE, EFIT2006, EFIT-ITM, EQUINOX

Equilibrium codes and linear MHD stabilityEquilibrium : CAXE, CHEASE, HELENA, FINESSE, (DINA)Mapping : COTRANS, JMC MHD Stability : CASTOR, KINX, MISHKA, PHOENIX

Free-boundary direct equilibrium solvers CREATE, CEDRES, EFIT

3D equilibrium solversVMEC

Equilibrium toolboxFLUSH


equilibrium and MHD Stability

• Standardise contributed codes to become independent of machine /diagnostic data.

– Use only external geometry data (from database)

– Definition of interfaces between codes and machine and diagnostics

• Validation and Verification

– compare equilibrium and MHD stability codes on benchmark case

– Apply codes to a relevant experimental problem/data

– Ongoing discussion – independent experimental metrics challenging to find

diagnostic(1)description

machine description

diagnostic(2)description

magnetics MSE

equilibrium description

equilibrium reconstruction

high resolution equilibrium

code spec.parameters

code spec.parameters

equilibrium description

MHD stability code spec.parameters

MHD output description


Machine independent EFIT_ITM

• EFIT has been adapted to use the ITM structures and to use external geometry information– A unique version of EFIT can now be used

for ITER, Tore Supra, JET, etc– Using only TF tools for

Data storage, accessand data structures

Validation effort underway


• Objective: Non linear MHD phenomena• Working to deliver a module for sawteeth, NTMs, RWMs and

ELMs• Sawtooth module ready for adaptation for

Integrated Modelling Project 2New Leader: M. Ottaviani

Deputy: S. Sharapov

Sawtooth model (Porcelli) already implemented in JETTO, ASTRA and CRONOS. Applied to the prediction of the sawtooth crash time, and comparison with JET data.

F. Porcelli et al., FEC 2006, Chengdu

IMP#2 is being restructured with a stronger focus on the delivery of tools.


Integrated Modelling Project 3Leader: D. Coster

Deputy: V. Basiuk, D. Kalupin, V. Parail , G. PereverzevObjective:

–To provide the computational basis for a modular transport code, taking account of the core, the pedestal and the scrape-off layer. Ultimately, to enable the simulation of complete tokamak scenarios, e.g. for ITER

The goal of IMP3 is to deliver a transport code framework that is based on the scientific workflow environment to be provided by ISIP:

• Adopt a modular approach to the construction of a transport code

• Communication between modules to be via the ITM agreed data-structures.

•The verification and validation would be a joint effort between the originating IMP providing modules and IMP3

• Ultimate aim is to have a complete set of modules to enable the simulation of a full discharge (ie a “tokamak simulator”).

Kinetic Plasma Description

IMP1 IMP2 IMP4

NBI Heating Modules

ICRH Heating Modules

ECRH Heating Modules

IMP?

IMP5

1d, fluid core

IMP1 IMP2 IMP4

NBI Heating Modules



IMP3

IMP5P

has

e 0

Ph

ase

1

Ph

ase

2

1d, fluid core

IMP1 IMP2 IMP4

NBI Heating Modules



Kine

tic P

lasma

De

scriptio

n

IMP3IMP5

European Transport Solver - ETS


Common interface to transport models

CALL ANOMALOUS(MODEL, PROFILES, GEOMETRY,TRANSPORT,[DIAG],ifail)

Derived types:Standardized Inputs: PROFILES, GEOMETRY defined in generic modules

(type definitions and allocations,…)Standardized Output: TRANSPORT defined in same generic modules

(fluxes + eff.diff for transport channels)Model dependent data: WEILAND; GLF23, RITM, EDWM in specific model

dependent module (MMM95 under testing) [DIAGNOSTIC]: Optional diagnostic output supplied in model dependent

formats.

Simple and extensible interface: New models need to supply 1. Default settings for options2. A mapping to actual model call (by template)3. Derived types for model specific inputs/outputs (may be empty!)

developed with JET IM project

Next step: Point-wise interface “profiles” -> “dimless”; collaborate with Lehigh/FacetsFor corrected GLF23 pointwise interface?


Data structure defined for 1-D core transport code

For every equation the substructure contains the information about :

value (1-D profile of computed quantity); the source (any comments about source of 1-D profile);

flag (integer number commenting how the 1-D profile was obtained: computed, imported from different code, taken from experiment, etc.); the boundary (substructure, specifying the type of the boundary conditions and the value);

source_term (substructure, including the 1-D profile of the total source and describing how it was obtained);

transp_coef (substructure, including 1-D profiles of total D and V, and describing the source of these quantities );

flux (substructure, containing predictive and interpretative fluxes);

time_deriv (1-D profile of the integral of the time derivative term);

codeparam (substructure, containing all internal parameters used by the transport solver)


Integrated Modelling Project 4Leader: B. Scott

Deputy: M. OttavianiObjective:To develop a suite of unified, validated codes to provide quantitative predictions for the linear properties of a range of instabilities, including: ion-temperature-gradient (ITG) modes, trapped electron modes (TEM), trapped ion modes (TIM), electron-temperature-gradient (ETG) modes, micro-tearing modes, etc.

• Three Formalised tasks:• Catalogue Codes, standardise documentation• Code verification and benchmarking• Development and exchange of elements of theoretical basis for model

• Benchmarking campaign has started (will run for 2 years)• Cyclone like base case and (unique to Europe) edge• also comparing neoclassical and turbulence code equilibria


IMP#4: Code Verification & Benchmarking

Core and edge standard cases:• L-mode edge near 100 eV and 2 x 1013 cm-3

• Standard Cyclone base case for core (local and global modes)

First results reported at the EPS meeting in Rome 2006

global GEMR code differs at higher ne because it corrects for gradient relaxation


Edge benchmarking

first series:

cold ion four field model in ASDEX-Upgrade L-Mode cases

mainly flux tube codes, one global one in “thin atmosphere" domain

0.915 < r/a< 1.0

• non-periodic: gradient relaxes, buffer diffusion zones -> dissipation

• periodic: obviously non-viable for global case, admits unphysical radial jet flows

beta scan problematic enough to expose differences among the codes


IMP#5, heating current drive and fast particles

Leader: T. HellstenDeputy: Y. Peysson, F. Zonca

- Objective: develop the computational basis for a modular package of codes simulating heating, current drive and fast particle effects

- Area covered: ECRH, ICRH, NBI, LH, alpha particle and fast particle interaction with instabilities

- Goal: self-consistent calculations validated against experiments

- Priority: realistic modelling applicable to ITER standard and advanced scenarios


Integrated Modelling Project 5: strategy/outlook

Short term (~6 months): Identify the most effective data structure for linking codes between them on the ITM Platform (almost done). Some benchmarking activity (to start soon).

Medium term (~2/3 years): fast code development for basic ITER modelling

•ICRF, one code available, but some extension needed•ECRH/ECCD, In good shape, several codes available.•Benchmarking of these codes on ITM-TF platform.

Long term (> 3 years): Development of modular advanced codes for comprehensive modelling and integration in specific problems.

•Several ICRF full wave codes available.•Developments of 3D Fokker-Planck modelling.•Improvements in plasma response functions.•Description of AE modes etc. in Fokker-Planck treatment.…


Predictive modelling of ITER scenarios

Task on predictive modelling of ITER scenarios within the ITM TF – starting NOW– EU modelling tools already exists and are being

employed towards scenario assessments and development (e.g. JETTO, CRONOS, ASTRA) for core plasma and B2.5/EIRENE and EDGE2D/EIRENE for the SOL.

– Implementation as a Task within ITM TF gives• Access to physics experts and codes • Access to high level experimentalists (by call implementation)

– Explore limitations of current codes and needs for a European Transport Solver for ITER

– See V. Parail talk this meeting-

Leader: V. Parail


V&V and QA in General

• Level 0 – Ad Hoc approach (mainly at individual code level)• Non systematic approach, metrics and reporting left to individuals• No monitoring and forced adherence to any standards

• Level 1 – Consistent V&V (codes or packages )• Following predefined procedures

• Detailed consistent reporting• Verified operation• Critical assessment of “performance” • Critical assessment of experimental applicability

• Openly reported in standardized formats• Level 2 – Consistent QA (@ Organization or TF level)

• A superset of requirements over level 1 relating to• Software Management• Software Engineering

Detailed procedure with checkpoints guaranteeing conformanceto quality and reliability goals

• Level 3 – QA procedure under nuclear licensing requirements• Possible ITER requirement for some operator codes ??


Oversimplified view

Qualification

Verification

Val

idat

ion

Computational model

Conceptual model

Plasma

Data Validity

Qualification: Is the physics description adequate?

Verification:Are the equations implemented and solved for correctly?

Validation: Do we have a reliable and sufficiently accurate description of the plasma?

Data Validity:Is our measured data asufficient representation of reality?

Code benchmarking: (C2C)A tool in both V&V and physics exploration

TF V&V procedures: EFDA-TF-ITM(04)-8

From Winter 1970’s – still valid!!!

Collaborations

• Recognized collaborations for ITM outside EFDA– ITER (EFDA/ELE, ITPA)– US BPO( CPES, CSWIM, FACETS)– JP BPSI (Broader Approach)– Kepler (UCSD)– Further collaborations as they are formalized under

EFDA

– EUFORIA (under final negotiation)• EU FP7 Capacities project aimed at enhancing edge and core

transport and turbulence modelling for ITER in EUROPE– Workflow, grid and visualization

» May accelerate developments on portals/workflows» Will USE UAL as basis for data access

– Designed to amend ITM-TF activities towards grid and HPC – EGEE, PACE, …


Vision - Aims - Goals




IMP1: the on-going tasks

• adaptation of high resolution equilibrium, mapping and linear MHD

stability codes to the ITM data structures. – Status

• High resolution equilibrium codes CHEASE (H. Lutjens), HELENA (G. Huysmans, C. Konz), and CAXE (S. Medvedev) have been adapted to use ITM data structures.

– codes have common interface– codes verification ongoing

• Linear MHD Stability codes CASTOR (C. Konz), KINX (S. Medvedev), MISHKA (G. Huysmans) have been adapted to use ITM data structures.

– Verification on synthetic test case and ITER test case– CASTOR, MISHKA-1 and MISHKA-D have been combined into new framework

ILSA (C. Konz, E. Strumberger, EPS2006)

– To be done• Verification on more synthetic benchmarks

-0.1900

-0.1916

MISHKA-1 (.v=0)

-0.2252CAXE

-0.2230-0.2205HELENA

-0.2230CHEASE

CASTOR (=0)

KINX(=0)Soloviev

-0.1900

-0.1916

MISHKA-1 (.v=0)

-0.2252CAXE

-0.2230-0.2205HELENA

-0.2230CHEASE

CASTOR (=0)

KINX(=0)Soloviev

-2.5e-4CAXE

-2.52e-4-3.5e-4HELENA

-2.57e-4CHEASE

MISHKA-1KINXITER scenario #2

-2.5e-4CAXE

-2.52e-4-3.5e-4HELENA

-2.57e-4CHEASE

MISHKA-1KINXITER scenario #2

“Complete” set of Equilibrium tools and MHD analysisReady to be launched on platform/Gateway


Edge Code Benchmarking

• ITPA activity: – code-code comparison

• Phase I: pure D, no drifts• Phase II: pure D, drifts• Phase III: D+C, no drifts• Phase IV: D+C, drifts

– SOLPS-EDGE2D/NIMBUS • Phase I completed successfully

(reported on at PSI 2004)

• Phases II and III in progress

– SOLPS-UEDGE• Phase I underway• Phase II expected to start soon

• Additional code-experiment– JET: SOLPS-EDGE2D/NIMBUS– AUG/D3D: SOLPS-UEDGE

EDGE2D D+C

EDGE2D D

SOLPS

SOLPS

EDGE2D


Using MDS+ to connect SOLPS to ASCOT

• ITPA standard pedestal MDS+ tree created based on AUG shot 17151 (provides the equilibrium data)

• 2D background (ne, Te, Ti, …) written as MDS+ tree by SOLPS (IPP-Garching)

provides input to ASCOT. Due to open field lines, special care must be taken with the Monte Carlo collisions

• Planned: output of ASCOT to be saved to a MDS+ tree

ASCOT: (Accelerated Simulation of Charged Particle Orbits in a Tokamak) T. Kurki-Suonio, L. Aho-Mantila, J. Heikkinen, V. Hynönen, T. Kiviniemi, A. Salmi, S. Sipilä, V. Tulkki

Distribution of electrons reaching outer target

Documents

ITPA NAKA, October 2007. P. Strand Plans and status of the European Task Force Integrated Tokamak Modelling Presented by: Pär Strand TF Leader : P. Strand,