Nonlinear Extended MHD Simulation for Magnetic Plasma Confinement

C. Sovinec, C. Kim, H. Tian, Univ. of WI-MadisonD. Schnack, A. Pankin, Science Apps. Intl. Corp.-San DiegoD. Brennan, MIT S. Kruger, Tech-X Corp.E. Held, Utah State University X. Li, Lawrence-Berkeley Lab.D. Kaushik, Argonne Nat. Lab. S. Jardin, Princeton Pl. Phys. Lab.

and the NIMROD TeamMath and Computer Science Division

Argonne National LaboratoryJuly 7, 2005

As the development of magnetic plasma confinement approaches conditions for ignition, ...

Joint European Torus,European Fusion

Development Agreement

DIII-D,General Atomics Corporation

Alcator C-mod,Massachusetts

Institute of Technology

TFTR,Princeton Plasma

Physics Laboratory

… the need for predictive simulation increases.

planned International Thermonuclear Experimental Reactor (ITER)

• Fusion power: 500 MW

• Stored thermal energy: 10s of MJ

Critical ‘macroscopic’ topics include:

1. Internal kink stability2. Neoclassical tearing

excitation and control3. Edge localized mode

control4. Wall-mode feedback[2002 Snowmass Fusion Summer

Study]

Outline• Introduction• Macroscopic plasma dynamics

• Characteristics• Simulation examples

• PDE system• Computational modeling

• Numerical methods• High-order spatial representation• Time-advance for drift effects

• Implementation• Conclusions

Macroscopic Plasma Dynamics

• Magnetohydrodynamic (MHD) or MHD-like activity limits operation or affects performance in all magnetically confined configurations.

• Analytical theory has taught us which physical effects are important and how they can be described mathematically.

• Understanding consequences in experiments (and predicting future experiments) requires numerical simulation:

• Sensitivity to equilibrium profiles and geometry

• Strong nonlinear effects

• Competition among physical effects

Fusion plasmas exhibit enormous ranges of temporal and spatial scales.

• Nonlinear MHD-like behavior couples many of the time- & length-scales.

• Even within the context of resistive MHD modeling, there is stiffness and anisotropy in the system of equations.

Characteristic Lengths in ITER (m)10-5 10-4 10-3 10-2 10-1 100 101 102 103 104

iongyroradius

electrongyroradius

ionskin depth

effective particlemean free path

equilibriumgradient

plasmacircumference

electronskin depth

Characteristic Times in ITER (s)10-12 10-10 10-8 10-6 10-4 10-2 100 102 104

Alfven wavepropagation

electron plasmaoscillation

electrongyromotion

driftrotation global resistive

diffusionenergy

turnover

iongyromotion

Examples of Nonlinear Macroscopic Simulation1) MHD evolution of the tokamak internal

kink mode (m=1, n=1)• Plasma core is exchanged with cooler

surrounding plasma.

M3D simulation of NSTX [W. Park]

Evolution of pressure and magnetic topology from a NIMROD simulation of DIII-D

Examples of Nonlinear Macroscopic Simulation (continued)2) Disruption (Loss of Confinement) events

• Understanding and mitigation are critical for a device the size of ITER.

Simulated event in DIII-D starts from an MHD equilibrium fitted to laboratory data.

Resulting magnetic topology allows parallel heat flow to the wall.

simulation & graphics by S. Kruger and A. Sanderson

Examples of Nonlinear Macroscopic Simulation (continued)

3) Helical island formation from tearing modes• Being weaker instabilities, tearing modes are heavily influenced by non-MHD effects.• Tearing modes are usually non-disruptive but lead to significant performance loss.• Slow evolution makes the system very stiff.

R

Z1.5 2

-1

-0.75

-0.5

-0.25

0

0.25

0.5

0.75

1

Resistive tearing evolution in toroidal geometry generating coupled island chains.

Magnetic Island Width vs. Time

time (s)

w(c

m)

0 0.01 0.02 0.03 0.04 0.05 0.06 0.071

2

3

4

5

6

7

simulationw(t)=1.22ηΔ' t+w(0)

Examples of Nonlinear Macroscopic Simulation (continued)4) Edge localized modes

• Strong gradients at the open/closed flux boundary drive localized modes.• Heat transport to the wall occurs in periodic events—can be damaging if not controlled.

Nonlinear coupling in MHD simulation leads to localized structures that are suggestive of bursty transport.

Medium-wavenumber modes are unstable.

simulation by D. BrennanMHD description is insufficient, however.

PDE System: a fluid-based description

BJVVV×=⎟

⎠⎞

⎜⎝⎛ ∇⋅+∂∂

tnmi ( )ViΠ⋅∇−∇− p

( ) 0=⋅∇+∂∂ n

tn V

ααααα VV ⋅∇−=⎟

⎠⎞

⎜⎝⎛ ∇⋅+∂∂ nTT

tTn

23

αα Q+⋅∇− q

( ) ⎥⎦⎤

⎢⎣⎡ +×−∇−××−∇=

∂∂ JBVBJB ηe

1 pnet

( ) ( )[ ] ⎟⎠⎞

⎜⎝⎛ ⋅∇−∇+∇≡×⋅+−+⋅×=Π VIVVWbWbbIbbIWb

32,ˆˆˆ3ˆˆ3ˆ

4Tii

gv eBpm

( ) i1

i ˆ5.2 TeBp ∇×+ − b

( ) e1

e ˆ5.2 TeBp ∇×− − b

( )[ ] iiii Tn ∇⋅−−= ⊥+ bbIbbq ˆˆˆˆ|| χχ

( )[ ] eeee Tn ∇⋅−−= ⊥+ bbIbbq ˆˆˆˆ|| χχ

α=i,e

BJ ×∇=0μ

Evolution equations:

Closure relations (for collisional conditions):

( )( )bbIbWb ˆˆ3ˆˆ2|| −⋅⋅=Π iip τ

PDE System (continued) Fluid models of macroscopic MHD activity in MFE plasmas are characterized by extreme stiffness and anisotropy.• Stiffness: Time-scales that impact nonlinear MHD evolution include

• Parallel particle motion leading to parallel thermal equilibration over flux surfaces in 100s of nanoseconds to microseconds.• MHD wave propagation over global scales in microseconds.• Magnetic fluctuations and tearing in hundreds of microseconds tomilliseconds.• Nonlinear profile modification and transport in tens to hundreds of milliseconds.• Global resistive diffusion over seconds.

• Anisotropy: Magnetization of nearly collisionless particles leads to• Effective thermal diffusivity ratios, , reaching and exceeding 1010.• Shear wave resonance that allows nearly singular behavior of MHD modes.

Both properties make the numerical solution of fluid models challenging when applied to MFE conditions, and they must be addressed when developing an algorithm for studying fusion MHD.

⊥χχ||

Computational Modeling

Challenges:• Anisotropy relative to the strong magnetic field

• Distinct shear and compressive behavior• Extremely anisotropic heat flow

• Magnetic divergence constraint• Stiffness arising from multiple time-scales• Weak resistive dissipationHelpful considerations:• Linear effects impose the time-scale separation• Typically free of shocks

Modeling: Spatial Representation

• The NIMROD code (http://nimrodteam.org and JCP 195, 355, 2004) uses finite elements to represent the poloidal plane and finite Fourier series for the periodic direction.• Polynomial basis functions may be Lagrange or Gauss-Lobatto-Legendre. Degree>1 provides

• High-order convergence without uniform meshing• Curved isoparametric mappings

SSPX

R

Z

1 1.5 2 2.5

-1

-0.5

0

0.5

1

Packed mesh for DIII-D ELM study

LDX

Modeling: Spatial Representation (continued)

ln(h)ln

[||d

iv(b

)||/

||b

||]

-5 -4 -3 -2-8

-7

-6

-5

-4

-3

-2bilinearbiquadraticbicubic

ln(h)ln

[||d

iv(b

)||/

||b

||]

-5 -4 -3 -2-8

-7

-6

-5

-4

-3

-2h1

h2

h3

• Polynomials of degree>1 also provide• Control of magnetic divergence error

Magnetic divergence in a tearing-mode calculation.

Scalings show convergence rates expected for first derivatives.

BEB⋅∇∇+×−∇=

∂∂

divbtκ

( )( )

( )( ) ( )

∫

∫

∫

∗⋅×Δ−

⎭⎬⎫

⋅∗×∇−⋅∇∗⋅∇⎩⎨⎧

Δ=

⎭⎬⎫

Δ⋅∇∗⋅∇Δ+⎩⎨⎧

Δ⋅∗

cEs

Ecbcx

bcbcx

dt

dt

tgd

divb

divb

κ

κ

‘Error diffusion’ is added to Faraday’s law:

for all vector test functions c*.

The ratio of DOF/constraints is 3 in the limit of large polynomial degree.

• Polynomials of degree>1 also provide• Resolution of extreme anisotropies (Lorentz force and diffusion)

x

y

-0.5 -0.25 0 0.25 0.5-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

x

y

-0.5 -0.25 0 0.25 0.5-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

T and B fields from

an 8×8 mesh of bicubic elements.

Simple 2D test:

• Homogeneous Dirichlet boundary conditions on T

• Heat and perpendicular current have sources.

• Analytically, the solution is independent of χ||,

• The resulting measures the effective , including the numerical truncation error.

)cos()cos(2 2 yx πππ

)cos()cos(),( 1 yxyxT ππχ −⊥=

( )0,01−T⊥χ

h

|χef

f-1|

0.1 0.2 0.3

10-4

10-3

10-2

10-1

100

101

χ|| = 109

Diffusion error for bicubic,

biquartic, and

biquinticelements.

Reproducing Transport with Magnetic Islands

χ|| / χperpw

(cm

)108 109 1010

1

2

3

4

567 sim. data & fit

analytic Wc

Critical island width vs. χ||/χperpWc shows where diffusion time-scales match [Fitzpatrick, PoP 2, 825 (1995)].

R

Z

1.5 2

-1

-0.75

-0.5

-0.25

0

0.25

0.5

0.75

1

With anisotropy, heat transport across magnetic islands is a competition between parallel and perpendicular processes.

Modeling: Time-advance algorithms

tj tj+3/2tj tj+1tj+1/2

Vj+1Vj

Bj+1/2nj+1/2, Tj+1/2 nj+3/2, Tj+3/2

Bj+3/2

• Stiffness from fast parallel transport and wave propagation requires implicit algorithms.• Semi-implicit methods for MHD have been refined over the last two decades (DEBS, XTOR, NIMROD).• The underlying scheme is leapfrog,

( ) ( )[ ]{ } ( ) ( )VVBVJBBVVL Δ⋅∇+∇⋅Δ∇+×Δ×∇×+××Δ×∇×∇=Δ 0000000

1 pp γμ

with the following implicit operator in the velocity advance to stabilize waves without numerical dissipation.

Modeling: Time-advance algorithms (continued)• A new implicit leapfrog advance is being implemented for modeling two-fluid effects (drifts and dispersive waves).

( ) ( ) 2/12/1i

2/12/1i 2

121 ++++ ×=ΔΠ⋅∇+ΔΔ−⎟

⎠⎞

⎜⎝⎛ ∇⋅Δ+Δ∇⋅+ΔΔ jjjjjj tL

tnm BJVVVVVVV

( )jjjjj pnm VVV i2/12/1

i Π⋅∇−∇−∇⋅− ++

( )2/11121 +++ ⋅⋅−∇=Δ∇⋅+

ΔΔ jjj nn

tn VV

( ) 12/12/1112

321

21

23 +++++ ⋅∇−∇⋅−=Δ⋅∇+⎟

⎠⎞

⎜⎝⎛ Δ∇⋅+ΔΔ jjjjj nTTnTT

tTn

ααααααααα VVqV

( ) 2/12/1 ++ +⋅∇− jj QT αααq

( ) JBJBJBVBΔ×∇+×Δ+Δ××∇+Δ∇⋅+

ΔΔ +++ η

211

21

21 2/12/11 jjj

net

( ) ⎥⎦⎤

⎢⎣⎡ +×−∇−××−∇= +++++ 2/12/11

e2/12/11 jjjjj p

neJBVBJ η

• A fully time-centered approach is also possible.

Analysis shows the leapfrog with implicit magnetic advance to be numerically stable and accurate on two-fluid waves.

k

Im(o

meg

a)

0 1 2 3 4 5-6E-15-5E-15-4E-15-3E-15-2E-15-1E-15

01E-152E-153E-154E-155E-156E-15 k

Re(

omeg

a)

0 1 2 3 4 50

0.5

1

1.5

2

2.5

3

3.5

θ =0.04π Δt =1, cs2/ vA

2=0.1. θ =0.46πk

Im(o

meg

a)

0 1 2 3 4 5

-6E-15

-5E-15

-4E-15

-3E-15

-2E-15

-1E-15

0

1E-15

2E-15

3E-15

4E-15 k

Re(

omeg

a)

0 1 2 3 4 50

0.5

1

1.5

2

2.5

3

3.5

Modeling: Implementation• Algebraic systems from 2D and 3D operations are solved during each time-step (~10,000s over a nonlinear simulation).• 3D systems result from nonlinear fluctuations in toroidal angle.

• Toroidal couplings are computed with FFTs in matrix-free solves.• 2D systems represent coupling over the FE mesh only, and matrixelements are computed.

Example sparsity pattern for a small mesh of biquartic elements—after static condensation but before reordering.

• They are also generated for preconditioning the matrix-free solves.

Modeling: Implementation (continued)• Solving algebraic systems is the dominant performance issue.• Iterative methods scale well but tend to perform poorly on ill-conditioned systems.• Collaborations with TOPS Center researchers Kaushik and Li led us to parallel direct methods with reordering→SuperLU (http://crd.lbl.gov/~xiaoye/SuperLU/).

procs

time

pers

tep

(s)

20 40 60 80 1000

2

4

6

8

10

12

14

16

18 factoring (3 comps)solve (3 comps)total (3 comps)factoring (6 comps)solve (6 comps)total (6 comps)

SuperLU DIST

procs

time

pers

tep

(s)

20 40 60 80 1000

10

20

30

40

50

60

70

80

90

100factoring (3 comps)solve (3 comps)total (3 comps)factoring (6 comps)solve (6 comps)total (6 comps)

cg with line-Jacobi preconditioning

SuperLU improves NIMROD performance by a factor of 5 in nonlinear simulations.

Conclusions

The challenges of macroscopic plasma modeling are being met by developments in numerical and computational techniques, as well as advances in hardware.

• High-order spatial representation controls magnetic divergence error and allows resolution of anisotropies that were previously considered beyond reach.

• SciDAC-fostered collaborations have resulted in huge performance gains through sparse parallel direct solves (with SuperLU).

• Algorithm development is proceeding for two-fluid modeling.

Help with matrix-free non-Hermitian solves is welcome!