1

New Paradigms in High-Accuracy

Numerical MHD (ADER = Arbitrary DERivatives in space and time;

Multi-dimensional Riemann Solvers)

By

Dinshaw Balsara (Univ. of Notre Dame)

2

Goals of Talk

ADER = Arbitrary DERivatives in space and time

Based on a Lax-Wendroff/Cauchy-Kowalewskya type procedure –

trading time derivatives for spatial derivatives.

Will describe a Galerkin-like formulation of ADER in pedagogic detail.

Interested in somewhat lower orders (no more than 4) but v. interested

in practical implementation on world’s fastest supercomputers.

Implementation-friendly details described in our papers & upcoming

text-book on Computational Astrophysics.

Schemes must be robust, positivity-preserving. Instantiated for WENO.

Also present Multi-dimensional HLL Riemann Solver.

3

Talk Outline

1) Challenges Posed by Motivating Astrophysical Applications: Met by

ADER-WENO Algorithms

2.a) ADER in Brief (Recall Lax-Wendroff procedure.)

3) Multidimensional HLL Riemann Solver

4) Some MHD Results for ADER-WENO schemes

5) Higher Order Scheme FAQ’s

6) Conclusions & Table of Contents of Upcoming Text on Computational

Astrophysics

2.b) If time permits: Rapid strategy for obtaining numerical fluxes (&

electric fields)

4

Motivating Applications of Interest Star formation ISM Turbulence

Planet formation

And many, many others…..

5

1) Challenges Posed by Motivating Applications 1) Applications are large (>10243 zones) & have fully-developed

turbulence high spatial and temporal accuracy for smooth flow with a

minimum of dispersion.

2) Strong shocks at random locations shock-capturing.

3) Have to run on PetaScale caliber parallel machines Low inter-

processor communication.

4) Magnetohydrodynamics (MHD) has its own divergence-free

requirements for B Formulated multi-d Riemann Solvr, Balsara (2010)

5) Has to be fast Scheme has to prove its own benefits vis a vis TVD

schemes, which are blindingly fast and rather accurate these days!

6) Very beneficial to have a single stage update for AMR applications

Need space-time interpolatable solution Finite Volume ADER schemes

are a natural. (Classical RK3 & RK4 aren’t SSP; cannot be used to

interpolate in time.)

6

Motivating ADER Methods

I.e. This is just a clean way of replacing time derivatives with spatial

derivatives

Recap the Cauchy-Kowalewskya Procedure:

Suppose I want to evolve the following simple PDE : u u 0 u + u = 0

Say that at the beginning I have the 3rd order spatial variation of u,

I.e. I have

t x t xa a

2

0

2 2

0

1 : u , w + w + w

2

1 1I want 3rd order in space-time : u , w + w + w + u + u + u

2 2

Obtaining u is easy : u u

Obtaining u is also possible : u u

Obtaining u can al

x xx

x xx t xt tt

t t x

xt xt xx

tt

x t x

x t x t xt t

a

a

2so be done : u u u u u utt xt tx t x xxx xa a a a a a

7

2) ADER In Brief ;

STEP 0) Say PPM or WENO has provided a 3rd or higher order

reconstruction

STEP I) Start with the reconstructed initial polynomial within each zone

expressed in a higher order modal basis set (i.e. orthogonal Hermite

polynomials) (1d here for simplicity; modes shown with caret):-

2

0 x xx

1ˆ ˆ ˆu (x, t=0) = w + w x + w x

12

We can use the above to obtain the spatial

representation of the fluxes at t=0;

i.e. evaluate only once:

t xGoverning Equation : u + f(u) = 0

2

0 x xx

1ˆ ˆ ˆf (x, t=0) = f + f x + f x 12

How? Answer: a) Evaluate u1 = u(x=0,t=0), u2 = u(x=.5,t=0),

u3 = u(x=-.5,t=0). b) Find f1 = f(u1), f2 = f(u2), f3 = f(u3) . c) Obtain

modal representation of flux. 3rd order in 1D is first non-trivial case.

1 2 3

4 5

6

x

t

x=0.5 x= 0.5

t=0

t=1

8

STEP I, cont’d)

We wish to obtain the space-time representation of the conserved vars.

within a zone; I.e. we want to involve the dynamical equation– the PDE:-

2 2

0 x xx t tt xt

1ˆ ˆ ˆ ˆ ˆ ˆu (x, t) = w + w x + w x + u t + u t + u x t

12

Similarly, we wish to obtain a space-time representation of the fluxes:

(the fluxes don’t need to be saved, so they don’t take up storage)

2 2

0 x xx t tt xt

1ˆ ˆ ˆ ˆ ˆ ˆf (x, t) = f + f x + f x + f t + f t + f x t12

t tt xt t tt xtˆ ˆ ˆˆ ˆ ˆWe start the with : u u = u = 0 and f f = f = 0iteration

t xGoverning Equation : u + f(u) = 0

V. Imp. Questions: 1) Given u(x,t), how to obtain f(x,t)?

2) How to obtain the improved space-time representation? t tt xtˆ ˆ ˆi.e. u , u & u

9

STEP II) Pick a space-time element. Pick a set of nodal points in a

space-time element. 1d shown here; has been extended to multi-d.

u (xi , ti ) can be evaluated at each such nodal point “i = 1, .., 6”.

STEP III) Use nodal values to find nodal fluxes fi = f ( u (xi , ti ) ) .

STEP IV) Obtain modal representation of fluxes:

Notice: Finite Difference-like forms

0 1 2 3

x 2 3

xx 2 1 3

t 1 2 3 4 5 6

tt 1 2 3 4 5 6

xt 2

f̂ = 4f + f + f 6 ;

f̂ = f f ;

f̂ = 2 f 2 f + f ;

f̂ = f 2 f 2 f + 2 f + 2 f f ;

f̂ = 2 f f f + f + f f ;

f̂ = 2 f

3 4 5f f + f Notice: t=0 variables only

evaluated once; saves float pts.

1 2 3

4 5

6

x

t

Nodal points at

different time levels

Space-time element in (x,t)

Spans [-.5,.5]x[0,1]

x=0.5 x= 0.5

t=0

t=1

10

STEP V) Use space-time basis functions as test functions to derive

update equations:

2

1 2 3x,t = t ; x,t = t ; x,t = x t

Make Galerkin, i.e. weak-form integration over the space-time element :-

t x

1/2,1/2 , 0,1

x, t u x,t + f x,t dx dt = 0 for = 1, 2, 3i

x t

i

This gives us the update equations:-

t x tt xt xt xxˆ ˆ ˆˆ ˆ ˆu = f ; u = f 2 ; u = 2f

Go back to STEP II and iterate!

For an Nth order scheme, even in multi-dimensions, this process can be

shown to converge in (N1) iterations. – Picard Iteration

ADER can now be used in Predictor-Corrector fashion to obtain a

one-step update scheme for conservation laws. We describe Predictor.

11

Flowchart for Initializing ADER-CG Iteration

2

0 x xx

1ˆ ˆ ˆStart with : u (x, t) = w + w x + w x

12

u (xi , ti =0) evaluated at each nodal point “i = 1, .., 3”.

x1=0.5 ; x2 = 0.0 ; x3=0.5

Use nodal values, u (xi , ti ) to find nodal fluxes fi = f ( u (xi , ti )), i=1,..,3

i i

0 1 2 3 x 2 3 xx 2 1 3

ˆUse nodal fluxes f to obtain modal fluxes f

ˆ ˆ ˆf = 4f + f + f 6 ; f = f f ; f = 2 f 2 f + f

t tt xt t tt xtˆ ˆ ˆˆ ˆ ˆWe start the with : u u = u = 0 and f f = f = 0iteration

12

Flowchart for ADER-CG Iteration

2 2

0 x xx t tt xt

1ˆ ˆ ˆ ˆ ˆ ˆStart with : u (x, t) = w + w x + w x + u t + u t + u x t

12

u (xi , ti ) evaluated at each nodal point “i = 4, .., 6”.

(x4, t4) = (-.5,.5) ; (x5, t5) = (0.5,0.5) ; (x6,t6) = (0,1)

Use nodal values, u (xi , ti ) to find nodal fluxes fi = f ( u (xi , ti ) )

i i t 1 2 3 4 5 6

tt 1 2 3 4 5 6 xt 2 3 4 5

ˆ ˆUse nodal fluxes f to obtain modal fluxes f : f = f 2 f 2 f + 2 f + 2 f f ;

ˆ ˆf = 2 f f f + f + f f ; f = 2 f f f + f

t x tt xt xt xxˆ ˆ ˆˆ ˆ ˆUpdate equations : u = f ; u = f 2 ; u = 2f

N-1 iterations for an Nth order scheme.

13 13

MHD is different, Reason: The magnetic field evolves according to

Faraday’s law, i.e. a Stokes-law type equation. Notice dualism in fluxes. 1

+ c = 0 ; = ; B-field satisfies constraint: = 0t c

BE E v B B

x

y z

Zone center (i,j,k)

x, 1/2, ,B i j k

y, , 1/2,B i j k

z, , , 1/2B i j k

x, , 1/2, 1/2E i j k

y, 1/2, , 1/2E i j k

z, 1/2, 1/2,E i j k

x

y

z

z, 1/2, 1/2,E i j k

Motivating Multi-D

Riemann Solver

14

3) Multidimensional HLL & HLLC Riemann Solvers

Task: Build 2d HLL & HLLC Riemann Solvers for : 0t x y U F G

Patterned after the three central ideas in the 1D HLL Riemann solver:

1) Use a simple wave model with extremal waves in each of the two

principal mesh directions.

2) The resolved state is a constant state(s); Likewise, for the resolved

flux(es).

3) The rest of the 2d Riemann solver is to be obtained via applications

of the integral form of the conservation law.

x

t

U* UR UL

SRT SLT

T

Recapitulate 1D HLL RS

for : 0t x U F

15

x

y

SRTSLT

SUT

SDT

Q

A

MCN

O

R

B

D

RUULUU

LDU RDU

RUF

RUG

LUF

LUG

RDF

RDG

LDF

LDG

Let 4 zones come together at the vertex O. Assume structured mesh.

Let the solutions from the 4 zones that come together at the vertex be:

ULU -- Read as ULeftUp URU -- Read as URightUp

ULD -- Read as ULeftDown URD -- Read as URightDown

x x x x

1 1 1 1

x x x x

y y y y

1 1 1 1

y y y y

S max , , , , ,

S min , , , , ,

S max , , , , ,

S min , , , , ,

N N N N

R RU RD LU RU LD RD

L LU LD LU RU LD RD

N N N N

U RU LU RD RU LD LU

D RD LD RD RU LD LU

U U U U U U

U U U U U U

U U U U U U

U U U U U U

Extremal speeds for 2d

wave model:

(Notice: 1d RP is built in)

Putting it in perspective:

Monge Cone Monge Pyramid

for HLL.

OK, because it introduces a little

more dissipation.

16

y

t

CD

T

F

HLL

DFHLL

UF

SUTSDT

Space-Time depiction of wave model x=0 slice of the 2d wave model

Observe that the dark and light grey regions will only have a 1d RP

It is by this device that the 2d RP collapses to the 1d RP when needed.

Now integrate over space-time prism with base QRNM to get Resolved

State U*. Similarly, integrate over prism with base QDCM to get Flux F*.

Q

MN

R

x O

AB

C

D

y

t

U*, F*,G*

Resolved state is constant state

17

The resolved state is given by:

The two resolved fluxes are given by

(Notice weighted 1d HLL fluxes + multi-d term):

The Riemann solver is provably positivity preserving for density.

Don’t have a handle on which states should be free of cavitation, but

positivity preserving aspect for the pressure is also demonstrated for

large variation in variables.

S S + S S S S S S

S S S S

S S + S S

S S S S

RU R U LD L D RD R D LU L U

R L U D

RU LU U RD LD D RU RD R LU LD L

R L U D

U U U UU

F F F F G G G G

HLLE HLLE

HLLE HLLE

S S S S

S S S S S S S S

S SS S

S S S S S S S S

U D R LU D RU LU LD RD

U D U D R L U D

U DR LR L RU LU LD RD

R L R L R L U D

F F F G G G G

G G G F F F F

18

Notice, this RS is invoked at edges, not face-centers. It returns two

fluxes, x-flux and y-flux, for every invocation.

Does permit larger CFL numbers. Stable for least a CFL of 0.65, and

often much higher, on 2d problems.

2d test problems from Schultz-Rinne, Collins & Glaz (1993).

19

4)Some Hydro & MHD Results with ADER-WENO Accuracy Analysis for Hydrodynamics:-

V. Accurate Advection:-

3rd order 4th order

20

Accuracy Analysis for MHD:-

Dissipation-free

Propagation

of Alfven Waves :-

21

MHD Rotor, 3rd Order Scheme:-

MHD Near-Infinite 3D Blast Problem, 4th Order Scheme:-

22

But What About Speed? Notice: At the same cost, ADER gives one

higher order than RK!

Scheme

Riemann Solver 2nd Order

Zones/sec

3rd Order

Zones/sec

4th Order

Zones/sec

ADER-WENO-MHD Linearized 60,186 33,424 10,396

RK-WENO-MHD Linearized 33,475 14,057 3,051

ADER-WENO-Euler Linearized 154,674 67,025 20,348

RK-WENO-Euler Linearized 92,448 31,598 8,119

2.5 GHz Nehalem, single processor

5) Higher Order Scheme FAQs

And What About Memory Usage? 3rd order uses 1.67 times as much

memory as 2nd order; 4th order uses 1.87 times as much memory as 3rd

order.

23 23

And What About Scalability?

Results from RIEMANN code

running with CHOMBO

framework. (And yes,

CHOMBO does do Div-free

AMR-MHD!)

RK involves much more

communication!

Do the methods take well to

Adaptive Mesh Refinement?

Do they scale well with AMR

to PetaFlop Scales?

24

But What About Shock-Dominated 3D Turbulence Problems?

Compensated velocity spectrum Compensated magnetic spectrum

25

6) Conclusions

1) ADER formulation for Euler & MHD flow displayed.

2) Efficient strategies for evaluating numerical fluxes and electric fields

shown. Multidimensional Riemann solver removes one of the

fundamental stumbling blocks in numerical MHD.

3) Resulting schemes have superb accuracy and advection properties

without compromising on shock-capturing.

4) Resulting schemes are fast, have low reasonably memory footprint and

are highly parallelizable.

5) Several applications have already been carried out.

6) Advantages of higher order even extend to large, 3D, shock-dominated

turbulence problems.

7) All, including codes available as a text-book on Computational Astro

26

Table of Contents : Computational Astrophysics Text-Book

Chapter 1: Overview of Partial Differential Equations Relevant to Astro.

Chapter 2: Finite Difference Approximations – Stability Theory

Chapter 3: Scalar Advection and Linear Hyperbolic Systems

Chapter 4: Nonlinear Conservation Laws; The Scalar Case

Chapter 5: The Hydrodynamical Riemann Problem

Chapter 6: Eigenstructure and Approximate Riemann Solvers for

Hyperbolic Systems

Chapter 7: Multidimensional Schemes for Nonlinear Hyperbolic

Systems; 2nd and Higher Order

27

Chapter 8: The Inclusion of Non-Ideal Terms and Stiff Source Terms

Chapter 9: A Little on Other Hyperbolic PDEs of Interest

Chapter 10: Multigrid Methods for Elliptic and Parabolic PDEs

Chapter 11: Systems Requiring Matrix Inversion and Some Modern

Linear Algebra Methods and Packages

Chapter 12: Adaptive Mesh Refinement

28

2.b) Rapid strategy for obtaining numerical

fluxes (& electric fields) Dumbser, Kaser & Toro (2008), Balsara et al. (2009), Balsara et al. (2011)

Say for the governing equation :

We have obtained the space-time representation of “u” and “f”,

i.e. within each zone we have:

t x u + f(u) = 0

2 2

0 x xx t tt xt

1ˆ ˆ ˆ ˆ ˆ ˆu (x, t) = w + w x + w x + u t + u t + u x t

12

2 2

0 x xx t tt xt

1ˆ ˆ ˆ ˆ ˆ ˆf (x, t) = f + f x + f x + f t + f t + f x t12

Question: How do we rapidly obtain the numerical flux after we

have obtained the space-time representation?

29 29

1/2 ; 1/2 ; 1/2 ; 1/2 ; 1/2

1/21/2

Consider the v. simple case of the HLL flux:

The numerical flux is then given by :

HLL R L R Li L i R i R i L i

R L R L R L

HLL HLL

ii

f t f t f t u t u t

f f t

1

0

1/2 ; 1/2 ; 1/2 ; 1/2 ; 1/2

If we freeze and in some intelligent way (described later) then we can write:

t

L R

HLL R L R L

i L i R i R i L iR L R L R L

dt

f f f u u

1

; 1/2; 1/2

0

1

; 1/2; 1/2

0

1

; 1/2; 1/2

0

1

; 1/2; 1/2

0

Define:

;

;

;

;

L iL i

t

R iR i

t

L iL i

t

R iR i

t

f f t dt

f f t dt

u u t dt

u u t dt

x

t i i+1

; 1/2 ; 1/2;

L i L iu f

; 1/2 ; 1/2

; R i R i

u f

; 1/2

; 1/2

1/ 2, &

1/ 2,

L i i

L i i

u t u x t

f t f x t

; 1/2 1

; 1/2 1

1/ 2, &

1/ 2,

R i i

R i i

u t u x t

f t f x t

RL

t=0

t=1

30 30

Above example showed how time-averaging can be done. For

multidimensional problems, the averaging can be done in space & time.

Can also be done for other Riemann solvers.

Question: How does one obtain L and R ?

x

t i i+1 i-1

; 1/2bL iu ; 1/2bR iu

L R

; 1/2 ; 1/2 11/ 2, 1/ 2 and 1/ 2, 1/ 2

are honest to goodness physical states at the space-time barycenters of the faces.

They can be used to obtain and

bL i i bR i i

L R

u u x t u u x t

These ideas extend to multidimensions; they can also be used to yield

the electric fields for MHD.