Stability Analysis Algorithms for Large-Scale Applications

Stability Analysis Algorithms for Large-Scale Applications

Andy Salinger, Roger Pawlowski, Ed Wilkes

Louis Romero, Rich Lehoucq, John Shadid

Sandia National Labs

Albuquerque, New Mexico

Computational Challenges in Dynamical Systems

Fields Institute, Dec. 6, 2001

Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy under contract DE-AC04-94AL85000.

Supported by DOE’s MICS and ASCI programs

Elevator Talk (Lift Talk)

We’re developing a library of stability analysis algorithms that work with massively parallel engineering analysis codes.

The main research issues are developing algorithms that are relatively non-invasive (easy to implement) and that work reasonably well with approximate iterative linear solvers.

With this “LOCA” software, we’ve been able to track bifurcations of 1 Million unknown PDE discretizations.

What are you working on these days?

Why Do We Need a Stability Analysis Capability?

Nonlinear systems exhibit instabilities, e.g

• Multiple steady states

• Ignition

• Symmetry Breaking

• Onset of Oscillations

• Phase Transitions

These phenomena must be understood in order to perform computational design and optimization.

Current Applications: Reacting flows, Manufacturing processes, Microscopic fluids

Potential Applications: Electronic circuits, structural mechanics (buckling)

Delivery of capability:LOCA libraryExpertise

The Targeting of Large-Scale Applications Codes Restricts the Choice of Algorithms

Requirements: Stability analysis algorithms must be scalable and relatively non-invasive:

• Must work with iterative (approximate) linear solvers

• Should avoid or limit: Requiring more derivatives Changing sparsity pattern of matrix Increasing memory requirements

Targeted Codes: Newton’s Method, Large-Scale, Parallel

• Navier-Stokes & Reaction-Diffusion, Free Surface Flows, Molecular Theory, Structural Mechanics, Circuit Simulation

LOCA:The Library of Continuation Algorithms

Arclength continuation

Turning point (fold) tracking

Pitchfork tracking

Phase transition tracking

rSQP optimization hooks (Biegler, CMU)

Residual fill (R)

Jacobian Matrix solve (J-1b)

Mat-Vec multiply (Jb)

Set parameters ()

LOCA Algorithms LOCA Interface

LOCA:The Library of Continuation Algorithms

Arclength continuation

Turning point (fold) tracking

Pitchfork tracking

Phase transition tracking

rSQP optimization hooks (Biegler, CMU)

Eigensolver: ARPACK driver for Cayley transform

Residual fill (R)

Jacobian Matrix solve (J-1b)

Mat-Vec multiply (Jb)

Set parameters ()

Fill mass matrix (M)

Shifted Matrix Solve (J+M)

LOCA Algorithms LOCA Interface

LOCA:The Library of Continuation Algorithms

Arclength continuation

Turning point (fold) tracking

Pitchfork tracking

Phase transition tracking

rSQP optimization hooks (Biegler, CMU)

Eigensolver: ARPACK driver for Cayley transform

Hopf tracking

Residual fill (R)

Jacobian Matrix solve (J-1b)

Mat-Vec multiply (Jb)

Set parameters ()

Fill mass matrix (M)

Shifted Matrix Solve (J+M)

Complex matrix solve (J+iM)

LOCA Algorithms LOCA Interface

Q: Can General Bifurcation Algorithms Scale to ASCI-Sized Problems?

• Large problem sizes require iterative linear solves• The less invasive bordering algorithms require inversion of

matrices that are being driven singular

R 0=

Jn 0=

n 1=

Turning Point BifurcationJ 0 Rp

Jn x J Jpn

0 t0

xn

p

R–Jn–

1 n–

=

Full Newton Algorithm

Ja R–=

Jb R– p=

Jc Jn – xa J– n=

Jd Jn – xb Jp– n=

p 1 n– c– d =x a p b+=n c p d+=

Bordering Algorithm

Bordering Algorithm for Hopf tracking

f x 0=

Jy Mz+ 0=

Jz My– 0=

lty 1– 0=

ltz 0=

J 0 0 0f------

Jy x

-------------Mz

x--------------------+ J M Mz

Jy -------------

Mz --------------------+

Jz x

-------------My

x--------------------– M– J My–

Jz -------------

My --------------------–

0 lt

0 0 0

0 0 lt

0 0

xyz

f–Jy– Mz–

Jz– My+

1 lty–

ltz

=

J M– M J

g

h

Jy -------------

Jy x

-------------b Mz

--------------------- Mz

x---------------------b+ + +

Jz -------------

Jz x

-------------b My

---------------------– My

x---------------------b–+

=

Ja f–=

Jbf------–=

J M– M J

e

f

Jy x

-------------a Mz

x---------------------a+

Jz x

-------------a My

x---------------------a–

=

J M– M J

c

d

Mz

My–=

ltdl

te l

td l

tf l

tc–+

lthl

tc l

tgl

td–

--------------------------------------------=

1 lte l

tg+ +

ltc

------------------------------------–=

y y– e– g– c–=

z z– f– h– d–=

x a b+=

Eigenvalue Approx with Arnoldi, ARPACK 3 Spectral Transformations have Different Strengths

Complex Shift and Invert Cayley Transform v.1 Cayley Transform v.2

Lehoucq and Salinger, IJNMF, 2001.

Stability of Buoyancy-Driven Flow: 3D Rayleigh-Benard Problem in 5x5x1 box

MPSalsa (Shadid et al., SNL):

•Incompressible Navier-Stokes

•Heat and Mass Transfer, Reactions

•Unstrucured Finite Element (Galerkin/Least-Squares)

•Analytic, Sparse Jacobian

•Fully Coupled Newton Method

•GMRES with ILUT Preconditioner (Aztec package)

•Distributed Memory Parallelism

200K node meshpartitioned for320 Processors

At Pr=1.0, Two Pitchfork Bifurcations Located with Eigensolver

Eigenvector at Pitchfork

No Flow

2D Flow

3D Flow

5 Coupled PDE’s,50x50x20 Mesh:275K Unknowns

Three Flow Regimes Delineated by Bifurcation Tracking Algorithms

Codimension 2 BifurcationNear (Pr=0.027, Ra=2050) Eigenvectors at Hopf

Rayleigh-Benard Problem used to Demonstrate Scalability of Algorithms

ScalabilityScalabilityEigensolver: 16MContinuation:

16MTurning Point:

1MPitchfork:

1MHopf: 0.7M

Steady Solve 5 Minutes

Eigenvalue Calculation (~5)

10-20 Minutes

Pitchfork Tracking

25 Minutes

Hopf Tracking 80 Minutes (p=200)

275K Unknowns: 128 Procs

CVD Reactor Design and Scale-up:Buoyancy force can lead to undesirable flows

Chemical Vapor Deposition of Semiconductors: GaN, GaAs

Good and bad flows are found to coexist at certain values of (Ra, Re)

Good Flow

Bad Flow

30500 Unknowns

Good and bad flows are found to coexist at certain values of (Ra, Re)

30500 Unknowns

Tracking of bifurcation leads to design rule

Ra 1.75Re0.5

1100Re---------+

=

Ideal gas curves collapse onto Boussinesq for good choice of To

Boussinesq

Pawlowski, Salinger, Romero, Shadid 2001

Optimization Algorithms, such as rSQP, Need Same Calls as Bifurcation Algs

Collaboration with Biegler, CMU

Operability Window for Manufacturing Process Mapped with LOCA around GOMA

Slot Coating Application

Family of InstabilitiesFamily of Solutions w/ Instability

Steady Solution (GOMA)

back pressure

bac

k p

ress

ure

LOCA+Tramonto: Capillary condensation phase transitions studied in porous media

Bifurcation Diagram (a.k.a. Adsorption Isotherm)Density contours aroundrandom cylinders

LOCA+Tramonto: Capillary condensation phase transitions studied in porous media

Liquid

Vapor

Partial Condensation

Phase diagram

Tramonto: Frink and Salinger, JCP 1999,2000,2002

Counter-terrorism via PDE Optimization: Find fluxes at 16 surfaces to match data at 25 sensors

rSQP Exact

1.954 2.0

0.032 0.0

-0.012 0.0

-0.014 0.0

-0.006 0.0

0.042 0.0

-0.017 0.0

0.003 0.0

0.002 0.0

-0.002 0.0

4.990 5.0

0.057 0.0

0.312 0.0

0.133 0.0

0.944 1.0

0.049 0.0

1

2

5

6724 State variables, 16 design variables, x0=y0=0

88 rSQP Iterations, f=1.5e-6 , 30 sec / iter

Re=10

Flow Transport

Fluxes

Summary and Future Work

Powerful analysis tools has been developed to study large-scale flow stability applications:

– General purpose algorithms in LOCA linked to massively parallel finite element codes.

– Bifurcations tracked for 1.0 Million unknown models– Singular formulation works semi-robustly

Future work : Support common linear solvers (e.g. Aztec, Trilinos, PetsC, LAPACK)Implement more invasive, non-singular (bordered) formulationsMultiparameter continuation (Henderson, IBM)New application codes, e.g. buckling of structures

J a

bt c