The Applied Partial Differential Equations Integrated Software Infrastructure Center

(APDEC)

Goal: Develop algorithms and software for simulating multiscale problems on structured grids, based on block-structured adaptive mesh refinement (AMR) and embedded boundary (EB) representations of irregular boundaries.

Applications-driven approach: end-to-end development of software tools to meet the needs of specific OSC scientific problems.

ParticipantsAPDEC Principal Investigators

P. Colella, J. Bell (LBNL), L. Diachin (LLNL), R. Samtaney (PPPL), M. Berger (NYU), R. Leveque (Univ. of Washington), M. Minion

(Univ. of North Carolina), E. G. Puckett (Univ. of California-Davis), C. Rutland (Univ. of Wisconsin).

Science CollaboratorsMagnetic Fusion: S. Jardin (PPPL).Combustion: R.Cheng (LBNL); C. Rutland is funded by both

APDEC and BES SciDAC.Accelerator Modeling: R. Ryne, W. Leemans, E. Esarey (LBNL).

SciDAC ISIC CollaboratorsMultigrid solvers: R. Falgout (LLNL / TOPS).CCA: J. Ray (SNL-CA / CCTTSS).

Applications Development, FY 2001 - 2004

Combustion: developed AMR algorithm for turbulent combustion with detailed chemistry and transport, and performed high-resolution simulations of laboratory-scale hydrocarbon flames.

Magnetic Fusion: developed AMR capability for MHD, and applied it to obtain new scientific results in the areas of pellet injection for tokomaks, magnetic reconnection, and MHD shock refraction.

Accelerator Modeling: developed preliminary version of AMR-PIC for Vlasov-Poisson, and coupled it to MaryLie/Impact (ML/I) code for beam dynamics.

General-purpose AMR solvers for elliptic, parabolic and hyperbolic PDEs, with easy-to-use APIs for physics-dependent information.

EB AMR finite-volume solvers for elliptic, parabolic and hyperbolic problems in complex geometries and a variety of grid-generation tools for embedded-boundary grids.

Support infrastructure for particle methods on AMR grids. Node-centered Poisson solvers for AMR-PIC, including support for complex geometries based on the Shortley-Weller method.

Interoperability tools, including a framework-neutral data interface for AMR data, interfaces to hypre and other solver packages, and a prototype CCA componentization of AMR elliptic solvers. AMR visualization and data analysis package.

Software Development, FY 2001 - 2004

Application Goals, FY 2004 - 2006

Use the development of a collection of specific application simulations to drive the development and hardening of the APDEC software suite. Application goals were developed in close consultation with stakeholders, with both the development of general tools and meeting specific modeling needs considered.

Magnetic fusion: The overall goal is to increase the fidelity of the AMR MHD codes in for problems in magnetic fusion. The capabilities we propose developing are driven by important issues for the design of ITER, i.e. fueling a tokomak and stability of burning plasmas. Also, developing AMR algorithms and software to deal with the strong anisotropies present in these problems represents a substantial extension of our current capabilities.Pellet injection: anisotropic diffusion, high-fidelity and efficient treatment of plasma boundary. Magnetic reconnection: 2-fluid model, including Hall term; implicit treatment of magnetosonic, Alfven waves.

Applications Goals (cont’d)

Accelerator modeling: AMR-PIC for Vlasov-Poisson enhances a core capability for accelerator modeling, and will be completed early in the renewal period. Fluid simulation tools for laser-driven plasma-wakefield accelerators meets critical need in an area in which there are few tools available. Beam dynamics: complete the development of a robust AMR-PIC capability for AST electrostatic PIC codes, including support for complex geometries. Laser-driven plasma-wakefield accelerators: develop fluid dynamics simulation capabilities for gas jet injection, laser-driven shock dynamics.

Combustion: develop a simulation capability for an ultra-low NOx burner, combining the existing AMR combustion capability with an EB AMR algorithm for viscous incompressible flow in the nozzle.It is difficult to obtain and interpret multidimensional data in flows with swirl, while such flows are the most common way to stabilize flames. Combining combustion and complex geometry in AMR represents a substantial increment in capability over what is currently available.

Software Goals, FY 2004 - 2006Performance tuning of embedded boundary software: new aggregate irregular stencil operations for serial efficiency, and new load-balancing approaches for improving parallel scalability (needed for combustion, gas jet problem).

Extension of the APDEC infrastructure to support AMR on mapped grids,parabolic solvers for problems with strong anisotropies (needed for MHD modeling).

Increased interoperability within the APDEC software suite (e.g. mixing spatial dimensions) and between APDEC software and other packages (e.g. framework-neutral data alias for particles) (needed for all applications).

Software Goals (cont'd)

Implement new analysis-based AMR algorithms for constant-coefficient elliptic solvers. Combination of FFTs, multipole methods and Anderson’s method of local corrections leads to more efficient algorithms with increased robustness, vastly reduced parallel communication requirements, and better coupling to PIC (needed for all applications).

3D MLC calculation of Poisson's equation, with 256^3 mesh broken into 64^3 blocks. Imageon right shows solution error through slice at mid-plane (joint work with S. Baden, G. Balls, UCSD.)

Advanced Algorithm DevelopmentGoal: to investigate issues in algorithm design with a potential impact on SciDAC applications in the 3-5 year time horizon.

• Higher order methods; spectral deferred corrections (UNC).

• Sharp interface algorithms for compressible jets (UCD).

• Higher-order spatial discretizations for embedded boundaries (NYU, LLNL, Univ. Washington).

• Improved methods for generating surface data from CAD data (LLNL).

• Novel discretization methods for mapped grids. (Univ. Washington).

• Improved model fidelity for spray combustion (Univ. Wisconsin).

Staffing Profile for APDECCurrent staffing profile:

Applications Development: 2.65 FTE (2 FTE at LBNL, .65 FTE at PPPL). An additional .35 FTE has been provided to PPPL by the SAPP program, for which a separate proposal will be submitted for the renewal.

Software Development: 5.66 FTE (4.66 FTE at LBNL, 1 FTE at LLNL).

Advanced Algorithm Development: 1 FTE at LLNL.

University researchers contribute to both software development and advanced algorithm development efforts (primarily the latter).

For the renewal period, there will be a shift of 3 FTE of effort from software development to applications development. These will mainly consist of the current members of the software development team: their intimate knowledge of the algorithms and software will be essential to the successful development of the new applications capabilities.

The remaining software development efforts will focus on delivering specific capabilities that are required by the various applications efforts, as well as supporting the existing software base.

Broader ImpactsOther applications projects that are using APDEC software.

Flame propagation in type 1A supernovae (S. Woosley, UCSC).

Cosmology: galaxy formation (F. Miniati, MPA-Garching); semi-local strings (J. Borrill, LBNL)

NASA CT Program: multiphase flow in microgravity environments; star formation.

Time-dependent Ginzburg-Landau models for phase-field dynamics (F. Alexander, LANL).

AMR-PIC for heavy-ion fusion simulations (A. Friedman, D. Grote, J.-L. Vay, LBNL / LLNL).

Cell modeling (A. Arkin, P. Schwartz, LBNL; D. Adalsteinsson, Univ. of North Carolina).

Large-deformation solid-fluid interactions. (G. Miller, UC Davis).

Micro-fluidic simulation for BioMEMS devices (D. Trebotich, LLNL).

Broader Impacts (cont.)

Mesoscale atmospheric modeling (C. Bono, LBNL).

Flow in San Francisco bay (M. Barad, UC Davis; E. Ateljevich, California DWR).