Organization of Talk: TSI Project Description TSI-ISIC Collaborations Ongoing TSI-SDM Projects A Look Ahead

http://www.phy.ornl.gov/tsi/

Investigator Team

Radiation Transport/Radiation Hydrodynamics

Blondin (NC State) Bruenn (FAU) Hayes (UCSD) Mezzacappa (ORNL) Swesty (SUNYSB)

Nuclear Structure Computationsfor EOS and Neutrino-Nucleus/Nucleon Interactions

Dean (ORNL, UT) Fuller (UCSD) Haxton (INT, Washington) Lattimer (SUNYSB) Prakash (SUNYSB) Strayer (ORNL, UT)

Linear System/Eigenvalue Problem Solution Algorithms for Radiation Transport and Nuclear Structure Computation

Dongarra (UT, ORNL) Saied (UIUC, NCSA) Saylor (UIUC, NCSA)

Visualization

Baker (NCSA) Toedte (ORNL)

Cross-Cutting Team Long-Term CollaborationsStructured like SciDAC

Supernova Science

Blondin Bruenn Fuller Haxton Hayes Lattimer Meyer (Clemson) Mezzacappa Swesty

TOPS

TOPS

SDM

CCAPERCTSTT

GoalGoal Ascertain the explosion mechanism(s). Reproduce supernova phenomenology (element synthesis; neutrino, gravitational wave, and gamma ray signatures; neutron star kicks; gamma ray burst connection)

RelevanceRelevance Dominant source of many elements in the Universe. Given sufficiently well developed models, serve as laboratories for fundamental nuclear and particle physics that cannot be explored in terrestrial laboratories. Driving application in computational science (radiation transport, hydrodynamics, nuclear physics, applied mathematics, computer science, visualization).

ParadigmParadigm Result from stellar core collapse and bounce in massive stars. Radiatively driven (perhaps some are MHD driven, or both).

Need Boltzmann SolutionNeed Angular DistributionNeed Spectrum

“Gray” Schemes InadequateSpectrum ImposedLimited Angular Information (Few Moments)Parameterized (No First Principle Solution)

The bar is high! (10% effects can make or break explosions.)

Janka and Mueller

Herant et al.

Mezzacappa et al.

Fryer and Heger

Burrows, Hayes, and Fryxell

Swesty TSITSIYear 1Year 1

TSITSIYear 2Year 2

TSITSIYear 3Year 3

TSITSIYear 2Year 2

Gra

y M

od

els

Gra

y M

od

els

Sp

ace

Sp

ace

Neutrino EnergyNeutrino Energy

Past Transport in 2D ModelsD: DiffusionFLD: Flux-Limited DiffusionMGFLD: Multigroup FLDMGBT: Boltzmann Transport

What is the Recipe for Explosion?

Are there multiple mechanisms? Neutrino-driven supernovae MHD-driven supernovae Supernovae driven by both neutrinos and MHD effects

One mechanism for a class of stars?

Is the mechanism tailored to the individual star?

Neutrino Heating

Convection

Rotation

Magnetic Fields

GeneralRelativity

HydrodynamicsExplicit Differencing

Reactive FlowsNewtonian

General Relativistic

RadiationTransport

Implicit DifferencingMGFLD

PreconditionersSparse System Solvers

MGBTPreconditioners

Sparse System Solvers(Matrix Free)

Nuclear,Weak Interaction

PhysicsThermodynamics

(Composition),Neutrino Sources and Interactions

Supernova Science

Implicit Time DifferencingExtremely Short Neutrino-Matter Coupling Time ScalesNeutrino-Matter EquilibrationNeutrino Transport Time Scales

Nonlinear Algebraic EquationsLinearizeSolve via Multi-D Newton-Raphson Method

Large Sparse Linear Systems

ISIC Collaborations: TOPSISIC Collaborations: TOPS

Progress:Sparse Approximate Inverses for 2D MGFLD (Saylor, Smolarski, Swesty; J. Comp. Phys.)ADI-Like Preconditioner for Boltzmann Transport (D’Azevedo et al.; Precond 2001, NLAA)

AGILE-BOLTZRAN, V2D codes turned over to TOPS for analysis and development.

Boltzmann Equation nonlinearintegro-PDE

Memory Requirements (assuming matrix-free methods): 10s Gb up to 1/2 Tb

TSI Code: F90 + MPI Code Object-Oriented Design for Interoperability and Reuse Application Framework:

IBEAM = Interoperability Based Environment for Adaptive Meshes NASA HPCC-Funded Project (PI: Swesty)

AMR: PARAMESH

ISIC Collaborations: CCTTSSISIC Collaborations: CCTTSS

Goal: Develop our framework to be CCA-compliant.Initiated discussions with ANL, LLNL, and ORNL members of CCTTSS.

Assess Code Performance on Parallel Platforms Identify Code Optimizations to Increase Performance

TSI Code SuiteHydrodynamics:

VH-1 (PPM)ZEPHYR (Finite Difference)

Neutrino Transport:AGILE-BOLTZTRAN: 1D General Relativistic Adaptive Mesh

Hydrodynamics with 1D Boltzmann TransportV2D: 2D MGFLD Transport Code V3D: 3D MGFLD Transport Code (Under Development)2D/3D Boltzmann Code (Under Development)

Re

su

lts

fo

r V

H-1

Re

su

lts

fo

r V

H-1

ISIC Collaborations: PERCISIC Collaborations: PERC

VH-1 numerical hydrodynamics algorithmscales well.

Adaptive Quadratures (Direction Cosines) for Multidimensional Radiation Transport

Results for 1D Boltzmann Transporton Milne Problem (D’Azevedo):

Greatest challenge to completing 3D Boltzmann simulations is memory.Minimize number of quadratures to minimize memory needs while maintaining physical resolution. (Also important for 1D/2D MGBT.)Optimization Problem

ISIC Collaborations: TSTTISIC Collaborations: TSTT

Extended Core

Compact Core

Identify Optimal Paths in Our Collaborative Visualization Server-Client ModelMaximize Bandwidth along these Paths (Not Achieved Using Current Protocols)

Collaboration with Supporting Base Projects: NetworkingCollaboration with Supporting Base Projects: Networking

Participated in ORNL Workshop on DoE High-Performance Network R&D and Applications

Convey TSI Needs to Networking Team Participate in White Paper to Define and Develop Interface between Efforts

ISIC Collaborations: SDMISIC Collaborations: SDM

Use PROBE environment for staging data between simulation platforms and end-user visualization platforms. Develop new data analysis techniques/tools tailored to our application, allowing (a) data reduction and (b) discovery potential. Use of agent technology for distributed data analysis (data analysis must be done in parallel to achieve reasonable throughputs).

QuickTime™ and aCompact Video decompressor

are needed to see this picture.

QuickTime™ and aCompact Video decompressor

are needed to see this picture.

QuickTime™ and aCompact Video decompressor

are needed to see this picture.

QuickTime™ and aCompact Video decompressor

are needed to see this picture.

Latest TSI 2D/3D Models:

Hydrodynamics only.Focused on understanding 2D/3D flow and its coupling to shock wave.Convectively stable.

2D model exhibits bipolar explosion (due to nonlinear flow-shock interaction).

3D model exhibits similar “long-wavelength” behavior. Key finding.New “rolling” flows identified.

AAS Meeting; Ap.J. SubmittedAAS Meeting; Ap.J. Submitted

2D Model2D Model

3D Model3D Model

SDM: Data NeedsSDM: Data Needs

3D Hydrodynamics Run5 Variables (Density, Entropy, Three Fluid Velocities)1024 X 1024 X 1024 Cartesian Grid1000 Time Steps

20 Terabyte Dataset

StingrayRS/6000 S80

MarlinRS/6000 H70

Origin 2000Reality Monster

ExternalEsnet Router

IBM and Compaq SupercomputersProbeProduction

Production HPSS

ProbeHPSSCAVE

Other ProbeNodes

Bulk Storage

Data Reduction, pre-Vis ManipulationRendering

SDM: PROBESDM: PROBE

Utilize PROBE until data manipulations, partitioning of manipulations, and bandwidths are known. PROBE is adaptable!

SDM: Data AnalysisSDM: Data Analysis Data Reduction Scientific Discovery

Original density distribution at finaltime “slice.”

Density distribution at last slice reconstructed from 30 principalcomponents. (First slice reconstructed from 3!)

ServerServer ClientClient

PCA Data Reduction Networking Technologies

PCA Data ReconstructionIntegrate into collaborative visualization?

New Windows on the Universe?

Scientific DiscoveryCan we use current data analysis tools to better understand and better quantify supernova physics?Can we develop new tools that will provide a new view of supernova physics?

Team of Agents Divides Up Data

SDM: AgentsSDM: Agents

Current data analysis techniques performed on a 10 Gb dataset would take 3 years to complete!

Need for distributed data analysis.

Agents perform analysis on subsets of data.Merge results via peer-to-peer agent collaboration and negotiation.

+ GUI/environment for the selection and (distributed) use of data analysis tools and the display of pre- and post-processed data.

Both data analysis and visualization can employ agent technology.

SDM: A Look AheadSDM: A Look Ahead

We have a testbed! Existing 2D/3D datasets. Three TSI nodes: NCSA, NCSU, ORNL.

Testbed for collaborative visualization tools.Testbed for networking.

PROBE being used to postprocess the data. PCA has been used successfully for data reduction. Agents have been used in a cross-platform demo utilizing this data.

Continue to explore possibilities.Continue extensive interactions between TSI modelers and SDM data analysts.

Can we integrate data analysis and agent technology (distributed data analysis) with collaborative visualization?

Will existing tools/new tools lead to scientific discovery?New views on the data?Better quantification of supernova dynamics?