http://www.phy.ornl.gov/tsi/

TeraScale Supernova Initiative:TeraScale Supernova Initiative:A Networker’s ChallengeA Networker’s Challenge

11 Institution, 21 Investigator, 34 Person, Interdisciplinary Effort ascertain the core collapse supernova mechanism(s)understand supernova phenomenology

e.g.: (1) element synthesis, (2) neutrino, gravitational wave, and gamma ray signatures provide theoretical foundation in support of OS experimental facilities (RHIC, SNO, RIA, NUSEL)develop enabling technologies of relevance to many applications

e.g. 3D, multifrequency, precision radiation transportserve as testbed for development and integration of technologies in simulation “pipeline”

e.g. data management, networking, data analysis, and visualization

Explosions ofExplosions ofMassive StarsMassive Stars

Relevance:Relevance:Element ProductionElement ProductionCosmic LaboratoriesCosmic LaboratoriesDriving ApplicationDriving Application

With ISIC and other collaborators:With ISIC and other collaborators:77 people from 24 institutions involved.77 people from 24 institutions involved.

Need Boltzmann SolutionNeed Angular DistributionNeed SpectrumNeed Neutrino Distribution

Fluid Instabilities Rotation Magnetic Fields

6D RMHD Problem!6D RMHD Problem!Need these to few percent accuracy!

f (r,μ,E)f (r,θ,μ1,μ2,E)f (x,y,z,μ1,μ2,E)

Spherical SymmetrySpherical Symmetry

AxisymmetryAxisymmetry

No SymmetryNo Symmetry

R(r,μ,E,μ',E')R(r,θ,μ1,μ2,E,μ1

',μ2',E')

Example: Boltzmann transport equation for spherical symmetry.Example: Boltzmann transport equation for spherical symmetry.

R(x,y,z,μ1,μ2,E,μ1',μ2

',E')

Dominant Computation:Nonlinear, integro-partial differential equations for the radiation distribution functions.

3D Hydrodynamics Run5 Variables (Density, Entropy, Three Fluid Velocities)1024 X 1024 X 1024 Cartesian Grid1000 Time Steps

43 Terabyte Dataset

“The flea on the tail on the dog…”

Multidimensional Neutrino Data

f (x,y,z,μ1,μ2,E)

Data Management Data Management

13 Petabyte Dataset

Eν(x,y,z,E)r F ν(x,y,z,E)

r F ν (x,y,z,E)

t P ν(x,y,z,E)

⎛

⎝ ⎜

⎞

⎠ ⎟ ~3 Petabyte Dataset

...in weeks to months on a PF platform.

NetworkingNetworking

Raw Bandwidth Needs43TB (Hydro Data Only):

@1Gbps→ ~5days

@10Gbps→ 10hrs

@100Gbps→ 1hr

@1Tbps→ ~5minutes

What about the radiation field data?

Need end-to-end dedicated paths/bandwidth, on demand.

Interactive visualization.Real-time collaboration.

Need protocols that provide this capability.None exist that will give 10 Gbps throughputs and stable control.

Work with Nagi Rao (ORNL).

Eν(x,y,z,E)r F ν(x,y,z,E)

r F ν (x,y,z,E)

t P ν(x,y,z,E)

⎛

⎝ ⎜

⎞

⎠ ⎟

@ 3 PB!

Bulk Data Transfer NeedsBulk Data Transfer Needs Needs for Collaborative VisualizationNeeds for Collaborative Visualization

Addressing Bulk Data Transfer Needs:Addressing Bulk Data Transfer Needs:

Logistical Networking Light Weight Low Level Deployable … Solution

New Paradigm Integrate storage and networking. Multi-source, multi-stream.

QuickTime™ and aAnimation decompressor

are needed to see this picture.

Atchley, Beck, and Moore (2003)

Data transfer rates 200-300 Mbpsusing TCP/IP!

Limit set by ORNL firewall.Greater rates expected

outside firewall,other protocols (e.g., Sabul).

Direct impact on TSI’s ability to do work!

Without putting in place the needed computational science infrastructure, our science will simply be inaccessible in the future.

Significant progress has been made in the areas of linear solvers, performance analysis and optimization, data management and analysis, networking, and visualization.

In particular, Logistical Networking has provided an easily deployable solution to our current bulk data transfer needs and has had a significant impact on TSI’s current ability to do science.

TSI’s future data management and networking needs are daunting. TSI will generate hundreds of TeraBytes of simulation data per simulation within the next two years. What then?

Meeting these needs will require every new idea. Investment now in networking technologies will allow us to meet these needs.