Upload
delphia-haynes
View
228
Download
5
Embed Size (px)
Citation preview
Towards the 3D multiphase flow simulation
of collapsing columns and pyroclastic flow propagation
T. Esposti Ongaro(1) , A. Neri(1), C. Cavazzoni(2), G. Erbacci(2), G. Macedonio(3) , M.V. Salvetti(4), M. de’ Michieli Vitturi(5) andF. Beux(6)
(1) Centro per la modellistica fisica e pericolosità dei processi vulcanici – INGV – Pisa - Italy(2) High Performance Computing Division - CINECA –
Casalecchio di Reno (Bologna) - Italy(3) Osservatorio Vesuviano – INGV- Napoli - Italy(4) Dip.to di Ingegneria Aerospaziale – Università degli Studi di Pisa – Italy(5) Dip.t di Matematica - Università degli Studi di Pisa – Italy(6) Scuola Normale Superiore – Pisa- Italy
The main objective of the EU-funded EXPLORIS project consists in the quantitative assessment of explosive eruption risk in densely populated EU regions through the development of volcanic risk facilities such as:
1. 3D Simulation codes (fallout and pyroclastic flows)2. Vulnerability functions and databases3. Probabilistic risk assessment protocols
Framework
Why do we need the 3D modelling?
Fundamental physical processes are intrinsically 3D Examples: Turbulence, fluid dynamic instabilities, etc.
Mt. St. Helens, 1980 (courtesy of B.Voight)
Boundary conditions are also mostly 3D Examples: Volcanic topography, wind, etc.
The Red Zone around the Somma-Vesuvio complex (from Pareschi et al, 2000)
The PDAC model (Neri et al., 2003)
Mass conservation
Gas components conservation
Momentum balance
Enthalpy balance
+ Constitutive equations (e.g. EOS, stress tensors, LES turbulence closure, etc.)
resulting in 3D in [5*(n+1) + (m-1)] independent equations(e.g., 16 equations using n=2 particulate phases and m=2 gas components )
COMPUTATIONALLY VERY DEMANDING!
Code re-engineering
• Implementation of a new modular structure in Fortran95 (easier to use and to mantain)
• Parallelization of the 3D code by adopting a Domain Decomposition approach (using “ghost cells”)
• Adoption of Message Passing Interface (MPI) (guarantees efficiency and portability)
• Improvement of code numerical accuracy by implementing Second-Order-Upwind schemes with limiters (Van Leer, minmod, Ultra Sharp)
• Implementation of “Immersed Boundary” techniques (more accurate description of the flow field over the topographic profile)
Parallelization strategy
topography proc 0
proc 1
proc 2
proc 3
proc 4
proc 5
proc 0
proc 1
proc 2
proc 3
proc 4
proc 5
proc 6
topography
proc 0 proc 2proc 3
proc 1 proc 3proc 4
proc 5
topography
The computational domain was partitioned into sub-domains with equal number of active cells (“domain decomposition” approach)
The load balancing and the scalability depend on the sub-domain topology
1) 2) 3)
Domain Decomposition into:
1) Horizontal layers
2) Vertical columns
3) 3D blocks
Benchmarks performed on a linux cluster with PIV (3.06GHz) and myrinet interconnection network
Scalability of the parallel code The total execution time (T) is the sum of the computation (Tc) and communication (Tm) times: T = Tc + Tm
The Speed-up, S(N), is the ratio between the execution time with 1 and N CPUs S(N) = T(1) / T(N). The ideal Speed-up is linear.
The saturation limit depends on the size of the computational domain and on the sub-domains topology
Estimated execution time for a 3D simulation on a volcano topography (1283 computational cells)
1 3 10 30
100
300
1000
3000
processors
seco
nd
s /
10
0 t
ime
step
s Linux (PIV 3.06GHz)SGI Altix (Itanium2 1.3GHz)IBM p690 (Power4 1.3GHz)
1 simulated second -> 200 sec / 128 CPU30 min. eruption -> 360000 sec / 128 CPU (4 days)
The code has been ported on several supercomputers and PC clusters
The performance depends strongly on the optimization of the algorithm and on the latency time of the interprocessor network
643 Cells
• Composition = Rhyolite (2% wt. H2O), Vent Diam. = 127 m, Mass flow-rate = 3.8 x 107 Kg/s (from Papale et al. 1998)
• Grain-sizes = Two classes of 30 and 500 microns in equal amount• Domain size = (1.2 x 1.2 x 2.0) Km3
• Non-uniform rectilinear mesh = 100 x 100 x 70 (cell size: 5-100 m)
Gas Temperature Isosurfaces (400, 800, 1000°K)
6s 10s 14s
3D Simulation of a axisymmetric volcanic jet
(!) circular vent is discretized by using the finest uniform cells and applying a technique for flux correction in partially filled cells
2D vs 3D axisymmetric jets
6 sec 14 sec
Isolines of the Log10 of the total particle fraction (-5:-2:-0.375) + Vector flow field of the gas velocity
• The comparison between 2D/3D is carried out on a non-uniform mesh (5-100 m) by using a First-Order-Upwind scheme• A Second-Order-Upwind scheme with minmod limiter is also shown (on a uniform 12.7 m mesh)
Simulation of a 3D volcanic jet at Vesuvius
Vent conditions for a sub-Plinian eruption from Todesco et al. (2002)
Gas temperature isosurfaces (400,800,1000°C),t= 25 s
Physical Domain: (8.0x8.0x8.0) km3
Non-uniform rectilinear mesh: 1283 cells
Horizontal resolution: 5-250 m
Simulation of a 3D dome explosion (t=20s)
UTM Georeferenced map: isolines of the gas temperature
Isosurfaces of the total particle fraction (10-3, 10-5)
Initial conditions: P=3 bar, T=1100°C, Solid vol. fract.=0.7, L=50 m
Topography from the Vesuvius DEM
Conclusions
• A first fully 3D version of the multiphase flow code PDAC has been completed and tested
• Benchmarks on the 3D parallel version of the code have shown that it is now possible to run a 3D simulation on a parallel supercomputer in a time comparable to that required for a 2D simulation on a single CPU
• Preliminary simulation results suggest that a more realistic description of the dynamics of volcanic dispersal processes can be obtained by using fully 3D models
Numerical simulation of the pyroclastic flow propagation along the Northern Sector of Vesuvius (run LL_w2_t950 of Todesco et al. 2002): isolines of the Log10 of the total particle fraction. Zoom of the gas velocity vector field near the ground
The Immersed Boundary technique• The “Immersed Boundary” technique consists in the introduction of a pseudo-force in the momentum equations to mimic the presence of a no-slip boundary
• This technique is suited to accurately model the boundary conditions on a generic solid wall even using a rectilinear mesh
• The use of the Immersed Boundary technique on a 2D numerical simulations produces an effect analogous to a decrease of the artificial roughness due to the steps
steps imm.b.
Code optimizationsExecu
tion t
ime
(rela
tive t
o F
77
code)
recover the performances of the F77 original code
code evolution
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
F77 Serial F90 Serial 2D Parallel 3D Parallel Unrolling Memoryaccess
Parameterdependent
sub.
Slower
Faster
Numerical simulation of volcanic column formation at Vesuvius
Run L_w2_t950 in Todesco et al. 2002 (mfr = 5x107, 2% H2O)
• Domain size: (8.0x8.0x8.0) Km3
• 1283 rectilinear non-uniform mesh Horizontal resolution: 5-250 m • NO Immersed Boundaries • The internal crater topography strongly drives the decompression
Gas temperature isosurfaces (400, 800 and 1000°C) at t= 26 sec.