Embed Size (px)
Citation preview
May 17, 2007
Algorithmic/Algorithmic/ExascaleExascale Simulation Simulation Opportunities in Modeling of Nuclear Opportunities in Modeling of Nuclear Fuel RodsFuel Rods
Phani Kumar V.V. Phani Kumar V.V. NukalaNukala, , SrdjanSrdjan SimunovicSimunovicComputer Science and Mathematics DivisionComputer Science and Mathematics DivisionOak Ridge National LaboratoryOak Ridge National Laboratory
Steve Steve ZinkleZinkleMaterials Science and Technology DivisionMaterials Science and Technology DivisionOak Ridge National LaboratoryOak Ridge National Laboratory
Larry Larry OttOttNuclear Science and Technology DivisionNuclear Science and Technology DivisionOak Ridge National LaboratoryOak Ridge National Laboratory
Phani K. Nukala Oak Ridge National Laboratory
Nuclear Fuel CycleNuclear Fuel Cycle
Uranium OreUranium OreUranium Oxides (UUranium Oxides (U3OO88))U235 (0.7%), U238U235 (0.7%), U238
Uranium Uranium hexaflouridehexaflouride (UF(UF66))
Uranium Uranium hexaflouridehexaflouride (UF(UF66))U235 (>3.5%), U238U235 (>3.5%), U238
Uranium dioxide (UOUranium dioxide (UO22))
HH22SOSO44
ConversionConversion
EnrichmentEnrichmentHigh Temp. High Temp. sintering furnacesintering furnace
Brittle ceramic Brittle ceramic pelletspellets
Phani K. Nukala Oak Ridge National Laboratory
Nuclear Fuel AssemblyNuclear Fuel Assembly
Reactor vessel and internals
Fuel assembly
Fuel Rod
Pellet
Phani K. Nukala Oak Ridge National Laboratory
Nuclear Fuel RodNuclear Fuel Rod
3.6
to 3
.8m
The fuel pellet undergoes significant physical changes during burn upModeling of pellet deformation requires
Fine resolution, and 3D discretizationCoupling multi-physics models at multiple scales
Phani K. Nukala Oak Ridge National Laboratory
Complex MOX Fuel Pellet Behavior Seen in Complex MOX Fuel Pellet Behavior Seen in Historical LMFBR StudiesHistorical LMFBR Studies
Cross-section of LMFBR fuel and description of evolution
FuelSwelling due to fission products, irradiation damagephase changes, microstructure evolution Thermal expansion, conductivity, heat generationCracks and changes in mechanical properties
Phani K. Nukala Oak Ridge National Laboratory
Multiple PhenomenaMultiple Phenomena
Thermal conductivityFission gas formation, behavior and releaseMaterials dimensional stability– Restructuring, densification,
growth, creep and swellingDefect formation & migrationsDiffusion of species
Pu and Am redistributionMechanical propertiesThermal expansionSpecific heatPhase diagramsFuel-clad gap conductanceRadial power distributionFuel-clad chemical interactions
Dynamic properties:Changes with radiation effects,temperature, and time
Phani K. Nukala Oak Ridge National Laboratory
Fuel Modeling is inherently MultiFuel Modeling is inherently Multi--scalescale
nm-μmLength scale
Tim
e sc
ale
ps-n
sns
-μs
μs-s
s-ye
arde
cade
s
Å-nm μm-mm mm-m
Point defect properties Ef, Em,…(Ab initio,
Quantum chemical)
Defect & fission product production(Molecular dynamics,
Fission track thermal spike)
Defect & fission product migration, bubble nucleation(Kinetic Monte Carlo)
Irradiated fuel chemistry & phase stability
(thermodynamic & chemical rate theory codes)
Fuel performance(thermal conductivity,
fission product accumulation, fuel swelling codes)
Fuel-claddinginteractions(fuel swelling,
fuel-clad chemistrycodes)
-900
-800
-700
-600
-500
-400
-300
-200
-100
0
FUN
CTI
ON
MU
O2
500 1000 1500 2000 2500 3000T
UO2+x
50 GWd/MT MOX
g.b.g.b.
0
20
40
60
80
100
120
0.0001 0.001 0.01 0.1
60°C Irradiation
300°C Irradiation
The
rmal
Con
duct
ivity
(W/m
-K)
Damage Level (dpa)0
Aluminum Nitride
Silicon NitrideMD simulation of He mobility in UO2 gb
Columnar grains Equiaxed grains
Temperature
Phani K. Nukala Oak Ridge National Laboratory
Current StateCurrent State--ofof--thethe--Art (NRC Codes)Art (NRC Codes)
Axi-symmetric models (2D)
Coarse discretization
Heuristic multi-physics models
Fundamental nuclear physics occurs at the grain size level (10-20 μm) in the fuel pellet
Phani K. Nukala Oak Ridge National Laboratory
Simulation & Modeling MotivationSimulation & Modeling Motivation
Predictive capabilityReduce fuel development and qualification time by a factor of 3Deliver increasingly accurate simulation capabilities for fuel life cycle performance assessmentAddress safety concerns during steady state and transients
centerline fuel melting and wastage of cladding Address fuel rod behavior in Design Basis Accident scenarios such as loss-of-coolant, reactivity insertion, etc
TRU fuelsExtreme temperatures, irradiation rates, damageNew chemistry, physics, material issues that are not considered previously
Phani K. Nukala Oak Ridge National Laboratory
Motivation for High Performance ComputingMotivation for High Performance Computing
Multi-physics couplingNeutronics, chemistry, fission products transport, heat transfer, thermo-mechanics, thermal-hydraulics etc.
Modeling of multiple scales in space and timeFrom irradiation point defects, chemistry, microstructure evolution to transport, conductivity and constitutive properties at macro-scaleDevelop physics-based macroscopic models
Computational efficiencyProvide comparable turn-around time with new codesDesign optimization and sensitivitiesVarious loading/boundary conditions scenarios
Phani K. Nukala Oak Ridge National Laboratory
High-fidelity, physics-based simulation of nuclear fuel performance requires models with increased spatial and temporal resolution
Computational tools for simulation of macroscopic fuel rod models (engineering and design)
Computational tools for developing physics-based models (first-principles, molecular dynamics, dislocation dynamics, kinetic Monte Carlo, discrete lattice models)
Algorithms for coupling multi-physics based models across length and time scales
OutlineOutline
Phani K. Nukala Oak Ridge National Laboratory
Relevant Computational SimulationsRelevant Computational Simulations
Fuel rod / fuel bundle simulationsEngineering and design Continuum mechanics based finite element simulations
Fuel pellet simulationsMacroscopic: Finite elements
Micro/Mesoscopic: Phase-field, Kinetic Monte Carlo, Discrete dislocation simulations, Discrete lattice models, Molecular dynamics
First-principles based simulations: Density functional theory and others
3.6
to 3
.8m
May 17, 2007
Modeling of Fuel RodModeling of Fuel Rod
Phani K. Nukala Oak Ridge National Laboratory
Driver
Heat generation and transfer
Neutronics
Thermo-mechanics/hydraulics
Radiation damage
Chemistry
Fission products
Code Integration Code Integration -- MultiMulti--physics Modelphysics Model
Base capabilityDepending on the level of coupling one of the modules can become Driver, e.g. FEM Thermo-mechanics module can host chemistry data
Phani K. Nukala Oak Ridge National Laboratory
ThermoThermo--Mechanical CodesMechanical Codes
General-purpose system for large-scale analysisThermal, and mechanics
Employs a hierarchical domain decomposition method (HDDM) to efficiently utilize massively parallel computer resources
Partition into non-overlapping sub-domainsAnalyze sub-domains (fine problem)Enforce compatibility between sub-domains
(coarse problem)
Phani K. Nukala Oak Ridge National Laboratory
Domain Decomposition
Ω1 Ω2 Ω3 Ω4 DomainProcessing
Local ProblemΩ1 Ω2 Ω3 Ω4
Coarse ProblemΩ1 Ω2 Ω3 Ω4
Interface ProblemΩ1 Ω2 Ω3 Ω4
FETI-DP
Global Problem
Global Problem
Local Problem
FETIFETI--DP/HDDMDP/HDDM
Phani K. Nukala Oak Ridge National Laboratory
Scaling on CrayScaling on Cray--XT4XT4
Scaling up to 6000 cores
20% peak efficiency
Solved up to 300M problem
Number of coresNumber of cores10001000 60006000
May 17, 2007
Modeling of Fuel PelletsModeling of Fuel Pellets
Phani K. Nukala Oak Ridge National Laboratory
Modeling Brittle Fracture of Fuel PelletsModeling Brittle Fracture of Fuel Pellets
Fracture due to thermal mismatch and high thermal gradients
Chemistry/Transport/Diffusion of fission products
Misorientation dependent low angle grain boundary fracture strength
Time dependent evolution of grain structures and recrystallization
Phani K. Nukala Oak Ridge National Laboratory
HighHigh--performance computingperformance computing
High-performance computing Processing time
L = 64 on 128 3 hours
L = 100 on 1024 12 hours
L = 128 on 1024 3 daysL = 200 on 2048 20 days (est.)
16,384
8,192
4,096
2,048
1,024
512
256
128
64 8 16 32 64 128 256 512 1,024Number of processors
Wall
-clo
ck ti
me (
seco
nds)
per 1
,000 b
onds
bro
ken
100^3benchmark
IBM BG/L
Cray XT3
Phani K. Nukala Oak Ridge National Laboratory
Simulation of Nuclear Fuel RestructuringSimulation of Nuclear Fuel Restructuring
Simulation of fuel swelling due to fission products accumulation in fuel matrix
Courtesy T.J. Courtesy T.J. BartelBartel, SNL, SNL
May 17, 2007
ExascaleExascale RequirementsRequirements
Phani K. Nukala Oak Ridge National Laboratory
Fuel Rod/Bundle SimulationFuel Rod/Bundle Simulation
Solving a 300M problem took 300 seconds on 6000 cores
Simulation involves 50 steps with 10 iterations per step
Total time ~ 500 x 300 = 150,000 sec ~ 42 hours
Assuming a constant efficiency of 20%, a 100,000 core simulation requires approximately
(6000/100000) x (42 / 0.2) ~ 12 hours on 500 TF machine
A bundle of 40 rods requires 12 hours on 20 PF machine
All this only with thermo-mechanics and no coupling with other multi-physics models
Phani K. Nukala Oak Ridge National Laboratory
Fuel Pellet SimulationFuel Pellet Simulation
Solving a 1B problem took 1000 seconds on 8000 cores
Simulation involves 50 steps with 10 iterations per step
Total time ~ 500 x 1000 = 500,000 sec ~ 6 days
Assuming a constant efficiency of 20%, a 200,000 core simulation requires approximately
(8000/200000) x (6 / 0.2) ~ 1.2 days on 1 PF machine
All this only with thermo-mechanics and no coupling with other multi-physics models; No coupling of multiple scales
Phani K. Nukala Oak Ridge National Laboratory
Additional Details Additional Details
Assumed perfect scaling with 20% efficiency!
Addition of multi-physics models: chemistry, fission products transport, and neutronics will deteriorate the efficiency
Necessity to run various loading/boundary conditions, and sensitivity studies will require capacity computing with each simulation running at 1 PF.
Phani K. Nukala Oak Ridge National Laboratory
Final Thoughts Final Thoughts ……
Fuel and cladding behavior is governed by Coupled multi-physics phenomenaFeedback from multiple scales
High-fidelity simulations aiming to resolve these multiple scales and couple multi-physics models demand capability computing
Development of physics-based models requires higher resolution and larger system sizes
Coupling of time scales in these multi-physics models requires algorithmic advances
Faster turn around times, multiple loading/boundary conditions scenarios, and sensitivity studies require capacity computing with each simulation running at a PF.
May 17, 2007
Algorithmic Challenges: Coupling Algorithmic Challenges: Coupling Multiple Time Scales in MD SimulationsMultiple Time Scales in MD Simulations
Phani K. Nukala Oak Ridge National Laboratory
MTS Propagator
( ) Slowh/2
NFasth/N
Slowh/2
MTSh ΦΦΦ=Φ ∗
ReversibleAt least, second-order accurate
Step 1: ½ kick
Step 2: vibration N steps
Step 3: ½ kick
( ) ( )nSlow1 yy Φ=
( )(1)Fast
(2) yy Φ=
( )(2)Slow1n yy ∗+ Φ=
MTS MethodMTS Method
Phani K. Nukala Oak Ridge National Laboratory
Stabilized MTS Method AnalysisStabilized MTS Method Analysis
( ) 22
21
22
21 ωωpqqωωq >>=+−=
MTS exhibits parametric resonance, whereas SMTS is stable for any time step
0 1 2 3 4 5 60.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
k δt/Tfast
Eig
enva
lue
mag
nitu
des
λ1
λ2 MTSMTS
0 1 2 3 4 5 60.9999
1
1.0001
k δt/Tfast
Eig
enva
lue
mag
nitu
des
λ1
λ2
SMTSSMTS
Phani K. Nukala Oak Ridge National Laboratory
Stabilized MTS Method AnalysisStabilized MTS Method Analysis
( ) 22
21
22
21 ωωpqqωωq >>=+−=
Collision of eigenvalues leading to instability of MTS is clearly visible
−1 −0.5 0 0.5 1−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Real
Imag
λ1
λ2
−1.5 −1 −0.5 0 0.5 1 1.5 2−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Real
Imag
λ1
λ2
SMTSSMTSMTSMTS
Phani K. Nukala Oak Ridge National Laboratory
FermiFermi--PastaPasta--UlamUlam (FPU) Problem(FPU) Problem
Different behavior over different time scalesTime scale ω-1: almost-harmonic motion of stiff springsTime scale ω0: motion of soft nonlinear springsTime scale ω: energy exchange among the stiff springsTime scale ωN, N>1: O(ω-1) deviations in oscillatory energy
( ) ( ) ( ) ( )∑∑∑=
+=
−=
− −+−++=n
0i
42i12i
n
1i
212i2i
2n
1i
22i
212i qqqq
4ωpp
21qp,H
Phani K. Nukala Oak Ridge National Laboratory
FermiFermi--PastaPasta--UlamUlam (FPU) Problem(FPU) Problem
( )2jn
22jnj xωy
21I ++ +=
Oscillatory energyOscillatory energy( )
( )2qqx
2ppy
12i2ijn
12i2ijn
−+
−+
−=
−=
68 69 70 71 72 73 740.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
0.6
Time
I1
I2
I3
0 50 100 150 200 2500
0.2
0.4
0.6
0.8
1
1.2
1.4
Time
I1
I2
I3
I
I1
I2
I3
h = 0.00025h = 0.00025 ωω = 50= 50
Phani K. Nukala Oak Ridge National Laboratory
FPU ProblemFPU Problem
h = 0.03h = 0.03
0 1 2 3 4 5 60
0.05
0.1
0.15
0.2
0.25
h ω/π
(Δ H
) max
SMTSSMTS
0 1 2 3 4 5 6 70
0.05
0.1
0.15
0.2
0.25
h ω/π
(Δ I)
max
Parametric resonance?No parametric resonance effects in the HamiltonianResonance in the oscillatory energy
Max error in the Max error in the HamiltonianHamiltonian
Max error in the Max error in the oscillatory energyoscillatory energy
Phani K. Nukala Oak Ridge National Laboratory
Final Thoughts Final Thoughts ……
Fuel and cladding behavior is governed by Coupled multi-physics phenomenaFeedback from multiple scales
High-fidelity simulations aiming to resolve these multiple scales and couple multi-physics models demand capability computing
Development of physics-based models requires higher resolution and larger system sizes
Coupling of time scales in these multi-physics models requires algorithmic advances
Faster turn around times, multiple loading/boundary conditions scenarios, and sensitivity studies require capacity computing with each simulation running at a PF.
