Embed Size (px)
Citation preview
Xiaoxia Li
Group of HPC & Cheminformatics
Institute of Process Engineering
Chinese Academy of Sciences, Beijing
GTC 2016 San Jose, Californiae, 7 April, 2016
Outline
2
Reaction mechanisms of coal pyrolysis?1
GPU-enabled ReaxFF MD (GMD-Reax)2
Pyrolysis of coal, biomass, polymer 3
4 Concluding remarks and perspective
China is the largest producer& consumer of coal
China has much more coal, less oil
Background
3
Mechanism still hardly accessible Experimentally, hard to detect and replicate the free radical initiation at high temperature in lab Computationally with QM, extremely high computing cost, limited model scale: ~100 atoms
ReaxFF MD
(Reactive molecular dynamics)
Reaction mechanism ?
Overview of ReaxFF
4
ReaxFF MD: reactive force field + molecular dynamics
by van Duin (Penn state), Goddard (Caltech) et al.
for bond breaking and forming with parameters based on experiments and QM
(quantum mechanics approach)
Faster than DFT (widely used QM) for models > 1000 atoms
No priori knowledge of reaction pathways required
A comprehensive knowledge on multiple reaction pathways of coal pyrolysis is not available !
ReaxFF MD is promisingfor coal pyrolysis simulation
Publications on ReaxFF MD Subject searching hits from Web of Science
Can large coal model simulated efficiently with ReaxFF?
HPC Programs of ReaxFF - supercomputer/cluster F-ReaxFF, Univ. South. California, 2007 (parallel )
PuReMD, Purdue Univ., 2011 (single node performance )
In LAMMPS, Sandia National Lab. (open source)
FORTRAN code (precise, based on van Duin’s original code)
C code (2011, faster , based on PuReMD)
In commercial software
ADF (to enhance visualization, ~2011)
GULP, Materials Studio 6.0 (2012)
Is it practical to simulate large coal model (~10,000 atoms) on desktop workstation?
Desktop workstationis more preferable
5
ReaxFF MD on Desktop workstation?
6
Computational challenges – complexity of coal structure and pyrolysis ~10,000 atoms, state-of-the-art coal model scale
~1,000 atoms, practical scale for LAMMPS (Sandia National Lab) and ADF (Europe, a major player
of QM software) on single computational node
10 - 50 folds
slower than
classical MD
ReaxFF vs LJ potential
LAMMPS Benchmarks
2012: http://lammps.sandia.gov/
bench.html#potentials)
FORTRAN code
C code
7
MD ReaxFFMD
Dynamic atom charge
equilibration
Bond order dependency
Time-step 1 fs
Fixed atom charge
Overview of ReaxFF MD
Time-step0.1 fs
Computational cost of ReaxFF MD vs MD
8
ReaxFF MD vs MD
Similar computing loops, but
Time-step: 0.1 fs (ReaxFF MD) vs 1 fs (MD)
Atom charge: optimizing at each time-step (ReaxFF MD) vs fixed (MD)
Additional computing introduced in potential & its corrections
Taper + Morse for van der
Waals in ReaxFF
9
Thanks for the GPU & CUDA
Rapid development GPU computing since 2007
MD codes (major players and novel codes such as HOOMD)Stone, J.E., et al., GPU-accelerated molecular modeling coming of age. Journal of Molecular
Graphics and Modelling, 2010. 29(2): p. 116-125.
GPU infrastructure in IPE (in my office building)
Potential seen from GMD we created in 2009 - 2010 (a GPU
enabled code for MD)
Polyethylene crystalization
Mole-8.5 (GPU enabled) 1 Peta, Double
Top 500 Supercomputer
19th, 2010
33th, 2011
37th, 2012
55th, 2013
ReaxFF MD on Desktop workstation? GPU
10
GMD: a GPU enabled code for classical MD
Our first attempt using GPU
Performance is comparable with early version of GROMACS GPU
Application in polymer chain crystallization (Polyethylene as model)
PE models: 360,000 united atoms & 400,000 united atoms
Simiao Wang, et al. Two mechanisms of polymer chain crystallization within nanoglobule.
Polymer. 2013;54(15):4030-4036
GMD and its applications in polymer crystallization study
Students in GPU HPC companies (NVIDIA, Sugon) and more10 folds larger model scale than
that simulated in CPU cluster
GMD-Reax: ReaxFF MD on GPU
11
GPU works for MD the first GPU code for ReaxFF MD (C2050)
Its implementation – tough job
Constrained coding closely linked with GPU hardware
faster memory limited, global memory access latency, and more
GMD-Reax: ReaxFF MD on GPU
12
Our approach Most of computations on GPU
Faster SFU for some bond order based corrections (early version)
T thread for charge evaluation/time-step – bottle neck
Finely tuned data access for computation, and more
13
GMD-Reax: performances GMD-Reax on one C2050 achieved up to 16 times speedup against the
LAMMPS’ codes on 8 CPUs (~fastest on CPU, Sandia National Lab &
Purdue Univ)
Zheng, M.; Li, X.; Guo, L., Algorithms of GPU-enabled reactive force field (ReaxFF) molecular dynamics. Journal of Molecular Graphics and Modelling 2013, 41, (April), 1-11
Single precision
14
GMD-Reax: performances GMD-Reax on one C2050 achieved up to 8 times speedup against the
LAMMPS’ codes on 8 CPUs (~fastest on CPU, Sandia National Lab &
Purdue Univ)
Double precision
15
GMD-Reax: performance & impact
GMD-Reax(Ours, DP)
PuReMD-GPUs(Purdue Univ.)
Notes
Systems BenchmarkedAmorphous coal pyrolysis
systems(4976 – 27 283 atoms)
Bulk water systems(6540 – 50 097 atoms)
Amorphous silica(6000 – 48 000 atoms)
Coal models are more complex than bulk water or silica systems, of which all energy terms must be computed in potential evaluation of ReaxFF MD
Tesla C2075 has more global memory than Tesla C2050
Hardware of GPU Tesla C2050 Tesla C2075Speedups against
PuReMD in LAMMPS(1 CPU core)
4.5 – 14.0(complex coal models)
7.1 – 16.6 (water)5.8 – 11.4 (silica)
Speedups against PuReMD in LAMMPS
(8 CPU cores)
1.5 – 4.0(complex coal models)
2.0 – 2.9 (water)
1.5 – 2.1 (silica)
The only two GPU codes available have comparable
performance, ours even better
Ours published ~ 1.5 year earlier
Ours：Journal of Molecular Graphics and Modelling
2013, 41, (April), 1-11
Top 5, NVIDIA GPU Award, 248th ACS meeting, 2014
PuReMD-GPUs： Journal of Computational Physics2014, 272(Sept), 343-359
ReaxFF MD of coal pyrolysis
16
Challenges – complexity of coal structure and pyrolysis
Coal model construction?
Computing scale discrepancy?
Lack of reaction analysis ability for revealing mechanism
LAMMPS, ADF analysis tool (?) number of molecules (formula based) ~ time
Manual analysis is a must?
Manual analysis is not practical for revealing the complex reaction mechanism of coal pyrolysis
n-dodecane (C6H14) pyrolysis:
1279 species, 5056 reactions
17
What we need to do?
Reaction analysis - discovering the bonding and species changes
3D chemical structure processing
Automatic perception of atomic connectivity, bonding type, species, reaction
VARxMD: the first reaction analysis tool for ReaxFF MD
Jian Liu, Xiaoxia Li et al., Journal of Molecular Graphics and Modelling2014, 53(9):13-22
18
VARxMD: the first reaction analysis tool for ReaxFF MD
Allowing for “direct” observation of chemistry events computationally
What we have – detailed reaction list
All reactions
Product evolution & underlying
reactions
2D & 3D Reaction details
19
VARxMD: the first reaction analysis tool for ReaxFF MD
Allowing for “direct” observation of chemistry events computationally
What we have – a view of all reaction sites
20 Reaction site – bond breaking or forming highlighted
VARxMD: the first reaction analysis tool for ReaxFF MD
What we have – a 3D view of a reaction with reaction sites highlighted
New methodology for large scale ReaxFF MD
21
Xiaoxia Li et al., Molecular Simulation,2015, 41(1-3), 13-27
GPU high performance computing We created the first GPU-enabled codes
Cheminformatics approachWe created the first reaction analysis tool
New methodology applications
22
Large scale ReaxFF MD simulations
Coal pyrolysis (~10,000 atoms) Liulin coal model: C14782H12702N140O690S37, 28,351 atoms, second
largest ever simulated
Pyrolysis of polymer (HDPE) (150x8, 7216 atoms)
Pyrolysis of biomass
15,920 atoms for lignin
7572 atoms (C2160H3612O1800)
Pyrolysis and oxidation hydrocarbon fuel
10,828 atoms for bio-oil
Tingting Zhang, Xiaoxia Li, et al. Energy and Fuels 2016, just accepted
Mo Zheng, Ze Wang, Xiaoxia Li, et al. Fuel, 2016. 177: p. 130-141
Xiaolong Liu, Xiaoxia Li, et al. Polymer Degradation and Stability 2014, 104(June), 62-70
Mo Zheng, Xiaoxia Li, et al. Energy and Fuels 2014, 28(1), 522-534
Typical time for one
condition is
one week
(GMD-Reax)
New methodology applications
23
Coal pyrolysis simulations - large scale coal models
ModelsModel scale
(atoms)Chemical formula Simulation time
Bituminous model (proof-of- concept) 4976 C2417H2235N41O240S43
~ 7000 min
(5 days)
Hailaer brown coal model 12,335 C5752H5422N8O1137S16
~ 2400 min
(<2 days)
Hailaer brown coal model 27,809 C12996H12228N18O2561S36
~ 6000 min
(4 days)
Liulin bituminous coal model 13,498 C7068H5968N78O351S33
~ 2800 min
(2 days)
Liulin bituminous coal model 28,351 C14782H12702N140O690S37
~ 6300 min
(4.5 days)
13C-NMR spectra of Liulin coal
Ultimate Analysis (wt % daf)
C 88.4
H 4.8
O 5.2
N 0.94
S 0.46
Proximate analysis ( wt% )
Moisture 0.66
Ash 11.32
Volatile 20.64
Proximate and Ultimate Analysis of Liulin Coal
Fugu subbituminous coal model 23,898 C11995H10362N159O1366S15
~ 6000 min
(4.0 days)
24
Coal pyrolysis simulations – evolution of overall spectrum products
High temperature and short time pyrolysis favor the maximum amount of tar generation
Liulin bituminus coal
New methodology applications
25
Representative products/precursors in coal tar
Advantage of our
VARxMD & large
scale models
(~30,000 atoms)
Naphthalene, methyl-naphthalene and dimethyl-naphthalene are representative products in
Liulin coal pyrolysis observed by Py-GC/MS
Simulated observation within 87.5 ps agree with Py-GC/MS
Liulin bituminous coal pyrolysis
Py-GC/MS, up to 20,000 K/sheating rate
26
No Reactions involving H3C• Reactions involving HO•
1 C312H259O16N3SC281H228O16N2S +C30H28N + CH3 C312H259O16N3SC307H253O15N3S + HO + C5H5
2 C312H259O16N3S C275H220O12N2S +2HO+CHO2 +
C30H28N+C5H5+CH3
C312H259O16N3S C281H229O14N2S+ CHO + C30H28N +
HO
3 C312H259O16N3SC291H236O12N3S +CHO+HO + CHO2 +
C18H17 + CH3
C275H219O14N2+HSC275H215O11N2S+2H2O+ HO
4 C312H259O16N3SC310H254O13N3S +HO + CHO2+ CH3 C65H51O3NC65H50O2N + HO
5 CH3 + C22H22OCH4+ C22H21O C28H28O+ HOC28H27O+H2O
6 C281H226O14N2S+2HO+CH3C282H230O16N2 + HS C272H215O11N2S+HO+C24H22ON+ C13H8 C13H7+
C47H37O2N+C225H179O10NS+ H2 + C24H21ON
7 CH3+ C24H23ON CH4+ C24H21N+ HO HO+ C16H16 H2O+C16H15
8 CH3+ C276H225O14N2S+C30H28NCH4+ C306H251O13N3S+
HO
HO+ C193H154O12NS+CH3H2O+ C35H30O2N +
C159H125O10+HS
H3C• and HO•consumption
Coal pyrolysis reaction mechanisms - by the unique VARxMD
Complex radical reactions newly revealed
Coal pyrolysis is initialized by thermal decomposition at bridged bonds of coal structure to produce unstable
radicals such as HO• and H3C•
H3C• and HO•generation
New methodology applications
27
Coal pyrolysis simulations – correlation of radicals and products
at low T, H3C fluctuating – few CH4 generated at high T, H3C decreasing - increased production of CH4
Earlier maximum and then decreasing of HO – increasing of H2O with T
New methodology applications
28
HDPE pyrolysis simulations Reproduce comprehensive reaction mechanism & weight loss – time prediction
New methodology applications
ReaxFF MD simulation Py-GC/MS experiment150x8, 7216 atoms
29
Cellulose pyrolysis simulations Product evolutions & major reaction pathways
New methodology applications
6*60 1,4-β-D-glucopyranoses7572/17664 atoms
30
Lignin pyrolysis simulations Three pyrolysis stages & reaction mechanism
New methodology applications
40xC160H180O58 15 920 atoms
Summary & perspective
31
New methodology for ReaxFF MD: GPU computing + cheminformatics
GMD-Reax – first GPU code of ReaxFF MD, much faster
VARxMD – a novel tool, unravel of complex detailed reactions
Large scale pyrolysis simulation of polymer, biomass & coal
reaction mechanisms revealed hardly accessible experimentally or by QM, or by small
scale simulations
Methodology application perspective
GMD-Reax can be used in other ReaxFF MD applications for combustion, catalysis etc.
VARxMD can be applied too
Approaching to more real process of pyrolysis and combustion
Working with models of 100,000 atoms on one single workstation with GPUs
Acknowledgment
Grants of NSFC (21373227, 91434105), MPCS-2012-A-05
Hard work of
Xiaofang Tao
Xiaolong Liu
Prof. Fengguang Nie
Dr. Xianjie Qiao
Junyi Han
Song Han
Tingting Zhang
Mingjie Gao
Chunxing Reng
Zimin Wang
Dr. Zhaojie XiaWucheng Tang
Dr. Mo Zheng
Jian Liu Xiaomin Gong
Dr. Ze WangProf Li Guo Prof Wenli Song
