Upload
annabella-hargrove
View
224
Download
0
Tags:
Embed Size (px)
Citation preview
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
A community modeling environment: A community modeling environment: geodynamic integration of multi-scale geodynamic integration of multi-scale
geoscience datageoscience data
Mian LiuMian Liu11, Huai Zhang, Huai Zhang1,21,2, ,
Youqing YangYouqing Yang11, Qingsong Li, Qingsong Li11, Yaolin Shi, Yaolin Shi22
1-University of Missouri-Columbia
2-Computational Geodynamic Lab, CAS, China
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Motivation 1:
Exponentially increase of multi-scale observational data that need to be integrated and interpreted within a self-consistent geodynamic framework
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
EarthScope Annual Data Volume
Data volumes over next 10 years
GPS: ~7.7 TB
BSM/LSM: ~10.5 TB
Seismic: ~120 TB
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Motivation 2: • Advance of computer hardware
(especially PC clusters and grid computers) and software engineering have provided unprecedented computing power;
• Data infrastructure have made integrating multiscale data both easy and necessary.
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
So we built the data So we built the data cyberinfrastructures, now cyberinfrastructures, now
what? what?
Internet
Data Grid
Physical model
HPCC
GEON
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Internet
Data Grid
Physical model
HPCC
Free scientists from coding Free scientists from coding to do science, or whatever to do science, or whatever
they do bestthey do best
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
• Earth Simulator-GeoFEM project • Geoframwork• QuakeSim• SCEC Community Modeling Environment • CIG (Computational Infrastructure for
Geodynamics )
Some current efforts on geodynamic Some current efforts on geodynamic computationscomputations
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
More than one way to do it …
• Develop specific type of models (e.g., mantle convection);
• Use plug-in modules in a general system to generate specific type of models (wave, fluid, structure, etc.)
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Multi-Purpose/Multi-physics Parallel FE Simulator/Platform for Solid Earth
Different finite element model can be plugged into this system
http://geofem.tokyo.rist.or.jp
Example: Earth Simulator- GeoFEM project
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Won’t it be nice if we can have a general, flexible
community modeling system? • Not all geological needs can fit into the
pigeonholes;
• Need integration with data CI;
• Scalable for parallel and grid computation
Wouldn’t it be nice if all (or most of) these can be automated?
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Examples of commercial FE code generation systems
• PED2D (http://members.aol.com/pde2d)
• FEPG (Finite Element Program Generator) (http://www.fegensoft.com/english/index.htm)
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
The devil is in the details …The devil is in the details …
PDE2FEMsystem
FEM meshes generator
Geometric modeling of real application
Graph partition (Metis4.0, Pmetis3.0) and Data partition
GES,PDE,CDE, SDE etc. element subroutine generators
GCN, NFE etc. nonlinear algorithms generators
Libs for PDEs, shape functions and other software packages
Theoretical and application documents for users
Server-Client communication system
PFEPG Server administration tools
Common Gateway Interface via www browsers for users
User level communication protocol
MPI
Data structure and optimization
Dynamic load-balancing of each node in parallel computer
Sequential solvers
Krylov subspace iterative solvers and preconditioners
Blas,blacs,pblas,splib,fepglib,
AZsolv, MUMPSArpack, SuperLU
LMDDM and LMDDA algorithm kernel subroutines
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
disp u vcoor x yfunc funa funb func shap %1 %2gaus %3mass %1load = fu fv $c6 pe = prmt(1)$c6 pv = prmt(2)$c6 fu = prmt(3)$c6 fv = prmt(4)$c6 fact = pe/(1.+pv)/(1.-2.*pv)funcfuna=+[u/x]funb=+[v/y]func=+[u/y]+[v/x]stifdist =+[funa;funa]*fact*(1.-pv)+[funa;funb]*fact*(pv)+[funb;funa]*fact*(pv)+[funb;funb]*fact*(1.-pv)+[func;func]*fact*(0.5-pv)
*es,em,ef,Estifn,Estifv,
*es(k,k),em(k),ef(k),Estifn(k,k),Estifv(kk),
goto (1,2), ityp1 call seuq4g2(r,coef,prmt,es,em,ec,ef,ne) goto 32 call seugl2g2(r,coef,prmt,es,em,ec,ef,ne) goto 33 continue
DO J=1,NMATEPRMT(J) = EMATE((IMATE-1)*NMATE+J)End doPRMT(NMATE+1)=TIMEPRMT(NMATE+2)=DTprmt(nmate+3)=imateprmt(nmate+4)=num
Other element matrix computing SubsPDE expressionContains information of the physical model, such as variables and equations for generating element stiffness matrix.
Fortran Segmentscodes that realize the physical model at element level.
variables
equation
Automated Code Generator
Step 1: From PDE expression to Fortran segments
Segment 1
Segment 2
Segment 3
Segment 4
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Step 2: From algorithm expression to Fortran segments
do i=1,k do j=1,k estifn(i,j)=0.0 end do end do do i=1,k estifn(i,i)=estifn(i,i) do j=1,k estifn(i,j)=estifn(i,j)+es(i,j) end do end do
U(IDGF,NODI)=U(IDGF,NODI) *+ef(i)
defistif Smass Mload Ftype emdty lstep 0
equationmatrix = [S]FORC=[F]
SOLUTION Uwrite(s,unod) U
end
Algorithm ExpressionContains information for forming global
stiffness matrix for the model.
Fortran Segmentscodes that realize the physical model at global level.
Stiffness matrix Segment 5
Segment 6
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
SUBROUTINE ETSUB(KNODE,KDGOF,IT,KCOOR,KELEM,K,KK, *NUMEL,ITYP,NCOOR,NUM,TIME,DT,NODVAR,COOR,NODE,#SUBET.sub *U) implicit double precision (a-h,o-z) DIMENSION NODVAR(KDGOF,KNODE),COOR(KCOOR,KNODE), *U(KDGOF,KNODE),EMATE(300),#SUBDIM.sub *R(500),PRMT(500),COEF(500),LM(500)#SUBFORT.sub#ELEM.subC WRITE(*,*) 'ES EM EF ='C WRITE(*,18) (EF(I),I=1,K)#MATRIX.sub L=0 M=0 I=0 DO 700 INOD=1,NNE ……… U(IDGF,NODI)=U(IDGF,NODI)#LVL.sub DO 500 JNOD=1,NNE ………500 CONTINUE700 CONTINUE ……… return end
Program StencilFortran Segments generated
Step 3: Plug Fortran segments into a stencil, forming final FE program
Segment 1
Segment 2
Segment 4
Segment 3
Segment 5
Segment 6
…………..
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Examples
• Western US tectonics
• Deformation of Asian continent
• Stress evolution and strain localization in the San Andreas Fault
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
A Preliminary Finite Element Model of Active A Preliminary Finite Element Model of Active Crustal Deformation in the Western USCrustal Deformation in the Western US
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
The Power of GEON Cluster NodeThe Power of GEON Cluster NodeOriginal series model (single CPU)
•Less than 3000 elements
•Three layers in R-direction
•2 min for per time step
Preliminary parallel model (16-nodes, 32 CPUs)
•More than 800,000 unstructured elements
•Major Faults and more deformation zones
•Subduction of Juan de Fuca slab
•21 layers in R-direction
(x 40vertical topographic exaggeration)
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Automatic domain decomposition Automatic domain decomposition for parallel computingfor parallel computing
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
The model now allows simulation of large scale The model now allows simulation of large scale continental deformation with unprecedented detailcontinental deformation with unprecedented detail
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
The model now allows simulation of large scale The model now allows simulation of large scale continental deformation with unprecedented detailcontinental deformation with unprecedented detail
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Ongoing Effort:Ongoing Effort: Toward a new 3D model of Toward a new 3D model of continental deformation in Asiacontinental deformation in Asia
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Loading the San Andreas Fault by relative PA-NA motion
•Fully 3D•Dynamic•Plastic-viscoelastic•Co-seismic/interseismic cyclesfrom seconds to 104 years•Parallel computing on PC clusters
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Comparison of predicted surface velocity and GPS data
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Predicted rate of plastic strain energy release outside the SAF
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Dream on…
• Integrating the community modeling environment with the geoscience data cyberinfrastructure;
• Grid computation and data integration;
• Automated (optimized?) work flow management (the Kepler system?)
www.geongrid.orgCYBERINFRASTRUCTURE FOR THE GEOSCIENCES
0
0
0
zzzyzxz
yyzyyxy
xxzxyxx
fzyx
fzyx
fzyx
Automatic sourcecode generator
funcfuna=+[u/x] ………funf=+[u/y]+[v/x]
………dist =+[funa;funa]*d(1,1)+[funa;funb]*d(1,2)+[funa;func]*d(1,3)+[funb;funa]*d(2,1)+[funb;funb]*d(2,2)+[funb;func]*d(2,3)+[func;funa]*d(3,1)+[func;funb]*d(3,2)+[func;func]*d(3,3)+[fund;fund]*d(4,4)+[fune;fune]*d(5,5)+[funf;funf]*d(6,6) load = +[u]*fu+[v]*fv+[w]*fw-[funa]*f(1)-[funb]*f(2)-[func]*f(3)-[fund]*f(4)-[fune]*f(5)-[funf]*f(6)
PDEs
Complete source code
FEM Modeling Language
Data Grid (GEON and others)
Physical model
Model results
HPCC
Data =>???
SWF
SWF