Embed Size (px)
DESCRIPTION
Enabling Science with Grid Technology. Jamie Rintelman , Kerstin Kleese-Van Dam, Rik Tyer STFC-Daresbury Laboratory; Daresbury, Cheshire, UK. Who am I?. STFC Daresbury laboratory eScience Dept - Computational Science and Engineering Dept Liaison Chemist - PowerPoint PPT Presentation
Citation preview
Daresbury Laboratory
Enabling Science with Grid Technology
Jamie Rintelman, Kerstin Kleese-Van Dam, Rik Tyer
STFC-Daresbury Laboratory; Daresbury, Cheshire, UK
Daresbury Laboratory
Who am I?
STFC Daresbury laboratory – eScience Dept - Computational Science
and Engineering Dept Liaison
Chemist– Specializing in Quantum Chemistry,
electronic structure theory, GAMESS
Daresbury Laboratory
•Traditional Way of Working•eMinerals Grid Computing
Framework – Background on eMinerals Program– Grid Computing Framework
• Input preparation• Monty - bulk job submission• RMCS - integrated compute/data/metadata
framework• Rgem - analysis of results
•Scientific Examples– QDGA– BTG– eMinerals
Daresbury Laboratory
Traditional Way of Working I
Files on local computer
Check for available remote resource
Comp01
Comp02
Comp04
Comp03
Daresbury Laboratory
Traditional Way of Working I
Files on local computer
Check for available remote resource
Comp01
Comp02
Comp04
Comp03
Daresbury Laboratory
Traditional Way of Working I
Files on local computer
SCP files over
Comp01
Comp03
Daresbury Laboratory
Traditional Way of Working I
Run job
Comp01
Comp03
Output files on remote resource
Daresbury Laboratory
Traditional Way of Working I
Check on progress (many times?)
Comp01
Comp03
Output files on remote resource
Daresbury Laboratory
Traditional Way of Working I
SCP files back
Comp01
Comp03
Output files on remote resource
Daresbury Laboratory
Traditional Way of Working I
Files on local computer
Comp01
Comp03
Daresbury Laboratory
Traditional Way of Working II
Collaborator asks for files
Email collaborator with files or with location of files
Find filesPlace them on ftp server, or put in
Daresbury Laboratory
eMinerals / RMCS framework
A new way of working
Daresbury Laboratory
eMinerals Project
•NERC funded•Collaborators throughout the UK•Pragmatic approach to development = Science Driven
Daresbury Laboratory
The eMinerals team
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
Daresbury Laboratory
eMinerals Project -
ScopeSet of tools to facilitate scientific work
– Building and configuring grids – Job submission tools – Data management – Metadata management – Data processing / Information extraction
– Simulation output visualization
Daresbury Laboratory
eMinerals Project -
ScopeSet of tools to facilitate scientific work
– Building and configuring grids – Job submission tools – Data management – Metadata management – Data processing / Information extraction
– Simulation output visualization
Daresbury Laboratory
RMCS Framework
Components:– Input Preparation– Bulk Job Submission– Running Jobs– Analysis of Results
Additional Building Blocks:– Storage Resource Broker (SRB) - Data Storage and Collaborative Sharing
– AgentX - XML data, sharing between programs, metadata capture (developed by Phil Couch, STFC)
Daresbury Laboratory
Input preparation
Bespoke scripts to automate generation of input files for parameter sweep type calculations
Template
inputSimulation
SimulationSimulation
Simulation
Daresbury Laboratory
Monty - Bulk Job
Submission-set up structure in SRB for staging of input files and binary; storage of output files-set up structure in database for metadata capture-submit jobs to RMCS
SRB
MetadataDatabase
RMCS
Monty
Daresbury Laboratory
Metadata Database
Study
Data Set
Data Objects (i.e. “parameters”)
Data Set
Metadata in database are divided into study, data set, and data objectStudy = entire projData Set = group of calculationsData Objects = piece of data from each calculation
Daresbury Laboratory
Daresbury Laboratory
RMCS - integrated compute/ data/ metadata framework
3 tier modelClient tools
GUIs (SOAP library), Command Line Tools (gSOAP)
RMCS Server
The Grid
Daresbury Laboratory
“The Grid” so far
eMineralsMiniGrid
Northwest Grid
National Grid Service
The Grid
Cambridge Condor Pool
Scarf Cluster (coming soon)
Daresbury Laboratory
RMCS - integrated compute/data/metadata
framework
1. Meta-schedule2. Stage input files and binary3. Run job/submit to batch queue4. Transfer output to SRB5. Use Rcommands + AgentX to put
metadata into database /extract XML data if available
Daresbury Laboratory
Rgem - Collect Results
-Analyze Results-Collect parameters from a chosen dataset -> tab separated file -> graph
Data Set
Data Objects
Daresbury Laboratory
Scientific Examples
•Quantum Directed Genetic Algorithm•Transition metal oxides, Perovskites•eMinerals
Daresbury Laboratory
Quantum Directed Genetic Algorithm
(QDGA)Marcus Durant (Univ of Northumbria), Jens Thomas
(STFC-Daresbury)
The QDGA project uses a genetic algorithm to try and determine an optimal catalyst for the conversion of nitrogen (N2) to hydrazine (N2H4)
Daresbury Laboratory
Template
Input
GAMESS-UK
Daresbury Laboratory
Template
Input
GAMESS-UK
BespokeScripts
DFT Transition
State Search
DFT Transition
State Search
DFT Transition
State Search
DFT Transition
State Search
Daresbury Laboratory
Template
Input
GAMESS-UK
Monty
SRB
Input files + binary
Output files
Create directory structure
BespokeScripts
DFT Transition
State Search
DFT Transition
State Search
DFT Transition
State Search
DFT Transition
State Search
Daresbury Laboratory
Template
Input
GAMESS-UK
Monty
SRB
Input files + binary
Output files
Create metadata containers in database
BespokeScripts
DFT Transition
State Search
DFT Transition
State Search
DFT Transition
State Search
DFT Transition
State Search
MetadataDatabase
Daresbury Laboratory
Template
Input
GAMESS-UK
MontyRMCS
Submit jobs via RMCS
SRB
Input files + binary
Output files
BespokeScripts
DFT Transition
State Search
DFT Transition
State Search
DFT Transition
State Search
DFT Transition
State Search
MetadataDatabase
Daresbury Laboratory
Template
Input
GAMESS-UK
DFT Transiti
on State
Search
DFT Transiti
on State
Search
DFT Transiti
on State
Search
DFT Transiti
on State
Search
RMCS
SRB
Input files
GAMES-UK binary
MetadataDatabaseNWGrid
(Daresbury, Manchester, Liverpool, Lancaster)
Stage input files + binary
Daresbury Laboratory
Template
Input
GAMESS-UK
DFT Transiti
on State
Search
DFT Transiti
on State
Search
DFT Transiti
on State
Search
DFT Transiti
on State
Search
RMCS
SRB
Input files
GAMES-UK binary NWGrid
(Daresbury, Manchester, Liverpool, Lancaster)
Metaschedule
Submit to batch queues
MetadataDatabase
Daresbury Laboratory
Template
Input
GAMESS-UK
DFT Transiti
on State
Search
DFT Transiti
on State
Search
DFT Transiti
on State
Search
DFT Transiti
on State
Search
RMCS
NWGrid(Daresbury, Manchester, Liverpool, Lancaster)
Transfer output to SRB
SRB
Input files
GAMES-UK binary
Output files
MetadataDatabase
Daresbury Laboratory
Template
Input
GAMESS-UK
DFT Transiti
on State
Search
DFT Transiti
on State
Search
DFT Transiti
on State
Search
DFT Transiti
on State
Search
RMCS
NWGrid(Daresbury, Manchester, Liverpool, Lancaster)
SRB
Input files
GAMES-UK binary
Output files
MetadataDatabase
Using AgentX and Rcommands, place metadata in database
Daresbury Laboratory
Template
Input
GAMESS-UK
DFT Transiti
on State
Search
DFT Transiti
on State
Search
DFT Transiti
on State
Search
DFT Transiti
on State
Search
NWGrid(Daresbury, Manchester, Liverpool, Lancaster)
SRB
Input files
GAMES-UK binary
Output files
Rgem
Collect and plot Total Energy from each
optimized geometry
MetadataDatabase
Daresbury Laboratory
Template
Input
GAMESS-UK
DFT Transiti
on State
Search
DFT Transiti
on State
Search
DFT Transiti
on State
Search
DFT Transiti
on State
Search
NWGrid(Daresbury, Manchester, Liverpool, Lancaster)
SRB
Input files
GAMES-UK binary
Output files
Rgem
Collect and plot Total Energy from each
optimized geometry
RMCSMonty
BespokeScripts
MetadataDatabase
Daresbury Laboratory
Transition metal oxides; Perovskites
(e.g. LaMnO3)Band Theory Group; W. Temmerman, M. Lueders, L. Petit, R.
Tyer
XMLdata
SimulationSimulation
SimulationSimulation
XMLdataXML
dataXMLdata
AgentX Database
RGem
RC
omm
ands
Use of RMCS Framework with XML output allows each of these steps to be linked together seamlessly
Daresbury Laboratory
“Grand Challenge” science and the eMinerals VOLevel of theory
Adsorbing surface
Contaminant
Quantum Monte Carlo
Large empirical models
Linear-scaling quantum mechanics
Organic molecules
Halogens
Metallic elements
Cla
ys,
mic
as
Alu
min
osi
lica
tes
Na
tura
l org
an
ic m
att
er
Ph
osp
ha
tes
Ca
rbo
na
tes
Oxi
de
s/h
ydro
xid
es
Su
lph
ide
s
Daresbury Laboratory
Some Recent eMinerals Projects
•Calculation of compressibility of diopside (CaMgSi2O6) between 0 and 22 Gpa - Andrew Walker•Equation of State of Silica Glass - Andrew Walker•Adsorption of Polychlorinated Dibenzo-p-Dioxins (PCDDs) onto Mineral Surfaces - Kat Austen
Daresbury Laboratory
GCOMS: (Proudman Oceanographic
Laboratory) Global Coastal Ocean Modelling
System
37% of the Earth’s population live within 100km of the coast
Shelf seas are 7% of ocean area but account for up to 30% of production
Shelf seas modify and transport terrestrial inputs: freshwater, nutrients, pollutants
Strong role in dense water formation, mixing on slopes etc.
Shelf seas are omitted or poorly resolved in global ocean models ………….... but they are a disproportionately important part of the earth system
Daresbury Laboratory
acknowledgements
Cambridge eMinerals Group - Prof. Martin Dove, Kat Austen, Andrew Walker, Richard BruinSTFC eScience - Rik Tyer, Kerstin Kleese Van Dam, Rob Allan, Phil CouchSTFC CSED - Jens Thomas, Martin Lueders, Walter Temmerman
