Upload
alan-sill
View
57
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Introduction to the Open Grid Forum community and the document production process, as well as several primary application arenas for OGF specifications, given at the co-located International Conference on Cloud and Autonomic Computing (CAC 2014), IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO 2014) and the IEEE International Conference on Peer-to-Peer Computing (P2P’14) conferences, September 8-12, 2014 at Imperial College in London, UK.
Citation preview
©2013 Open Grid Forum
The Open Grid Forum: History, Introduction and Process
Alan Sill VP of Standards, OGF
Senior Scientist, High Performance Computing Center Adjunct Professor of Physics
Texas Tech University
1
FAS* Co-located Conferences September 8-12, 2013
© 2014 Open Grid Forum FAS* 2014: SAOS, P2P, CAC, OGF London, UK Sep 9, 2014
About the Open Grid Forum:
Open Grid Forum (OGF) is a leading global standards development organization operating in the areas of cloud, grid and related forms of advanced distributed computing. The OGF community pursues these topics through an open process for development, creation and promotion of relevant specifications and use cases. OGF actively engages partners and participants throughout the international arena through an open forum with open processes to champion architectural blueprints related to cloud and grid computing. The resulting specifications and standards enable pervasive adoption of advanced distributed computing techniques for business and research worldwide.
2
© 2014 Open Grid Forum FAS* 2014: SAOS, P2P, CAC, OGF London, UK Sep 9, 2014
History and Background
• OGF began in 2001 as an organization to promote the advancement of distributed computing worldwide.
• Grid Forum --> Global Grid Forum --> GGF + Enterprise Grid Alliance --> formation of OGF in 2005.
• Mandate is to take on all forms of distributed computing and to work to promote cooperation, information exchange, best practices in use and standardization.
• OGF best known for a series of important computing, security and network standards that form the basis for major science and business-based distributed computing (BES, GridFTP, DRMAA, JSDL, RNS, GLUE, UR, etc.).
• Have also been working on cloud and Big Data standards (OCCI, WS-Agreement, DFDL, etc.) for several years.
• Cooperative work agreements with other SDOs in place.3
© 2014 Open Grid Forum FAS* 2014: SAOS, P2P, CAC, OGF London, UK Sep 9, 2014
OGF Standards
OGF has an extensive set of applicable standards related to advanced distributed grid and cloud computing and associated storage management and network operation:
- Job Submission and Workflow Management (JSDL, BES) - Federated Identity Management (FedSec-CG) - Virtual Organizations (VOMS) - Managing the Trust Eco-System (CA operations, AuthN/AuthZ) - Network Management (NSI, NML, NMC, NM) - Secure, fast multi--party data transfer (GridFTP, SRM) - Service Agreements (WS-Agreement, WS-Agreement Negotiation) - Data Format Description (DFDL) - Cloud Computing interfaces (OCCI) - Distributed resource management APIs (DRMAA, SAGA, etc.) - Firewall Traversal (FiTP) - (Many others under development)
4
© 2014 Open Grid Forum FAS* 2014: SAOS, P2P, CAC, OGF London, UK Sep 9, 2014
Starting Point: OGF Documents
5
http://ogf.org/documents
© 2014 Open Grid Forum FAS* 2014: SAOS, P2P, CAC, OGF London, UK Sep 9, 2014
OGF Document Types
• Informational: To inform the community about a useful idea or set of ideas.
• Experimental: To inform the community about a useful experiment, testbed or implementation of idea or set of ideas.
• Community Practice: To inform the community of common practice or process, with the objective to influence the community.
• Recommendations: To document a specification, analogous to an Internet Standards track document. Recommendations are initially designated as "proposed," and following further experience and review may become full recommendations.
• Further information including guidance and advice contained in GFD.152 at: http://ogf.org/documents/GFD.152.pdf
6
© 2014 Open Grid Forum FAS* 2014: SAOS, P2P, CAC, OGF London, UK Sep 9, 2014
Example: Worldwide LHC Computing Grid
7
~450,000 cpu cores ~430 Pb storage
Typical data transfer rate: ~12 GByte/sec
Total worldwide grid capacity: ~2x WLCG across all grids and
VOs
The Role of Standards for Risk Reduction and Inter-operation in XSEDE
XSEDE: The Next Generation of US National Supercomputing
Infrastructure
Cloud and grid standards now power some of the largest academic supercomputing infrastructures in the world!
Blacklight Shared Memory 4k Xeon cores
!Darter 24k cores !Nautilus Visualization Data Analytics !Keeneland CPU/GPGPU
Stampede 460K cores w. Xeon Phi >1000 users Upgrade in 2015
Wrangler Data Analytics
Trestles IO-‐intensive 10k cores 160 GB SSD/Flash !Gordon Data intensive 64 TB memory 300 TB Flash Mem
Open Science Grid High throughput 124 sitesBlue Waters
Leadership
SuperMIC 380 nodes – 1PF (Ivy bridge, Xeon Phi, GPU)
Over 13 million service units/day typically delivered as of 2014 across all XSEDE supercomputing sites (about 3 million core hours/day), totaling about 1.6 billion core hours per year
Yellowstone Geosciences
US National Cyberinfrastructure
Promote an open, robust, collaborative, and innovative
ecosystem
Adopt, create and disseminat
e knowledge
Extend the impact of cyber-
infrastructurePrepare
the current and next
generation
Provide technical
expertise and support services
Collaborate with other CI groups and
projects
FutureGrid*
Maverick Visualization Data Analytics
Comet “Long Tail Science” 47k cores/2 PF High throughput
ACI-‐REF Campus sharing, NSF Cloud (shared)
Grids
Credit: Irene Qualters, US
National Science Foundation
www.egi.euEGI-InSPIRE RI-261323 www.egi.euEGI-‐InSPIRE RI-‐261323
EGI international presenceStorage Value (yearly increase)
Disk (PB) 235 PB (+69%)
Tape (PB) 176 PB (+32%)
Value (yearly increase)
CPU cores 361,300 across 53 countries (1.44 M job/day)
Standards-based international collaboration
EGI Federated Cloud: A successful standards-based international federated cloud infrastructure
Credit: David Wallom
Chair EGI Federated Cloud Task Force
Members •70 individuals •40 institutions •13 countries
Stakeholders •23 Resource Providers •10 Technology Providers •7 User Communities •4 Liaisons
Technologies •OpenStack •OpenNebula •StratusLab •CloudStack (in evaluation)
•Synnefo •WNoDeS
BSC
CNRS
LMU
OeRC
Masaryk
TUD
IFAE
Cyfronet
100%IT
CESNET
RADICAL
SRCE
DANTE
FZJ
GRNET
GWDG
STFC
SARA
KTH
INFNFCTSG
EGI.eu
Imperial
CESGACETA
IFCA
IGI
IPHC
IN2P3
SZTAKI
IISAS SixSq
Standards •OCCI (control) •OVF (images) •X.509 (authN) •CDMI (storage - under development)
(Updated July 2014)
© 2014 Open Grid Forum FAS* 2014: SAOS, P2P, CAC, OGF London, UK Sep 9, 2014
Continuing series… !
Oriented towards REAL DEVELOPMENT
!Past events
co-sponsored by many open source and standards-related
organizations including
OGF, DMTF, SNIA, OASIS, ETSI, OCEAN and OW2
Developer-oriented in-person
standards and software testing
series
Easy to get involved and join
in events as developers or
project researchers
Cloud Plugfest 11 just completed!
http://cloudplugfest.org
Cloud Plugfest Developer Series:
12
More events in planning pipeline.
© 2014 Open Grid Forum FAS* 2014: SAOS, P2P, CAC, OGF London, UK Sep 9, 2014
Conclusions and Future:
• OGF actively engages many partners and participants throughout the international arena through an open forum with open processes to promote best practices and standards in advanced distributed computing.
• OGF occupies an important role in standards and software development with significant uptake in advanced distributed computing, including cloud, grid, networking and large-scale data processing, transfer and handling.
• OGF documents support a variety of flexible architectures for advanced scientific, community and business uses.
• OGF’s experience has enabled distributed computing built on these architectures to provide more flexible, efficient and utility-like global infrastructures. Join us!
13