OGF Standards Development

© 2007 Open Grid Forum

OGF Standards DevelopmentChris Smith, VP of Standards, OGF

DMTF Alliance Partner Technical SymposiumPortlandJuly 17th, 2007

2© 2007 Open Grid Forum

Overview

• OGF History & Mission

• Organization of OGF

• Groups & Documents

• Standards Function

• Technical Strategy for Standards

• OGF & DMTF Alliance


History

• Birthed in high performance computing community in 1998

• Merged with European & Asian grid efforts in 2001

• 52 groups and 81 documents

• Birthed in enterprise data center community in 2004

• Raised awareness of grids in “end-user” organizations

• 5 groups and several important documents

Merger completed, June ’06; OGF Launched September of ‘06

http://www.ggf.org http://www.gridalliance.org


Open Grid Forum

Events & Forums

Community Practice

Industry Standards

Bring communities together to

share, innovate, workshop and

outreach

Leverage expertise & experience of the

community to enable successful

building and operating of grids

Align with/influence other SDOs and/or

develop specifications that

lead to interoperable software standards


Events & Activities

•Events enable our “Open Forum” mission• Assembles grid topic experts from around the world• Provides opportunity for grid professionals to network together• Allows buyers and sellers to interact• Provides a venue for major grid projects to collaborate• Enables cross-OGF alignment on technical strategy• Delivers relevant content to interested parties• Enables collaboration for the development of specifications


Events & Activities• 3 major events a year: Winter, Spring, Fall• Spread roughly equally by region (US, Europe, APAC)

• Driven by attendance and host offers

• Several types of content:• Group Sessions: BoFs and chartered group f2f meetings

• eScience Program: refereed workshops and in-depth sessions on grid innovations and best practices

• Enterprise Program: end-user deployment-focused sessions and requirements gathering

• Tutorials: hands-on sessions; educational

• Shorter, more focused events also possible• Occasional webcasts


Organization

President

BoD

Operations

Editor

Enterprise

eScience Standards Marketing Regional

Nominating Committee

Technical Strategy Committee

Advisory Committee

OGF Overview Document available at:

http://www.ogf.org/rotate_headers/rotate_launch.php


Functions/Areas/Groups/Chairs

Standards Function

Standards CouncilVP, StandardsArea Directors

Program ManagerOGF Editor

Man

ag

em

ent

Application

Secu

rityArchitecture

Infrastructure

Data Compute

Standards Areas

Liaisons

Areas managed by Area Directors

Areas consolidate Groups of like

focus

Groups are led by Chairs

Functions consolidate Areas

of like focus


Example: SAGA-Working Group

Standards Function

Standards CouncilChris Smith, VP

Steven Newhouse, Dieter Kranzlmueller, AD’sJoel ReplogleGreg Newby

Application Area

Standards Areas Each group has an email list (

[email protected]

) and a GridForge

project

Simple API for Grid Applications (SAGA-WG)

Co-Chaired by:

Shantenu Jha , Thilo Kielmann , Tom Goodale

Note that SAGA-WG is one of many groups in

this Area


Group Types

• Working Groups (WG)• Crisp focus on development of a specification or guideline• Clear milestones based on delivery of drafts and publication dates

• Research Groups (RG)• Focus can be broader than WG, but must be clear• Milestones based on delivery of drafts, organization of workshops, and workshop reports• Technology exploration (e.g., Semantic Grid-RG)

• Community Groups (CG)• Explore grid usage and requirements in a sector (e.g., Telco-CG)• Milestones based on delivery of requirements documents

• Full Listing:http://www.ogf.org/gf/group_info/areasgroups.php


How Groups Work

• Because every group has a different charter and leadership, every group is different

• Consistent things:• Work is announced on the group email list• Deliverables are normally one or more documents• Documents follow the same publication process (OGF Editor)• “Rough consensus and working code” is the rule of thumb• All work done under OGF IPR Policy

• Inconsistent things:• Meeting cadence• Use of GridForge (web-based collaboration tool) for group work • Group deliverables differ based on chartered work


OGF Editor

• Manages the publication process (GFD-C.1) and “pipeline”

• Ensures documents are consistently formatted

• Works with Area Directors/VPs to ensure technical viability of documents

• Drafts submitted to the “editor pipeline” • Editor project in GridForge• http://forge.ogf.org/sf/projects/ggf-editor


Deliverables

• Different Groups have different deliverables• Research groups may hold workshops and create proceedings

documents and/or “best practice” documents• Working groups will likely deliver specifications but may have

preliminary “informational” documents to lay the foundation• Community groups may hold workshops that capture

requirements which are documented and given to working groups

• Other groups may document a well-used process in the grid industry or in the OGF organization

• Different deliverables require different document types


OGF Document Types

• Informational• Informs the community of an interesting and useful

Grid-related technology, architecture, framework, or concept

• Specifies requirements related to a particular vertical application

• Experimental• Informs the community of the results of Grid-related

experiments, implementations, operational experience, or to propose an experimental specification


Document Types

• Community Practice• Inform and influence the community regarding an approach or

process that is considered to be widely accepted by consensus and practice in the Grid community or within the OGF organization

• Recommendations (2 stage)• Documents a particular technical specification or a particular set

of guidelines for the application of a technical specification. The recommendations documents are intended to guide interoperability and promote standard approaches.


Standards Function Overview

• Set in the context of of alignment with• e-Science• Enterprise

• Promote• Uptake• Interoperability

• Run a Process• Output in the form

of Standards

Managem

ent

Managem

ent

ApplicationsApplications

Secu

rity

Secu

rityArchitecture

Infrastructure

Data Compute

StandardsStandardsOGF Liaison

Technical Strategy CommitteeTechnical Strategy Committee


Standards Development

• Short Version of the Process• A group of people interested in creating a standard hold a

BoF (optional).• Grid Forum Steering Committee Approves a WG• In an open forum, the WG writes the specification• GFSG Review• Public Review• Final approval and publication

• Complex Open Process, …• but things can move very fast when there is energy

available.


Technical Strategy Overview

• Defines the overall OGF technical strategy for the development of standards

• Three-year timeframe from 2006 to 2010• Describes the output of the OGF standards working

groups as well as the requirements that drive them.• Technical Strategy Document

• Focus Areas• Goals• Alignment Process• Use Cases• Roadmap


Broad and Narrow Grids

• The concept allows for clearer communication encompassing all aspects and points of view on Grids.

• A Broad Grid is any collection of services• A Narrow Grid is defined by both the

technologies used, as well as the application focus. Some examples:• OGSA compliant Enterprise compute cluster• Web 2.0 Sensor Net linked to Google maps


Narrow Grids of Focus

• Collaboration Grids• Multiple institutions, secure, widely distributed, VOs• Collaborative agreements & commercial partnerships• Financial Model: Increase overall revenue

• Data Centre Grids• Centralized management of multiple platforms• Aggregation of enterprise resources and applications• Financial Model: Reduce Total Cost of Ownership (TCO)

• Cluster Grids• Networks of Workstations, Blades, etc.• Cycle scavenging, Homogeneous workload• Financial Model: Lower marginal costs


Goals of the TSC

• Identify use-cases, patterns, and scenarios• Publish documents• Design and drive the standards process.• Identify and highlight core architectural

standards • Encourage software developers

• From open source community and the commercial • To develop and implement standards

• Hold regular alignment summits


Technical Strategy Process

OGF Technical Strategy & Roadmap

Requirements Workshops

Uses Cases

Requirements

Best Practices

Architectures

Specifications

Milestones

OGF Events

OGF Document Series

Alignment & Prioritization

Open forum for grid innovation and outreach

Open standards for grid software interoperability

Alignment & Prioritization

Technical Strategy Committee Standards Groups

& Workshops


A More Refined View

EGR-RG

Financial

Telco

Pharma

EDA

SN-CG

Vendors

RequirementsSolicitation

Applications

Compute

Data

Infrastructure

Management

Architecture

Security

StandardsGroups

RequirementsRollup, Analysis

&Prioritization(EGR-RG)

TSCGAP Analysis

What WGs are doing i.e. WG roadmap

Prioritized Req and Req Patterns

Req

Req and Req Patterns

RequirementsSpecs

BestPractices

Req

Best PracticeWorkshops

Req

Best Practices

• Overall standards roadmap• Gap analysis of WG roadmap vs. prioritized Req•Recommended actions


Technical Strategy & Roadmap

“Moon Shot” Goal

The Open Grid Forum should commit all its availableresources to the goal that before this decade is out,

commercial and academic organizationswill build real operational grids using

Open Grid Forum defined components.


High Value Use Cases

• Grid APIs

• Job Submit

• File Movement

• Application Provisioning

• Data Provisioning and Data Grids

• Grid Security


• Concept - hold BOF to form…• Working Group - generates a…• Draft Specification - public comment…• Proposed Recommendation - implementations…• Interop - experience documents…• Full Recommendation - GFSG review…• Product - implemented in product…• Deployment - customer use of standard

Lifecycle of a Standard


Roadmap Snapshot

Capability Specif ication Status Milestone 1 Milestone 2 Area SAGA Draft Feb 07: Spec Dec 07: Product Applications Grid APIs DRMAA Product Jan 07: Full Rec Dec 07: Deploy Applications

Grid RPC Interop Jan 07: Full Rec Dec 07: Deploy Applications JSDL 1.0 Product Feb 07: Deploy Compute OGSA-BES Draft Mar 07: Spec Dec 07: Product Compute

Job Submit

HPC Profile Draft Mar 07: Spec Dec 07: Product Compute DMI WrkGrp Dec 06: Spec Dec 07: Product Data ByteIO Interop Oct 06: Spec Dec 07: Product Data

File Movement

GridFTP Product Dec 06: Deploy Data WS-DAI Spec 74 Aug 07: Product Mar 08: Deploy Data WS-DAIR Spec 76 Aug 07: Product Mar 08: Deploy Data

Data Provisioning and Data Grids

WS-DAIX Spec 75 Aug 07: Product Mar 08: Deploy Data CDDLM Spec 69 Aug 07: Product Mar 08: Deploy Management Application

Provisioning ACS Spec 73 Aug 07: Product Mar 08: Deploy Management OGSA-AuthN Concept Security Authentication OCSP Profile Draft Operations OGSA-AuthZ WrkGrp Jan08:Spec Security Authorization Distributed Audit Concept Security OGSA-SBP-Core Draft Aug 07: Product Mar 08: Deploy Architecture OGSA-SBP-SecChan Draft Aug 07: Product Mar 08: Deploy Architecture

Web Services Protocol Security

Grid Extended Interaction Profiles

Concept Security


There is a Gap Analysis too

• The Maturity column indicates the rough State of the Art with respect to each capability.

• Categories:• Out of Scope for the OGF• Gap that OGF should be investigating• Area of Grid Research• Evolving area either in OGF or some other SDO• Capability with Mature specifications or solutions.


Gap AnalysisCategory Capability Working Group or Comment Maturity

Multiple Security Infrastructures OGSA Auth-Z Evolving Perimeter Security Solutions Firewall I ssues RG Research Virtual Organization VOMS work applies Gap Encryption Existing technology is currently adequate Out of Scope Certifi cation CA Ops WG Evolving Authentication OGSA-AuthN Evolving Authorization OGSA-AuthZ Evolving

Security

Web Service Protocol Security OASIS/WSS, OGSA Secure Channel Mature Instantiate New Services CDDLM-WG, degenerate workflow Evolving Deployment ACS-WG, CDDLM-WG Evolving Provisioning ACS-WG, CDDLM-WG Evolving Service Level Management GRAAP-WG Evolving Notification OASIS/WS-Notification, WS-Eventing Mature Messaging OASIS/WS-Notification, WS-Eventing Mature Logging Service Related to metering, see below Gap Service and Resource Monitoring Grid Monitoring Architecture Evolving Metering and Accounting UR-WG and RUS-WG More needed Evolving Policy WS-Policy Evolving Policy Management Management standards needed for policy Gap Administration Community practices needed Gap Systems Management Reference Model-WG Evolving

Operations

Aggregation of Services and Resources See OASIS WS-ServiceGroup Mature OGSA-Naming WS-Naming-WG, GFS-WG Evolving Resource Discovery OASIS/WSDM Mature Resource Brokering RSS-WG On Hold Job Management OGSA-BES-WG, JSDL-WG. Mature Choreography, Orchestration and Workflow OASIS/BPEL, OGSA Workflow Design Team Mature Resource Virtualization GridVirt-WG, CDDLM-WG Evolving Information Model DMTF/CIM, GLUE-WG Mature CPU Scavenging Proprietary Solutions Exist Mature Legacy Programs ACS-WG Evolving

Resource Management

Reservation GRAAP-WG, GSA-RG Evolving Data Movement DMI-WG, Grid-FTP Evolving Data Access GFS-WG and DAIS-WG Mature Data Integration DAIS-WG Evolving Data Management Storage Network-CG, OGSA-Data-WG Evolving Data Provisioning Continuation of EGA data work, OGSA-Data Gap

Data

Metadata OASIS/WSRF-RMD, Evolving Application Debugging Gap Application Development Application APIs SAGA-WG,GridRPC-WG,GridCPR-WG,DRMAA-WG Mature Communication Protocols HTTP/SOAP, Mature Architecture Reference Model-WG, OGSA-WG Mature Grid Semantics Semantic Grid-RG Research

Foundations

Grid Fabric OASIS/WSRF, NM-WG, NML-WG Mature Fault Tolerance Implementation Property Mature Load Balancing Implementation Property Mature Failure Recovery Implementation Property Evolving

System Properties

Self-Management Implementation Property Research


TS&R Document Status

• Like all OGF documents it is always accessible in Gridforge

• forge.gridforum.org/sf/projects/tsc

• Has been through its Public Comment period

• To be published in time for OGF21 in October 2007


OGF & DMTF Alliance

• Intention of the alliance is to cross-leverage each organization’s domain strength• DMTF focus on distributed management

infrastructure and information/data modeling of resources

• OGF focus on the deployment, operation and access to resources in a Grid


Alliance Benefits

• Collaboration on broadening CIM to encompass Grid concepts

• Leverage technical and marketing resources in both DMTF and OGF

• Coordinate development processes and priorities between DMTF and OGF to ensure DMTF modeling standards address Grid technology, and that OGF Grid services address management requirements

• Education about technologies and activities across the organizations, and participation in each others events


Current Work Register

• Review / Comment (1Q2007)• DSP 0225: URI Format for Published XML Schema• DSP 0230: WS-CIM Mapping Specification

• Extrapolate from the OGSA-BES work and submit to DMTF a general container model (current work in progress)

• Identify the base set of grid resources that need to be advertised that applications can consume (via requirements) and update the CIM as needed including how best to map/leverage information models. (current work in progress)


Liaisons

• Tom Roney, OGF liaison to the DMTF• [email protected]

• Mike Baskey, DMTF liaison to the OGF• [email protected]


Other OGF Sessions

Information Modeling of Grid Resources: the OGF GLUE WG Approach• Sergio Andreozzi• Tuesday, 3pm

OGSA Information Model Updates• Ellen Stokes• Wednesday, 10am

OGSA Reference Model• Paul Strong• Wednesday, 1pm


Questions?