Grid Computing & Tuple Space Grid Computing & Tuple Space

Presented byPresented by

Nelson ChuNelson Chu

What is Grid?What is Grid? Computing Grid is a very large distributed Computing Grid is a very large distributed

computing network, Virtual Organizations computing network, Virtual Organizations (VO)(VO)

Grid is an infrastructure to collaborate many Grid is an infrastructure to collaborate many diverse computing resources in a network to diverse computing resources in a network to accomplish a task(s).accomplish a task(s).

What is Grid?What is Grid?

Classification: Classification: Computational GridComputational Grid (eg. WestGrid) (eg. WestGrid)

Desktop GridDesktop Grid (eg. SETI@HOME) (eg. SETI@HOME)

Data GridData Grid (eg. Earth Systems Grid)(eg. Earth Systems Grid)

Global Grid Forum (GGF), Globus, etc. Global Grid Forum (GGF), Globus, etc. Open Grid Services Architecture - OGSAOpen Grid Services Architecture - OGSA Web Service Resource Framework - WSRF Web Service Resource Framework - WSRF

(OGSI)(OGSI)

ESG

Fabric: Physical resources

Resource & Connectivity Protocols: secured access

Collective Services: brokering, monitoring

User Applications/Tools

Grid ArchitectureGrid Architecture

Fabric layer: resource management Fabric layer: resource management e.g. computational resources, storage e.g. computational resources, storage resources, and network resources. resources, and network resources.

Connectivity layer: Communication and Connectivity layer: Communication and authentication authentication

Collective layer: global-naming, brokering, Collective layer: global-naming, brokering, scheduling, workload and workflow scheduling, workload and workflow management, and data replication. management, and data replication.

Application layer: user defined tools and Application layer: user defined tools and applicationapplication

OGSA Architecture

OGSA provides a service oriented architecture to simplify grid services development and to build a

transparent grid structure

Physical resources (for example: storage, network, and computer) and logical resources services

Web services (including WSRF)

OGSA architect services (such as security management, data management, workflow

management, file system management, etc.)

Grid applications

Research Opportunities: GridResearch Opportunities: Grid

SecuritySecurity Multi-institutional users and heterogeneous resources Multi-institutional users and heterogeneous resources Authentication & authorization mechanismsAuthentication & authorization mechanisms Different local name spaces, credentials, and/or Different local name spaces, credentials, and/or

accountsaccounts Trust mechanisms: domain-based, user-based, service-Trust mechanisms: domain-based, user-based, service-

based, network-based, and scheduler-basedbased, network-based, and scheduler-based

Fault ToleranceFault Tolerance Detect and notify a fault occurrenceDetect and notify a fault occurrence Handle any failureHandle any failure

Research Opportunities: GridResearch Opportunities: Grid

SchedulingScheduling Workflows in a service-based environmentWorkflows in a service-based environment Split and distribute jobs to resourcesSplit and distribute jobs to resources Time, cost, performance, completeness, fairness, Time, cost, performance, completeness, fairness,

effectiveness, and security policies. effectiveness, and security policies.

Resource discoveryResource discovery System selectionSystem selection Job submission: resources reservation and Job submission: resources reservation and

supporting operationsupporting operation Monitoring and cleaning upMonitoring and cleaning up

Current Research TrendsCurrent Research Trends

Scheduling, Security, Scheduling, Security, Fault ToleranceFault Tolerance

Replica location services Replica location services

Metadata catalog servicesMetadata catalog services

Reliable file transfer servicesReliable file transfer services

Globus Toolkit 3 / 4: Globus Toolkit 3 / 4: Replica Catalog, GridFTP, Replica Catalog, GridFTP, and (GASS) and (GASS) Global Access to Secondary Storage Global Access to Secondary Storage ..

Globus, IBM, Oracle 10g, Sun, etc.Globus, IBM, Oracle 10g, Sun, etc.

Research Opportunities: DataResearch Opportunities: Data

Data VersioningData Versioning Global name space: data characterization, Global name space: data characterization,

heterogeneous systemsheterogeneous systems Data SecurityData Security Fault tolerance: recoveryFault tolerance: recovery New challenges: inter-domains replication, New challenges: inter-domains replication,

data synchronization, replica registration, data synchronization, replica registration, querying, and meta-catalog. querying, and meta-catalog.

Grid EnvironmentGrid Environment

Resources collaborationResources collaboration

Loosely coupling Loosely coupling Time, space, destinationTime, space, destination

Communication allows significant degree of Communication allows significant degree of interoperability, synchronization, and interoperability, synchronization, and extensibilityextensibility

Simple, flexible, and powerful mechanismSimple, flexible, and powerful mechanism

Grid System Grid System

Open issues: data synchronization, fault Open issues: data synchronization, fault tolerance, persistence, concurrency, tolerance, persistence, concurrency, transparency, etc. transparency, etc.

Tuple Space paradigm Tuple Space paradigm Simple operations provide powerful inter-Simple operations provide powerful inter-

process mechanism for communication and process mechanism for communication and synchronization. synchronization.

Flexible for function expansion: ordered tuple, Flexible for function expansion: ordered tuple, leasing, … leasing, …

Tuple SpaceTuple Space

Developed in a parallel and distributed Developed in a parallel and distributed programming system, Linda, at Yale Uprogramming system, Linda, at Yale U

Basic element ~ TupleBasic element ~ Tuple

Tuple: a vector of typed values / fields / Tuple: a vector of typed values / fields / variablesvariables

Operations: Writing, Reading, Consuming &Operations: Writing, Reading, Consuming &

MatchingMatching

Tuple SpaceTuple Space

TS Vs DBSTS Vs DBS

SpacesTwo

SpacesOne

A

C

Read

Take “B” … … … wait/abort

Write

Processes

Producers

Consumers

Tuple SpaceTuple Space

Globally shared memory space Globally shared memory space (Communication buffer)(Communication buffer)

Tuples are associatively addressed by Tuples are associatively addressed by matchingmatching

Loosely couplingLoosely coupling

Mechanism for inter-process communication Mechanism for inter-process communication and synchronizationand synchronization

Simple, flexible, powerful, …Simple, flexible, powerful, …~> Grid Resource Collaboration~> Grid Resource Collaboration

Tuple Space on the GridTuple Space on the Grid

Service RegistrationService Registration Service DiscoveryService Discovery Service LookupService Lookup Job SchedulerJob Scheduler Data MgmtData Mgmt

Tuple Spaces

(Services / Tasks / Data)

Consumers

Producers / Processes

Tuple Space on the Grid Tuple Space on the Grid with Java Technologywith Java Technology

Grid Enabled

• Resource Mgmt (GRAM)

• Security Feature

• Fault Tolerant

• Data Mgmt

• Look up Service

• Discovery Mgmt

• Leasing

• Plug and Work

• Resources / Services Collaboration

Grid Job SchedulingGrid Job Scheduling

Eg) WestGrid, …Eg) WestGrid, …

Grid IScheduler/

Resources Broker

Resource

Resource

Meta-Scheduler Grid II

Centralized System

• Load balance

• Synchronization

• Fault Tolerance

• Policy

• Meta-Scheduler

Tuple Spaces Based Gird Job SchedulerTuple Spaces Based Gird Job Scheduler

Resource 1 Agent

Resource N Agent

Registrar

Job

Desc.Resources Required

Job

Desc.Resources Required

Result

SecurityCheck()

PolicyCheck()

TakeJob()

WriteResult()

Resources List

Register()

JobSpace

JobSubmit()

Registrar

ResourcesList()

ResourcesListExchange()

JobExchange()

Domain II

Users

Figure 1: Tuple Spaces Based Grid Scheduler

GetResult()

LeaseRenew ()

Fault Tolerance ManagementFault Tolerance Management Leasing Structure for Dynamic ParticipationLeasing Structure for Dynamic Participation On-Demand Service On-Demand Service Automatic Load Balance Automatic Load Balance Global Name SpaceGlobal Name Space Security model ?Security model ?

View 3

View 1

Query… …SearchView()

GetJobTicket()

QueueTicket()

GridFTP() / TransferSchedule()

SecCheck()

PolicyCheck()

TakeJob()

UpdateView()

EventNotify()

UpdateData()

… …

PriorityCheck()

New Data

Derived DataRenewL()

TicketBox

ViewSite

OpenSpace

Figure 2: Data Access Management with Tuple Spaces

Registrar RegistrarResourcesListExchange()

Domain II

Resource Agent

EventNotify()

Update()

Update()

EventNotify()

Register()

JobExchange ()

Replica

Ticket:View(s)/Query(ies)CertificateUser informationJob description… …

Gird Data Gird Data Management System Management System

with Tuple Spaceswith Tuple Spaces

ReferencesReferences

Ian Foster, Carl Kesselman, and Steven Tuecke, “The Anatomy of Ian Foster, Carl Kesselman, and Steven Tuecke, “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, Intl J. the Grid: Enabling Scalable Virtual Organizations”, Intl J. Supercomputer Applications, 2001Supercomputer Applications, 2001

Ian Foster, Carl Kesselman, Jeffrey M. Nick, and Steven Tuecke, Ian Foster, Carl Kesselman, Jeffrey M. Nick, and Steven Tuecke, “The Physiology of the Grid: An Open Grid Services Architecture for “The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration”, http://www.globus.org, 2002Distributed Systems Integration”, http://www.globus.org, 2002

Gregory T. Byrd, and Praveenkumar Cheruvu, “Tuple Space Gregory T. Byrd, and Praveenkumar Cheruvu, “Tuple Space Computing on the Grid”, North Carolina State UniversityComputing on the Grid”, North Carolina State University

K.A. Hawick, H.A. James, and L.H. Pritchard, “Tuple-Space Based K.A. Hawick, H.A. James, and L.H. Pritchard, “Tuple-Space Based Middleware for Distributed Computing”, Technical Report DHPC-Middleware for Distributed Computing”, Technical Report DHPC-128, University of Wales128, University of Wales

Globus, http://www.globus.orgGlobus, http://www.globus.org

IBM, www.ibm.com/gridIBM, www.ibm.com/grid

DiscussionsDiscussions

Thank you !Thank you !