Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team

Genesis II“Open Source, OGSA Implementation”

Genesis II:Mapping Grids into the Local File System:

Access, RNS, and ByteIO

Andrew Grimshaw

Genesis II Team

University of Virginia

“Open Source, OGSA Implementation”


Outline

• Background

• Genesis II - The specs in action


Classic three layer view

API’s & interfaces, e.g. SAGA, NFS, CIFS

Standard portypes

Resources Layer

Grid Services Layer

Access Layer


Classic 3-layer name scheme

…File replica 2File replica 2

WS-name EPRWS-name EPRFile replica 1File replica 1

File replica mFile replica m

RNS file name 1RNS file name 1

RNS file name nRNS file name n

…

Human names Abstract name:EPI, rebinding

WS-Names are WS-Addresses with optionalEPI and resolver EPR

This is essentially a table


Four specs

• RNS – directory service that maps strings to EPRs <<EPR> XSD-any>– Can build directed graphs, including trees– Leaves can be most anything, web pages, ByteIO

endpoints, DMI endpoints, BES resources• WS-Naming – A profile on WS-Addressing that

supports identity and rebinding of addresses – migration, failure, and replication transparency

• ByteIO – think POSIX file/steam, read, write, stat• BES – Accepts JSDL documents and executes

them


Outline

• Background

• Genesis II – The specs in action


User’s First

• A large percentage of a grid’s target audience is unable or unwilling to learn new interaction abstractions

• Instead of asking the user to adapt to the grid, we should adapt the grid to the user


User Abstractions

• One of the most ubiquitous user interaction abstractions is the file system– Drag-and-drop– Double Click– /proc filesystem– Named pipes

• RNS, ByteIO, and WS-Naming provide the foundation for building these abstractions


(Most) everything is a file or directory

• Files and directories can be accessed without knowing anything about Grids, Web Services, or anything thanks to OGRSH/IFS– OGRSH/IFS map the Grid into the file system

• BES resources, queues are directories– “ls” to list the jobs, “cat” a job to see its state– “cp” a JSDL file into the directory -> start the job up– A shell script can start jobs by copying

• Genesis II containers are directories– “ls” to see the services and porttypes

• IDP are files/directories• IS’s are directories

– “cp” a query file to IS, creates result• RDBMS’s will be directories• The user can access all of these services without dealing with Web

Services!!


Genesis II - Data

• FTPd for Windows

• OGRSH for Linux (shim)

• IFS for Windows

• ExportDir

• Replicated ExportDir


Localhost Fdpd

• Fully secure– My X.509– SSL– No bits in

the clear


Windows IFS


OGRSH (Linux)


Using RNS to name non-file-system components

• BES resources are also RNS directories

• We can schedule a job on a resource simply by “dropping” it into the directory


Export Directory

• Map a Unix or Windows file system into the Genesis II RNS name space so that others can securely Create/Read/Write/Destroy


Data publisher

Mapping data into the GridData clients Data clients

LinuxWindowsWindows

• Links directories and files from source location to data grid directory and user-specified name

• Presents unified view of the data across platforms, locations, domains, etc.

• Data publisher controls authorization policy.

Data publisherData publisher


Export Directory Tool


Create an Export DirSelect local path


Create an Export DirSelect target RNS path


Moral of the story

• RNS allows us to place arbitrary resources into a traditional directed graph/tree structure

• OGRSH/IFS map RNS namespaces into the local file system

• Users can interact with the grid without knowing anything about grids

Documents

Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team