1

Grid DAIS:Database Access and Integration Services

Greg RiccardiFlorida State University riccardi@cs.fsu.edu

2

Overview of Presentation Goals of DAIS Conceptual model of Grid database

access Examples of client-service interactions

Discovery and creation of servicesAsynchronous query processing and datasetsUpdating from datasets

Representing Sky Query in DAIS Other topics/issues

3

Goals of DAIS The group seeks to promote standards for the

development of grid database services, focusing principally on providing consistent access to existing, autonomously managed databases. Provide service-based access to existing data management

systems. Accommodate several widely used data management paradigms

(e.g., relational, object, XML) within a consistent framework. Provide sufficient information about itself to allow the service to

be used given the specification of the service and the metadata provided by the service.

Peacefully coexist with other Web and Grid Service standards. Be orthogonal to Grid authentication and authorization

mechanisms. Support higher-level information-integration and federation

services.

4

Desirable Properties of DAIS Systems OGSI/A compliant

Letter and Spirit Plugability/Extensibility

Different kinds of data resources Many access mechanisms Evolvable

Easy to understand and apply Existing standards/designs Tooling

GridServices and WebServices applicable Supports current technology Access AND integration

Integration of different models at the data level Implementable Integrateable into customer scenarios Technology independent

5

The Model – External Artifacts

DBMS

DB

Resultset

External data resource manager

External data resource

External data set

External = external to the OGSI compliant grid

6

The Model – Logical Artifacts

DBMS

DB

Resultset

data resource manager

data resource

data activity session

data request

data set

7

Data Resource Manager External data resource manager (edrm)

A data management system such as a relational database management system or a file system

Data resource manager (drm) A grid service that represents the external data

resource manager Binds to an existing edrm Supports management operations such as start and

stop Mainly out of scope of DAIS. A place holder for

interaction with other working groups

8

Data Resources External Data Resource (edr)

A data construct managed by the external data resource manager, for example, a database or a directory structure.

An external data resource manager may manage many external data resources

Data Resource (dr) A grid service that represents an external data resource Represents the point of contact to the data structures managed

by the edrm. Exposes meta-data about the structure of the edr Defines the the target for queries across the edr Can act as a notification source for notifications associated with

the edr Is bound to existing or newly created edr Has similarities with a data set. More of which later.

9

Data Sets External Data Set (eds)

Data logically separated from an external data resource manager

Could be a snapshot (query) of a relational database or data generated by some process prior to being inserted into a database

Will be typed and identifiable Data Set (ds)

A service wrapper for the eds Exposes meta data about the type, description, format of the eds Immutable Exposes simple data access operations depending on the type

of data. getAllData, createIterator, getTuple, getFile, getByte, etc.

Can be moved while maintaining its handle and data identity Can be copied or replicated while maintaining its data identity Can be delivered to a data manager for persistence

Query and update could be supported

10

Putting It Together Logical Artifact = Service

ds

eds

dr

edredrm

drm das

External world

DAIS world

create

bind/create

create

bind

create

create

bind/create

requester

data request

locate access data

11

Exploiting The Logical Artifacts: Data Sets

dsdr

edr

das

ds ds dr

edr

das

queryinsert/update

create

move copy

referencelaunch launch

GSH GSH

move service

reference create

copy service

reference create

target details

target details

Analyst1

12

Client-Server Interaction Patterns

A G

Q

S + R

AG

Q + D

S

RC

A G

Q + U

S

A G

Q

S

UP

Retrieve Update/Insert

AG

Q

S

DCR

A G

Q + D

S

IP

U

1.

2.

3.

4.

6.

5.

Pipeline

A

G = PQ1 + D

S1 U/R

G = CS2

Q2

A

G = PQ1

S1 U/R

G = CS2

Q2 + DI

7.

8.

13

Examples of client-service interactions Discovery and creation of services

createcreate

A dr

das

Create

GSH

Query

Result

drmCreate

GSH

Database

ExternalDataResourceManager

11

2

23

14

Examples of client-service interactions Asynchronous query processing and

datasets

Database

crea

te

A das

dsC

Query

Id

Id

Get

Result

3

1

2

5

4

15

Examples of client-service interactions Updating from datasets

Database

ds

dasAUpdate(Id)

Status

Get Result2

1

3

16

Example of performance estimation

Database

preparedasA

Query

Status

Estimate

Statistics

17

SkyQuery Cross Match Query

18

Cross Match Estimation in DAIS

ClientQuery Manager

Query

StatisticsSpatialQuery

StatisticsPr

epar

e

das

Database

Prep

are

das

DatabasePr

epar

e

das

Database

SpatialQuery

SpatialQuery

Statistics

19

Cross Match in DAIS

ClientQuery Manager

Query

Id1Run

Query

Result

MatchId1

MatchId2 Id2Id3

getId3

Que

rycr

eate

das

ds3

Database

Que

rycr

eate

das

ds2

DatabaseQ

uery

crea

te

das

ds1

Database

20

Other topics and issues for DAIS Data provenance management Transaction management Fault tolerance Security, logging, auditing Supporting many concurrent users Establishing the identity and provenance

of datasets Creating pipelines and other workflows Querying streams of data