Embed Size (px)
Citation preview
A PPARC funded project
AstroGrid Architecture
Consortium Meeting,Leicester,3rd Nov 2003
03-Nov-2003 AstroGrid Architecture 2
AgendaGoals reminderGeneral points Approach Registries & Identifiers
ComponentsNext?
Note: presentation template
03-Nov-2003 AstroGrid Architecture 3
…
03-Nov-2003 AstroGrid Architecture 4
GoalsA working datagrid for key UK databases OK: data centres to install
AstroGrid
High throughput datamining facilities for interrogating those databases
??: GDW
A uniform archive query and data-mining software interface
OK: DataAccess, GDW
The ability to browse simultaneously multiple datasets OK
A set of tools for integrated on-line analysis of extracted data
OK: more in AG2
A set of tools for on-line database analysis and exploration
OK: more in AG2
A facility for users to upload code to run their own algorithms on the datamining machines
??: AG2 / GRID
An exploration of techniques for open-ended resource discovery
OK: Registry
03-Nov-2003 AstroGrid Architecture 5
Goals: additionalWorkflow constructionDistributed virtual storage spaceData access rightsUser identification, profiling, …Group permissioningEffective usable GUI
Complete VO infrastructure
03-Nov-2003 AstroGrid Architecture 6
…
03-Nov-2003 AstroGrid Architecture 7
General: Approach
InfrastructureOpenInteroperabilityComponent-basedDeploymentLeading-edgeIncremental development
03-Nov-2003 AstroGrid Architecture 8
General: Approach: Infrastructure
Other VO projects: Tools: SkyNode, Cone Search, … Registry
AstroGrid MySpace Community Workflow Portal Access Security
03-Nov-2003 AstroGrid Architecture 9
General: Approach: Open
Open Source Implement IPL/OPL & Copyright notices Upload releases to nescforge
Deployment choices Choice of component installation Open interfaces
Server / Browser access Run from anywhere
Open Science Free access to all documents Open engagement
03-Nov-2003 AstroGrid Architecture 10
General: Approach: Interoperability
IVOA engagement Resource Registry
Content Identifiers Schema
ADQL
Deployment Registry content SkyNode & ADQL
03-Nov-2003 AstroGrid Architecture 11
General: Approach: Interoperability
Lead IVOA Data access Access policy Authentication & Authorisation Workflow Component interoperability Community MySpace (VOSpace) Grid-based components
03-Nov-2003 AstroGrid Architecture 12
General: Approach: Component-based
Plug-n-Play1. Install2. Configure3. Operate
Pick-n-Mix Choose components for environment
Mix-n-Match Use components from other VO projects
Goal:Minimum component interaction Asynchronous calls Call once only Pass maximum info
03-Nov-2003 AstroGrid Architecture 13
General: Approach: Deployment
03-Nov-2003 AstroGrid Architecture 14
General: Approach: Leading-edge
Not bleeding-edgeLatest (mostly proven) technologies Cocoon XML messaging XQuery SOAPy Beans (?) Eclipse IDE
even if only by us
03-Nov-2003 AstroGrid Architecture 15
General: Approach: Incremental development
Quarterly releasesWeekly integration
Daily Write Test Commit Build Integrate
03-Nov-2003 AstroGrid Architecture 16
…
03-Nov-2003 AstroGrid Architecture 17
General: Registries & Identifiers
Registries Resource Community MySpace
Identifiers AuthorityID ResourceKey Others: datasets, files, tables, …
03-Nov-2003 AstroGrid Architecture 18
General: Registries & Identifiers: Registries
Resource Data: archive, catalog, spectra, … Service: ace, topcat, vot-merge, … Others: registry, authority, …
Community Person Group
MySpace Items: files, tables, …
03-Nov-2003 AstroGrid Architecture 19
General: Registries & Identifiers: Identifiers
XML structure: AuthorityID ResourceKey IVOA standard
String equivalent: ivo://AuthorityID/ResourceKey
03-Nov-2003 AstroGrid Architecture 20
General: Registries & Identifiers: Identifiers
Sub-identifier Extend XML structure Add to string after:
# ?
ivo://AuthorityID/ResourceKey#... ivo://AuthorityID/ResourceKey?...
Ignored by resource registry!!
03-Nov-2003 AstroGrid Architecture 21
General: Registries & Identifiers: AuthorityID
Like domain name: astrogrid.org community.astrogrid.org sdss.jhu vizier.u-strasbg.fr
Owned by: Organisation Person Registry
Allocated to resource by one registry
03-Nov-2003 AstroGrid Architecture 22
General: Registries & Identifiers: ResourceKey
Like path name: J/A+A/385/87 services/SIAP 1XMM
Chosen by userUnique across VOPermanent Obsolete names flagged
03-Nov-2003 AstroGrid Architecture 23
General: Registries & Identifiers: Community – Person & Group
Identity same as resource: community.star.le.ac.uk
Tony.Linde community.astrogrid.org
groups/AGLI u-strasbg.fr
Genova
Replication? User can choose details to be replicated Default?
Identity, name , affiliation, email
03-Nov-2003 AstroGrid Architecture 24
General: Registries & Identifiers: MySpace
Extend resource identifier
As string: vosp://community.star.le.ac.uk/Tony.Linde#
pub/documents/articles/Strategies for AstroGrid
<ItemID><PersonID>
<AuthorityID>community.star.le.ac.uk</AuthorityID><ResourceKey>Tony.Linde</ResourceKey>
</PersonID><ItemPath>pub/documents/articles</ItemPath><ItemName>Strategies for AstroGrid</ItemName>
</ItemID>
03-Nov-2003 AstroGrid Architecture 25
General: Registries & Identifiers: MySpace
AuthorityID Community registryResourceKey PersonPerson metadata: msManager resource
msManager msRegistry Metadata for all items:
Physical location of msServer
msServer Maps name to internal representation Mediates access to item Allocates & initiates transfer, add, delete…
<ItemID> <PersonID> <AuthorityID>community.star.le.ac.uk</AuthorityID> <ResourceKey>Tony.Linde</ResourceKey> </PersonID> <ItemPath>pub/documents/articles</ItemPath> <ItemName>Strategies for AstroGrid</ItemName></ItemID>
03-Nov-2003 AstroGrid Architecture 26
…
03-Nov-2003 AstroGrid Architecture 27
Component ArchitectureRegistrySecurityCommunityPortalWorkflowMySpaceDataAccessApplicationsMessaging
03-Nov-2003 AstroGrid Architecture 28
Components: Registry
Schema IVOA Working Draft: RegWp03 V0.9: still waiting Extensions?
SkyNode DataAccess components Services: TopCat, ACE, …
Harvesting: OAI http GET
03-Nov-2003 AstroGrid Architecture 29
Components: Registry
IVOA commitment Use schema Replicate with other registries
VizieR NVO others
Fully working by 01-Jan-2004 NVO demos at AAS, Atlanta, 4-8th Jan Remote access via browser Add/Update/Delete resources via GUI
03-Nov-2003 AstroGrid Architecture 30
Components: Registry: Issues
Storage XML native XML via RDBMS RDBMS
Query XQuery
Real XQuery (FLWOR) rather than XPath SQL
03-Nov-2003 AstroGrid Architecture 31
Components: Security
Resource has access policy attached Part of metadata (schema extension) Written using ODRL?
Policy identifies Access rights (RWED?) Default: RE Per Group
Members inherit Group permissionsGroup may restrict members’ permissions
03-Nov-2003 AstroGrid Architecture 32
Components: Security
Service call / Data query: Includes user identity & permissions
Certification / Encryption Look at GSI User may upload certificate Components have own certificates Component exchange based on:
Component certificate User certificate
03-Nov-2003 AstroGrid Architecture 33
Components: Security: Issues
Certification implementationComponent ‘trust’Fallback Unidentified user Non-trusted component
03-Nov-2003 AstroGrid Architecture 34
Components: Community
Community registryRelationship to MySpaceGroups Contain people (PersonID) Creator has full admin rights over group
Can delegate rights to others Including people from another community
‘Community’ is meaningless Only people & groups have meaning, IDs
03-Nov-2003 AstroGrid Architecture 35
Components: Community: Issues
Registry schema extensionsRegistry implementation From resource registry?
Replication of user metadata If? How?
03-Nov-2003 AstroGrid Architecture 36
Components: Portal: Issues
Adding application linksIntegrating appletsDescribing application GUIs XForms Service metadata: pointer to xforms
structure
Scripting application GUIs XForms?
03-Nov-2003 AstroGrid Architecture 37
Components: Workflow
Add component from tasklist to workflow Appropriate GUI presented
Set parameters Design query …
Save & Resubmit Templates Task groups
Add to workflow Change parameters
Query Number of records
BPEL description
03-Nov-2003 AstroGrid Architecture 38
Components: Workflow
Add generic resource JES selects optimum resources at:
Submission Run time
Task I/O links Identify mismatch Insert converter
Progress tracking User notification Use profile in community registry
03-Nov-2003 AstroGrid Architecture 39
Components: Workflow: Issues
Effective GUI (non drag-n-drop)Adding services to ‘toolbar’ Discover via registry Standard + user-selectable?
03-Nov-2003 AstroGrid Architecture 40
Components: MySpace
Identifier as per IVOA standard, extendedmsManager msRegistrymsServer manages items Mediates access Provides access method
03-Nov-2003 AstroGrid Architecture 41
Components: MySpace: Issues
Access via msServerAccess rightsIdentifier valid for database tables?
03-Nov-2003 AstroGrid Architecture 42
Components: DataAccess
Common interfacePlugins for databases
03-Nov-2003 AstroGrid Architecture 43
Components: DataAccess: Issues
FITS file access Wizard to set up service
03-Nov-2003 AstroGrid Architecture 44
Components: Applications
Web service interface/wrapperShould not wrap: Existing web services http GET services Any other callable service
03-Nov-2003 AstroGrid Architecture 45
Components: Applications: Issues
Describing GUI for appsAdding GUI-based services to Portal
03-Nov-2003 AstroGrid Architecture 46
Components: Messaging
Message QueueMessage LogIssues?
03-Nov-2003 AstroGrid Architecture 47
…
03-Nov-2003 AstroGrid Architecture 48
Next?Complete architecture docs Revise after today Update with interfaces Add use cases & sequence diags
Get feedback & complete
