View
219
Download
0
Category
Tags:
Preview:
Citation preview
17/1/2005 1
NGS StatusStephen Pickles
Technical Director, GOSC
(with thanks to Andy Richards, NGS Co-ordinator)
CERN, 17/1/2005
http://www.ngs.ac.ukhttp://www.grid-support.ac.uk
17/1/2005 2
Outline
• Users– registrations, usage, helpdesk queries
• Services & Middleware– what we offer today– managing change
• Expansion– “joining” the NGS
17/1/2005 3
Users
17/1/2005 4
Gaining AccessNGS core nodes
• data nodes at RAL and Manchester• compute nodes at Oxford and
Leeds
• free at point of use• apply through NGS web site• accept terms and conditions of use• light-weight peer review
– 1-2 weeks• to do: project or VO-based
application and registration
• all access is through digital X.509 certificates
– from UK e-Science CA– or recognized peer
National HPC services
• HPCx
• CSAR
Must apply separately to research councils
Digital certificate and Conventional (username/ password)
access supported
17/1/2005 5
User registrations so far
Number of Registered NGS Users
0
20
40
60
80
100
120
140
160
180
04 April 2004 24 May 2004 13 July 2004 01 September2004
21 October2004
10 December2004
Date
Nu
mb
er o
f U
sers
NGS UserRegistrations
Linear (NGS UserRegistrations)
17/1/2005 6
UKGSC: Queue Totals For Month Prior To (10:11 AM, 26/11/2004)
Notes•Certification still dominant query type•General contains GGUS related queries from EGEE/GGUS helpdesk(A separate project for this has now been created)•LCG savannah generated queries are increasing through this helpdesk
Helpdeskhttp://www.grid-support.ac.uk
support@grid-support.ac.uk
17/1/2005 7
NGS Core Services
Globus, SRB, Oracle, OGSA-DAI, and others
17/1/2005 8
NGS Core Services: Globus
• Globus Toolkit version 2– GT 2.4.3 from VDT 1.2
• Job submission (GRAM)• File transfer (GridFTP)• Shell (GSI-SSH)• Information Services (MDS/GIIS/GRIS)
– Information providers from GLUE schema– Use BDII implementation of MDS2 (as does EGEE)
17/1/2005 9
NGS Core Services: SRB
• Storage Resource Broker from SDSC
• Location transparent access to storage• Metadata catalog• Replica management
• Clients on compute nodes• Servers on data nodes
• Issues/to do:– licensing– MCAT replication and failover
17/1/2005 10
NGS Core Services: Oracle
• Oracle 9i database• Only on data nodes
• Populated by users/data providers• Infrastructure maintained by NGS
database administrators
• Accessed directly or via OGSA-DAI
17/1/2005 11
NGS Services: OGSA-DAI
• Developed by UK e-Science projects OGSA-DAI and DAIT• OGSA-DQP (Distributed Query Processing)• Experimental service based on OGSI/GT3 on Manchester
data node only– containment: 2 cluster nodes reserved for development and
production– will consider WS-I and WSRF flavours when in final release
• Uses Oracle underneath• User-provided Grid Data Services validated on test system,
then transferred to production during scheduled maintenance
• Early users from e-Social Science (ConvertGrid)• Established liaisons with OGSA-DAI team
17/1/2005 12
NGS Core Services: other
Operated by GOSC for NGS and UK e-Science programme
In production:• Certificate Authority• Information Services (MDS/GIIS)• MyProxy server• Integration tests and database• Cluster monitoring• LCG-VO
In testing:• VOMS• EDG Resource Broker
In development:• Accounting
– using GGF Usage Record standard for interchange, software from MCS project and TeraGrid• Portal
17/1/2005 13
Managing middleware evolution
• Important to coordinate and integrate this with deployment and operations work in EGEE, LCG and similar projects.
• Focus on deployment and operations, NOT development.
UKCampus
andother Grids
OMII
ETF
NGSOther software sources Software with proven
capability & realistic deployment experience
‘Gold’ services
Prototypes &specifications
Feedback & future requirements
EGEE…
Deployment/testing/advice
Operations
17/1/2005 14
Maintaining Compatibility
• Operating a production grid means valuing robustness and reliability over fashion.
• NGS cares about:– alignment/compatibility with leading international Grid efforts– special requirements of UK e-Science community– easy migration/upgrade paths– proven robustness/reliability– based on standards or standards-track specifications
• NGS cannot support everything• Everyone wants service-oriented grids
– but still settling out: WS-I, WS-I+, OGSI, WSRF, GT3, GT4, gLite• Caution over OGSI/WSRF has led to wide convergence on GT2 for
production grids and hence some inter-Grid compatibility– but there are potentially divergent forces at work
• Significant changes to NGS Minimum Software Stack will require approval by NGS Management Board on conservative time scales
17/1/2005 15
Expansion
Adding new resource providers
17/1/2005 16
ExpansionResource providers join the NGS by• Defining level of service commitments through
SLDs• Adopting NGS acceptable use and security policies• Run compatible middleware
– as defined by NGS Minimum Software Stack– and verified by compliance test suite
• Support monitoring and accounting
Two levels of membership1. Affiliation
• a.k.a. connect to NGS2. Partnership
17/1/2005 17
AffiliationAffiliates commit to:• running NGS-compatible middleware
– as defined in NGS Minimum Software Stack– this means users of affiliate’s resources can access these using
same client tools they use to access NGS• a well-defined level of service and problem referral
mechanisms– SLD approved by NGS Management Board and published on
NGS web-site• providing technical, administrative, and security (CERT)
contacts• providing an account and mapping for daily compliance
tests (GITS++)• accepting UK e-Science certificates• maintaining baseline of logs to assist problem resolution
17/1/2005 18
PartnershipPartners commit to same as affiliates, plus:• making “significant” resources available to NGS users
– creation of accounts/mappings– in future, VO support, pool accounts, etc
• recognise additional CAs with which UK e-Science programme has reciprocal agreements
• publish/provide additional information to support resource discovery, brokering
• ability to compile code for computational resources
Bristol and Cardiff going through certification process now.• supported by “buddies” and NGS-Rollout list• useful feedback on viability of NGS Minimum Software
Stack
17/1/2005 19
BenefitsAffiliation:• NGS brand
– certified NGS-compatible• better integrated support for local users who also access
NGS facilities• assistance/advice in maintaining NGS-compatibility over
time
Partnership:• higher brand quality• representation on NGS management board
– either direct, or through regional or functional consortia
NGS brand must be valuable to make this work.
17/1/2005 20
Recent Developments and Summary
17/1/2005 21
Recent Developments
• NGS newsletter• OMII helpdesk• Resource Broker (LCG) available for early adopters• TeraGrid
– UK certificates accepted on TeraGrid– Looking at INCA monitoring with Jenny Schopf
• Trial of accounting software from MCS project– using GGF Usage Record draft standard (as EGEE)
• Meeting with Open Science Grid’s iGOC• Collecting user exemplars
– Agreed form of acknowledgement• Workshop towards an NGS user induction course. Need:
– Easier access (e.g. user interface machines and portals)– T-infrastructure
17/1/2005 22
Summary
TODAY• 4 core nodes operational• 150 Users registered (50 since 1 September ’04)• Grid enabled – Globus v2 (VDT distribution v1.2) at present• BDII information service (GLUE + MDS Schemas)• Data Services – SRB, Oracle, OGSA-DAI• Growing base of user applications• MyProxy and CA services• VO Management Software – LCG-VO• User support: Helpdesk
Next…• Other Middleware [gLite/OMII etc…]• NGS Portal• Resource Broker• SRB production service• Accounting• Continued expansion
Recommended