Upload
donna-tran
View
42
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Manuel Rubio del Solar CETA-CIEMAT EELA Tutorial, Mérida, 07-10.11.2006. gLite Information System(s). This presentation is based on LCG and gLite manuals and previous EELA tutorials. Outline. Grid Information Systems Overview gLite Information Systems GLUE Schema Grid elements - PowerPoint PPT Presentation
Citation preview
E-infrastructure shared between Europe and Latin America
gLite Information System(s)
Manuel Rubio del SolarCETA-CIEMATEELA Tutorial, Mérida, 07-10.11.2006
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
This presentation is based on LCG and gLite manuals and previous EELA tutorials
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
Outline
• Grid Information Systems Overview• gLite Information Systems• GLUE Schema
– Grid elements
• LCG Information System– Hierarchy– LDAP model
• R-GMA– Overview
– Server Architecture– Consumer-Producer model
– User interfaces
• References
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
Grid Information Systems Overview
• Collect information of grid resources– Discovering new added resources – Monitoring load and health status
• Publish these information– Periodically updated– Well know data model
• Used by– Users searching a concrete resource– WMS allocating and managing jobs– Other monitoring services
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
gLite Information Systems
• Basic data model.– Grid Laboratory Uniform Environment (GLUE) Schema.
• Two architectures in glite3– LCG Information System
BDII over Globus MDS. OpenLDAP interface. Preserves LCG-2 compatibility.
– Relational Grid Monitoring Architecture (R-GMA) Based on the GMA standard from the Grid Global Forum Information in SQL relational databases Web Services. Future replacement of LCG IS.
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
GLUE Schema
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
GLUE Schema
• A schema of objects and attributes describing Grid resources and its relationships.– Originally a EU-DataTAG and US-iVDGL coordinated effort. – Current participants: EGEE, OSG, Globus and NorduGrid. – A way to describe Grid info
Statically and dynamically supplied Hierarchically represented Independently of the framework (LDAP, XML, SQL…)
• Actual release (1.2) is mapped into– LDAP– XML– Old ClassAd (used by Condor Matchmaking)
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
Site
A collection of resources owned by the same organization and managed by the same administrator. Contains info on the location, the administrator, the web homepage and so on.
Service
The description of a deployed Web Service. Contains the URI endpoint of the WS, the WSDL document, the list owners and so on.
StorageElement Cluster
1 1 1
*
*
*
Site Element
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
Cluster
A set of heterogeneous resources. Contains information on shared temporary directories.
SubCluster
A set of similar resources. Contains the number of Logical and Physical CPUs.
1
*
Host
Contains detailed static information of the type of hosts and related installed software. Data deal with the type of CPU architecture, memory sizes, the operating system installed as well as the type of network adapter. Furthermore it contains some information on performance mesures obtained by executing well known benchmark softwares.
Location
Information on installed softwares, their path and version
1
*
ComputingElement
1
*
Cluster Element
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
ComputingElement
Abstraction of a queue of jobs
Policy
Contains info on configuration policies. MaxWallClockTime, MaxRunningJobs, MaxCPUTime . . .
AccessControlPolicyBase
Set of rules defining access control policy rules
Info
Static information on the resource that deal with the type of Loca scheduler adopted, the default Storage Element and so on.
VOview
View for a given Virtual Organization. Contains authorization details for VO members and the amount of available resources.
State
Dynamic information on the status of this queue such as the number of free CPUs and the Estimated Traversal Time (ETT)
Job
Information on jobs in this queue, its owner, its local and global ID and its status
*
*
Computing Element
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
LCG Information System
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
LCG Information System Levels
• Resource level: Grid Resource Information Server (GRIS)– One GRIS on top of each CE, SE, RB, MyProxy (no WNs).
– Sensors and scripts get status of concrete resources statically (e.g. GlueCEUniqueID) or dynamically (e.g. GlueCEStateWaitingJobs)
• Site level: Grid Index Information Server (GIIS) – Compiles all the information of the different GRISes in a site.
– LCG-2 recommends using a BDII instead of a GIIS Improves robustness and stability. Called the site BDII.
• Top level: Berkeley DB Information Index (BDII) – Keeps all Grid information about the VOs (generally only one).
– Stores information from local BDIIs or GIISes in its database.
– Only queries sites that are included in a configuration file.
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
LCG Information System Hierarchy
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
A LDAP Model
• Way of collecting info– Pull model (higher level servers periodically query lower level
servers)– All servers are based on LDAP
Inherit hierarchical structure (tree-like) LDAP Data Information Format (LDIF)
• Users get info with– Generic applications
ldapsearch (MDS: 2135 and BDII:2170 ports) Graphical (BDII web; LDAP GUIs) Always can get information about specific resources (maybe more
up-to-date) by querying directly the site BDIIs, GIISes or GRISes.
– Querying VO info with lcg-infosites or lcg-info
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
LDAP Browser showing BDII Info
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
Relational-GMA
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
R-GMA Overview
• Added from EDG Project• Based on the GMA standard from the GGF• Information in SQL relational databases (a DB per VO)• Query syntax is a SQL subset• Simple consumer-producer model• Web Services oriented• CLI and Web user interface• Allows self-logging applications• Future replacement of LCG IS
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
RGMA Overview
• R-GMA offers a global view of the VO information– In one large relational DB: virtual database.– Registry stores localization tuples (database rows) published by
producers: Standard Tables: CE state in GLUE Schema (by R-GMA-GIN) Applications specific tables (e.g. self-logging with Log4j) Access by SQL queries through a WS interface.
– Always updated Consumers get information directly from the resource Can be cached by a secondary producer from single o multiple
producers to improve access.
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
R-GMA Server Architecture
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
R-GMA Server Description
• R-GMA Servers– Server (one or several) – Schema Server (There is one common global schema server for the grid)– Registry Server (one or several) – Browser (shows grid information via web browser)
• R-GMA Clients– Generic Client (generic set of APIs for different languages and CLIs) – Servicetool (used by all services to publish its existence and status)– Site Publisher
Client to publish the existence of a site. Each site must have one R-GMA site publisher.
– GadgetIN (GIN) Client to extract information from MDS and to republish it to R-GMA. used by the CE.
– Data archiver (flexible archiver) Client to make the data that is coming from the R-GMA site-publisher,
servicetool and GIN constantly available. By default the GLUE tables and service tables are archived, however this can be
configured.
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
R-GMA Servers Per Node Type
– The gLite R-GMA Server should deployed the first as part of a gLite grid, because all services require it to publish service information.
Deployment module Contains Used / included by
R-GMA Server R-GMA server
R-GMA registry server
R-GMA schema server
R-GMA browser
R-GMA site publisher
R-GMA archiver
R-GMA servicetool
MON-box
R-GMA client RGMA client APIs UI, WN, SD
R-GMA servicetool R-GMA servicetool R-GMA server,VOMS, LB, WMS, CE, DC, FTS, FTA, Hydra, I/O-Server,Torque Server
R-GMA GIN R-GMA GadgetIN CE
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
Registry
Producer Consumer
Store
Loc
atio
n
Look up Location
Execute or Stream data
name ID birth Group
SELECT * FROM people WHERE group=‘HR’
Tom 4 1977-08-20
HR
R-GMA Producer- Consumer model
• The Producer stores its location (URL) in the Registry.
• The Consumer looks up producer URLs in the Registry.
• The Consumer contacts the Producer to get all the data.
• Data passes directly from the producer to the consumer: (does not through the registry)
• The user or any user application does not need to know the registry; this will be handled directly by the consumers and producers.
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
Consumer
Producer 1
Registry
TableName
Value 1 Value2
Value 3 Value 4
TableName
Value 1 Value 2
TableName URL 1
TableName URL 2
• The Consumer will get all the URLs that could satisfy the query.
• The Consumer will connect to all the Producers.
• Producers that can satisfy the query will send the tuples to the Consumer.
• The Consumer will merge these tuples to form one result set.
Producer 2TableName
Value 3 Value 4
Multiple Producers
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
CPULoad (Producer 3)
CH CERN ATLAS 1.6 19055611022002
CH CERN CDF 0.6 19055511022002
CPULoad (Producer 1)
UK RAL CDF 0.3 19055711022002
UK RAL ATLAS 1.6 19055611022002
CPULoad (Producer 2)
UK GLA CDF 0.4 19055811022002
UK GLA ALICE 0.5 19055611022002
CPULoad (Consumer)
Country Site Facility Load Timestamp
UK RAL CDF 0.3 19055711022002
UK RAL ATLAS 1.6 19055611022002
UK GLA CDF 0.4 19055811022002
UK GLA ALICE 0.5 19055611022002
CH CERN ALICE 0.9 19055611022002
CH CERN CDF 0.6 19055511022002
Select * from CPULoad
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
ServiceURI VO type emailContact sitegppse01 alice SE [email protected] RAL
gppse01 atlas SE [email protected] RAL
gppse02 cms SE [email protected] RAL
lxshare0404 alice SE [email protected] CERN
lxshare0404 atlas SE [email protected] CERN
ServiceStatusURI VO type up statusgppse01 alice SE y SE is running
gppse01 atlas SE y SE is running
gppse02 cms SE n SE ERROR 101
lxshare0404 alice SE y SE is running
lxshare0404 atlas SE y SE is running
Result Set (Consumer)
URI emailContact
gppse02 [email protected]
SELECT Service.URI Service.emailContact FROM Service S, ServiceStatus SS WHERE (S.URI= SS.URI and SS.up=‘n’)
Joins
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
R-GMA user interfaces
• R-GMA CLI
• Web interface
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
References
• GLUE Schema– http://glueschema.forge.cnaf.infn.it/
• LCG-2 User Guide– https://edms.cern.ch/file/454439//LCG-2-UserGuide.html
• gLite 3.0 User Guide– https://edms.cern.ch/file/722398/1.1/gLite-3-UserGuide.pdf
• R-GMA home page– http://www.r-gma.org/
• R-GMA in EGEE– http://hepunx.rl.ac.uk/egee/jra1-uk/
EELA Tutorial, Mérida, 14-16.11.2006
E-infrastructure shared between Europe and Latin America
Questions…