The GRIDCC Project: providing a real-time GRID for distributed instrumentation F. Asnicar 1, L. Del Cano 1, G. Maron 2, R.Pugliese 1, C.Scafuri 1 on behalf of the GRIDCC Collaboration 1 Sincrotrone Trieste ELETTRA, Trieste, Italy, 2 INFN, Legnaro, Italy ICALEPCS'05 Geneva, Switzerland, October 2005 ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Outline Introduction The GRIDCC project Pilot Applications The GRIDCC architecture The Instrument Element The Execution Services The Multipurpose Collaborative Environment Status of the Art Requirements and first design choices The Work ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Project goals... the GRIDCC project extends the state of the art of computing Grid technologies, by introducing the handling of real-time constraints and interactive response into the existing Grid middleware build a widely distributed system that is able to remotely control and monitor complex instrumentation these new applications introduce requirements for real-time and highly interactive operation of GRID resources. ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Project goals (cont.) verify the feasibility of a Grid-based remote control of systems requiring real-time response with real applications running on existing Grid test beds over both national and international network infrastructures (e.g. GEANT). integrate a grid of instruments into existing Grid infrastructures that provide the computational power and storage needed for the applications ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Project in 3 steps Development of generic Grid middleware, based on existing building blocks (Grid Services) which will allow the remote control and monitoring instrumentation such as distributed systems. Testing of the middleware on challenging applications to validate it both in terms of functionality and quality of service: European Power Grid Geo-hazards Remote Operation of an Accelerator Facility High Energy Physics Experiment Dissemination of the new software technology to encourage a wide range of enterprises to evaluate and adopt our Grid-oriented approach to real-time control and monitoring of remote instrumentation. ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese GRIDCC project landscape Supporting Services Virtual Ctrl. Room Virtual Ctrl. Room Diagnostics Instrument 1 Instrument 2 Instrument 3 Use of the Grid technology, as extension of the Web Service Technologies, to develop a widely distributed control system with access to grid enabled computing and data storage facilities StorageElement ComputingElement ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Participants Participant nameCountry Istituto Nazionale di Fisica NucleareItaly Institute Of Accelerating Systems and ApplicationsGreece Brunel UniversityUK Consorzio Interuniversitario per TelecomunicazioniItaly Sincrotrone Trieste S.C.P.A (Elettra)Italy IBM (Haifa Research Lab)Israel Imperial College of Science, Technology & MedicineUK Istituto di Metodologie per lAnalisi ambientale Consiglio Nazionale delle Ricerche Italy Universita degli Studi di UdineItaly Greek Research and Technology Network S.A.Greece ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Pilot Application: Power Grid In electrical utility networks (or power grids), the introduction of very large numbers of embedded power generators often using renewable energy sources, creates a severe challenge for utility companies. GRIDCC technology would allow the generators to participate in a Virtual Organization, and consequently to be monitored and scheduled in a cost-effective manner. The test bed (by means of computer simulation and emulation) will demonstrate the performance of the emulat system under various conditions, ranging from light power system loading (where energy economics is most important) to power system emergency conditions (where overloaded power circuits necessitate co- ordinated generator control to avoid power black-outs). ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Pilot application: Geo-hazards Remote Operation of Geophysical Monitoring Network Objective: To enable real-time processing of data acquired by heterogeneous (multi- parametric, distributed, remote or unmanned, etc.) geophysical network exploiting GRID capabilities. Monitoring of fluid and gas migration processes in volcanic areas Monitoring of diffusion processes of contaminant plumes and the study of groundwater circulation system in landslide bodies. ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Pilot Application: High-Energy Physics: control and monitor of experiments The so-called Run Control and Detector Control Systems of the experiment are charged with supervising the full configuration of the detector, but also with monitoring the data read out, their analysis and on-line interpretation. Monitoring this detector, and potentially changing settings as a result of analysis on the monitoring data, is a complex task shared by a few hundred people distributed in geographically distributed laboratories. This task requires continuous analysis and display of large amounts of data generated by the detector and in the past was done in a counting room near to the detector. In the context of GRIDCC, this application will be made to run in a completely distributed fashion, over the Grid. ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Pilot Application: Far Remote Operations of ELETTRA ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Pilot Application: Far Remote Operations of ELETTRA Far remote operation of an accelerator facility (i.e. the ELETTRA Synchrotron Radiation Facility) involves: maintenance of the accelerator and its trouble shooting, the repair of delicate equipment understanding and pushing performance limitations performing commissioning and set ups routine operations All these activities are based on large amounts of information, which are at present accessible only at the accelerator site. ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese GRIDCC Architecture IE VCR IE ExeS IE CE IE SE IMS SecS PS VCR WfMS WMS AgrS AutS TGS PolR VIGS ACM IMDM IMSpx RS LPS CollabS ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese The role of the Instrument Element ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese The Architecture of the Instrument Element ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese The Execution Service Arhitecture ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese An ExampleWorkflow: 1BM ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Multipurpose Collaborative Environment The MCE is a groupware which provides general purpose services to control remote instrumentation, manage experimental activity. The system will be used to implement the VCR for the different applications of the GRIDCC project through customization and integration with application specific services. The MCE will be based on a core groupware application (e.g., authentication, management of the VO users and instruments, monitor of the instrument status, job control, etc) and a set of plug-ins general (e.g., chat, file browser, notebook, video conference) specific to the particular application (e.g., accelerator control, instrument control) ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese MCE General Requirements Support for general distance collaboration and, at the same time, access to remote control and monitoring of scientific instrumentation Support of different classes of users: MCE users, MCE administrators, MCE grid service providers Modularity and flexibility: different VCR application support MCE will mainly rely upon web-based interfaces allowing users to access its resources at any location through a standard web-browser Partial support of PDA and mobile devices ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese What is a Collaboratory? A laboratory without walls in particular, the core capabilities that constitute a collaboratory can be seen as technologies to link: People to people (e.g., electronic mail, and tools for data conferencing, such as VRVS) People to information (e.g., the World Wide Web and digital libraries) People to facilities (e.g., status of remote instruments) to enhance utilization by expanding access to resources ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese State of the Art: HCI perspective Collaboration Tools Communication Support Systems Shared Workspaces and Applications Electronic Notebooks Meeting and Decision Support Systems Remote access and Control Systems ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese State of the Art: technological and architectural perspective Collaboration Portals GRID Information Portal GRID Application Portal Portal Framework Collaboration Tools Emerging Standards (JSR-168, WSRP,) ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese First design choices ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese The Work ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese The Work ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese The Work ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese The Work ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese The Work ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese More info on ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese What is a Collaboratory? The term collaboratory was coined by William Wulf by merging the words collaboration and laboratory, and defined as ... Centre without walls, in which researchers can perform their research without regard to geographical location - interacting with colleagues, accessing instrumentation, sharing data and computational resource, and accessing information in digital libraries. ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Reporting I ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Project Timing 1 23Years ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese Human Resources IASABRNL 3 CNIT 4 ELETTRA 5 IBM 6 Imperial 7 INFN 1 IMAA 8 UNIUD 9 GRNET 10 TOTAL PARTNER S Research/innovation activities WP1: System Architecture WP2: Real time Web Service WP3: Grid Enabled Instrumentation WP4: Real-time workload149 WP5: Cooperative Env WP6: System integ. & deply WP7: Dissemination Total research/innovation Consortium management activities WP8: Management Total consortium management ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese State of the Art: Technical and Architectural Perspective PRODUCTTYPETECHNOLOGY CHEFCollaboration PortalJakarta Jetspeed Global-MMCSCollaboration PortalOpenH323, NIST SIP stack, VIC and RAT, Narada Brokering, Java Media Framework. Jetspeed, Java Applets, ActiveX, Apache Batik ElettraVirtual Collaboratory (EVC)Collaboration Portalweb / tomcat / struts / webmin / vnc / axis Upper Atmospheric Research Collaboratory (UARC) Collaboration Portal1st version: NeXTStep 2nd version: Java Applets Pervasive Collaborative Computing Environment (PCCE) Collaboration PortalOld: Modified IRC server. PCCE server. Java Swing User Interface New: Based on Jabber XMPP protocols, BPEL4WS and grid services Space Physics and Aeronomy Research Collaboratory (SPARC) Collaboration PortalBuilt as a thin client (e.g. HTTP), migrating toward CHEF-based system (e.g. portlet technology) Core2000Collaboration PortalWeb technology. Mbone tools GENIUSGRID Application PortalEnginFrame: Apache, HTTPS, Java, XML, rfb (remote frame buffer) GridIce, tightVNC OGCEGRID Application PortalJetspeed, CHEF, OGSA/OGSI, Globus CoG, GridSphere. MyProxy, GridFTP, NaradaBroker, Xmessages, Java JMS HotPageGRID Information PortalGP-IR (XML-centric GridPort Information Repository Database), SOAP::Lite (Perl) InfoPortalGRID Information PortalPerl/CGI Web technology. Globus Monitoring and Discovery Service, XML database. PHP-Nuke ICALEPCS'05 Geneva, Switzerland, October Roberto Pugliese State of the Art: Techical and Architectural Perspective PRODUCTTYPETECHNOLOGY LCPortalGRID Application PortalGridPort, MyProxy, Globus. GridSphere GRBGRID Application PortalPerl/CGI Web technology. C API to Globus. GridSphere GridPortGRID Application PortalPerl/CGI Web technology, Globus JetSpeed PPDGGRID Application PortalsVDT (Virtual Data Toolkit): Globus toolkit, Condor-G, SRM, SAM, GridFTP, JIM, Public Key Infrastructure (X.509 certificates), MonaLisa GPDKPortal FrameworkJava: Servlets, JSP GridSpherePortal FrameworkPortlets, standards, OGSA. Starts from Websphere, Jetspeed, GPDK, role based access control, Castor JDO JSR-168 standard JetSpeedPortal FrameworkJava: Applets, Servlets, JSP. XML JSR-168 standard AccessGridCollaboration toolsPython, SoapPy, OpenSSL, Globus Toolkit VRVSCollaboration toolsMbone tools (VIC & RAT). H.323 Java J2EE + XML PCCECollaboration toolsJabber Instant Messenger Collaboratory for Macromolecular Crystallography Collaboration toolsNX Industry software Molecular Interactive Collaborative Environment (MICE) Collaboration toolsStandalone application. Java, CORBA, ICE (Interactive Collaborative Environment)