Можливості співпраці з PLGrid (Co-operation possibilities with PLGrid)

CGW’12, Krakow, Poland, October 22-24, 2012 http://www.plgrid.pl/

ЗМІСТ

• Консорціум PL-Grid • Базові показники поточного проекту PLGrid Plus • Розроблені сервіси і наукові дослідження, що

виконуються• Як можна побудувати співпрацю?

Consortium and the PL-Grid Project• ACC CYFRONET AGH – Academic

Computer Centre Cyfronet AGH, Krakow (coordinator)

• ICM UW – Interdisciplinary Centre for Mathematical and Computational Modelling, Warsaw University, Warsaw

• PSNC – Poznan Supercomputing and Networking Center, Poznan

• TASK – Academic Computer Centre, Gdansk

• WCNS – Wroclaw Centre for Networking

and Supercomputing, Wroclaw. Consortium created (Jan. 2007) with goal:

Significant extension of amount of computing resources provided to the Polish scientific community

Consortium and the PL-Grid Project

The main aim of the PLGrid Plus project is to increase potential of the Polish Science by providing the necessary IT services for research teams in Poland, in line with European solutions.

Consortium and the PL-Grid ProjectRank Site System Cores Rmax

TFlop/s Rpeak TFlop/s

88 Cyfronet Zeus - Cluster Platform 3000 BL 2x220, Xeon X5650 6C 2.66 GHz, Infiniband, HP

15264 128.8 162.4

279 Gdansk CI Task Galera Plus - ACTION Xeon HP BL2x220/BL490 E5345/L5640 Infiniband, ACTION

10384 65.6 97.8

296 ICM Warsaw Boreas - Power 775, POWER7 8C 3.84 GHz, Custom, IBM

2560 64.3 78.6

298 PCSS Rackable C1103-G15, Opteron 6234 12C 2.40 GHz, Infiniband QDR, SGI

5640 63.9 136.4

348 Grupa Allegro Cluster Platform 3000 BL 2x220, Xeon L5420 4C 2.50 GHz, Gigabit Ethernet, HP

10748 59.1 107.5

360 WCSS Supernova - Cluster Platform 3000 BL2x220, X56xx 2.66 Ghz, Infiniband, HP

Resources: 230 Tflops, 3600 TB users: 1000+ jobs/month: 500,000-1,500,000

6348 57.4 67.5

Zeus once again high on the TOP500 list

• In the November 2011 edition of the TOP500 list of the most powerful computers in the world, the Zeus cluster from ACC Cyfronet AGH took a high, 88 place, thereby maintaining the first place in Poland.

• At present, Zeus operates in the following configuration:

• 162,41 Tflops of theoretical computing power (actual power: 128,79 Tflops),

• 22 TB (terabytes) of RAM,• memory storage capacity of 1.8 PB

(petabytes).• Zeus is using the Scientific Linux (SL)

operating system and is assigned to the scientific computations.

Project PLGrid Plus

• Budget: total ca. 18 M€ including funding from the EC: ca.15 M€ • Duration: 1.10.2011 – 30.9.2014 • Five PL-Grid Consortium Partners; Coordinator: Academic Computer

Centre CYFRONET AGH • Preparation of specific computing environments, i.e. solutions, services

and extended infrastructure (including software), tailored to the needs of different groups of scientists.

• These domain-specific solutions will be created for already identified 13 groups of users, representing the strategic areas and important topics for the Polish and international science:

• AstroGrid-PL, HEPGrid, Nanotechnologies, Acoustics, Life Science, Chemistry and Physics, Ecology, SynchroGrid, Energetics, Bioinformatics, Health, Materials, and Metallurgy.

Project PLGrid Plus

Access to the software packages is provided by:

Glite Unicore QCG

New Services in PLGrid Plus

• Cloud Computing for Polish Science – new computing paradigm foreseen as a natural extension of the current Infrastructure offer

• Platform for supporting e-Science, resulting from the need for an international cooperation between various disciplines of scientific domains

• Production infrastructure oriented towards domain specific services, tools, environments and software packages

• Professional support for specific disciplines and topics important for Polish e-Science

• Visualisation of the scientific results via shared infrastructure servers equipped with possibility of binding domain specific visualisation tools

Activities in Domain Grids in general

Integration Services -National and International levels -Dedicated Portals and Environments -Unification of distributed Databases -Virtual Laboratiories -Remote Visualization -Service value = utility + warranty -SLA management

Computing Intensive Solutions -Specific Computing Environments -Adoption of suitable algorithms and solutions -Workflows -Cloud computing -Porting Scientific Packages

Data Intensive Computing -Access to distributed Scientific Databases -Organization of Scientific Databases -Data discovery, process, visualization, validation…. -4th Paradigm of scientific research

Instruments in Grid -Remote Transparent Access to instruments -Sensor networks

Organizational -Organizational backbone -Professional support for specific disciplines and topics

Innovative Infrastructure Environment PL-Grid extensions

• Efficient Resource Allocation – Grid Resource Bazaar, mobile access to the infrastructure, new security

modules and other tools for users and systems administrators, management of users request

• Experimental Workbenchs – GridSpace2 platform extension for supporting for new domains and integration

with new grid/cloud services – InSilicoLab – integrated environment for chemists and biologists

• Tools and Middleware – Migrating Desktop, VineToolkit and gEclipse tools integration with various PL-

Grid domain services – QStorMan Toolkit – extension for domain requirements on optimization of

data access – QosCosGrid continuation of development – Liferay Portal framework(s) – adoption to specific needs – HelpDesk Portal for the users (specialized versions)

QCG-Icon

• QCG-Icon is a program written specifically for the Windows platform, designed to enable the selected applications installed on the computing resources of the PL-Grid project, and available through the QosCosGrid services. In developing the program special emphasis was put on the following fact: using an application installed in the grid environment should be as intuitive as using a locally installed application.

• At the moment, using the QCG-Icon it is possible to run scripts written in MATLAB language, in R language and the simulations of the NAMD software. In the future, more applications will be made available.

• To run the QCG-Icon, the Java environment - at least in version 1.6 (JRE 6) - is required. Additionally, it is necessary to gain access to the QCG infrastructure in the PL-Grid project.

The Key File System (KeyFS) software

• The Key File System (KeyFS) software, installed on UI (User Interface) machines in the PL-Grid infrastructure, can provide a user with the grid credentials (private key and user certificate) immediately after registering the user in the portal and generation of the Simple CA certificate.

• In this case, the new user does not need to perform the entire sequence of actions associated with the collection of portal PKCS12 package (copying PKCS12 to the selected UI, conversion to a format acceptable by the grid tools, ensuring appropriate access rights, etc.). KeyFS will provide the files, including certificate and private key, to the user's UI, a moment after registering an account in the PL-Grid portal.

Vine Toolkit has been deployed on the PL-Grid infrastructure

• Vine Toolkit is a set of the portal tools allowing to create web applications, which enable remote running and controlling the computer simulations. Vine Toolkit also allows to secure control of access to users' files and data.

• On 4.10.2011 Vine Toolkit has been deployed on the PL-Grid infrastructure. The software is available at the http://vinetoolkit.org/ web site.

Migrating Desktop

• Migrating Desktop is a user friendly graphical interface that allows easy use of PL-Grid project's computing resources.

• Provided environment is not dependent on the operating system and hardware. Migrating Desktop enables execution of applications and tools, file management, personalization of the environment.

• On 19.05.2011 Migrating Desktop has been deployed on the PL-Grid infrastructure. The software is available at the http://desktop.plgrid.pl web site.

GridSpace2 Virtual Laboratory

• GridSpace2 is a novel virtual laboratory framework enabling researchers to conduct virtual experiments on grid-based resources.

• It is a set of integrated components, which enable combining distributed computing resources and data sources into single applications. Within the Virtual Laboratory we provide an integrated environment for the construction, use and presentation of applications by combining modules, components, services, libraries or other software packages available in the PL-Grid.

• The service is available at https://gs2.plgrid.pl

g-Eclipse has been deployed on the PL-Grid infrastructure

• g-Eclipse is a platform which offers tools to customize grid users applications, to manage remote resources and to support the development cycle of new grid applications.

• PL-Grid users can benefit from a g-Eclipse user-friendly graphical interface designed for accessing grid infrastructure and integrate their applications with g-Eclipse to hide the grid heterogeneity and make usage of PL-Grid resources simple.

• The software is available in the PL-Grid Software Repository.

Book : New York : Springer, 2012

• This state-of-the-art survey describes the experience and the scientific results obtained by project partners as well as the outcome of research and development activities carried out within the Polish Infrastructure for Information Science Support .

• The 26 papers are organized in topical sections on: eclipse parallel tools platform integrated with QosCosGrid, the migrating desktop, science gateways based on the vine toolkit, the gridspace experiment platform, and the InSilico-Lab environment.

Fits to European e-Infrastructure Plans

HEP

Acoustics Multimedia Noise Monitoring System and Simulation

Monitoring station

Terrain model

Database Noise Impact on Hearing

Noise MapSound propagation modeling

Nanotechnology Application structure

Prototype

AuxEx server

Database Client session

KDM Auxiliar Experimentorum

Synchrogrid

• Synchrogrid builds services for the synchrotron radiation scientific community • It is expected that the establishment of the Research Centre of Polish Synchrotron

(2014) will influence the rapid rise in the users of synchrotron radiation to 1,000 people

• Involved institutions: – Jerzy Haber Institute of Catalysis and Surface Chemistry Polish Academy of Sciences – AGH University of Science and Technology – Jagiellonian University – University of Silesia in Katowice – Adam Mickiewicz University

• Launching a synchrotron makes its radiation easily accessible • The world witnessed the rapid increase of the number of scientists using

synchrotron radiation when the first synchrotron was opened in their country

Bioinformatics Processing of DNA sequencing data

• High-throughput GS FLX Instrument (Roche Diagnostics)

• UNICORE Commandline Client (UFTP)

• Target System Storage (PL-Grid) – data storage

• Target System (PL-Grid) – workflow execution

Health Processing of medical images

• UNICORE Rich Client • UNICORE Command line

Client (UFTP) • Web access • UNICORE Storage (PL-Grid)

– secure storage for medical images

• Target System (PL-Grid): – dedicated software to

process medical images

An example result of the matching procedure visualized via volume rendering. Red image represents reference dataset that is transformed in the registration process, while the green image is the current object under examination.

Metallurgy

▪ Creation of Grid Service, which supports creation of Statistically Similar Represen-tative Volume Element (SSRVE) by parallelization of optimization procedure allowing massive parallel calculations on grid infrastructure by parallelization of optimization procedure allowing massive parallel calculations on grid infrastructure ▪ Support of engineers in demanding calculations for optimization of the metallurgical process of profiles extrusion, realized by Implementation of parallel parts of Finite Element Method ▪ Material engineers support in numerical simulations of static recrystallization by using Monte Carlo approach.

Ecology KIWI Eye (1.0) observation set

A set for automatic photo observations

Functionality: • Remote control of the

camera • Remote viewing of the

video from the camera • High resolution photos

(DSLR type camera with interchangeable lenses)

• Scene and measuring points definition

• Workflow-type management software and remote access

AstroGrid-PL • The Virtual Observatory (VObs) is an

international astronomical community-based initiative aiming to provide transparent and distributed access to data available worldwide. The VObs consists of a collection of data centers each with unique collections of astronomical data, software systems and processing capabilities.

• Service goals: – Setup National VObs Data Center,

integration of Polish data and join international efforts

• Workflow Environment – Provide astronomers with workflow

environment – Enable easy resource switching: desktop –

cluster – grid – Library of template scenarios for popular

activities – Dedicated support for selected projects – Kepler environment is being tested...

Ukraine at CGW-12

Generated proposals

• gSLM and FedSM Projects (UK): “Workflow as service orchestration of web- and grid services in Service-

oriented e-infrastructure architecture based on WS-BPEL standards and tools”

• EUDAT Project (GE): “Multidisciplinary parallel access to heterogeneous distributed relational

databases based on their semantic wrapping“• PLGrid Plus Project (Poland): “Large-Scale Computing Techniques for Complex System Simulations”• VENUS-С Project (Geneva):"Interdisciplinary complex of optimal mathematical modeling in cloud

environment".

BOOK

Researchers who do not have Polish citizenship, should first contact the right person from the university or research institute in Poland, they cooperate with, and with his/her help go through the registration process.

Table of contents

• 1.PL-Grid: Foundations and Perspectives of National Computing Infrastructure

• 2. Integrating Various Grid Middleware Components and User Services into a Single Platform3. Towards Professional Service Operations in Grids

• 4. New Capabilities in QosCosGrid Middleware for Advanced Job Management, Advance Reservation and Co-allocation of Computing Resources – Quantum Chemistry Application Use Case

• 5. Seamless Access to the PL-Grid e-Infrastructure Using UNICORE Middleware

• 6. User-Oriented Provisioning of Secure Virtualized Infrastructure• 7. Flexible and Extendable Mechanism Enabling Secure Access to e-

Infrastructures and Storage of Confidential Data for the GridSpace2 Virtual Laboratory.

Table of contents

• 8. SARA – System for Inventory and Static Security Control in a Grid Infrastructure

• 9. ACARM-ng: Next Generation Correlation Framework• 10. Security Best Practices: Applying Defense-in-Depth Strategy to

Protect the NGI_PL• 11. Automation of System Monitoring Based on Fuzzy Logic or Rules;

Comparison of Two Designed Approaches with Regard to Computational Infrastructures

• 12. A Toolkit for Storage QoS Provisioning for Data-Intensive Applications

• 13. Implementation of Service Level Management in PL-Grid Infrastructure

• 14. Highly Integrated Environment for Parallel Application Development Using QosCosGrid Middleware

Table of contents

• 15. User-Friendly Frameworks for Accessing Computational Resources Online Web-Based Science Gateway for Nanotechnology Research

• 16. Scripting Language Extensions Offered by the GridSpace Experiment Platform

• 17. Managing Entire Lifecycles of e-Science Applications in the GridSpace2 Virtual Laboratory – From Motivation through Idea to Operable Web-Accessible Environment Built on Top of PL-Grid e-Infrastructure

• 18. GridSpace2 Virtual Laboratory Case Study: Implementation of Algorithms for Quantitative Analysis of Grain Morphology in Self-assembled Hexagonal Lattices According to the Hillebrand Method

• 19. Examining Protein Folding Process Simulation and Searching for Common Structure Motifs in a Protein Family as Experiments in the GridSpace2 Virtual Laboratory

Table of Contents

• 21. InSilicoLab – Managing Complexity of Chemistry Computations 22. Ab Initio Molecular Dynamics Simulations of Ketocyanine Dyes in

Organic Solvents23. Polish Contribution to the Worldwide LHC Computing24. PL-Grid e-Infrastructure for the Cherenkov Telescope Array Observatory25. Training in the PL-Grid as Key Component to Attract Users to Grid e-Infrastructures, 26. Dissemination Activities Conducted within the PL-Grid Project as a Successful Means of Promotion of Its Offer and Achievements

матеріал розміщено на 3 мовах в новому підрозділі "іноземні публікації"http://grid.kpi.ua/index.php/uk/foreign-publications/55-pl-grid.html

Software

Software

Software