View
28
Download
3
Category
Preview:
DESCRIPTION
GridSphere’s Grid Portlets A Grid Portal Development Framework. Jason Novotny Lawrence Berkeley National Laboratory Jason Novotny, Michael Russell, Oliver Wehrens The GridSphere Project www.gridsphere.org. The Purpose Of This Talk. - PowerPoint PPT Presentation
Citation preview
2004 1Michael Russell <michael.russell@aei.mpg.de>
GridSphere’s Grid PortletsA Grid Portal Development Framework
Jason Novotny
Lawrence Berkeley National Laboratory
Jason Novotny, Michael Russell, Oliver WehrensThe GridSphere Project
www.gridsphere.org
2004 2Michael Russell <michael.russell@aei.mpg.de>
The Purpose Of This Talk
The purpose of this talk is to provide a brief overview of the Grid Portlets web application.
We will describe the basic ideas behind Grid Portlets, what functionality it currently provides and development plans
2004 3Michael Russell <michael.russell@aei.mpg.de>
GridSphere’s Grid Portlets
The GridSphere portlet container is designed to be web application independent. Indeed, one of the key advantages of the Portlet API is the reuse of web applications.Thus, the GridSphere portlet container does not contain any support for using Grid technologies.Instead, GridSphere’s Grid related functionality is contained in a web application we call Grid Portlets.Grid Portlets, together with the GridSphere portlet container, offers a generic Grid portal and can be used to develop application-specific Grid portal applications.
2004 4Michael Russell <michael.russell@aei.mpg.de>
The GridLab Project
Funded by the EU (5+ M€), January 2002 – December 2004Application and Testbed oriented
Cactus Code, Triana Workflow, all the other applications that want to be Grid-enabled
Main goal: to develop a Grid Application Toolkit (GAT) and set of grid services and tools...:
resource management (GRMS), data management,monitoring, adaptive components, mobile user support, security services,portals,
... and test them on a real testbed with real applications
2004 5Michael Russell <michael.russell@aei.mpg.de>
GridLab Is An Architecture
2004 6Michael Russell <michael.russell@aei.mpg.de>
And A Global Effort
PSNC (Poznan) - coordination AEI (Potsdam) ZIB (Berlin) Univ. of Lecce Cardiff University Vrije Univ. (Amsterdam) SZTAKI (Budapest) Masaryk Univ. (Brno) NTUA (Athens)
Sun MicrosystemsCompaq (HP)
ANL (Chicago, I. Foster) ISI (LA, C.Kesselman) UoWisconsin (M. Livny)
collaborating with:Users!
EU Astrophysics Network,
DFN TiKSL/GriKSL
NSF ASC Project
other Grid projectsGlobus, Condor,
GrADS,
PROGRESS,
GriPhyn/iVDGL,
Most of the other European Grid Projects (GRIDSTART)
GWEN
2004 7Michael Russell <michael.russell@aei.mpg.de>
GridLab Testbed Snapshot
2004 8Michael Russell <michael.russell@aei.mpg.de>
GridLab Goals
Get Computational Scientists using the “Grid” and Grid services for real, everyday, production work (AEI Relativists, EU Network, Grav Wave Data Analysis, Cactus User Community), all the other potential grid appsMake it easier for applications to make flexible, efficient, robust, use of the resources available to their virtual organizationsDream up, prototype, and test new application scenarios which make adaptive, dynamic, wild, and futuristic uses of resources.
2004 9Michael Russell <michael.russell@aei.mpg.de>
What Do Our Users Need?
Application oriented environment
Flexible, easy-to-use, simple interfaces
Efficient and effective use of resources
Robustness, fail-safety, adapability
The ability to work in distributed teams
Support for mobile working environments
2004 10Michael Russell <michael.russell@aei.mpg.de>
What Do Our Users Want?
Larger computational resources Memory/CPU
Faster throughputCleverer scheduling, configurable scheduling, co-scheduling, exploitation of un-used cycles
Easier use of resourcesPortals, grid application frameworks, information services, mobile devices
Remote interaction with simulations and dataNotification, steering, visualization, data management
Collaborative toolsNotification, visualization, video conferencing, portals
Dynamic applications, New scenariosGrid application frameworks connecting to services
2004 11Michael Russell <michael.russell@aei.mpg.de>
Many Application Scenarios!
Dynamic Stagingmove to faster/cheaper/bigger machine
Multiple Universecreate clone to investigate steered parameter
Automatic Convergence Testingfrom initial data or initiated during simulation
Look Aheadspawn off and run coarser resolution to predict likely future
Spawn Independent/Asynchronous Taskssend to cheaper machine, main simulation carries on
Application Profilingbest machine/queuechoose resolution parameters based on queue
Dynamic Load Balancinginhomogeneous loads
multiple grids
Portal User/virtual organisation interface to the grid.
Intelligent Parameter Surveysfarm out to different machines
Make use ofRunning with management tools such as Condor, Entropia, etc.
Scripting thorns (management, launching new jobs, etc)Dynamic use of eg MDS for finding available resources
2004 12Michael Russell <michael.russell@aei.mpg.de>
Where Grid Portals Fit In
GridLab needs an effective way to deliver its tools and services to its target users.
We want to avoid having to develop new client-side technologies and methods for deploying them.
Given the ubiquitous Web browser and the evolution of Grid standards and technologies, a Grid portal provides an obvious solution to this problem.
2004 13Michael Russell <michael.russell@aei.mpg.de>
Developing Grid Portals
Grid web application development still remains a tedious task with little in the way of reusable components, forcing developers to constantly “re-invent” the wheel.
Often difficult and hard to maintain glue code must be written connecting the portal to Grid services, due to lack of/evolving standards.
Lack of real usability has made it difficult to test and evaluate user interfaces.
A portal is only as good as the underlying deployed infrastructure…. portal development often involves debugging underlying middleware
2004 14Michael Russell <michael.russell@aei.mpg.de>
What is a Grid Portal?
“A portal is a web based application that commonly provides personalization, single sign on, content aggregation from different sources and hosts the presentation layer of Information Systems”(JSR 168)
Grid Portals build upon the familiar Web portal model, such as Yahoo or Amazon, to deliver the benefits of Grid computing to virtual communities of users, providing a single access point to Grid services and resources.
2004 15Michael Russell <michael.russell@aei.mpg.de>
Early Grid Portal Projects
Grid-Port: Perl based framework developed by Mary Thomas and Steve Mock at San-Diego Supercomputing Center (SDSC)
Grid Portal Development Toolkit (GPDK):Developed by Jason Novotny at Lawrence Berkeley National Laboratories (LBNL)
Astrophysics Simulation Collaboratory (ASC):Developed by Michael Russell at University of Chicago
2004 16Michael Russell <michael.russell@aei.mpg.de>
Grid Portals Work Package
Within the GridLab Project, the Grid Portals Work Package (WP4) is responsible for developing a Grid portal for the GridLab Project.
The GridSphere Project was created by WP4 as it saw a need in the general community for a Grid portal framework.
WP4 is both developing and deploying GridSphere to serve target user communities in the GridLab Virtual Organization.
2004 17Michael Russell <michael.russell@aei.mpg.de>
GridSphere 2.0
2004 18Michael Russell <michael.russell@aei.mpg.de>
Personalized Environment
2004 19Michael Russell <michael.russell@aei.mpg.de>
Single Sign-On Capabilities
2004 20Michael Russell <michael.russell@aei.mpg.de>
Submit Jobs
2004 21Michael Russell <michael.russell@aei.mpg.de>
Perform File Transfers
2004 22Michael Russell <michael.russell@aei.mpg.de>
Manage Resources
2004 23Michael Russell <michael.russell@aei.mpg.de>
Grid Portal Design Requirements
We require a portal that is generic and not tied to any particular set of Grid technologies. The GridLab Testbed, for example, is Globus 2.X based although integartion with OGSA well under way
Develop a set of portlets and portlet services that provide a higher level model of Grid concepts.The Grid portlet services should provide a generic API to which the Grid portlets are written.
2004 24Michael Russell <michael.russell@aei.mpg.de>
Grid Portlets API
Our Grid Portlets API attempts to provide a high-level model of Grid concepts that
Abstracts the developer from particular Grid technologies and infrastructure.
Is extensible, can build upon its concepts to provide more complex services, resources and tasks.
Supports multiple implementations of the API.
Configurable at run-time via administrative interfaces
The Grid Portlets API builds from basic concepts to more advanced ones…
2004 25Michael Russell <michael.russell@aei.mpg.de>
Basic Concepts
UsersUsers should get a personalized a view of the Grid based upon their access rights to resources and their preferences.
CredentialsCredentials are required to authenticate users to resources and authorize their rights to utilize them.
ResourcesResources are anything that can be “utilized” (by users, portlet services and other resources).
TasksMany operations are quite naturally modeled as tasks. Workflow is not provided yet, but is desired in future releases.
2004 26Michael Russell <michael.russell@aei.mpg.de>
More Advanced Concepts
JobsJob submission to LSF, PBS, etc…
Job migration from one resource to another
Files & Related TasksPhysical file browsing (list, transfer, etc.)
Logical and replica file browsing
Information ResourcesLDAP/MDS information providers
Web service based providers
Other types…
2004 27Michael Russell <michael.russell@aei.mpg.de>
Simple Resource Diagram
2004 28Michael Russell <michael.russell@aei.mpg.de>
Hardware Resource Model
Hardware resources are utilized for performing computations, storing files, etc….Hardware resources contain…
Service resources (such as a Grid FTP resource)Software resources (such as Globus client tools)Hardware accounts (allowing people to login and perform operations in a shell).
2004 29Michael Russell <michael.russell@aei.mpg.de>
Job Resource Model
Job resources (such as Globus Gatekeepers) are service resources that enable the remote clients to submit jobs on computing resources.Job resources provide access to job schedulers (such as PBS).Job schedulers contain queues with different resource capabilities and usage policies.
2004 30Michael Russell <michael.russell@aei.mpg.de>
Some Job Resources
In Grid Portlets, we’ve modeled two types of service resources as job resources, both of which have different capabilities for supporting job submission.GTX Gatekeepers provide the ability to submit jobs directly to job managers running on hardware resources using the GRAM protocol.GridLab Resource Manager Services (GRMS)s are web services that provide resource brokering capabilities and support job submission with GRAM as well as Condor.
2004 31Michael Russell <michael.russell@aei.mpg.de>
Simple Task Diagram
2004 32Michael Russell <michael.russell@aei.mpg.de>
How Portlets Are Developed
In Grid Portlets, the presentation layer is made up of portlets which contain reusable UI components.
The business logic layer is implemented as a collection of portlet services which provide some high-level capabilities and make use of external resources.
2004 33Michael Russell <michael.russell@aei.mpg.de>
Job Submission Portlet
For example, the job submission portlet uses a number of UI components, including file browser components (i.e. for specifying locations of an executable).
These JSP components utilize a number of portlet services which provide access to job resources, file resources and so on…
2004 34Michael Russell <michael.russell@aei.mpg.de>
Current Status
Grid Portlets is still in prototype development mode.
Most of what is their works, but API is still evolving.
Documentation is still poor… Mike promises to improve it
Source code is available via anonymous CVS.
Will move towards a regular development and release cycle in the coming months.
Mike in Brazil ;-) but plans on continuing to enhance grid portlets
2004 35Michael Russell <michael.russell@aei.mpg.de>
Current Functionality
Credential Retrieval & ManagementSupport for retrieval & storing with MyProxy
Resource Registry & ManagementResources can be described in XML and online
Resource Information ProviderResource discovery and update with MDS2
Job Submission & MigrationGRAM job submission with Java CoG 1.1GRMS 1.9 job submission and migration
File Browsing & TransferringSupport for GridFTP with Java CoG 1.1
2004 36Michael Russell <michael.russell@aei.mpg.de>
What’s Next?
Enhancements to UI component model… These enhancements may be incorporated into GridSphere portlet container, but we are still reviewing JSF.
Many enhancements planned to existing portlets, including better integration among portlets and reuse of UI components.
Progress already underway to make JSR 168 compliant
Please submit feature requests and additions to bugtracker and mail list
2004 37Michael Russell <michael.russell@aei.mpg.de>
Planned Development
Support for logical files & replica managementSupport for GridLab Data Management Services
Support for Globus Replica Catalogue
Enhancements to information gatheringPortlets for managing use of information providers
More resource information providersSupport for GridLab Adaptive Information Service
Support for Network Weather Service
2004 38Michael Russell <michael.russell@aei.mpg.de>
How To Obtain Grid Portlets
Grid Portlets can be obtained from CVS….
Required Software:GridSphere 2.0.2 or higher.
Tomcat 4.1.29 or higher (Tomcat 5.X supported)
OGSA 3.0.2 (Plan to move OGSA 3.2)
MyProxy
cvs -d :pserver:anonymous@portal.aei.mpg.de:/home/rep
ository co gridportlets
2004 39Michael Russell <michael.russell@aei.mpg.de>
Configuring Grid Portlets
Configuration amounts to specifying what resources Grid Portlets makes available to users.
The first thing you should do is specify what MyProxy resource your portal will use for retrieving / storing Grid credentials.
If want to use our MDS2 information provider, then you should specify what GIIS resource you want to use.
Otherwise, you should make sure to specify the hardware and service resources that make up your Grid…
2004 40Michael Russell <michael.russell@aei.mpg.de>
How To Specify Resources
There are 2 ways to add resources to Grid PortletsOnline with the Resource Registry Portlet
Offline with Resources.xml
2004 41Michael Russell <michael.russell@aei.mpg.de>
How To Get Involved
Join one of gridsphere mail lists. Ask questions, provide feedback / comments.
Submit bugs and feature requests to http://bugs.gridsphere.org.
Because we are a small group, getting involved as a developer means demonstrating some knowledge / commitment to the project. Best way is to email ideas / patches to the list. If you’re loud enough, we’ll probably give you commit access and a few tasks!
2004 42Michael Russell <michael.russell@aei.mpg.de>
Projects Using Grid Portlets
AEI Numerical Relativity Portal
CCT Numerical Relativity Portal
NRC Portal
Unversity of Melborne, Australia
SDSC…
2004 43Michael Russell <michael.russell@aei.mpg.de>
Conclusion
Grid Portlets contains the Grid functionality in the GridSphere Project. It provides a framework for developing Grid portals.
It is still in prototype development mode, but we are already using it in the GridLab Project and in several other projects with which we are affiliated.
We will make announcements on our website at www.gridsphere.org as the status of Grid Portlets changes.
Feel free to try it out and send us comments.
Recommended