Embed Size (px)
Citation preview
Building Grid Portals with OGCE: Big Red Portal and
GTLABMehmet A. Nacar, Jong Youl Choi, Marlon
Pierce, Geoffrey FoxCommunity Grids Lab
Indiana University
IntroductionThis is a two-part talk on OGCE activities
Part 1: using OGCE’s current grid portal release to build a basic gateway for IU’s BigRed Jong Youl Choi Zero to Done in 2 months. YMMV
Part 2: simplifying the process for building portals like this using tag libraries. Mehmet A. Nacar Finishing up ~2 years of software development
Big Red supercomputer is part of TeraGrid at Indiana University 2048 cluster nodes, 4 terabyte memory
For more information on Big Red, see the Monday tutorial.
2
Portlets for Big Red We have developed several portlets for Big Red based on
the OGCE software Improved batch job submission for LoadLeveler (IU) and PBS
(NCSA). MEME job submission
Interactive and batch Includes both GridFTP and GRAM clients Job tracking.
MOAB queue monitoring for both entire machine and the specific user.
Access to existing IU resources like the IU Knowledge Base. These portlets also work with other TeraGrid resources.
We tested across IU and NCSA resources.
Meme Portlet
5
The MEME portlet uses OGCE portal libraries to upload and download files, submit jobs, and monitor their progress.
Dashboard Portlet
6
The dashboard portlet allows users to track jobs on the selected resource. The user can view either his own set of jobs or get information on all submitted jobs.
OGCE IFrame Portlet can be used to integrate external sites.
OGCE GPIR portlet can interoperate with TeraGrid GPIR services.
Manage TeraGrid MyProxy credentials with the OGCE ProxyManager portlets.
OGCE file management client portlets interact with TeraGrid GridFTP servers.
In addition to MEME, general purpose batch and interactive job submission to GRAM, WS-GRAM is supported.
Using TeraGrid Resources
Although utilizing Big Red, the portlets can be used with any other gateway in TeraGrid
Certificate username and local system user names may be different. To avoid confusion, we submit a whoami command to retrieve
correct username at first place. Aside: Why can’t we have just one user name for all TeraGrid
resources? GSI-enabled SSH users: see
http://communitygrids.blogspot.com/2007/06/teragrid-single-sign-on.html
Dashboard portlet facilitates this schema by using GRAM service.
13
Problems with Portlet Development
Grid portlets typically wrap each single Grid capability in a separate portlet
Problem is that Grid portlets need to combine these operations Portlets are entire web applications, so we need a component model for
portlets: reusable portlet parts JSF uses Inversion of Control pattern to implement its Model-
View-Controller architecture Remove dependencies on the Servlet API. Backing beans are just beans, so can be reused more easily outside of
web and portlet applications. JSF also provides an extensible framework (tag libraries) Apache JSF portlet bridge allows you to convert standalone
JSF applications (development phase) into portlets (deployment phase).
Grid Tag Libraries and Beans (GTLAB)
GTLAB provides common components for building portlets using tags and reusable parts.
The goal of GTLAB to simplify Grid portlet development Enable rapid development
GTLAB capabilities include Grid operations with XML based tags within Java Server Faces (JSF) framework.
Grid tag libraries are built using JSF custom component development techniques
Grid tags are interfaces to backing Grid beans End users pass values to Grid beans by using tag attributes.
We build on Java CoG 4’s abstraction layer. Each backing Grid bean has equal capability with a portlet
application in case of Grid portlet approach.
15
GTLAB Example
<html>
<body>
<f:form>
<o:submit id=”test” action=”next_page” />
<o:myproxy id=”pr” hostname=”gf1.ucs.indiana.edu” port=”7512” lifetime=”2” username=“mnacar” password=”***” />
<o:jobsubmit id=”task” hostname=”cobalt.ncsa.teragrid.org” provider=”GT4” executable=”/bin/ls” stdout=”tmp/result stderr=”tmp/error” />
</o:submit>
</f:form>
</body>
</html> 16
• Grid tags are associated with Grid services via Grid beans• Grid Beans wrap the Java COG Kit (version 4)
• We show an example JSF page section below.• This allows you to develop new Grid portlets with no additional Java code.
Grid Tags Associated Grid Beans Features
<submit/> ComponentBuilderBean Creating components, job handlers, submitting jobs
<handler/> MonitorBean Handling monitoring page actions
<multitask/> MultitaskBean Constructing simple workflow
<dependency/> MultitaskBean Defining dependencies among sub jobs
<myproxy/> MyproxyBean Retrieving myproxy credential
<fileoperation/> FileOprationBean Providing Gridftp operations
<jobsubmission/> JobSubmitBean Providing GRAM job submissions
<filetransfer/> FileTransferBean Providing Gridftp file transfer
ResourceBean Describes common properties among all tags and beans. Passing values given by standard visual JSF components.
How to integrate GTLAB
depends
depends
Input
Multi -staged task
Task DFile Transfer
Task AFile Operation
(mkdir )
Task BFile Transfer
Task CJob Submission
Output
depends
18
Multitask provides a simple Directed Acyclic Graph (DAG)
This example demonstrates a composite Grid job using multi-staged multitask
GTLAB handles lifecycle of DAG within JSF application
How to prepare application pages
Developers embed Grid tags snippet into JSF page These components are non-visual and are not displayed in
HTML. Resource bean provides bridging with form inputs and GTLAB
framework. <h:outputText value="Taskname: "/>
<h:inputText value="#{resource.taskname}" /> <o:multitask id="multi" persistent="true" taskname="#{resource.taskname}" />
Dynamic values to Grid tag attributes are provided by Resource bean.
Only visual component is <o:submit/> tag that is associated with action method of GTLAB.
19
GTLAB Dashboard PortletExample
<o:submit id=”track” action=”list_page” /> <o:multitask id=”dashboard” taskname=”track” persistent=”true” >
<o:myproxy id=”proxy” hostname=”gf1.ucs.indiana.edu” lifetime=”2” username=”#{resource.username}” password=”#{resource.password}” /> <o:jobsubmit id=”jobA” hostname=”cobalt.ncsa.teragrid.org” provider=”GT4” executable=”/bin/whoami” stdout=”tmp/result” stderr=”tmp/error” /> <o:jobsubmit id=”jobB” hostname=”cobalt.ncsa.teragrid.org” provider=”GT4” executable=”/bin/showq” stdin=”tmp/result” stdout=”tmp/list” stderr=”tmp/error” /> <o:dependency id=”depend” task=”jobB” dependsOn=”jobA” /> </o:multitask></o:submit>
20
Tracking and Managing Jobs
GTLAB manages lifecycles of jobs and monitor their status.
Grid operations are usually batch processes We provide callback mechanism to follow up the jobs GTLAB creates handlers for jobs and persistently stores them.
GTLAB handlers manages the job events such as stop, cancel or resuming the running jobs.
GTLAB provides archive for job metadata and allows managing the archive Handler tag helps to organize user’s job repository <o:handler id=”delete” action="#{monitor.delete}" > <f:param id="task" name="taskname“ value="#{task}"/> </o:handler>
21
GTLAB Related WorkGrid Portlets 1.3 of GridSphere
Now they are trying to decouple with GridSphere. It’s called Vine (Portlet Vine) as separate project
Grid Portlets 1.3 provide API and UI tags to build Grid portlets
RSF (Reasonable Server Faces) Derived from JSF, but it separates HTML pages and
backing beans RSF provides non-visual components unlike JSF Beans can be contained by Spring like containers.
Lifecycles of beans managed by SpringOGCE portlets
Packages Velocity, JSP and JSF portlets Provides portlet package for several Grid applications
such as Globus, Condor, SRB and GPIR
More InformationGTLAB version 1.0 Beta release available at
http://grids.ucs.indiana.edu/users/manacar/GTLAB-website
See link from main OGCE web site http://www.collab-ogce.org
Contact OGCE: [email protected] me: [email protected] Mehmet: [email protected]
23
Conclusion and Future WorkWe have described a science gateway to IU’s
Big Red supercomputer. Initial effort was to build the portal based on OGCE
portletsDescribed the process for creating Grid portlets
using GTLAB. We gained rapid development by using reusable
componentsWe are investigating abstracting our approach to
support different workflow engines.Investigate integrating AJAX with our session
management and monitoring. Please come to the Web 2.0 BOF Tuesday evening.
24
Acknowledgments
This work is supported by the National Science Foundation’s Information Technology Research (NSF grant ITR-0428774, 0427264, 0426867 VLab) and Middleware Initiative (NSF Grant SCI 0330613) programs.
