Exposing Computational Resources Across Administrative Domains The Condor Shibboleth Integration Project a scalable alternative for the computational grid
09/13/20052 Since the early days of mankind the primary motivation for the establishment of communities has been the idea that by being part of an organized group the capabilities of an individual are improved. The great progress in the area of inter-computer communication led to the development of means by which stand-alone processing sub-systems can be integrated into multi-computer communities. Miron Livny, Study of Load Balancing Algorithms for Decentralized Distributed Processing Systems., Ph.D thesis, July 1983.
09/13/20053 The single biggest road-block to the grid THE grid will not happen if resource owners do not bring their resources to the table Resource owners must know their resource is secure Security mechanisms must be scalable
09/13/20054 Grids focus on site autonomy One of the underlying principles of the Grid is that a given site must have local control over its resources, which users can have an account, usage policies, etc. Grids: The Top Ten Questions, Jennifer M. Schopf and Bill Nitzberg
09/13/20055 Resources must be attracted to the system Because users demand it! It is secure Rules for use of each resource can be established and enforced locally (WHO can do WHAT, and WHEN for HOW LONG?) It is scalable Administration of the access to the resource is no longer a daunting (if not impossible) task It is easy to set up
09/13/20056 Introduction to Condor In a nutshell, Condor is a specialized batch system for managing compute-intensive jobs. Like most batch systems, Condor provides a queuing mechanism, scheduling policy, priority scheme, and resource classifications. Users submit their compute jobs to Condor, Condor puts the jobs in a queue, runs them, and then informs the user as to the result.
09/13/20057 Communities benefit from Matchmakers..... someone has to bring together community members who have requests for goods and services with members who offer them. Both sides are looking for each other Both sides have constraints Both sides have preferences eBay is a matchmaker Condor is a matchmaker
09/13/20058 Condors Power A user submits a job to Condor Condor finds an available machine on the network and begins the job Definition of available is highly configurable If a machine becomes unavailable, job is checkpointed until the resource becomes available, or is migrated to a different resource Condor does not require an account on the remote machine
09/13/20059 Migrating jobs to other Pools Flocking Flocking is Condor's way of allowing jobs that cannot immediately run (within the pool of machines where the job was submitted) to instead run on a different Condor pool. Condor-C Condor-C allows jobs in one machine's job queue to be moved to another machine's job queue. These machines may be far removed from each other, providing powerful grid computation mechanisms, while requiring only Condor software and its configuration Condor-C is highly resistant to network disconnections and machine failures on both the submission and remote sides.
09/13/200510 DAGMan DAGMan (Directed Acyclic Graph Manager) is a meta-scheduler for Condor. It manages dependencies between jobs at a higher level than the Condor Scheduler
09/13/200511 Disadvantages Even with Flocking and Condor-C, administrative scalability doesnt exist System requires excessive communications between administrators, exchanging of host names and/or IP addresses
09/13/200512 What is Shibboleth? Internet2 Middleware Shibboleth leverages campus identity and access management infrastructures to authenticate individuals and then sends information about them to the resource site, enabling the resource provider to make an informed authorization decision.
09/13/200513 Shibboleth Goals Use federated administration as the lever; have the enterprise broker most services (authentication, authorization, resource discovery, etc.) in inter-realm interactions Provide security while not degrading privacy. Attribute-based Access Control Mike Gettes, Duke University
09/13/200514 Shibboleth Components: Identity Provider Users home site, where authentication credentials and attribute info is stored. Handle Server (HS) provides a unique and potentially anonymous identity for the user. User authenticates using sites existing technology (LDAP, Kerberos, WebISO, etc). Attribute Authority (AA) responds to requests about the user from the target. Retrieves user attributes from sites existing identity store typically a user directory such as LDAP. Both implemented with Apache, Tomcat and Java servlets/JSP.
09/13/200515 Shibboleth Components: Service Provider Protects the target application, enforces authentication and authorization. Assertion Consumer Service (ACS) maintains state information about the users unique numerical identifier (handle) Shibboleth Attribute Requester (SHAR) makes requests for attributes to the users Identity Providers Attribute Authority Co-located with application web server as server module. Implementations currently exist for Apache (UNIX and Windows) and IIS (Windows).
09/13/200516 Typical Access Flow 1. User attempts to access new Shibboleth-protected resource on target site application server. 2. User is redirected to Where are you From Server (WAYF), selects home site (origin site). Only necessary once per user session. 3. User is redirected to origin site Handle Server (HS) and authenticates with their local credentials (eg. Username/password)
09/13/200517 Typical Access Flow (cont.) 4. Handle server generates unique numerical identifier (handle) and redirects user to target sites ACS. 5. Target ACS hands off to SHAR, which uses the handle to request attributes from the user origin sites Attribute Authority. 6. Users AA responds with an attribute assertion, subject to Attribute Release Policies (ARP). 7. Target site uses the returned user attributes for access control and other application-level decisions.
09/13/200518 Federations Associations of enterprises that come together to exchange information about their users and resources in order to enable collaborations and transactions Built on the premise of Initially Authenticate locally, act globally Now, Enroll, authenticate and attribute locally, act federally. Federation provides only modest operational support and consistency in how members communicate with each other Enterprises (and users) retain control over what attributes are released to a resource; the resources retain control (though they may delegate) over the authorization decision. Mike Gettes, Duke University
09/13/200519 What if we Integrated Shib with Condor? Condor would function exactly as it now does Flocking (or eventually Condor-C) would use Shibboleth as its authentication model Classified Ads would include either user attributes or Shib Unique Identifiers This brings us to the Condor- Shibboleth Integration Project!
09/13/200520 Project Goals Primary goal is to create a scalable, expandable grid-aware universal workflow management tool for computational grids that doesn't require or exclude use of Globus grid map files. Scalable. Functions across unrelated administrative domains Tied to the Federated Authentication Model. No ties to Globus Certificates and Grid map files. Expandable. Design for relatively simple computational grids, but do not exclude connection to future projects requiring expanded grid services (Globus). Grid-Shib project already in progress could become a future connection.
09/13/200521 Project Goals Phase I: Shib Enabled Condor Web portal Shibboleth was originally designed as a web services federated authentication tool, fat clients not yet available. Phase II: Shib Enabled Condor Fat Client Extending the existing submit client model with Shib elements. There are already other open source projects in the works which will utilize Shibboleth in a fat client model, so we should be sure our work now will be compatible with the fat client model when it is supported by Shibboleth.
09/13/200522 Project Goals Impact Condor as little as possible. Identify key components that must be changed, mostly at the execute end. Some work can be performed by preprocessing scripts at the submit end. Ensure that changes made at execute end will not interfere with other modes of Condor use. Ensure that changes made at execute end will also work with eventual fat client version.
09/13/200523 Web based Condor Scheduler node Grid Portal must be running condor_schedd and have $(FLOCK_TO) populated with grid resources. Grid Portal must be configured as a Shibboleth Server User creates a job. Uploads Condor submit script, or Portal tools could help create simple submit scripts.
09/13/200524 Conceptual Workflow Model User logs in to Grid Portal Web Site (Phase I) Shibboleth web server module detects that no S