Embed Size (px)
Citation preview
Virtual WorkspacesState of the Art and Current Directions
Borja SotomayorUniversity of Chicago (Dept. of CS)
Kate KeaheyANL/UC
Tim FreemanANL/UC
Index
Virtual Workspaces Use Cases GT4 Workspace Service Current Directions
Index
Virtual Workspaces Use Cases GT4 Workspace Service Current Directions
What is a workspace?
What do I do if I want to “run something in the Grid”? We have to encapsulate our work in a job abstraction. This abstraction imposes a lot of restrictions on the
resource consumer. For example: Resource consumers can specify what libraries their
jobs require, but they (usually) can't have those libraries dynamically installed when the job is submitted. If the library is not already available in the resources, they generally have to make a prior arrangement with resource providers to make sure that software is installed.
However, these restrictions are usually perfectly acceptable.
But... what if they're not?
Magic!
What is a workspace?
In some cases, resource consumers need execution environments to be dynamically deployed on remote resources.
The virtual workspace is an abstraction for such an execution environment.
Dream up an ideal environmentfor your jobs (or anything else!)
2GB memory500MB scratch diskSoftware:FOO, BAR, BAZ
Ideal environment is deployed and can be managed
Run jobs in ideal execution environment
What is a workspace?
A virtual workspace is an execution environment that can be deployed dynamically and securely on the Grid.
Dynamic: Started and stopped on an on-demand basis. Secure: Environment encapsulated inside the workspace and the
user submitting the workspace are both trustworthy.
Of course, this is not an entirely new idea... It is possible to create custom execution environments by:
Dynamically setting up cluster nodes: CoD: http://www.cs.duke.edu/nicl/cod/ bcfg: http://trac.mcs.anl.gov/projects/bcfg2/
Providing access to existing installation Dynamic Accounts: http://workspace.globus.org/da/
Refining site configuration Pacman: http://www.archlinux.org/pacman/
What is a workspace?
Main two aspects of workspaces: Environment definition: Users get exactly the (software)
environment they need. Resource allocation: Provision and guarantee all the resources
the workspace needs to function correctly (CPU, memory, disk, bandwidth, availability), allowing for dynamic renegotiation to reflect changing requirements and conditions
Existing implementations either don't provide both, or...
Quality of life: Setting up a new software environment takes a long time, and still doesn't give the resource consumer full control.
Quality of service: Little or no enforcement.
Qua
lity
ofLife
Qua
lity
of
Serv
ice
What is a workspace?
What magic spell can we use?
Virtual Machines are a promising vehicle to achieve higher quality workspaces
Magic!
Virtual Machines
VM technology allows multiple virtual machines to run on a single physical machine.
Hardware
Virtual Machine Monitor (VMM) / Hypervisor
Guest OS(Linux)
Guest OS(NetBSD)
Guest OS(Windows)
VM VM VM
AppApp AppAppApp
Xen
VMWare
UML
Denali
etc.Performance: Paravirtualization (e.g. Xen) is very close to raw physical performance
Virtual Machines andGrid Computing
VMs have several qualities that make them an appealing technology in Grid systems: Security and isolation Customization of execution environment Resource control Site independence
Workspaces can be encapsulated in VM images, and dynamically deployed on VMM-enabled sites.
Index
Virtual Workspaces Use Cases GT4 Workspace Service Current Directions
Virtual labs
A CS department wants to teach a course on Parallel Programming.
Unfortunately, they do not have a computing cluster the students can use. And, even if they do, the cluster admin won't give the students root access.
Solution: Deploy a virtual workspace that provides the students with a ready-to-use cluster.
Event-driven applications
Some applications need large amounts of computational power the instant a specific event arrives.
Science comes at you fast!
Current solutions include job preemption or allowing urgent requests to skip to the front of the queue.
Solution: VMs allow seamless suspend/resume of execution environments. An “urgent environment” can be placed on standby, activated when the event arrives, and all other VMs can be paused.
Less draconian example:Fusion experiments
Strict software requirements
Resource consumers can have strict software requirements:
Jobs that will only run with a specific version of a library. Legacy software. Special access privileges
Managing multiple software requirements can be problematic for the resource provider
Conflicting software requirements. Switching between software environments can be costly (e.g.,
reimaging cluster nodes) Avoid malicious use of resources.
Solution: Encapsulate desired software environment inside a VM.
Index
Virtual Workspaces Use Cases GT4 Workspace Service Current Directions
GT4 Workspace Service
The GT4 Virtual Workspace Service (VWS) is a VM-based workspace implementation. GT4 WSRF frontend Xen-based, but other VMMs could potentially be
used. http://workspace.globus.org/
GT4 Workspace ServiceArchitecture
VWSNode
WorkspaceService
Poolnode
Resource Provider
ImageNode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
The workspace service has a WSRF frontend that allows users to deploy and manage
virtual workspaces
The VWS manages a set of nodes(typically a cluster).
This is called the node pool.
Each node must have a VMM (Xen)installed, along with the workspacebackend (software that manages
individual nodes)
VM images are stored in aseparate node.
GT4 Workspace ServiceTypical Deployment
Poolnode
ImageNode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Resource Provider
VWSNode
WorkspaceService
Users can interact directly with their
workspaces the same way the would with a
physical machine.
GT4 Workspace Service
Workspace Service is a Globus incubator project. http://workspace.globus.org/
Current release TP1.2.3 (04/20/07) Provides the functionality described in the previous slides
(managing a pool of nodes, and deploying single-machine workspaces to those nodes)
New in this release: Multiple partitions per VM (with some restrictions) HTTP staging Improved documentation Multiple bug fixes
Index
Virtual Workspaces Use Cases GT4 Workspace Service Current Directions
Workspaces + Batch Scheduling (I)
Workspaces allow us to dynamically deploy execution environments on a grid.
But... how do they fit into the model of batch-oriented computation? The “job” is the most widely used execution abstraction on grids.
In other words: Will it play in Peoria?
WorkspaceService
LRM
Option #1 – Use a workspace-aware LRM. This would be ideal, but there is no such thing. There are plenty of commercial
data center solutions, but they focus on resource leasing, not batch systems.
Option #2 – Use workspaces to deploy virtual compute clusters. Could be a valid option in certain cases (e.g., gliding in your own scheduler).
However, cluster admins have to give up their LRM, and the responsibility of providing an LRM rests on the job submitter.
Option #3 – Have LRMs and workspaces as separate components, but working together.
Workspace-awareLRM
LRM
LRM
Workspaces + Batch Scheduling (II)
We want the LRM to be able to create a workspace for jobs requiring a specific software environment.
There is a naïve way of doing this: Set the job prolog to “start VM” and the job epilog to “stop VM”. This adds overhead which can be excessive if not adequately managed [1]
Questions we want to answer: How can we efficiently combine resource leasing and batch
scheduling?
Can we use the suspend/resume functionality of VMs to improve cluster utilization, despite the overhead of using VMs? (partially addressed in [1])
How can we do this in such a way that cluster admins will be happy?
[1] A Resource Management Model for VM-Based Virtual Workspaces, Sotomayor, Borja. Masters paper, University of Chicago
Configuration Management Virtual workspaces without a collection of VM images to choose
from is like iTunes without the music.
However, VM images cannot be created just once and then used out-of-the-box.
VM images need to be configured to match the configuration of the site they are being deployed to (IP address, certificates, software in the image that depends on that information, ...)
Site admins need to establishing that a VM image is trustworthy.
VM images are large, and can use up valuable storage resources.
We need mechanisms that address these concerns. Otherwise, only site-specific VM images will be deployable in practice.
Bradshaw et al. [2] discuss requirements and services (including Bcfg2 and Workspace Service) necessary to ensure scalable management and deployment of appliances (implemented as VM images)
[2] A Scalable Approach To Deploying And Managing Appliances, Bradshaw, R., N. Desai, T. Freeman, K. Keahey. TeraGrid 2007
STAR (I)
STAR is an experiment at the Relativistic Heavy Ion Collider (RHIC) at the Brookhaven National Laboratory.
Their software is an example of an application that requires a very specific software environment to work correctly, and which cannot easily be deployed on Grid sites.
Doug Olson (LBL): “tends to push the boundaries on what will actually compile”
“using rarely used features of the language”
“even just validating a new platform is a big job even when it all compiles.”
The STAR offline analysis software is about 1.3M lines of code, 2/3 C++, a bit under 1/3 Fortran, and a bit of C.
www.star.bnl.gov
STAR (II)
rBuilder to the rescue rPath was founded by ex-RedHat luminaries Software Appliances (and not just VMs) http://www.rpath.com/rbuilder/
Stu Gott, Ken Vandine, and Marty Wesley worked with OSG and Doug Olson to produce a STAR appliance with rBuilder
STAR image successfully deployed on Teraport Xen nodes.
We have also experimented with the AliEn software (from the ALICE experiment)
http://www.rpath.org/rbuilder/project/alien/ http://alien.cern.ch/twiki/bin/view/AliEn/Home
GridWay + Workspaces (I)
The GridWay Metascheduler (http://www.gridway.org/) is a very successful dev.globus project (first to escalate to a full project)
The GridWay team is interested in using workspaces for deployment of Virtual Infrastructures across sites. Dynamic deployment and reallocation of VMs between
different administration domains. GridHypervisor Project: http://www.gridhypervisor.org/ We are collaborating with the GridWay team, although
the lack of common funding is an impediment.
GridWay + Workspaces (II)
Workspace Service
GridFTP
GridHypervisor
Physical Infrastructure
Virtual Infrastructure
Middleware
Applications
Virtual Machine Control & Monitor
Globus Services
Service Overlay
VirtualizerVirtualizer
Service InteractionUsers
VM R
eque
st
Grid SiteManagement of Virtual Machines in Grid Infrastructures, Rubén S. Montero. OGF20.
GT4 Workspace Service
On the implementation side, we are working on adding the following to the Workspace Service in the next releases: Support for virtual clusters EC2 backend Workspace Marketplace
http://workspace.globus.org/vm/marketplace.html
Improved usability and documentation
Questions?
Borja SotomayorUniversity of Chicago (Dept. of CS)
Kate KeaheyANL/UC
Tim FreemanANL/UC
