Embed Size (px)
Citation preview
© 2003 Hewlett-PackardThe information contained herein is subject to change without notice
HP, HP Labs and Grid Computing
23 September 2003
Paul Vickers
HP Research LaboratoriesBristol, [email protected]
http://www.hpl.hp.com
page 29/23/2003
HP’s Approach to Grid Computinghttp://www.hp.com/go/grid
Applications
Man
ag
em
en
t an
d C
ontro
lHP C
on
sult
ing S
er v
ices
Grid-Enabled Resources (desktops, blades, clusters, servers, supercomputers,
UDC, storage)
HP Adaptive Enterprise
Web Services
Globus/OGSA
Grid ISV Partners
hp utility data center
is an integrated software and hardware system that enables
virtual provisioning of application environments to optimize asset
utilization and reduce administrative costs
1. wire onceall components are wired once to support virtual allocation of resources for the entire system
2. resource virtualization all networking, storage, and server components can be allocated and reallocated many times without having to rewire any physical components
3. utility controllersimple user interface allows administrators to architect new solutions, and activate them
internetinternet intranet
intranet
storage virtualization
network /servervirtualization
utility
contro
ller
server pool
NAS pool
load balancer pool
firewall pool
switching pool
switching pool
storage pool
page 49/23/2003
Press release, Sept 4, 2003:
HP Advances Grid Strategyhttp://www.hp.com/hpinfo/newsroom/press/2003/030904b.html
• “We see the shared computing vision soon turning into reality as commercial enterprises more aggressively seek the agility and cost benefits the grid affords," Shane Robison, HP CTO
• HP Utility Data Center (UDC) delivers many grid capabilities to commercial customers today
• All HP systems to be Grid-enabled
• Enterprise Grid Consulting: architecture, pilots, deployment and support
• Web Services Management Framework standards contribution, Grid Management Software building on HP OpenView
• HP Labs prototypes: Grid Topology Designer, SmartFrog
page 59/23/2003
Quote from the founders of the Grid
“HP understands the commercial potential of the Grid and is making contributions to help insure its long-term viability and deployment. HP is focused on a open, industry-standard, and heterogeneous approach to Grid, as embodied in the Globus toolkit. This is what we always envisioned as the only way to truly make it work for everyone."
Ian Foster, Carl Kesselman, Steve Tuecke
page 69/23/2003
Challenges and requirements for Commercial Grids
• Organisational politics: “server hugging”
• Commercial Grade Service – security, reliability, flexibility, Service Level Agreements, billing
• Virtualization decoupling applications from resources
• Federation of resource pools for sharing and collaboration within and across organizations
• Programming the Grid deploying services
• Management of Grid services and systems
page 79/23/2003
HP’s management software strategyAutomating management is key to agility
Discrete partitioned Integrated clustered Virtualized federated
Utilization
Real-time business agility
Business efficiency
Business stability
Reso
urc
es
Serv
ices
Busi
ness
pro
cess
es
Man
ag
em
en
t &
con
trol
• Network management• Operations control• Client, server,
and storage management• Fault and performance • Integrated Console
• Service-level management
• Map infrastructure to services
• Automate IT workflow processes
• Meter/Metrics
• Workload Management
• Web services management
• Zero-Latency
• Complete data center virtualization
• Balance, schedule, and allocate resources based on business priorities and impact
• Optimize utilization and performance of applications
• Fully-federated Grid architecture
page 89/23/2003
The importance of Management
• “HP is focussed squarely on the management and execution of Grid services”, Carly Fiorina, HP CEO, OracleWorld, 11/9/3
“As our customers have learned over the past 20 years through trial and error, management cannot be an afterthought“
Carly Fiorina, BEAWorld, 3/3/3.
page 99/23/2003
HP Labs: A Global Research Organization
Palo Alto
Bristol
IsraelJapan
Cambridge
India
• ~800 employees worldwide• ~5% of $4b hp r&d budget
page 109/23/2003
HP Labs Planetary Computing Vision
• • We envision a world where distributed services execute on a utility that dynamically and securely allocates globally connected server and storage resources on demand …a global commercial Grid
page 119/23/2003
Grid/Utility Computing
TechnicalComputing
Grids
Enterprise Grids
Scientific/Commercial Grids: Converging Visions
Enable new application
s
Accelerate innovation
Adaptive Enterprise
Grid/Utility Computing
Utility Data
CenterFederated
Data Centers
Reduced cost of
ownership Business agility
Commercial Service
Levels
Virtual Organisation
s
page 129/23/2003
The Range of HP Labs’ Research in Utility/Grid Computing
securestorage
GFS
linux
webservices
HPTC
securenetworks
assetmgmt
thermalmodelling
SmartCooling
digitalmedia
bio-informaticstrading
QoS
thermalmetrics
the Gridapplication
ignition
federatedstorage
massivesystems
utilityeconomics
self-awaresystems
businessmetrics
performancemodelling
applicationpackaging
distributedmultimedia secure
virtualizationstorage
managementsecurity reliability
utility computing
page 139/23/2003
Grid Research: examples of HP Labs Technical Reports
• Making the UDC a Powerstation for the Enterprise Grid• Globus Grid and Firewalls: issues and Solutions in a
UDC Environment• A Grid Environment for Graphical, Interactive Sessions
to Remote Nodes • Specifying and Monitoring Guarantees in Commercial
Grids through SLAs• Complete set available at http://www.hpl.hp.com
page 149/23/2003
HP Labs Research in Utility/Grid Computing
Architecture Resource Pool
Resource Utility Service
GridServices
Service Portals
Applications
System
s M
anagement
Services
Managem
ent
Resource Allocation System
Service Deployment
System
Research Partnerships
HPL Resear
ch Target
UDC Evolutio
n
page 159/23/2003
HP Labs Research in Utility/Grid Computing: 1 of 3
Architecture Resource Pool
Resource Utility Service
GridServices
Service Portals
Applications
System
s M
anagement
Services
Managem
ent
Resource Allocation System
Service Deployment
System
Research Partnerships
HPL Resear
ch Target
UDC Evolutio
n
Deploying services on the utility
Grid/UDC integration
Application Partners
page 169/23/2003
Application Partners: Digital Media – 4:2:2Animation Film digitally rendered using a utility
service
page 179/23/2003
Application Partners: Particle Physics - CERN
ATLAS
CMS
LHCb
page 189/23/2003
Large HP nodes on scientific computing grids
PNNL Molecular Science Computing Facility
• 11.8 TFLOPS peak HP Integrity + Quadrics
• Fastest computer for unclassified research in US
• Fastest Linux, largest Itanium2
• On DOE science grid
LeMieux at Pittsburgh Supercomputing Center
• 6 TFLOPS peak AS SC• Fastest computer for
academic research in US• 10 GB/s lambda link to
TeraGrid, interoperable with SDSC, NCSA, ANL, CACR
page 199/23/2003
Utility Data Centres and Linux Itanium Clusters in HP Labs Palo Alto and Bristol
• Build a large Adaptive Infrastructure at HP Labs:• production: consolidate HPL IT services in the UDC
• research: as a platform for collaborative experimentation HP-wide and with external partners
• 2 principal data centers- Bristol & Palo Alto– 175 production application servers for email, web,
HP library services, tool & database servers– 30 TB of SAN storage– >200 research servers
page 209/23/2003
HP Labs Research in Utility/Grid Computing: 2 of 3
Architecture Resource Pool
Resource Utility Service
GridServices
Service Portals
Applications
System
s M
anagement
Services
Managem
ent
Resource Allocation System
Service Deployment
System
Research Partnerships
HPL Resear
ch Target
UDC Evolutio
n
Deploying services on the utility
Grid/UDC integration
Application Partners
page 219/23/2003
Connecting HP Utility Data Center and Grid
• HP Labs has implemented a gateway between the grid and UDC using the Globus Toolkit– Advertises UDC resources to the grid– Accepts requests from grid users for UDC resources
• Translates between Globus RSL and UDC resource description
• Includes a drag-and-drop “Topology Designer” tool to design and submit applications to UDC via Globus without detailed resource description
Demonstrated at GlobusWorld conference Jan 2003
page 229/23/2003
Grid Interface for UDC:UDC as a Grid PowerStation
UDC/XMLInterface
Utility Data Center =programmable poolof data center resources
UDC GRAM =GlobusGatekeeper +UDC Adapter
UDCGRAM
UDCGRAM
Grid Service
discover request use manageresources
Benefits: enterprise security, availability and fine-grained dynamic resource allocation
page 239/23/2003
Portal – “Topology Designer”
UDC in an Enterprise Grid
Grid Service
Interface
UDCGRAM
UDCGRAM
UDCGRAM
UDCGRAM
GridGrid
Enterprise Grid:- Resource Sharing- Cross-domain
Management
Benefits: improved asset utilization, business agility , collaboration across a virtual organisation
page 249/23/2003
page 259/23/2003
page 269/23/2003
page 279/23/2003
HP Globus intraGrid of PCs, Linux servers, Alpha Clusters, UDCs
page 289/23/2003
HP Labs Research in Utility/Grid Computing: 3 of 3
Architecture Resource Pool
Resource Utility Service
GridServices
Service Portals
Applications
System
s M
anagement
Services
Managem
ent
Resource Allocation System
Service Deployment
System
Research Partnerships
HPL Resear
ch Target
UDC Evolutio
n
Deploying services on the utility
Grid/UDC integration
Application Partners
page 299/23/2003
SmartFrog
• An open standards tool for defining, packaging & deploying applications in the enterprise grid based on policies
• Automated deployment of application as a grid service
• Adaptive execution of the Service, automatically adjusting supply to demand
ServicePackages
SmartFrog distributed
deployment engine
Utility/Grid infrastructure
page 309/23/2003
page 319/23/2003
page 329/23/2003
page 339/23/2003
page 349/23/2003
page 359/23/2003
HP Labs Bristol Utility/Grid Computing Demonstration Services
• SmartFrog service package examples– HP Scalable Web Server– HP Utility Rendering Service
• Automatic service creation on the service utility– Two services automatically deployed onto utility
resources– Resource allocation, software installation, system
configuration and start-up– Clean service removal
• Adaptive behaviour– Adapting to varying workloads– Adapting to deal with failure
page 369/23/2003
HP Labs Bristol Utility/Grid Computing Demonstration Services
Frame Factory Service Package
Submit Service
s
Deploy/Ignite
Services
Web Server Service Package
Digital Animators Utility Provider
page 379/23/2003
Applications in Enterprise Grids
Technical Commercial
Modus Operandi batch dynamic grid services
Workload compute jobs transactional with varying demand
Lifetime duration of jobduration of application deployment
Example Utility Rendering Service
Scalable Web Server Utility Service
page 389/23/2003
• Fabric Management for Grid Computing– Large-scale, diverse, dynamic, complex
• UK e-Science Project with Edinburgh University• Combines HPL SmartFrog with Edinburgh’s LCFG• Demonstration at GGF8, Jun 25-27
– Deployment of Globus v3 infrastructure– Deployment of GPrint – an adaptive printing Grid
Service• Announced SmartFrog open source release to
GGF
