View
226
Download
0
Tags:
Embed Size (px)
Citation preview
Gridbus Middleware and Utility Grids:Building Autonomic and Market-Oriented Global Grids for
Delivering IT Services as the 5th Utility
Dr. Rajkumar Buyya
Grid Computing and Distributed Systems (GRIDS) LaboratoryDept. of Computer Science and Software EngineeringThe University of Melbourne, Australia
www.gridbus.org
Gridbus Sponsors
2
GRIDS Lab @ Melbourne
Youngest and one of the rapidly growing research labs in our School/University:
Founded in 2002 Houses:
Research Fellows/PostDocs Research Programmers PhD candidates Honours/Masters students
Funding National and International organizations Australian Research Council & DEST Many industries (Sun, StorageTek, Microsoft,
IBM, Microsoft) University-wide collaboration:
Faculties of Science, Engineering, and Medicine
Many national and international collaborations.
Academics Industries
Software: Widely in academic and industrial users.
Publication: My research team produces over 20% of our
Dept’s research output.
EducationR & D
+ Community Services: e.g., IEEE TC for Scalable Computing
3
(some) Books: Co-authored & edited
4
Agenda
Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services
Global Grids and Challenges Security, resource management, pricing models, …
Service-Oriented Grid Architecture and Gridbus Solutions
Market-based Management, GMD, Grid Bank, Alchemi Grid Service Broker
Architecture, Design and Implementation Performance Evaluation: Experiments in Creation
and Deployment of Applications on Global Grids A Case Study in High Energy Physics
Summary and Conclusion
5
4 Essential Utilities and Delivery Networks
(1) WaterWater
Distribution Network
(4) TelephoneTelecom Networks
(3) Gas
(2) Electricity
Power Grid
6
(5) Computing Grid: Delivering IT services as the 5th utility (Power Grid inspiration)
eScienceeBusiness
eGovernmenteHealth
MultilingualeEducation
…
7
Power Grid Inspiration: Seamlessly delivering electricity as a utility to users
8
Grid-like Vision
In 1969, Leonard Kleinrock, one of the chief scientists of the original ARPA project which seeded the Internet, wrote:
"As of now, computer networks are still in their infancy, but as they grow up and become sophisticated, we will probably see the spread of "computer utilities", which, like present electric and telephone utilities, will service individual homes and offices across the country“
Despite major advances in hardware and software systems over the past 35 years, we are yet to realize this vision. How far are we still from delivering computing as a utility?
Let us look into the ICT evolution and project the future.
9
Why Grid Computing Now?Let us look at the Evolution of ICT
10
Computing and Communication Technologies Evolution: 1960-2010!
* Sputnik
1960 1970 1975 1980 1985 1990 1995 2000
* ARPANET
* Email* Ethernet
* TCP/IP* IETF
* Internet Era * WWW Era
* Mosaic
* XML
* PC Clusters* Crays * MPPs
* Mainframes
* HTML
* W3C
* P2P
* Grids
* XEROX PARC wormCO
MP
UTIN
GC
om
mu
nic
ati
on
* Web Services
* Minicomputers
* PCs
* WS Clusters
* PDAs* Workstations
* HTC
2010
* e-Science
* Computing as Utility
* e-Business
* SocialNet
ControlCentralised Decentralised
11
2100
2100 2100 2100 2100
2100 2100 2100 2100
Personal Device SMPs or SuperComputers
LocalCluster
GlobalGrid
SERV ICES
+
PERFORMANCE
Inter PlanetGrid
•Individual•Group•Department•Campus•State•National•Globe•Inter Planet•Universe
Administrative Barriers
EnterpriseCluster/Grid
Computing is Scaling: Towards Inter-Planetary Level
12
What is Grid? (It means different things to different people)
IBM On Demand Computing
Microsoft .NET
Oracle 10g
Sun N1 – Sun Grid Engine
HP Adaptive Enterprise
United Devices and related companies: Harvesting Unused Desktop resources
13
What is Grid?[Buyya et. al.]
A type of parallel and distributed system that enables the sharing, exchange, selection, & aggregation of geographically distributed “autonomous” resources:
Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks, mobile devices, PDA, etc;
Software – e.g., ASPs renting expensive special purpose applications on demand;
Catalogued data and databases – e.g. transparent access to human genome database;
Special devices/instruments – e.g., radio telescope – SETI@Home searching for life in galaxy.
People/collaborators.
depending on their availability, capability, cost, and user QoS requirements.
Widearea
14
How does Grids look like?A Bird Eye View of a Global Grid
Grid Resource Broker
Resource Broker
Application
Grid Information Service
Grid Resource Broker
databaseR2R3
RN
R1
R4
R5
R6
Grid Information Service
15
Classes of Grid Services / Types of Grids
Computational Services – CPU cycles Pooling computing power: SETI@Home, TeraGrid,
AusGrid, ChinaGrid, IndiaGrid, UK Grid,… Data Services
Collaborative data sharing generated by instruments, sensors, persons: LHC Grid, Napster
Application Services Access to remote software/libraries and license
management—NetSolve Interaction Services
eLearning, Virtual Tables, Group Communication (Access Grid), Gaming
Knowledge Services The way knowledge is acquired, processed and
managed—data mining. Utility Computing Services
Towards a market-based Grid computing: Leasing and delivering Grid services as ICT utilities.
Computational Grid
Data Grid
ASP Grid
Interaction Grid
Knowledge Grid
Utility Grid
infra
stru
ctu
re
Users
16
How Are Grids Used?
High-performance computing
Collaborative data-sharing
Collaborative design
Drug discovery
Financial modeling
Data center automation
High-energy physics
Life sciences
E-Business
E-ScienceNatural language processing & Data Mining
Utility computing
17
e-Science Environment: Supporting Collaborative Science
Distributed instruments
Distributed computation
Distributed data
Peers sharing ideas and collaborative
interpretation of data/results
2100 2100 2100 2100
2100 2100 2100 2100
Remote Visualization
Data & Compute Service
Cyberinfrastructure
E-Scientist
18
Agenda
Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services
Global Grids and Challenges Security, resource management, pricing models, …
Service-Oriented Grid Architecture and Gridbus Solutions
Market-based Management, GMD, Grid Bank, Alchemi Grid Service Broker
Architecture, Design and Implementation Performance Evaluation: Experiments in Creation
and Deployment of Applications on Global Grids A Case Study in High Energy Physics
Summary and Conclusion
19
Grid Challenges
Security
Resource Allocation & Scheduling
Data locality
Network Management
System Management
Resource Discovery
Uniform Access
Computational Economy
Application Construction
20
Some Grid Initiatives Worldwide
Australia Nimrod-G Gridbus DISCWorld GrangeNet. APACGrid ARC eResearch
Brazil OurGrid, EasyGrid LNCC-Grid + many others
China ChinaGrid – Education CNGrid - application
Europe UK eScience EU Grids.. and many more...
India Garuda
Japan NAGERI
Korea...N*Grid
SingaporeNGP
USA Globus GridSec AccessGrid TeraGrid Cyberinfrasture and many more...
Industry Initiatives IBM On Demand Computing HP Adaptive Computing Sun N1 Microsoft - .NET Oracle 10g Infosys – Enterprise Grid Satyam – Business Grid StorageTek –Grid.. and many more
Public Forums Global Grid Forum Australian Grid Forum Conferences:
CCGrid Grid HPDC E-Science
http://www.gridcomputing.com
1.3 billion – 3 yrs
1 billion – 5 yrs
450million – 5 yrs
486million – 5 yrs
1.3 billion (Rs)
27 million
2? billion
120million – 5 yrs
21
Open-Source Grid Middleware Projects
22
The Gridbus Project @ Melbourne:Enable Leasing of ICT Services on Demand
WWG
Pushes Grid computing into mainstream
computing
Gridbus
23
The Gridbus Project @ GRIDS Lab, The University of Melbourne: Toolkit for Creating and Deploying e-Research Applications on Utility Grids
The Gridbus Project @ GRIDS Lab, The University of Melbourne: The Gridbus Project @ GRIDS Lab, The University of Melbourne: Toolkit for Creating and Deploying eToolkit for Creating and Deploying e--Research Applications on Utility GridsResearch Applications on Utility Grids
Gridbus
Distributed Data
http://www.gridbus.org
• Gridbus is a “open source” Grid R&D project with focus on Grid Economy, Utility Grids and Service Oriented Computing.
• Gridbus Middleware components include:– Alchemi: .NET-based Enterprise Grid
– Grid Market Directory and Web Services
– Grid Bank: Accounting and Transaction Management
– Visual Tools for Creation of Distributed Applications
– Grid Service Broker and Scheduling
– Workflow Management Engine
– GridSim Toolkit
– Libra: SLA-based Resource Allocation
24
Agenda
Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services
Global Grids and Challenges Security, resource management, pricing models, …
Service-Oriented Grid Architecture and Gridbus Solutions
Market-based Management, GMD, Grid Bank, Alchemi Grid Service Broker
Architecture, Design and Implementation Performance Evaluation: Experiments in Creation
and Deployment of Applications on Global Grids A Case Study in High Energy Physics
Summary and Conclusion
25
What do Grid players want?
Grid Consumers Execute jobs for solving varying problem size and complexity Benefit by utilizing distributed resources wisely Tradeoff timeframe and cost
Strategy: minimise expenses
Grid Providers Contribute resources for executing consumer jobs Benefit by maximizing resource utilisation Tradeoff local requirements & market opportunity
Strategy: maximise return on investment
26
What do Grid players require?
They need tools and technologies that help them in value expression, value translation, and value enforcement.
Grid Service Consumers (GSCs): How do I express QoS requirements ? How do I trade between timeframe & cost ? How do I map jobs to resources to meet my QoS needs? How do I manage Grid dynamics and get my work done? …
Grid Service Providers (GSPs) How do I decide service pricing models ? How do I specify them ? How do I translate them into resource allocations ? How do I enforce them ? How do I advertise & attract consumers ? How do I do accounting and handle payments? …
27
Principle 1: Service Oriented Architecture (SOA)
A SOA is a contractual architecture for offering and consuming software as services.
There are four entities that make up an SOA service provider, service registry, and service consumer (also known as service requestor).
The functions or tasks that the service provider offers, along with other functional and technical information required for consumption, are defined in
the service definition or contract.
provider
registry
consumer
contract
28
Principle 2: Market-Oriented (Grid) Computing- (a) Sustained Resourced Sharing and (b)
Effective Management of Shared Resources
Grid Economy
29
Market-based Systems = Self-managed and Self-regulated systems.
Complexity present in Grid systems is similar to one present in human economies.
30
Grid Node N
Service-Oriented Grid Architecture
Grid Service Consumer
Pro
gra
mm
ing
En
viro
nm
ents
Grid Resource Broker
Grid Service Providers
Grid Explorer
Schedule Advisor
Trade Manager
Job ControlAgent
Deployment Agent
Trade Server
Resource Allocation
ResourceReservation
R1
Misc. services
Information Service
R2 Rm…
Pricing Algorithms
Accounting
Grid Node1
…
Core Middleware Services
…
…
HealthMonitor
Grid Market Services
JobExec
Info ?
Secure
Trading
QoS
Storage
Sign-on
Grid Bank
Ap
pli
cati
on
s
Data Catalogue
31
Gridbus and Complementary Technologies – realizing Utility Grid
AIXSolarisWindows Linux
.NETGridFabricSoftware
GridApplications
Core GridMiddleware
User-LevelMiddleware
GridBank
Grid Exchange & Federation
JVM
Grid Brokers:
X-Parameter Sweep Lang.
Gridbus Data Broker
MPI
Condor SGE TomcatPBS
Alchemi
Workflow
IRIX OSF1 Mac
Libra
Globus Unicore ……Grid
MarketDirectory
PDB
CDB
Worldwide Grid
GridFabricHardware
……
PortalsScience Commerce Engineering ……Collaboratories
……
Workflow Engine
Grid Storage Economy
Gri
d E
con
om
y
NorduGrid XGrid
ExcellGrid
Nimrod-G
Gridscape
32
On Demand Assembly of Services: Putting Them All Together
ASP Catalogue
Grid Info Service
Grid Market Directory
GSP(Accounting Service)
GridbusGridBank
GSP(e.g., UofM)
PEGSP
(e.g., VPAC)
PE
GSP(e.g., IBM)
CPUorPE
Grid Service (GS)
(Globus)
Alchemi
GS
GTS
Cluster Scheduler
Job
8
GridResource Broker
2
Visual Application Composer
Application CodeExplore
data1
36
45
Resu
lts9 7
Results+
Cost Info
10
11
Bill
12Data Catalogue
33
On Demand Assembly of Services: Putting Them All Together
ASP Catalogue
Grid Info Service
Grid Market Directory
GSP(Accounting Service)
GridbusGridBank
GSP(e.g., UofM)
PEGSP
(e.g., VPAC)
PE
GSP(e.g., IBM)
CPUorPE
Grid Service (GS)
(Globus)
Alchemi
GS
GTS
Cluster Scheduler
Job
8
GridResource Broker
2
Visual Application Composer
Application CodeExplore
data1
36
45
Resu
lts9 7
Results+
Cost Info
10
11
Bill
12Data Catalogue
34
Alchemi: .NET-based Enterprise Grid Platform & Web Services
InternetInternet
InternetInternet
Alchemi Worker Agents
Alchemi Manager
Alchemi Users
Web Services
Web Services
•SETI@Home like Model•General Purpose•Dedicated/Non-dedicate workers•Role-based Security•.NET and Web Services•C# Implementation•GridThread and Job Model Programming•Easy to setup and use• Widely in use!
35
Some Users of Alchemi
Tier Technologies, USALarge scale document processing using Alchemi framework
CSIRO, AustraliaNatural Resource Modeling
The Friedrich Miescher Institute (FMI) for Biomedical Research, SwitzerlandPatterns of transcription factors in mammalian genes
Satyam Computers Applied Research Laboratory, IndiaMicro-array data processing using Alchemi framework
The University of Sao Paulo, BrazilThe Alchemi Executor as a Windows Service
stochastix GmbH, GermanyServing clients in International Banking/Finance sector
Many users in Universities: See next for an example.
36
Students' project gives old computers new life - 1/25/2005
37
On Demand Assembly of Services: Putting Them All Together
ASP Catalogue
Grid Info Service
Grid Market Directory
GSP(Accounting Service)
GridbusGridBank
GSP(e.g., UofM)
PEGSP
(e.g., VPAC)
PE
GSP(e.g., IBM)
CPUorPE
Grid Service (GS)
(Globus)
Alchemi
GS
GTS
Cluster Scheduler
Job
8
GridResource Broker
2
Visual Application Composer
Application CodeExplore
data1
36
45
Resu
lts9 7
Results+
Cost Info
10
11
Bill
12Data Catalogue
38
Agenda
Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services
Global Grids and Challenges Security, resource management, pricing models, …
Service-Oriented Grid Architecture and Gridbus Solutions
Market-based Management, GMD, Grid Bank, Alchemi Grid Service Broker
Architecture, Design and Implementation Performance Evaluation: Experiments in Creation
and Deployment of Applications on Global Grids A Case Study in High Energy Physics
Summary and Conclusion
39
A resource broker for scheduling task farming data Grid applications with static or dynamic parameter sweeps on global Grids.
It uses computational economy paradigm for optimal selection of computational and data services depending on their quality, cost, and availability, and users’ QoS requirements (deadline, budget, & T/C optimisation)
Key Features A single window to manage & control experiment Programmable Task Farming Engine Resource Discovery and Resource Trading Optimal Data Source Discovery Scheduling & Predications Generic Dispatcher & Grid Agents Transportation of data & sharing of results Accounting
Grid Service Broker (GSB)
40
Gridbus Broker Architecture
Grid Middleware
Gridbus Client Gridbus ClientGribus Client
Grid Info Server
Schedule Advisor
Trading Manager
Gridbus Farming Engine
RecordKeeper
Grid Explorer
GE GIS, NWSTM TS
RM & TS
Grid Dispatcher
RM: Local Resource Manager, TS: Trade Server
G
G
CU
Globus enabled node.A
L
Alchemi enabled node.
(Data Grid Scheduler)
DataCatalog
DataNode
Unicore enabled node.
$
$
$
App, T, $, Opt
(Bag of Tasks Applications)
41
Gridbus Broker: Separating “applications” from “different” remote service access
enablers and schedulers
Alchemi
Gateway
UnicoreData Store
Access Technology
Grid FTPSRB
-PBS-Condor-SGE
Globus
Job manager
fork() batch()
Gridbusagent
Data Catalog
-PBS-Condor-SGE-XGrid
SSH
fork()
batch()
Gridbusagent
Single-sign on securityHome Node/Portal
GridbusBroker
fork()
batch() -PBS-Condor-SGE-Alchemi-XGrid
Application Development Interface
Sch
ed
ulin
gIn
terfa
ces
Alogorithm1
AlogorithmN
Plugin Actuators
42
Gridbus Services for eScience applications
Application Development Environment: XML-based language for composition of task farming (legacy)
applications as parameter sweep applications. Task Farming APIs for new applications. Web APIs (e.g., Portlets) for Grid portal development. Threads-based Programming Interface Workflow interface and Gridbus-enabled workflow engine.
Resource Allocation and Scheduling Dynamic discovery of optional computational and data nodes
that meet user QoS requirements. Hide Low-Level Grid Middleware interfaces
Globus (v2, v4), SRB, Alchemi, Unicore, and ssh-based access to local/remote resources managed by XGrid, Condor, SGE.
43
Figure 3 : Logging into the portal.
Drug DesignMade Easy!
Click Here for Demo
44
Excel Plugin to Access Gridbus Services
Excel
ExcelGrid Add-In
ExcelGrid Runner
ExcelGridJob
ExcelGrid Middleware
Gridbus Broker
Enterprise Grid
2100
2100
2100
2100
2100
2100
2100
2100
45
Discover Discover ResourcesResources
Distribute JobsDistribute Jobs
Establish Establish RatesRates
Meet requirements ? Remaining Meet requirements ? Remaining Jobs, Deadline, & Budget ?Jobs, Deadline, & Budget ?
Evaluate & Evaluate & RescheduleReschedule
Discover Discover More More
ResourcesResources
Compose & Compose & ScheduleSchedule
Adaptive Scheduling Steps
46
Deadline (D) and Budget (B) Constrained Scheduling Algorithms
Algorithm Execution Time (D)
Execution Cost (B)
Compute Grid
Data Grid
Cost Opt Limited by D Minimize Yes Yes
Cost-Time Opt
Minimize if possible
Minimize Yes
Time Opt Minimize Limited by B Yes Yes
Conservative-Time Opt
Minimize Limited by B, jobs have guaranteed minimum budget
Yes
47
Gridbus Project: Some Applications and Users
Gridbus Project: Gridbus Project: Some Applications and UsersSome Applications and Users
http://www.gridbus.org
BioGrid: Molecular docking for Drug-discovery
BioGrid: Molecular docking for Drug-discovery
High Energy Physics: Particle Discovery
High Energy Physics: Particle Discovery
Melbourne University
NeuroScience: Brain Activity Analysis
NeuroScience: Brain Activity Analysis
Natural Resource ModelingNatural Resource Modeling
CSIRO Land and Water, Austraila.
Large Scale document processing
Large Scale document processing
Tier Technologies, USA.
Detection of patterns of transcription factors in mammalian genes
Detection of patterns of transcription factors in mammalian genes
48
Agenda
Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services
Global Grids and Challenges Security, resource management, pricing models, …
Service-Oriented Grid Architecture and Gridbus Solutions
Market-based Management, GMD, Grid Bank, Alchemi Grid Service Broker
Architecture, Design and Implementation Performance Evaluation: Experiments in Creation
and Deployment of Applications on Global Grids A Case Study in High Energy Physics
Summary and Conclusion
49
Case Study: High Energy Physics and Data Grid
The Belle Experiment KEK B-Factory, Japan Investigating fundamental violation
of symmetry in nature (Charge Parity) which may help explain “why do we have more antimatter in the universe OR imbalance of matter and antimatter in the universe?”.
Collaboration 1000 people, 50 institutes
100’s TB data currently
50
Case Study: Event Simulation and Analysis
B0->D*+D*-Ks
• Simulation and Analysis Package - Belle Analysis Software Framework (BASF)• Experiment in 2 parts – Generation of Simulated Data and Analysis of the distributed data
Analyzed 100 data files (30MB each) that were distributed among the five nodes within Australian Belle DataGrid platform.
51
Australian Belle Data Grid Testbed
Grid Service Broker
Replica Catalog
AARNET
NWS NameServer
VirtualOrganization
Analysis Request
Analysis Results
CertificateAuthority
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
GRIDS Lab, University of Melbourne
Dept. of Physics,University of Sydney
ANU, Canberra
Dept. of Computer Science, University of Adelaide
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Intel Pentium 2.0 Ghz, 512 MB RAM
Dept. of Physics,University of Melbourne
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
VPACMelbourne
52
Belle Data Grid (GSP CPU Service Price: G$/sec)
Grid Service Broker
Replica Catalog
AARNET
NWS NameServer
VirtualOrganization
Analysis Request
Analysis Results
CertificateAuthority
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
GRIDS Lab, University of Melbourne
Dept. of Physics,University of Sydney
ANU, Canberra
Dept. of Computer Science, University of Adelaide
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Intel Pentium 2.0 Ghz, 512 MB RAM
Dept. of Physics,University of Melbourne
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NA
G$4
G$4
Datanode
G$6VPAC
MelbourneG$2
53
Belle Data Grid (Bandwidth Price: G$/MB)
Grid Service Broker
Replica Catalog
AARNET
NWS NameServer
VirtualOrganization
Analysis Request
Analysis Results
CertificateAuthority
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
GRIDS Lab, University of Melbourne
Dept. of Physics,University of Sydney
ANU, Canberra
Dept. of Computer Science, University of Adelaide
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Intel Pentium 2.0 Ghz, 512 MB RAM
Dept. of Physics,University of Melbourne
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NA
G$4
G$4
Datanode
G$6VPAC
MelbourneG$2
34
31
38
31
30
3336
32
54
Deploying Application Scenario
A data grid scenario with 100 jobs and each accessing remote data of ~30MB
Deadline: 3hrs. Budget: G$ 60K Scheduling Optimisation Scenario:
Minimise Time Minimise Cost
Results:
SUMMARY OF EVALUATION RESULTS
Scheduling strategy Total Time Taken (mins.)
Compute Cost (G$)
Data Cost (G$)
Total Cost (G$)
Cost Minimization 71.07 26865 7560 34425 Time Minimization 48.5 50938 7452 58390
55
SUMMARY OF EVALUATION RESULTS
Scheduling strategy Total Time Taken (mins.)
Compute Cost (G$)
Data Cost (G$)
Total Cost (G$)
Cost Minimization 71.07 26865 7560 34425 Time Minimization 48.5 50938 7452 58390
Observation
Organization
Node details Cost (in G$/CPU-sec) Total Jobs Executed
Time Cost
CS,UniMelb belle.cs.mu.oz.au4 CPU, 2GB RAM, 40 GB HD, Linux
N.A. (Not used as a compute resource)
-- --
Physics, UniMelb fleagle.ph.unimelb.edu.au1 CPU, 512 MB RAM, 40 GB HD, Linux
2 3 94
CS, University of Adelaide
belle.cs.adelaide.edu.au4 CPU (only 1 available) , 2GB RAM, 40 GB HD, Linux
N.A. (Not used as a compute resource)
-- --
ANU, Canberra belle.anu.edu.au4 CPU, 2GB RAM, 40 GB HD, Linux
4 2 2
Dept of Physics, USyd
belle.physics.usyd.edu.au4 CPU (only 1 available), 2GB RAM, 40 GB HD, Linux
4 72 2
VPAC, Melbourne brecca-2.vpac.org180 node cluster (only head node used), Linux
6 23 2
56
Agenda
Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services
Global Grids and Challenges Security, resource management, pricing models, …
Service-Oriented Grid Architecture and Gridbus Solutions
Market-based Management, GMD, Grid Bank, Alchemi Grid Service Broker
Architecture, Design and Implementation Performance Evaluation: Experiments in Creation
and Deployment of Applications on Global Grids A Case Study in High Energy Physics
Summary and Conclusion
Alessandro Volta in Paris in 1801 inside French National Institute shows the battery while in the presence of Napoleon I
Fresco by N. Cianfanelli (1841) (Zoological Section "La Specula" of National History Museum of Florence University)
58
….and in the future, I imagine a WorldwidePower (Electrical) Grid …...
What ?!?!This is a mad man…
Oh, monDieu !
59
2006 - 1801 = 205 Years(Recent R&D: Delivering Internet services via Electric cables).
60
When are we going to get IT as the 5th utility?(water, electricity, gas, telephone, IT)
eScienceeBusiness
eGovernmenteHealth
MultilingualeEducation
…
61
Summary and Conclusion
Grids exploit synergies that result from cooperation of autonomous entities:
Resource sharing, dynamic provisioning, and aggregation at global level Great Science and Great Business!
Grids have emerged as enabler for Cyberinfrastructure that powers e-Science and e-Business applications.
SOA + Market-based Grid Management = Utility Grids
Grids allow users to dynamically lease Grid services at runtime based on their quality, cost, availability, and users QoS requirements.
Delivering ICT services as computing utilities. Grids offer enormous opportunities for realizing
e-Science and e-Business at global level.
62
Thanks for your attention!
We Welcome Cooperation in Research and Development!http:/www.gridbus.org
Gridbus Middleware:Building Autonomic and Market-Oriented Global Grids for
Delivering IT Services as the 5th Utility
Dr. Rajkumar Buyya
Grid Computing and Distributed Systems (GRIDS) LaboratoryDept. of Computer Science and Software EngineeringThe University of Melbourne, Australia
www.gridbus.org
Gridbus Sponsors
Backup Slides
65
Some Characteristics of Grids
Numerousresources
Different securityrequirements
& policies
Resources areheterogeneous
Geographicallydistributed
Different resourcemanagementpolicies
Connected byheterogeneous, multi-level networks
Owned by multiple organizations &
individuals
Unreliable resources and environments
Slide by Hiro
66
What are Grid benefits?
Resource sharing across multiple administrative boundaries Effective utilisation of the (existing) resources Dynamic provisioning
Application Acceleration Scalability Reliability Virtualisation:
applications, services, resources,…
Utility Grids:Autonomic and Market-Oriented Global Grids for Powering e-Science and e-Business Applications
Dr. Rajkumar Buyya
Grid Computing and Distributed Systems (GRIDS) LaboratoryDept. of Computer Science and Software EngineeringThe University of Melbourne, Australia
www.gridbus.org
Gridbus Sponsors