Agent-Based Grid Load-Balancing
Daniel P. SpoonerUniversity of Warwick, UK
Junwei CaoNEC Europe Ltd., Germany
Outline Grid Computing & Middleware Agent-Based Methodology Distributed Service Discovery Grid Load-Balancing Experimental Results Conclusions & Future Work
Grid Computing Computational Grids - blueprint for a
new computing infrastructure
Grid/Web Services - distributed system and application integration
Data/Service/Community Grids - large-scale resource sharing in VOs
Grid Middleware Resource Management & Scheduling Information Services (Monitoring & Discovery) Data Management and Access Application Programming Environments Security, Accounting, QoS ……
Condor, LSF, Ninf, Nimrod, ……Globus, Legion, DPSS, ……Java/Jini, CORBA, Web Services, ……
Grid Resource Management
Key challenges: Cross-domain Large-scale Dynamic QoS support
GRM
Agent-Based MethodologyAn agent is: A local grid manager An user agent A broker A service provider A service requestor A matchmaker
AAA A
Local ManagementProcessor 1Processor 2Processor 3Processor 4Processor 5Processor 6Processor 7Processor 8
2n-1
Performance Prediction Task models Hardware models
FIFO AlgorithmGenetic Algorithm Heuristic &
Evolutionary Near-optimal on
makespan, deadlines and idletime.
Service Discovery Pure data-pull No advertisement Full discovery Efficient when service
change more quickly
Pure data-push Full advertisement No discovery Efficient when
requests arrive more frequently
R/A
A U/A1 TaskInfo
2 PullSerInfo
3 PullSerInfo4 Return
R/A
R/A
A U/A3 TaskInfo
1 PushSerInfo
2 PushSerInfo4 Return
R/A
Centralised, not applicable for grid computing!
Optimisation Strategies Configurable data-pull or data-push Agent hierarchy Multi-step advertisement & multi-step discovery Efficient when frequencies of request arrivals
and service changes are almost the same
Distributed!balancing betweenadvertisement anddiscovery!
A
A A
A A
User
Agent Implementation
PerformancePrediction
FIFOScheduling
DatabaseManagement
MatchMaker
Advertisement Discovery
Task Model
Results
Sched. Time User Deadline
To Another Agent
Task Execution
TaskManagement
FIFO/GA Scheduling
Service Info.Provider
Current M
akespanService Info
Hw Model
Task ExecutionResourceMonitoring
Task Execution
Load Balancing Metrics Total makespan Average advance time of task execution
completions (required deadline - actual task completion time)
Average processor utilisation rate (busy time / total makespan)
Load balancing level (1 - mean square deviation of processor utilisation rates / average processor utilisation rate)
Total number of network packages for both advertisement and discovery
Experiment DesignS1
(SGIOrigin2000, 16)
S2(SGIOrigin2000, 16)
S4(SunUltra10,
16)S3(SunUltra10,
16)
S5(SunUltra5,
16)
S6(SunUltra5,
16)
S12(SunSPARCsta
tion2, 16)
S11(SunSPARCsta
tion2, 16)
S8(SunUltra1,
16)
S7(SunUltra5,
16)
S10(SunUltra1,
16)
S9(SunUltra1,
16)
Tasks:sweep3dfftimprocclosurejacobimemsortcpi
Experiment 1 FIFO
FIFO FIFO
FIFO FIFO
Experiment 2 GA
GA GA
GA GA
Experiment 3 GA
GA GA
GA GA
Task Execution
-1200
-1000
-800
-600
-400
-200
0
200
1 2 3
Experiment Number
ε (s
)
S1S2S3S4S5S6S7S8S9S10S11S12Total
Both GA and agents contribute towards the improvement in task executions.
Resource Utilisation
0
10
20
30
40
50
60
70
80
90
1 2 3
Experiment Number
υ (%
)
S1S2S3S4S5S6S7S8S9S10S11S12Total
Less powerful S11 & S12 benefit mainly from the GA.
More powerful S1 & S2 benefit mainly from agents.
Load Balancing
0
10
20
30
40
50
60
70
80
90
100
1 2 3
Experiment Number
β (%
)
S1S2S3S4S5S6S7S8S9S10S11S12Total
The GA contributes more to local grid load balancing.
Agents contribute more to global grid load balancing.
Total Makespan
The centralised pure data-pull can always achieve the best results
Distributed agents with the hierarchical model can also achieve reasonably good results0
20
4060
80100
120140
160
4 6 8 10 12 14 16 18 20
Agent Number
t (m
ins)
CentralisedDistributed
Network Package
The network overhead for the pure data-pull strategy to achieve better results is very high.
Distributed agent-based service advertisement and discovery can scale well.
0
5000
10000
15000
20000
25000
4 6 8 10 12 14 16 18 20
Agent Number
pack
ets
CentralisedDistributed
Conclusions An multi-agent paradigm provides a
clear high-level abstraction of grid resource management system.
Distributed service advertisement and discovery strategies can be used to improve agent performance.
Agent-based framework is scalable, flexible, and extensible for further enhancements.