Load Balancing in Distributed Systems Nalini Venkatasubramanian [email protected]

Global Distributed Systems and Multimedia

Load Balancing in Distributed Systems

Nalini [email protected]


2

Motivation

A given computer is overloaded Must decrease load and maintain certain

characteristicsScalabilityPerformance

• Throughput

Ideally, this should be done transparently Solution: Load Balancing


3

Introduction to Load Balancing

Distributed resource allocation Can be thought of as “distributed scheduling”

Deals with distribution of processes among processors connected by a network

Handles issues such as deciding which process should be handled by a given processor

Can also be influenced by “distributed placement”

Important in data intensive environments and applications

Data placement may force process placement


4

Load Balancing Relationships

Load Balancer Manages resources Resource assignment

depends on policy or policies in effect

Client Requests resources Requests services

Clients

LoadBalancer

Resources

Requestresources

Applypolicies


5

Load Balancing Overhead

Satisfy client resource access without imposing large amounts of overhead Performance

How well resources are managed

EfficiencyCost of accessing and using a resource obtained

through a load balancer


6

Load Balancing Issues

When to migrate processes or forward requests

Which processor should be chosen to handle a given process or request

Should processes be moved off a computer

How should searching for lightly loaded computer be performed


7

Load Balancing Issues (cont.)

When should load balancing decisions be made

What should be taken into account when making the above decisions

How should old data be handledShould load balancing data be stored

centrally, or in a distributed manner


8

Load Balancing Issues (cont.)

Should computers make decisions together

What is the performance/overhead tradeoff incurred by load balancing

Prevention of overloading a lightly loaded computer


9

Load Balancing Techniques

Basically two ways to perform load balancing Statically

Resource is allocated once

DynamicallyResource is allocated and managed (possibly

dynamically reallocated) to ensure balanced load


10

Static Load Balancing

Resource allocation is performed once Once resource is allocated it remains allocated

(for what duration??)

Scheduling decisions are made Deterministically Probabilistically


11

Static Load Balancing – cont’d

Advantages State generally need not be stored

Simplifies implementation Less network traffic due to load balancing related

messages Disadvantages

Poor resource utilization A given resource may be used much more than others

Does not adjust to fluctuations in the load Possible for resource to become overloaded


12

Static Load Balancing – cont’d

Example of Static Load Balancing Forwarding processes/requests to a given

computer based on dynamically assigned addresses (e.g. via DNS)

Web servers (e.g. CNN)UCI host ea.uci.edu is load balanced


13

Dynamic Load Balancing

Attempts to maintain a balanced load by managing resources while a resource is in use May involve the following

Process migrationDisabling further access to a resource until a later

timeAdding new resources “on-the-fly”


14

Dynamic Load Balancing Strategies

Distributed versus non-distributed Should load information be stored centrally or

across several hostsSimplicity versus overhead and reliability

Cooperative versus non-cooperative Should decisions be made by a single load

balancer or severalGlobally managed resources versus locally managed

resources


15

Dynamic Load Balancing Strategies - cont’d

Adaptive versus non-adaptive Should previous data effect scheduling

decisions

Preemptive versus non-preemptive Should a running process be preempted in

favor of another process, or for migration to another resource


16

Case Studies

Load Balancing for Web ServersLoad Balancing for Parallel ComputersLoad Balancing for Multimedia

Applications


17

Multimedia Applications

ElectronicCommerce

Entertainment

Distance Learning

Tele-medicine

VideoServers

Graphics Processing

Web Servers

Requirements - Availability, Reliability, Quality-of-Service, Cost-effectiveness, Security

http://www.starwars.com/episode-i/snapshot/1999/10/snapshot19991005.html


18

Multimedia Load Management

Primary focus resource optimization across streams and

resource management across servers.Quality of Service

• continuous delivery requirement• minor violations of performance requirements

Admission Control - resource reservation/negotiationMedia Delivery - Resource scheduling (CPU,Disk)

Resource Mgmt. Implies Admission ControlCaching, VCR Control, Server Selection, Data

Placement


19

Distributed MM Servers

Video Server TopologyPartitioned Server Externally Switched Fully Switched

Video File Placement Heterogeneous Workload - large/small, hot/cold Online Placement Good placement is important

dynamic replication time-consuming, dynamic load-bal complementary


20

Dynamic Load Balancing

Adapts to statistical fluctuations and changing access patterns

Dynamic MigrationDeals with poor initial placement

ReplicationDynamic Segment Replication

• partial replication (quick response, less expensive)Total Replication

• on-demand vs. predictive


21

Load Management of Distributed MM Servers

Adaptive SchedulingAssigns requests to servers based on demand and

load factors.Invokes replication-on-demand, request migration

Predictive PlacementInvokes dereplication

OptimizationsEager ReplicationLazy Dereplication


22

A Scalable Video Server Architecture

Distribution NetworkDistribution Network

DistributionController

DistributionController

Data SourceData

SourceData

SourceData

SourceData

SourceData

SourceTertieryStorageTertieryStorage...

control

requests data


23

Architectural View of a Networked MM System

Qos Broker and Load Management SystemQos Broker and Load Management System

NodeManager

NodeManager

NodeManager

NodeManager

...

Local Data Streaming


24

Resources in a Video Server

NetworkNetwork

ClientClient

CommunicationModules

Data ManipulationModules

StorageModules

ProcessingModule


25

Load Placement Scenario

DataSourceS2

DataSourceS1

Access Network

Clients

...

Storage: 8 objectsBandwidth: 3 requests

Storage: 2 objectsBandwidth: 8 requests


26

Characterizing Server Resource Usage

Ability to service a request on a server depends on:

resource availablecharacteristics of a request

Load factor(LF) for a request:represents how far a server is from request

admission threshold.LF (Ri, Sj) = max (Dbi/DBj , Mi/Mj , CPUi/CPUj , Xi/Xj)


27

Adaptive Scheduling

When the broker receives a request Ri for a video object Vi :

Consider only data sources that have a copy of Vi.Consider only data sources tha have sufficient

resources to support Ri.Chooser server for which LF (Ri, Sj) is a minimum.If no such server exists

• Reject request.• Perform replication-on-demand.• Perform request migration.


28

Predictive Data Placement

Determines when, where and how many replicas of a video object.

Initiated periodically.Results in an assignment of replicas to

data sources.Greedy algorithm that uses revenue

generated as a metric.


29

The Greedy Cost Placement Matrix

S1 S2

V1 min(N1,1/LF(R1,S1))*r1 min(N1,1/LF(R1,S2))*r1



max(PM(Vi,S1)) max(PM(Vi,S2))

PM(Vi, Sj) is the maximum revenue that can accrue from allocating Vi to Sj.

Greedy heuristic: Map(Vi,Sj) = 1 if PM(Vi,Sj) = a b max(PM(Va,Sb))


30

Optimizations

To minimize the overhead of replication Eager replication

Replication of video object in anticipationPerformed when server resources are free

Lazy DereplicationCritical nature of storage resourcesMark reusable resources, reclaim disk space laterIf disk blocks are not overwritten, can be reclaimed


31

Life of a video object


32

Performance Evaluation Policies

Policy Placement of Video Objects Request Scheduling

P1 Replication on-demand Adaptive

P2 Predictive placement Predictive

P3 Predictive placement Adaptive

P4 Predictive placement + eagerreplication

Adaptive


33

Performance Evaluation - Startup Latencies


34

Performance of the basic configuration

p1

p2

p4

p3


35

Performance Evaluation - Varying Replication BW


36

Performance Evaluation Summary

P1: entails high startup latency, requires high storage and replication bandwidth.

P2: Unacceptably poor performance.P3: Similar performance to P4 in many

cases. At low transfer bandwidths, P4 outperforms P3.

P4: Performs well in all cases.

Documents

Load Balancing in Distributed Systems Nalini Venkatasubramanian [email protected]