Internet Service Migration and Placement

Part 1

Instructor: Xiaodong Zhang

Xiaoning Ding11/08/2004

Outline Background OPUS: An Overlay Peer Utility Service

Overview Architecture Research issues

Model-based resource provisioning Overview Web service model Model-based resource allocator

Outsourcing Services & Utility-based services

Outsourcing services Customer-owned or leased

system. The service provider takes r

esponsibility for managing the customer’s IT and network system – the computing infrastructure – based on customer-defined service level agreements (SLA).

Billed on a monthly or fixed-fee basis.

Utility-based services The service provider owns

the infrastructure leases the infrastructure to

the customers pay for what you use Example: Internet data

center enabling ASPs to deliver ASP services

Utility & SLA

Utilities deliver IT resources (CPU, storage, and bandwidth) to hosted application and, ultimately, end users

much as the electric utility transparently delivers power on demand to customers.

Applications agree to Service Level Agreements (SLAs) with the utility

Dedicate fixed resources per application

Reprovision manually as needed

Overprovision for surges High variable cost of capacity

Static Provisioning

0

0 Time (two months)

Th

rou

gh

pu

t (r

equ

ests

/s)

Load Is Dynamic

World Cup soccer site • May-June 1998• Seasonal fluctuations• Event surges (11x)• ita.ee.lbl.gov

0

0 Time (one week)

Th

rou

gh

pu

t (r

eq

ue

sts

/s)

M T W Th F S SM T W Th F S S

Week 6 7 8Week 6 7 8

ibm.com external site• February 2001• Daily fluctuations• Workday cycle• Weekends off

Adaptive Provisioning

offer economies of scale

Network access Power and cooling Administration and

security Surge capacity

Overlay network and Mobile code Increasing number of important network

services are deploying overlays CDN, Replicated services, Storage systems... Dynamically map data and functions onto

network resources Programs and data will adaptively migrate

and replicate in response to changing network conditions, client access characteristics,... Programs dynamically run at optimal network

locations Data dynamically flow to where it is required.

Outline Background OPUS: An Overlay Peer Utility Service

Overview Architecture Research issues

Model-based resource provisioning Overview Web service model Model-based resource allocator

OPUS: An Overlay Utility Service

App demand(per network region)

Overlay node

Peering

Allocate nodes to services based on current demand

OPUS: Overview targeting utilities consisting of a distributed set

of thousands of server sites, each with potentially 1000's of individual machines, cooperating together to fulfill aggregate SLAs

Simultaneously hosts multiple distributed applications replicated web services application-layer multicast content distribution networks. ...

Opus tasks Resource allocation

Allocate resources among competing applications Maximize aggregate performance Based on changing application and network characte

ristics, SLAs Replica placement

Closely related to resource allocation Where to place individual application replicas Consider dynamically changing client access patterns,

network failures, etc.

Opus tasks

Overlay topology construction create overlays that meet application

requirements of performance, delay, and reliability

minimize consumed network resources Request routing

discover the service replica capable of delivering the highest quality of service

OPUS: Architecture

The service overlay

Each Opus site runs an instance of site manager coordinating resource usage at that site and exchange status summaries with other opus sites.

Interconnects all active nodes and provides overlay services

“Backbone” for coordinated, decentralized resource allocation and resource control

The service overlay

Assist the construction and maintenance of application overlay

Dynamic and self-healing Scalability issue

Hierarchical data dissemination in dicast Think globally but act locally

Adaptive per-application overlay Each application uses its application ove

rlay to Route internal application traffic Disseminate content Synchronize state information …

The topology and site allotments are subject to change by resource allocator

Security and isolation

Allocating resources to applications at the granularity of individual nodes

Future plan: using virtual machine Using VLAN to isolate traffic on the

wire

Research Issues

Overlay topology construction Resource allocation Scalable tracking of system characteristi

cs Reliability QoS guarantees

Overlay topology construction Emphasize scalability

Quantify the benefits of competing structures Develop scalable distributed constructing algorithms

Initial work A general overlay topology that enables dynamic

tradeoffs between network performance/reliability and cost

Focus on network cost and relative delay penalty (RDP) to characterize overlay topology

Two candidate overlay topologies: K-spanner and LAST.

Overlay topology construction

Overlay topology construction

Distributed algorithms for building and maintaining the topology Selectively probing using probabilistic techniq

ues and hierarchy Using partial, approximate and probabilistic kn

owledge of network infomation Having each node gradually migrate to its “pr

oper” location in the overlay.

Resource allocation

classical economic model Customers are associated with utility

functions specifying the value of the services result from a allotment. (concave functions)

Opus maximizes global value across all applications.

Optimal solution: the marginal value of an additional resource unit is in equilibrium across all customers.

Resource allocation

Allocated Resources

Th

rou

gh

pu

t (V

alu

e) App2

App1Gra

dient 2

Gradient 1

Resource allocation

Scalability consideration Adapt from economic resource allocation

Decentralized federation of autonomous local “markets” exchanging information to converge toward a global equilibrium

Celluar structure Cell: an entire Opus site or a portion of large sit

e Cells plan their internal allocation locally Cells operate to trade load or resources

Tracking system characteristics Nodes are partitioned into clusters of siz

e d. Each cluster elects an agent responsible f

or disseminating local cluster information

Agents from d adjacent clusters form second-level clusters

All nodes are organized into a tree called dicast tree. Height=logdN

Tracking system characteristics

Hierarchical data dissemination in dicast

Tracking system characteristics Data travels up the tree, and may be

aggregated with data from the nodes At each level of the tree, an overlay

propagates the data among all participating cluster members

Updates are buffered awaiting the arrival of further updates until a threshold is reached, and updates are aggregated

Each node may has exact information of “nearby” nodes in the same

cluster Aggregate information of remote cluster

Reliability QoS GuaranteesAddress network level failures Restricted flooding

Redundantly transmit the same data over multiple logical path

Minimizing the overhead Intermediate nodes re-evaluate the

reliability of the remainder of the path, and choose between forwarding redundant data and suppressing duplicate data

Reliability QoS Guarantees

0.96A

B J DS0.97

0.98

0.97

0.99

SAD: 0.96*0.98*0.99=0.931

SAD: 0.97*0.97*0.99=0.931SA and BD: (1-(1-0.96*0.98)*(1-0.97*0.97))

*0.99=0.987

Reliability QoS Guarantees To match the overlay topology with the

failure characteristics of underlying network Construct overlays with disjoint paths to

lower the failure correlation among logical overlay links

Collect statistical information about loss correlation

Use network topology information

Outline Background OPUS: An Overlay Peer Utility Service

Overview Architecture Research issues

Model-based resource provisioning Overview Web service model Model-based resource allocator

Overview Addresses the provisioning problem

Multiple competing services hosted by a shared server cluster (utility)

How much resource does a service need to meet SLA targets

Applications Static web content Heavily resource-intensive Predictable in average per-request resource

demands

System architecture

The use of the model

Web service model

Web service model

)1(1

11

1

T

MH

)2()1( HSs

)3()/(1

/

ss

ss

kR

α Zipf locality parameter

λ Offered load in requests/s

S Average object size

T Total number of objects

M Memory size for object cache

Rp CPU response time

H Object cache hit ratio

λs Storage request load in IOPS

Rs Average storage response time

Web service model

)4()/(1

/

ss

kR

μs,φPeak storage throughput in IOPS

RpCPU response time

H Object cache hit ratio

λsStorage request load in IOPS

RsAverage storage response time

R Average total response time

)5()1( HRRR sp

Model-based resource allocator Periodically invoked by the utility OS exe

cutive to adjust the allotments Focus on memory and storage resources,

ignore CPU constraints Output

an allotment vector for each service CPU share,Memory and storage allotment [M,

φ]

Model-based resource allocator Resource provisioning primitives

Candidate plans initial candidate allotment vectors

LocalAdjust modifies a candidate vector to adapt to local resource constraint or surplus

GroupAdjust modifies a set of candidate vectors to adapt to a resource constrait or surplus

Model-based resource allocatorGenerating Initial Candidates

Ρtarget Rp

Φ=μs

Rp, Φ, ρtarget Rs (4)

s

p

R

RRH

1

λ,H λs (2)

Φ=λs / ρ target

|Φ-Φdesired|<ε

H M (1)

References

Utility Computing White Paper: http://www.sun.com/service/utility/FINAL_UC_WP.pdf

Service Utilities: http://issg.cs.duke.edu/utilies.html D. G. Andersen, H. Balakrishnan, M. F. Kaashoek, and R. Morris, "Resili

ent Overlay Networks," in 18th ACM Symposium on Operating Systems Principles (SOSP), October 2001, pp. 131-145.

"OPUS: Overlay Utility Service", Rebecca Braynard, Dejan Kostic, Adolfo Rodriguez, Jeff Chase and Amin Vahdat, poster at 18th ACM Symposium on Operating System Principles (SOSP), Banff, Canada, October 2001. ( poster)

R. Braynard, D. Kostic, A. Rodriguez, J. Chase, and A. Vahdat. Opus: an Overlay Peer Utility Service. IEEE OPENARCH 2002.

Ronald P. Doyle, et. al., ``Model-based resource provisioning in a Web service utility", Proceedings of the 4th USENIX Symposium on Internet Technology, 2003.