1

Black-box and Gray-box Strategies for Virtual Machine Migration

Timothy Wood, Prashant Shenoy, Arun Venkataramani, and Mazin Yousif †

Univ. of Massachusetts Amherst†Intel

4th USENIX Symposium on Networked Systems Design & Implementation (NSDI 2007)

2

Introduction

• Operate application in data center.– Effective management of data center resources

while meeting SLAs• Virtualization

• Benefit of Virtualization– Application isolation– Server consolidation(multiplexing)– Handle workload dynamics

3

Motivation

• Efficient data center resource management– Live Migration

• However, detecting workload hotspots and initiating a migration is currently handled manually– Lacks the agility to respond to sudden workload

changes– Need consider multiple resource• CPU, network, and memory

4

Solution

• Automated black-box and gray-box strategies for virtual machine migration (Sandpiper)– Monitoring system resource usage– Hotspot detection– Determining a new mapping– Initiating the necessary migrations

5

The Sandpiper ArchitectureGathering

resource usage statistics on that

serverGathers processor, network and memory swap statisticsfor each VM Implements a

daemon to gatherOS-level statistics and application logs

Construct resource usage profiles foreach virtual server(Predict PM workload)

Monitors usage profiles to detect hotspots.Hotspot: any resource exceeds a threshold(or SLA violation) for a sustain period

Determine:What virtual servers should migrateWhere to move themHow much of a resource to allocate the virtual servers after migration

6

Black-box monitoring(1/4)

• VM workload usages is inferred solely from external observations.– From Domain-0.

• Monitoring parameter: – CPU usage– Network bandwidth– Memory swap rate

• Monitoring interval

7

Black-box monitoring(2/4)-CPU monitoring

• VM CPU usage can be determined by tracking scheduling events in the hypervisor.– Does not include VM’s disk IO and network CPU

overhead.• These kinds of overhead is count on Domain-0• Each VM is then charged:

– domain-0’s CPU usage*(VM IO request/ total IO requests)

• Assumption: the monitoring engine and the nucleus overhead is negligible

8

Black-box monitoring(3/4)-Network monitoring

• Background:– Domain-0 in Xen implements the network

interface driver– VMs access the driver via clean device

abstractions(virtual firewall-router (VFR) interface)• Monitoring engine can use the Linux /proc

interface VNIC’s usage– /proc/net/dev

9

Black-box monitoring(4/4)-Memory monitoring

• Challenge:– Domain-0 cannot directly monitor each VM’s

actual memory usage/utilization.• Only know the amount of memory assigned to the VM.

• Solution:– Observing swap activity in Domain-0 can infer the

working set sizes.[11]

[11] S. Jones, A. Arpaci-Dusseau, and R. Arpaci-Dusseau. Geiger: Monitoring the buffer cache in a virtual machine environment. In Proc. ASPLOS’06, pages 13–23, October 2006.

10

Gray-box monitoring

• Motivation:– Black-box monitoring is not feasible to “peek inside” a VM to

gather usage statistics.• Solution:

– Install a light-weight monitoring daemon inside each virtual server

– Use /proc interface to gather OS-level statistics• CPU, network, memory

– Application-level statistics• Daemon get statistics from function provided by application itself• E.g. web/database server: request rate, request drop rate, service

time

11

Profile Generation(1/2)

• Profile: a compact description of that server’s resource usage over a sliding time window W.

• Profile content:– Blackbox parameter:• CPU utilization, network bandwidth utilization, and

memory swap rate

– Graybox parameter:• memory utilization, service time, request drop rate and

incoming request rate. (assumption: web server-apache)

12

Profile Generation(2/2)

• Profile type:– Distribution profile:• The probability distribution of the resource usage over

the window W.

– Time series profile:• The temporal fluctuations and it is simply a list of all

reported observations within the window W.

13

• Goal:– Signaling a need for VM migration whenever SLA

violations are detected.• A hotspot is flagged only if thresholds or SLAs

are exceeded for a sustained time.– at least k/n most recent observations and the next

predicted value exceed a threshold.• Use time series profile• Formula: (auto-regressive family of

predictors-AR(1).)

Hotspot detection

14

Resource Provisioning

• Goal:– Ensures that the SLAs are not violated even in the

presence of peak workloads.• Estimate the peak CPU, network and memory

requirement of each overloaded VM

• Black-box provisioning• Gray-box provisioning

15

Black-box provisioning(1/3)

• Estimation of peak CPU&Network bandwidth needs:– Distribution profile• Use historical data to predict the peak.• Challenge: Estimation error!!

• Background:– Both the CPU scheduler and the network packet

scheduler in Xen are work-conserving.

16

Black-box provisioning(2/3)

• Estimation error:– Example:• Two virtual machines that are assigned CPU weights of

1:1(50% of each)• Assume that VM1 is overloaded and requires 70% of the

CPU to meet its peak needs.

Case 1 Case 20%

20%

40%

60%

80%

100%

70%50%

20% 50%

VM2VM1

17

Black-box provisioning(3/3)

• Solution of estimation error:– adds a constant Δ to scale up this estimate.

• Estimation of peak memory needs:– If swap activity exceeds the threshold.– Then the current allocation is deemed insufficient

and is increased by a constant amount Δm

18

Gray-box provisioning(1/3)

• The gray-box approach can access to application-level logs.– Ability to estimate the peak resource needs of the

application even when the resource is fully utilized.

• Estimating peak CPU needs:– An application model is necessary to estimate the

peak CPU needs.

19

Gray-box provisioning(2/3)

• Estimating peak CPU needs(cont.):– Applications such as web and database servers can be

modeled as G/G/1 queuing systems[23].– G/G/1 queuing system behavior[13]:

• = mean service time (obtain from server log)• d = mean response time of request(obtain by SLA)• = request arrival rate• = variance of inter-arrival rate(obtain from server log)• = variance of service time(obtain from server log)

[23] B. Urgaonkar, P. Shenoy, A. Chandra, and P. Goyal. Dynamic provisioning for multi-tier internet applications. In Proc. ICAC ’05, June 2005.[13] L. Kleinrock. Queueing Systems, Volume 2: Computer Applications. John Wiley and Sons, Inc., 1976.

20

Gray-box provisioning(3/3)

• Estimating peak CPU needs(cont.):– We can map the current CPU usage with , then the

peak CPU usage can be calculated:

• Estimating peak network bandwidth

– b = mean requested file size

21

Hotspot mitigation(1/3)

• Hotspot mitigation alg:– Goal:

• Determine which VM should be migrate to where to dissipate.

– Challenge:• NPHard--- multi-dimensional bin packing problem

– Bin=physical server, dimension=resource constraints

– Solution:• A heuristic which solve:

– Which overloaded VMs to migrate– Migrate to where such that migration overhead is minimized.

» Migration overhead can not be neglect

22

Hotspot mitigation(2/3)

• Hotspot mitigation alg(cont.):– Intuition:

• Move load from the most overloaded servers to the least-loaded servers,

• minimize data copying incurred during migration

– Volume: the degree of load along multiple dimensions in a unified fashion.

• where cpu, net and mem are the corresponding utilizations of that resource for the virtual or physical server

23

Hotspot mitigation(3/3)

• Hotspot mitigation alg(cont.):– volume-to-size ratio (VSR):

• Volume/Size(Size=the memory size of the VM)

– Migration decision:• Move highest VSR VM from the highest volume server and

determines if it can be housed on the least volume physical server.

– Swap decision(only consider 2-way swap):• Activate when simple migration cannot solve hotspot.• Swap the highest VSR VM on the highest volume hotspot server

with k lowest VSR VMs in lowest volume server– If a swap cannot be found, the next least loaded server is considered

• Note: a swap may require a third server(RAM issue)

24

Implementation• Virtualization platform:

– Xen• Sandpiper Control plane:

– Run on the control node(Python)• Profiling Engine:

– Use past 200 measurement to generate profile• Hotspot trigger:

– 3/5 (k/n) past reading+next predicted over threshold• Default threshold:

– 75%• Monitoring Engine:

• Gray-box monitoring daemon: – Linux OS daemon, Apache module(service time, request rate, drop rate, file size)

25

Evaluation Environment

• Data center:– 20 server(2.4Ghz pentium-4 servers)– Connected with gigabit ethernet– At least 1GB ram

• OS– Linux 2.6.16+Xen 3.0.2-3

• Workload generator– A cluster of Pentium-3 Linux servers

26

• Experiment 1 uses 3 physical servers and 5 VMs with memory allocations as following.

• All VMs run Apache serving dynamic PHP web pages.

• Use httperf to inject a workload

Experiment 1-Migration Effectiveness

27

Experiment 1-Migration Effectiveness(cont.)

t=166,Hotspot detected,VM1 has highest VSR

PM3 has lowest volume

t=362,Hotspot detected,VM4 has 2-nd highest VSR

(no PM has enough capacity to host VM3)

PM1 has lowest volume

VM3

In final phaseVM1 and VM5 the same Volume

But VM5 use smaller memoryPM2 has lowest volume

28

Experiment 2- Virtual Machine Swaps

VM ID RAM(MB) Host Machine CPU Load type

VM1 384 PM1 Steadily increase

VM2 384 PM1 Constant

VM3 256 PM2 Constant

VM4 256 PM2 Constant

• Experiment setting:

• As before, clients use httperf to request dynamic PHP pages.

29

Experiment 2- Virtual Machine Swaps(cont.)Hotspot detected on PM1.

The only viable solution is to swap VM2 with VM4.(3 party swap)

VM4 use smallest memory, so it is migrated twice.

Migration of VM4 is completed, VM2 start to be migrated to PM2.

Migration of VM2 is completed, VM4 start to be migrated to PM1.Migration overhead

30

Experiment 3- Mixed resource workloads

• Experiment setting:

• VM2 is database that stores its table in memory• PM2 has more physical memory

VM ID Description Host Machine

RAM(MB)

VM1 Network intensive PM1 256

VM2 Network intensive+ memory grow over time

PM1 256

VM3 CPU intensive PM2 256

VM4 CPU intensive PM2 256

31

Experiment 3- Mixed resource workloads(cont.)

• PM1 has a network hotspot and PM2 has a CPU hotspot

• Sandpiper swaps a network intensive VM for a CPU-intensive VM at t=130

32

Experiment 3- Mixed resource workloads(cont.)

• Sandpiper responds by increasing the RAM allocation in steps of 32MB every time swapping is observed;

• When no additional RAM is available, the VM is swapped to the second physical server at t=430.– Swap two Network-intensive VM(VM1 and VM2)

33

Experiment 4- Gray v. Black: Memory Allocation

• Goal:– Compare the effectiveness of the black- and graybox

approaches in mitigating memory hotspots• Using the SPECjbb 2005 benchmark generate memory usage.

• Settings:Host Machine

PM RAM(MB) VM ID VM RAM(MB) Description

PM1 384 VM1 256 Gray-box strategy

PM2 384 VM2 256 Black-box strategy

PM3 1024 none none Idle server(Wait for migration)

34

Experiment 4- Gray v.s. Black: Memory Allocation(cont.)

• Experiment Result:

• Observation:– The gray-box system can reduce or eliminate swapping without significant

overprovisioning of memory.

35

Experiment 4- Gray v.s. Black: Apache Performance

• Settings:•

• We use httperf to generate requests for CPU intensive PHP scripts on all VMs.

Host Machine

Hosting VM

Actual VM CPU requirement

PM1 VM1~2 70%

PM1 VM3 33%

PM2 VM4 7%

PM3 none None

36

Experiment 4- Gray v.s. Black: Apache Performance

• Black-box strategy error guess:

1 2

3 4

37

• Compare Gray-box strategy with Black-box strategy:

• Gray-box strategy can migrate VM3 to PM2 and VM1 to PM3 concurrently

Experiment 4- Gray v.s. Black: Apache Performance

38

Experiment 5-Prototype Data Center Evaluation

• Data Center environment– 16 servers that run a total of 35 VMs.– 1 additional server runs the control plane – 1 additional server is reserved as a scratch node for

swaps.• Settings:– Six physical servers running a total of 14 VMs to be

overloaded• four servers see a CPU hotspot and two see a network

hotspot

39

Experiment 5-Prototype Data Center Evaluation

• Result:– Sandpiper eliminates hotspots on all six servers by

interval 60. Migration overhead

40

Sandpiper overhead and scalability

• Sandpiper’s CPU and network overhead: – depends on the number of PMs and VMs in the

data center.– Overhead of Graybox strategy may affected by the

size of application-level statistics gathered

41

Sandpiper overhead and scalability(cont.)

• Nucleus overhead:– Network:• Each report uses only 288 bytes per VM• The resulting overhead on a gigabit LAN is negligible

– CPU usage:• Compare the performance of a CPU benchmark with

and without our resource monitors running.– On a single physical server running 24 concurrent VMs,

• Nucleus overheads reduce the CPU benchmark by approximately 1%.

42

Sandpiper overhead and scalability(cont.)

• Control Plane Scalability:– Source of computation complexity• Computation of a new mapping of virtual machines to

physical servers after detecting a hotspot

43

Conclusion&future work

• In this paper, we proposed Sandpiper, a automatic system which can:– monitoring and detecting hotspots– determining a new mapping of physical to virtual resources– initiating the necessary migrations

• We discussed a blackbox strategy and graybox strategy.• Evaluation showed we can bring rapid hotspot elimination in

data center environments.• Future work:

– Support replicated services • automatically determining whether to migrate a VM or to spawn a

replica.

44

Comment

• Advantage:– Good point to separate the monitoring strategy in

blackbox and graybox.– Sandpiper’s architecture and strategy may fit our “Plan A”

• Shortage:– The relationship of CPU utilization and request rate may

not be linear– The hotspot mitigation algorithm only consider average

the workload between physical machine• Should consider how to make PM get highest utilization without

hotspot