Virtualization Performance

H. Reza TaheriSenior Staff Eng.VMware

2Copyright © 2006 VMware, Inc. All rights reserved.

Three Key Properties of Virtualization

Partitioning• Run multiple operating

systems on one physical machine

• Fully utilize server resources• Support high availability by

clustering virtual machines

Encapsulation• Encapsulate the entire state

of the virtual machine in hardware-independent files

• Save the virtual machine state as a snapshot in time

• Re-use or transfer whole virtual machines with a simple file copy

Isolation• Isolate faults and security at

the virtual-machine level• Dynamically control CPU,

memory, disk and network resources per virtual machine

• Guarantee service levels

3Copyright © 2006 VMware, Inc. All rights reserved.

Hosted vs. Native Virtualization

Native/Hypervisor Hosted

• Device support is inherited from host operating system for maximum hardware compatibility

• Virtualization installs like an application rather than like an operating system

• Can run alongside conventional applications

• Maximum performance with lowest overhead using certified hardware

• Highly efficient direct I/O pass-through architecture for network and disk

• Highly secure micro-kernel virtualization layer—only 100Ks of lines of code versus 10–25 million lines of host operating system code

• Advanced features like VMotion available

(ESX Server)(Workstation, VMware Player, VMware Server, ACE, VMware Server)

4Copyright © 2006 VMware, Inc. All rights reserved.

ESX Server Architecture

Hardware

VM

Device Drivers

Storage Stack Network Stack

VM

VMM VMM

VMkernel

POSIX API

VMX VMX

Peripheral Device Drivers

ManagementAgents and Interfaces

Service Console

VMX VMX VM VM

VMM VMM

ResourceManagement

5Copyright © 2006 VMware, Inc. All rights reserved.

Virtual Machines in ESX Server

(Up to 2 CD-ROMs)

1-4 ports 1-4 ports

1-4 adapters1-4 adapters;

1-15 devices each

Up to 16GB RAM

1-2 drives

VM Chipset 1 CPU (4 CPUs with

VMware SMP)

6Copyright © 2006 VMware, Inc. All rights reserved.

Hot Migration: VMotion® Technology

VMotion Technology moves running virtual machines from one host to another while maintaining continuous service availability

• - Enables Continuous Workload Consolidation• - Enables Zero-Downtime Maintenance

7Copyright © 2006 VMware, Inc. All rights reserved.

Resource Pools

Aggregate collections of disparate hardware resources into unified logical resource pools

• Customer Impact• Failed server mean less resources

not a failed application

• Dedicated (virtual) infrastructure for each business unit; central IT retains control over hardware

• Delegation of resource and virtual machine management down to the business unit

• Management of an entire SOA application stack as a single entity

Servers, Storage, Networking

Business Unit

Department A Department B

Aggregate Resources

Resource Pool 2CPU 36GHz, Mem 58GB

Priority HIGH

Resource Pool 3CPU 12GHz, Mem 22GB

Priority LOW

CPU 48 GHz, Mem 80GB

8Copyright © 2006 VMware, Inc. All rights reserved.

VMware HA: Restart VMs if ESX Server fails

Virtual Machine

Virtual Machine

Virtual Machine

Virtual Machine

Virtual Machine

Virtual Machine

Virtual Machine Virtual Machine

X

VC

9Copyright © 2006 VMware, Inc. All rights reserved.

CPU Utilization Distribution

1

10

100

1000

10000

100000

0 20 40 60 80 100

% CPU Utilization

Nu

mb

er o

f S

yste

ms

Workload Characteristics

Consolidation targets are often <30% Utilized• Windows average utilization: 5-8%

•Linux/Unix average: 10-35%

10Copyright © 2006 VMware, Inc. All rights reserved.

Exchange performance in a virtualized environment

• Many workloads on VM run at/near native H/W speed

• Let’s focus on something that does not!

• Study with:• Exchange 2003

• WS 2K3 EE

• Loadsim 2003

• 4-way Dell servers

• Native supports 1.4X as many users

• Alternative observation: VM has 1.6X the CPU usage for the same # of users

• Where does the time go?

11Copyright © 2006 VMware, Inc. All rights reserved.

• 2-socket, Woodcrest system• Exchange 2003

• WS 2K3 EE

• Loadsim 2003

• Perfmon and Vtune on native

• VMware s/w and h/w monitoring tools on VM

12Copyright © 2006 VMware, Inc. All rights reserved.

Change in h/w profile

• Native CPU usage is ~60%

• VM CPU usage is ~90% at ~4% less throughput• 73% in the guest+monitor

• ~20 in hypervisor

• VM has 1.5X the CPU usage at 0.96% the throughput

• 1.38X more instructions; 1.09X the CPI• The product matches the 1.5X

• 2-3X increase in TLB misses

• 40-80% more L1 misses

• L2:• 34% more data misses

• 90% more imisses, but much fewer than dmisses