Performance and Sizing Considerations

Chin, Kar Wai

Global DC Services Delivery Manager

Email : karwai.chin@trw.com

Http://www.trw.comTRW Automotives

Performance and Sizing Considerations

Understanding what’s a Virtual Machine

Candidate Selections

Building your application on VM e.g. Exchange, SAP

Sizing/Performance consideration for your VM

Support from major ISV

Building a solid foundation with VMware Plan & Design

Anatomy of a Virtual Machine

Each Virtual Machine is a complete system encapsulated in a set of software files

Virtual MachinesUnmodified Application

Unmodified

(X86) Physical Server

ESX Server

Unmodified OS

Virtual Hardware

Candidate Selections

Non-Candidates

Poor-Candidates

Candidates – requires additional resources

Virtualizable Candidates

Candidate Selections

Non-Candidates

Application requires more resources than can be made available to a single Virtual Machine

more than 4 active NIC ports, more than 64GB RAM, more than 4 physical

CPUs

http://www.vmware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_config_max.phttp://www.vmware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_config_max.p

df

OS supportability

http://www.vmware.com/pdf/GuestOS_guide.pdf

Candidate Selections

Application which uses specialized peripherals not supported in Virtual Machines

such as fax server cards, modems, encryption cards, other specialized

PCI/expansion cards, some dongles;

Application which uses specialized peripherals for licensing which are not supported in VM’swhich are not supported in VM’s

such as fax server cards, modems, encryption cards, video capture cards,

other specialized PCI/expansion cards, some dongles;

Candidate Selections

Poor-Candidates

Applications that require a serial connector, or parallel port connector for the licensing

Can mount port to VM, but locked to specific host, No Vmotion

Tape Backup Server – SCSI attached

Only Adaptec SCSI adapters

Generally highly utilized - CPU, Memory, Network and Disk

True Real-time application – one clock cycle to one event

Applications that use hard-core 3D graphics

Candidate Selections

Candidates – requires additional resources

Applications that require a USB key for licensing

USB over IP e.g. AnywhereUSB (Digi)

Applications that use licensing to a specific MAC address

can set a static MAC address in the VM’s configuration file (.vmx)can set a static MAC address in the VM’s configuration file (.vmx)

What about the rest?rest?

>95% Applications

Are good candidates for running on VMware Infrastructure

Building your Application on VMApplication on VM

>95% of Applications Match or Exceed NativePerformance on VMware Infrastructure

Pro

po

rtio

n o

f A

pp

s P

erf

orm

ing

We

ll

100%ESX 2 ESX 3 ESX 3.5 Future

Overhead

VM CPU

• 30% - 60%

• 1 vCPU

• 20% - 30%

• 2 vCPU • 4 vCPU

• <2% - 10%

• 8 vCPU

• <10% - 20%

ESX Versions

Pro

po

rtio

n o

f A

pp

s P

erf

orm

ing

We

ll

VM Memory

IO

• 3.6 GB

• 800 MBits

• 64 GB

• 100,000 IOPS

• 9 GBits

• 256 GB

• >200,000 IOPS

• 40 GBits

• <10,000 IOPS

• 380 MBits

• 16 GB

Disk IO

per

Second

Database Requirementsvs. VI Capacity

VI supports 80X IO

throughput

requirement of

average 4-CPU

Support Large Oracle Databases

60K

80K

100K100,000

DB requirements

(4-CPU Oracle DB)

VICapacity

Source: VMware Capacity Planner analysis of > 700,000 servers in customer production environments

average 4-CPU

Oracle Database

20K

40K

1,200

Run SAP with <10% Overhead# SD

UsersSAP SD Benchmark

200

400

600

800

1000

300270

480445

815

732

10%

overhead

7%

overhead

10%

overhead

200

Native VMware Native VMware Native VMware

CentraIInstance

& DB

1 vCPU4 Gb

4 vCPU16 Gb

CentraIInstance

& DB

2 vCPU8 Gb

CentraIInstance

& DB

4 vCPU16 Gb

12-Dec-2007: SAP Support for VMware

[….]

Production Support for Windows and Linux

VMware now a SAP Global Technology Partner

Server Sprawl

DevelopmentTypical SAP Deployment

Training Dev

QA / Test Production

Virtual Machine

Windows 64 Bit

ApplicationServers

SAP HR

Virtual Machine

SuSE Linux

Application Servers

SAP ERP

Virtual Machine

Windows2000 Server

ApplicationServers

Test & Dev

Virtual Machine

Windows2003 Server

ApplicationServers

SAP CRM

Virtual Machine

Windows64 Bit

ApplicationServers

Test & Dev

Virtual Machine

Red HatLinux

Application Servers

SAP ERP

Virtual Machine

Red HatLinux 64 Bit

ApplicationServers

SAP BI

SAP BI Resource Pool Test & Dev Resource PoolSAP ERP Resource Pool

The Dynamic SAP Datacenter

SAP Test & Development Systems

VMware Infrastructure

SAP HR System SAP ERP Cluster SAP Business Intelligence

VMware ESX Server VMware ESX Server

VMware® VMotion™ Technology

Storage Network Servers

SAP Sizing - Guidelines

Start with small units. Smaller units are better manageable and load balancing is easier.

Scale-out instead of scale-up. e.g. deploy additional Application Servers if necessary

Virtualization may not fit every customer, but often leaving a highly utilized Database on a ‚“native“ host and virtualising the remaining components is a viable option.

SD Benchmark results are around 400 SD users for 2 vCPU with 8GB and around 700 SD users for 4 vCPU with16GB.

Sizing and Performance Performance

Considerations

Sizing Considerations

Basic Concepts - CPU

CPU Performance

Virtual CPUs map to physical CPU cores

Multi-vCPUs from a single VM must be scheduled simultaneously

Configure only as many vCPUs as neededImpose unnecessary scheduling constraints on ESX Server� Impose unnecessary scheduling constraints on ESX Server

� Waste system resources (idle looping, etc.)

� Idle vCPUs will compete for system resources

Unavailable pCPUs can result in VM “ready time.”

# of total vCPUs + 1 <= # of total cores per ESX hosts (High-Performance)

Ensure application is support for multithread if assign multiple Vcpu to single

VM

Sizing Considerations

CPU-0 CPU-1 CPU-2 CPU-3B - 0 E - 0 Schedule - 1

C- 0 C- 1 D- 0 D- 1 Schedule - 2

A - 0 A - 1 B - 0 E - 0 Schedule - 3

A - 0 A - 1

Contending for CPU resources !

Sizing Considerations

CPU-0 CPU-1 CPU-2 CPU-3A - 0 A - 1 B - 0 E - 0 Schedule - 1CPU-4 CPU-5 CPU-6 CPU-7C- 0 C- 1 D- 0 D- 1

Optimized VM ratio to pCPU on host!

Sizing Considerations

Basic Concepts - Memory

Memory Performance

Service Console: 272 MB

VMkernel: 100 MB+

Per-VM memory space overhead increases with:

VA

Per-VM memory space overhead increases with:� Number of VCPUs

� Size of guest memory

Do not over allocate – for intensive workloads

PA

MA

Sizing Considerations

Basic Concepts - Networking

Networking Performance

NIC teaming

Use separate NICs to avoid

contention

Use same virtual switch

VSwitch VSwitch

VirtualMachines

VirtualNICs

ESX Server

VM3

Use same virtual switch

to connect communicating VMs

Use vmxnet virtual device in guest

Requires CPU cycles

PhysicalSwitches

1000 Mbps

1000 Mbps

PhysicalNICs 1000

Mbps

Performance Considerations - VM Network

Virtual Switch

Separate multiple

subnet into different

virtual switch

Reduce network hoop

from each virtual

switches to improve

network efficiency

Proper planning on

network infrastructure

and VM guest IP

address and subnet

Standardize Virtual

Switch name for HA,

DRS and VMotion

purposes

Performance Considerations - VM NetworkEnsure Physical Network

Cards is slotted at PCI-X to

improve network

performance

Separate the VM network

traffic from Service Console

and VMkernel

Reside the VMs which

require connectivity to each

others into same virtual

switches to improve

efficiency

Multiple mode of the physical

NIC must match with the

switch configuration

Virtualization Storage Best Practices

Standardize the VMFS datastore size with storage best practices recommended by

different storage vendors

Avoid high intensive I/O workload VM guests access to the same datastore at the same

time

Assign separate Virtual Disks and Virtual Controller if require for better management

capability and performance purpose

Select RDM for huge storage requirement in single volume

Avoid extra multi path if do not require as it will impact the total numbers of LUNs to be Avoid extra multi path if do not require as it will impact the total numbers of LUNs to be

presented to each ESX hosts

Create separate VMkernal for Software ISCSI adaptor or select the right ISCSI HBA

cards for ISCSI storage protocol

Sufficient Swap Space require in each datastore to ensure the allow the memory over

commit on ESX servers

Round Robin

Round Robin

Load Balance across all

available paths

Improve I/Os throughput

Avoid Bottleneck on dedicated

path for performance

perspectiveperspective

Experimental at current ESX 3.5

Performance Considerations - Memory Reservation

Memory Reservation

Increase physical

memory respond

time

Improve data

processing efficiency

Share value impact Share value impact

the resources priority

to VM

Only reserve if

needed

Performance Considerations - CPU Reservation

CPU Reservation

CPU reserve for High

CPU contending VM

Improve performance

Reduce available CPU

processing power to

the rest of VM on

same ESX Hostsame ESX Host

Only reserve the

amount which needed

Shares value will

impact the

prioritization of VM in

the ESX host

Performance Considerations - Advance CPU

CPU Reservation

Keep the setting to

default for hyper

threaded core

sharing

Scheduling Affinity

allow specify Vcpu

to be assigned to

specify physical specify physical

Core in ESX

servers

Distribute workload

to underutilize

physical cores if

needed

Recommended to

keep as default

setting

Performance Considerations - DRS Best Practices

Reside VMs which require communication in same host will reduce communication

latency

Separate VMs with same functionality to different hosts to maintain maximum

availability

Right aggressive mode for migration threshold and fully automate

Enable EVC to improve VMotion Capability

MS Kerberos implementation allows a 5 minute tolerance

Active Directory operations are critically time dependent

File Replication Services (FRS) synchronizes scripts, database changes/updates, policies based, in part, on time-stamping

Time Synchronization – Why is it so important?

Time Synchronization

No CPU cycles needed – none given!

Clock drifts can be significant in a relatively short period

Idle cycles in a virtual machine is an Active Directory domain’s worst enemy

How do you combat time synchronization issues?

More than a 28 minute drift!

Time Synchronization–Option A – Using W32Time

Use Windows Time Service – NOT VMware Tools

Microsoft KB article # 816042 provides instructions for this process

1. Modify registry settings on the PDC emulator for the forest root domain:

HKLM\System\CurrentControlSet\Services\W32Time\Parameters

Change Type REG_SZ value from NT5DS to NTPChange NtpServer value from time.windows.com,0x1 to an external stratum 1 time source, i.e. tock.usno.navy.mil,0x1tock.usno.navy.mil,0x1

HKLM\System\CurrentControlSet\Services\W32Time\Config

Change AnnounceFlags REG_DWORD from 10 to 5

2. Stop and restart time service – net stop w32time ���� net start w32time

3. Manually force update � w32tm /resync /rediscover

Time Synchronization–Option B – VMware Tools

Modify Windows Time Service – Use VMware ToolsImplement Domain Controllers Group Policy to modify registry:

Modify

Enable ESX server NTP daemon to sync

with external stratum NTP sourceVMware Knowledge Base ID# 1339

Enable ESX server NTP daemon to sync with

external. Use VMware Tools time synchronization within the virtual machine

NOTE: VMware Tools time sync is designed to play “catch-up”, not slow down!

Time Synchronization - Summary

Use one method or the other

Do NOT use both!!!

Decisions should be based on current time management infrastructure or organization’s policies

Ensure Multiple NTP servers synchronize option is been Ensure Multiple NTP servers synchronize option is been configured on each ESX Host

Exchange and MS SVVP

Ong, Kok Leong

Senior Consultant, VMware ASEAN

e klong@vmware.com

m +60-12-4706070

w http://www.vmware.com/

Native VMware Infrastructure

Double Capacity of Exchange 2007 Hosts

8K mailboxes

8 VM

16K mailboxes

16 core

128 GB 16 core

128 GB

Proving to be Better Than Native

The experiment: 2,000 user Exchange multi-role VMs

The hardware:

IBM x3850 M2

4 x quad-core Intel Xeon 7350

128G RAM128G RAM

The criteria: LoadGen test pass

Scaling Exchange (Natively) On a Single Server

8 Cores Max

Single-Server Native Exchange Scaling

8K

10K

12K

14K

16K

Outside of Ma

ilbo

xe

s

8 core max

2 4 6 8 10 12 14 16

8K

6K

4K

2K

5M/mailbox perf.

threshold

Highest

Performance

Outside of

Microsoft

recommendations

Ma

ilbo

xe

s

CPUs (or cores)

Multi-VM Scaling of Exchange on VI3

8K

10K

12K

14K

16KM

ailb

oxe

sBuilding blocks stay within

Microsoft recommendations:

• 1,000 mailboxes/core

• 5MB/mailbox

Maximum performance!

• Three building blocks

breaks through pre-

virtual memory

boundaries

• Five building blocks

shatters pre-virtual

CPU limitations

5 MB/mailbox perf.

2 4 6 8 10 12 14 16

8K

6K

4K

2K

Ma

ilbo

xe

s

CPUs (or cores)

• Eight building blocks

enables 16,000

users

VI3 provides native-matching performance and

complete resource utilization

5 MB/mailbox perf.

thresholdEight core limit

recommended

by Microsoft

Exchange Sizing Considerations

Considerations

Do not over-commit pCPUs when running Exchange VMs.

Do not over-allocate vCPUs; try to match the exact workload.

If the exact workload is unknown, start with fewer vCPUs initially and increase

later if necessary.

Memory pages can be shared across VMs that have similar data (e.g. same

guest OS)guest OS)

Memory can be over-committed, (i.e. allocating more memory to VMs than is

physically available on the ESX Server)

Ensure heavily-used VMs not all accessing same LUN concurrently.

Microsoft Support and Licensing for VMware

MS Licensing Allows VMotion MS Support for Apps on ESXMS Licensing Allows VMotion

> Reassign licenses between servers as frequently as needed

> Covers 41 server apps including > Exchange 2007> SQL Server 2008> SharePoint 2007> Dynamics CRM 4.0

MS Support for Apps on ESX

> ESX validated as part of Microsoft SVVP program

> Microsoft ensures same technical support on ESX as on physical servers for Windows and the 33 apps

> Exchange 2007> SQL Server 2008> SharePoint 2007> Dynamics CRM 4.0

MS SVVP – What does validation mean?

Customers will now get corporate support from MS when running Windows Server on ESX 3.5 U2/U3

Windows Server 2008, Windows 2000 Server Service Pack 4, Windows Server 2003 Service Pack 2 and subsequent service packs are supported

Any version of Windows back to Windows Server 2000 SP4 is

VMware Confidential

also supported, including all OS roles such as Active Directory, File Services, etc

VMware has a premier support contract with MS and can

escalate issues to MS Support on a customer’s behalf.

MS VSSP – What about the MS Applications?

As part of VSSP, Microsoft has published a list of MS Applications that are supported. Article ID:957006

Microsoft Application Virtualization (App-V) Microsoft Operations Manager (MOM) 2005

Microsoft BizTalk Server Microsoft Search Server

Microsoft Commerce Server Windows Essential Business Server 2008

Microsoft Dynamics AX Windows Small Business Server 2008

Microsoft Dynamics CRM Microsoft SQL Server 2005 and SQL Server 2008

Microsoft Dynamics NAV Microsoft System Center Configuration Manager

Microsoft Exchange Server Microsoft System Center Data Protection ManagerMicrosoft Exchange Server Microsoft System Center Data Protection Manager

Microsoft Forefront Client Security Microsoft System Center Essentials

Microsoft Intelligent Application Gateway (IAG) Microsoft System Center Operations Manager

Microsoft Forefront Security for Exchange (FSE) Microsoft System Center Virtual Machine Manager

Microsoft Forefront Security for SharePoint (FSP) Microsoft Systems Management Server (SMS)

Microsoft Host Integration Server Microsoft Visual Studio Team System

Microsoft Internet Security and Acceleration (ISA) Server Microsoft Windows HPC Server 2008

Microsoft Office Groove Server Windows Server 2003 Web Edition

Microsoft Office Performance Point Server Microsoft Windows Server Update Services (WSUS)

Microsoft Office Project Server Windows Web Server 2008

Microsoft Office SharePoint Server and Windows SharePoint Services

Expanding ISV Ecosystem

McKesson

Additional Resources

Visit us on the web to learn more on specific apps

http://www.vmware.com/solutions/business-critical-apps/

Best Practices, Reference Architectures, and Case Studies

Microsoft Apps (Exchange, SQL, SharePoint)

Oracle

SAPSAP

Microsoft server software and supported virtualization environments

http://support.microsoft.com/kb/957006

http://www.windowsservercatalog.com/svvp.aspx?svvppage=svvp.htm

VMware - Plan and Design Service

Project Kickoff

Data Collection

Week 1 Week 2 Week 3 Week 4 Week 5

Comprehensive VI architecture design for implementation

Avoid implementation pitfalls that can stall or slow deployment

Accelerate the achievement of virtual infrastructure benefits

Capacity and Financial Analysis

Assessment Deliverable

Design Sessions

Creation of Deliverables• Architectural Design

• Assembly & Configuration Guide• Standard Procedures

• Test Plan & Rollout Plan

Final Recommendations

Week 6 Week 7 Week 8 Week 9 Week 10

Start with Capacity Planner

VMware Capacity Planner is…

Business and decision support tool

To perform faster, more accurate and benchmarked consolidation assessments

Capacity Planner provides:

Complete state of the datacenter (As-Is)Complete state of the datacenter (As-Is)

Comprehensive future state consolidation recommendation and roadmap (To-Be)

Thank You

ONG, Kok Leong

Senior Systems Consultant

e klong@vmware.com

m +60 (12) 4706070

w http://www.vmware.com