Phoenix VMUG Teams Topic: Disaster Recover of vCenter Server What if the vCenter Server Fails?

Phoenix VMUG Teams

Topic: Disaster Recover of vCenter Server

What if the vCenter Server Fails?

Agenda Topics

• Presentation Overview

• What is vCenter Server

• DR Team One Presentation

• DR Team Two Presentation

Presentation Overview

At our last meeting (11/2008) we challenged the Phoenix VMUG users to two create two teams of 5 members and come up with a DR plan for a vCenter Server

Today they will be presenting their findings

Our hopes are that you can use this presentation as a guide to develop your own DR plan for vCenter Server in your Environment

Presentation Overview

Presentation Outline

Matt Mancini – vCenter Server Overview

Team 1 Presentation Lead by Jason Schling of Macerich

Team 2 Presentation Lead by Duke Encinas of Scottsdale Insurance

What is Virtual Center

By a show of hands

How many users know what vCenter Server is?

How many users have an active and tested DR plan for vCenter Server?


The vCenter Server provides a convenient single point of control to the datacenter.

It provides many essential services such as access control, performance monitoring, and configuration.

It unifies the resources from the individual computing servers to be shared among virtual machines in the entire datacenter.

It accomplishes this by managing the assignment of virtual machines to the computing servers and the assignment of resources to the virtual machines within a given computing server based on the policies set by the system administrator.


VM Provisioning – Guides and automates the provisioning of virtual machines

Host and VM Configuration – Allows the configuration of hosts and virtual machines

Resources and Virtual Machine Inventory Management – Organizes virtual machines and resources in the virtual environment and facilities their management.

Statistics and Logging – Logs and reports on the performance and resource utilization statistics of datacenter elements, such as virtual machines, hosts, and clusters


Task Scheduler – Schedules actions such as VMotion to happen at a given time.

Consolidation – Analyzes the capacity and utilization of a datacenter’s physical resources. Provides recommendations for improving utilization by discovering physical systems that can be converted to virtual machines and consolidated onto

Distributed Services such as VMware DRS, VMware HA, and VMware VMotion. Distributed Services allow the configuration and management of these solutions centrally from vCenter Server.


Alarms and Event Management – Tracks and warns users on potential resource over‐utilization or event conditions.



Virtual Center supported requirements:•CPU- 2.0GHZ or higher Intel or AMD x86 processor•Memory- 2GB RAM minimum•Disk Storage space- 560MB free space min – 2GB recommended•Can be run as a VM

Software requirements:32 bit versions of operating system onlyWindows 2000 SP4 with update rollup 1Windows XP ProWindows 2003 Server SP1 or R2 Virtual Center Database Storage area for maintaining Virtual Center inventory as well as status of each VM and each managed host

Database formats:Oracle, Microsoft SQL Server, SQL Server Express

License –Server based license / Host Based


What if the server fails?

Team One Presentation

DR Team One PresentationTeam Members:

Jason Schling, Macerich

Nate Fyie, Macerich

Robert Cooley, Gradient Analytics

Charles Braffett, See it Our Way


Important concepts related to Virtual Center recovery

- VM’s are portable files

- Understand what your RTO and RPO are

- Some major areas that can affect your ESX environment

- Storage, Network, Backup and Recovery

- NAS or SAN?

- Storage protocols are important what do you use fiber, iSCSI or NFS?

- Backups. Do I use tape, disk to disk or is replication enough?


Important concepts related to Virtual Center recovery

- Must plan for different types of scenarios and meet your RTO and RPO

- Have you ever recovered from any of these, do you test frequently?

- How does the network impact my disaster recovery plan

- VLAN and server IP addressing in production and DR site

- Is my bandwidth pipe large enough for replication

- Must provide consistent storage, VM and network standards across the enterprise

- Must have good documentation


431 2 875 6

UID 1 2

HPProLiantDL585

G5

Data

431 2 875 6

UID 1 2

HPProLiantDL585

G5

431 2 875 6

UID 1 2

HPProLiantDL585

G5

431 2 875 6

UID 1 2

HPProLiantDL585

G5

431 2 875 6

UID 1 2

HPProLiantDL585

G5

UID 1 2

DL120 G5ProLiant

HP

(Production) (DR recovery)

MPLS WAN

Data Replication

Storage Network

ESX 1, 2, 3

Virtual Center 1

Off – SiteStorage of LTO tapes

LTO Tape Library

Storage Array 1 Fiber Channel Drives 10TB

ESX 4, 5

Storage Network

Storage Array 2SATA Drives 20TB

VM server AVM server BVM server CVM server DVM server EVM server …

VM server AVM server BVM server CVM server DVM server EVirtual Center

DR is now Production (RT 4 hours)

Break ReplicationRestore Snap ShotPresent storage

Rescan for volumesPerform VC recovery

RT 96 hours

VM Network

VM Network


Virtual Center Recovery Steps


Virtual Center Recovery Steps

Validate replication with snap shot validation through storage management interface

Provision storage in DR site for Read / Write access – present volumes to network

Validate networking configurations

Perform Virtual center recover

- Login to http service of ESX server that hosts VC DR at DR site- Power on ESX servers- Via VC add in all ESX servers at DR site- Add required networking- Add required storage, for each storage volume add to inventory the VM- Power on each recovered VM

Fail back to production when DR is over


Questions / answers?

VM’s are essential files and hardware independent

VM’s are easy to replicate / backup through third party software / process

Ability to take snap shots and recover from snap shots

Ability to clone snap shots for DEV / TEST to test DR

VM’s can recover from tape or disk

VM consolidation into standardized shareable storage

Virtual Center Recovery Overview

Team Two Presentation

DR Team Two Presentation


What do you want to do with Virtual Center?

Determine what functions are needed in your environment

How difficult will it be for your team to support Virtual Center?

Different modes: Host Based and License server


Host Based Advantages:

No license server to manage

Acceptable for a small number of ESX Hosts

Host Based Disadvantages:

Licenses not shared between hosts

Licenses are stored individually on ESX Server Hosts

Not all features are available, such as: VMotion, HA and DRS


License Server Mode advantages:

All functionality is available

All managed hosts need to be able to resolve and connect to License Server

LMTools (License Manager Tools) available to help reconcile issues with Virtual Center Licensing


License Server Mode Disadvantages:

Not enough licenses installed on the Virtual Center License Server will make it difficult to start ESX Hosts managed by Virtual Center

Difficult to set up in multiple tiered Security/Network environment


The Database is everything!

All performance information, Resource Pool information and Authentication information is stored in the database

“Losing the server that runs Virtual Center might result in a small period of downtime; However, losing the back-end database to Virtual Center could result in days of downtime and extended periods of rebuilding.” -Chris McCain, Mastering VMWare Infrastructure 3


What database works best for you?

SQL Server Express not recommended for more than 5 ESX hosts and more than 50 VMs. Also there is a 4GB database max

Oracle vs. Microsoft SQL Server 2005 Which is easier for you to manage?

Existing asset or new build?


Size the Database accordingly!

Select the Performance Statistics Collections Level

Best Practices: Past Day and Past Week - Level 3

Past Month and Past Year - Level 1

90% of information collected is for Performance Statistics

Add sufficient space for growth and backups

Use Virtual Center Database Calculator


Changing the collection cycle can have adverse effects:

Collection period by default is every five minutes

Change by 1 minute increases database size 20%

Infrequent collection times leads to invalid data


Virtual or Physical database?

Depends on your confidence level in Virtual Technology

Where do your strengths lie? Virtual or physical

Virtual advantages:

Snapshot, VMotion, Storage VMotion and Resource Pools

Virtual disadvantages:

Reliant on ESX Host being available

Minimum of two ESX Host cluster to eliminate single Point of Failure


Virtual Center Database options:

Clustered or stand alone? VMWare recommends Microsoft Cluster

Is clustering a new configuration for your environment?

Increases complexity, what risk does this mitigate?


Placement of Virtual Center:

Will Virtual Center server manage local only or remote ESX hosts as well?

Flat Network vs. Tiered DMZ Network Management network in place?

Virtual Center Server should be placed where it can connect to all ESX Hosts, especially if templates are used to create VMs


Recover Virtual Center:

If database is intact, redeploy new VM, install Virtual Center and point it to your Virtual Center Database

Consider building two Virtual Center servers One powered on, one powered off, both configured to point

to the Virtual Center database

Documents

Phoenix VMUG Teams Topic: Disaster Recover of vCenter Server What if the vCenter Server Fails?