© 2014 VMware Inc. All rights reserved.

VMware vSphere 6:Design and Upgrade Considerations

Chris ClancySr. Systems EngineerVMware, Inccclancy@vmware.com

10/21/2016

2

Agenda

1 vSphere 6 : Design Factors

2 vSphere 6 : Upgrade Considerations

3 Q & A (Throughout)

*

A few housekeeping items….

1. This session is for you – so please feel free to ask questions!

2. The agenda is tight for this session (~55 mins). I might ask to take some questions offline if we go too far off track, or we are running out of time.

3. This session is meant to cover the major design / upgrade elements of vSphere 6 architecture. (vCenter, Platform Services Controller, Certificates)

4. While this focuses on vSphere 6.0, I will make references to some newly announced vSphere 6.5 elements when possible.

5. I will be around for several hours beyond this session, please feel free to pull me aside to continue any dialogue around this topic.

Design Factors

Design Factors: Think about these firstWhile not an exhaustive list by any means, these represent the basics when investigating a potential migration / upgrade to vSphere 6. In reality, you would conduct a readiness assessment for all infrastructure elements that interact with your vSphere environment. • Existing vSphere Major and Update version(s)

• Current number of vCenters

• Number of datacenter locations

• Host Hardware Eligibility

• Current management database topology (i.e. SQL, Oracle)

• Existing VMware Solutions (i.e. vRealize, Horizon, etc)

• Storage and Availability Infrastructure– Including any VMware specific integrations (i.e. Third party backup, Array plug-ins, etc)

• Networking Infrastructure - Are you currently using vSS, vDS, Nexus 1KV?

• Third party software integrations and vSphere 6 suitability (i.e. VADP backup, management tools, etc)

• Your current SSO topology

Always Consider “RAMPS” Design Standards• Recoverability

• Backup and recovery of vCenter, PSC, associated VMware solutions, etc• Availability

• HA, database clustering, load balancing, etc • Manageability

• Windows vs. Appliance (Linux based) vCenter, web client, command line tools, etc• Performance

• Virtual machine resources, network speed, storage backend, etc• Security

• User roles/permissions, lockdown mode, etc

Some Items to Consider for RAMPS:

• vCenter(s)• Platform Services Controller(s)• Host(s)• Virtual Machine(s)• Storage• Virtual Networking

• Physical Networking• Certificates• Management Infra. Databases• Other VMware Solutions• Any other supporting solutions….

vCenter, PSC and Related Topologies

7

vCenter 6 Platform Choice

Metric / Feature vSphere 5.5Operating System Windows ApplianceHosts Per vCenter Server 1,000 100 or 1,000

Powered-ON VMs 10,000 10,000

Hosts per Cluster 32 32

Linked Mode Yes No

• Replication Technology Microsoft AD LDS / ADAM -

Mixed Platforms No No

vSphere 6.0Windows Appliance

1,000 1,000

10,000 10,000

64 64

Yes YesIn-House

(from PSC)In-House

(from PSC)Yes Yes

vCenter 6 Platform Choice (Continued)

• The question becomes which vCenter form factor should you use?• Remember that the two platforms are functionally identical• Make the decision based on your business needs

– Is there Linux experience?– Do you have VUM co-located with a Windows vCenter installation*?– Are licensing costs a concern?

* In vSphere 6.5 – VUM is bundled with the appliance

Options for protecting vCenter • Backup* (VDP / Third Party VADP)• Database Clustering (RAC / WSFC)• VMware HA**

• Hardware failure• Guest OS failure

• VMware SMP-FT• Hardware Failure• Relevant to vCenter sizes of 4 vCPU and below

• Windows Server Failover clustering for protecting vCenter server services

* vSphere 6.5 will include native backup/restore capabilities for vCSA

** vSphere 6.5 will include native HA capabilities

vCenter – New Deployment ArchitectureThe services are split between the Platform Services Controller and vCenter Server

• The Platform Services Controller includes:– vCenter Single Sign-On™– License service– Lookup service– Directory services (vmdir)– VMware Certificate Authority

• The vCenter installation includes:– vCenter Server– vSphere Web Client– Inventory Service– vSphere Auto Deploy™– vSphere ESXi Dump Collector– vSphere Syslog Collector (Windows) or

vSphere Syslog Service (Appliance)

PSC Server Host OS

Platform Services Controller

vCenter Server Host OS

vCenter Server

Platform Services Controller

• Available in appliance or Windows-based form factors

• 8 PSC maximum per common SSO domain

• 4 PSC maximum behind a load balancer• A maximum of 4 vCenters can point to a

given PSC

PSC Server Host OS

Platform Services Controller

vCenter – New Deployment Architecture

The services are split between the Platform Services Controller and vCenter Server

vCenter – Deployment Topologies

The services are split between the Platform Services Controller and vCenter Server

Architecture #1 – Embedded Deployment Model

• Sufficient for environments with:– Only a single site– No expansion past a single vCenter required

• Easiest to deploy and maintain

• Multiple standalone instances supported

• Replication between embedded instances not recommended.

vCenter Server Host OS

vCenter Server

Embedded PSC

Architecture #2 – External Deployment Model

• Sufficient for environments with:– Only a single site– Up to 4 vCenter Servers

• Multiple Platform Service Controller nodes locally

• vCenter interacts with the Platform Service Controller through a compatible load balancer

• Platform Service Controllers replicate state information between them and provide a single pane of glass view of the environment

• (Optional) vCenter instances can be clustered with Windows Server Failover Clusters (WSFC)

PSC Server Host OS

External PSC

PSC Server Host OS

External PSC

Load Balancer

vCenter Server Host OS

vCenter Server

vCenter Server Host OS

vCenter Server

Replication

Load Balancing and Platform Service Controllers• NSX, F5 Big-IP and Citrix Netscaler supported

• Important Note: You are assigning a weighted priority to individual PSC nodes behind a load balancer.

• Each node is essentially operating in an “Active / Passive” sense. However, replication is actively occurring between each of them on a routine basis.

• You aren’t actually balancing PSC traffic to each PSC node. You are sending that traffic to the more heavily weighted PSC.

• If a PSC node goes down, the lower weighted PSC node will start servicing requests as the vCenter is resolving traffic to the Virtual IP address of the load balanced pair.

• In short, a load balancer provides an automated failover to the “passive” node in the event of an active node failure.

• You have the option of manually flipping a vCenter over to a PSC node if you were not in a position to initially load balance them.

Architecture #3 – External Deployment Model Multiple Sites

PSC Server Host OS

External PSC

PSC Server Host OS

External PSC

vCenter Server Host OS

vCenter Server

vCenter Server Host OS

vCenter Server

PSC Server Host OS

External PSC

vCenter Server Host OS

vCenter Server

Provides Enhanced Linked Mode• Facilitated via Platform Services Controller• Maintains single pane of glass management• Replicates Licenses, permissions, tags

and roles

By Default• Each site is independent• PSC replication automated• Site awareness• No HA Shown

Site #1:New York

Common SSOM Domain and Replication

Site #2:San Francisco

Site #3:Toronto

Architecture #4: Platform Services Controller – Max Size

Common SSO DomainCommon SSO DomainCommon SSO Domain

PSC Server Host OS

External PSC

PSC Server Host OS

External PSC

Load Balancer

vCenter Server Host OS

vCenter Server

vCenter Server Host OS

vCenter Server

vCenter Server Host OS

vCenter Server

vCenter Server Host OS

vCenter Server

PSC Server Host OS

External PSC

PSC Server Host OS

External PSC

Load Balancer

vCenter Server Host OS

vCenter Server

vCenter Server Host OS

vCenter Server

vCenter Server Host OS

vCenter Server

vCenter Server Host OS

vCenter Server

PSC Server Host OS

External PSC

PSC Server Host OS

External PSC

Load Balancer

vCenter Server Host OS

vCenter Server

vCenter Server Host OS

vCenter Server

What Should You Use?• Build based on business requirements, thinking of the future. Do not over-engineer!

• If there is only a single small site or if there is no desire for Enhanced Linked Mode:– Use embedded nodes – Allows for simplicity in the environment – Reduces the administrative overhead of configuring the environment. – High Availability (HA) is provided by VMware HA.

• If there are multiple sites and/or vCenter and Enhanced Linked Mode will be used: – Use an external Platform Service Controller configuration– HA is provided by having multiple PSC, and load balancers as well as VMware HA.– The number of controllers and Load Balancers depends on the size of the environment:

VMware SolutionsWithout HA With HA

# PSC # PSC # Load Balancers

2 – 4 1 2 1

5 – 8 2 4 2

9 – 10 3 6 3

Certificate Authority and Operational Modes

21

22

• Provisions each ESXi host, each vCenter Server and vCenter Server service with certificates that are signed by VMCA

New vCenter Server solutions for complete certificate lifecycle management:

vCenter Server 6.0 - Certificate Lifecycle Management for vCenter and ESXi

While you can decide not to use VMCA in your certificate chain, you must use VECS to store all certificates, and keys for vCenter Server and services.

All ESXi certificates are stored locally on the host.

VMware Endpoint Certificate Service (VECS)

VMware Certificate Authority (VMCA)

• Stores all certificates and private keys for vCenter Server and vCenter Server services

• Managing VECS is done via vecs-cli

VMware Certificate Authority (vSphere 6.0)• vCenter architecture has changed substantially between 5.x and 6.0

– Consolidation of Solution Users has occurred– Fewer solution users and therefore fewer certificates– No longer need to replace certificates to be signed and secure

• Manage certificates in a wallet– Uses VMware Endpoint Certificate Store (VECS) to store certs– Certificates are no longer being stored on disk in various locations– Are centrally managed in VECS

VMware Certificate Authority (VMCA)

• Built into the Platform Services Controller– Issues CA signed Certificates to all solutions and

ESXi hosts

• Operates in one of several modes:– VMware Certificate Authority Self-Signed Root

Certificate (Default)– VMware Certificate Authority Enterprise

Certificate (Subordinate)– Custom – Hybrid

• Can be updated from the GUI for ESXi hosts, or command line

• vSphere 6.0 Update 1 now includes the PSC UI tech preview which allows for certificates to be replaced from a GUI.

25

Certificate modes with VMCA

• VMCA provides the Root certificate

• All vSphere certificates chain to the VMCA

• Regenerate certificates easily on demand

• “Self Signed”

VMCA Default VMCA Enterprise

• Replace VMCA CA cert with a subordinate CA certificate from your Enterprise PKI

• Upon removal of the old VMCA CA certificate, all old certificates will be regenerated

• Disable VMCA as CA

• Provide your own custom certificate for each solution

• More complicated, for highly security conscious customers

Custom

• Replacement of machine SSL certificates

• VMCA for Hosts and Solution Users

• Very popular with high security customers

• Recommended

Hybrid

VMware Certificate Authority - Should I use it?• Yes.

• Recommended configuration varies

• For many environments using default configuration is sufficient – All that is required is to download and install the VMware Certificate Authority Root Certificate to clients

• For environments that secure or have compliance requirements use the Hybrid CA mode– External facing management interfaces are secured with a minted certificate from your corporate PKI– Non-external facing resources (i.e. solutions users, ESXi hosts) are issued certificates directly from the

VMCA

• There are very few scenarios where custom (manual) configuration is recommended. This is usually reserved for the highest security but comes with the highest administrative overhead.

• See KB below for specific steps to replace certificates:

Replacing default certificates with CA signed SSL certificates in vSphere 6.0 (2111219)

Upgrade Considerations:

Quick Tips:

• No two client environments are the same

• It is important for you to carefully visit your “end state” design prior to ANY upgrade steps

• Use RAMPS for design decisions

• If you have concerns (or just want some stick time) – consider executing against lab equipment and/or non-critical systems first.

• Understand when migration windows will be necessary.

5.X Support Guidance

• 5.0 & 5.1 are currently End of General Support!

• 5.5 General Support Through 9/2018

Upgrade and Migration Resources• VMware KB Articles and Self-Service Resources…..Specifically….

• vSphere 6 Documentation Center (here)• Compatibility Guide (here)• Product Interoperability Matrix (here)• Update Sequence for vSphere 6 and Compatible VMware products (here)• Other useful KB articles are located in the appendix

• Your assigned VMware field Systems Engineer* and account team

• Your Technical Account Manager* (TAM – if you have one)

• VMware Professional Services**

* Advisory Role.** Design and Implement Role

Migration Strategies for vSphere 6• Build a new 6.0 vCenter and new 6.0 hosts ….. Then swing over workloads from the legacy

environment• Can be useful if major hardware refreshes coincide with your vSphere upgrade cycle.• Complexity ensues – as you will have to transpose various configurations (i.e. vDS)• Gets complicated when multiple VMware solutions / customizations are already in place.• Typically not seen in large very large environments

• Build new 6.0 vCenter and upgrade existing 5.x hosts. • Can be useful if you want to “clean slate” the vCenter topology and associated services. • Useful when the migration window is lengthy. You can run both environments in parallel and migrate

virtual workloads at your pace. • You will still need to carry over and re-configure certain elements of your old vCenter to the new

vCenter environment • Again, this is more complicated as the number of integrations with vCenter increase.

• Upgrade existing vCenter and hosts (The focus of this section)• Useful for keeping existing integrations intact• There are well documented procedures for exactly how to do this• This is generally the recommended practice.

Platform Sevices Controller: Know before you go….• The mode you enter the upgrade in is the mode you will end up on.

vCenter 5.1/5.5 with Embedded SSO vCenter 6.0 with embedded PSC.

• It is strongly recommended to get your 5.1 / 5.5 SSO topology in line with your desired 6.0 PSC topology.

• There are several reasons for this – generally related to flexibility in re-pointing vCenter to SSO instances.

vCenter Upgrade: Know before you go….• Please make sure your time is correct and sync’d across all relevant vCenter(s) and host(s).

• Please make backups of any relevant databases

• For vCenter – make sure you have disk space available.

• Make sure your certificates are valid. Resolve any existing certificate issues before proceeding

• Please use FQDN for everything!

• Ensure DNS is working properly, and that you can resolve vCenter / other management infrastructure host name(s) forwards and backwards.

• When possible, use snapshots before upgrading individual management components on virtual machines.

• Make sure the necessary ports are open for vCenter, SSO, Syslog, etc.

• Document EVERYTHING.

• vCenter 6.0 CANNOT manage vSphere 4.x hosts in the same cluster as 5.x or 6.x hosts!!

General Order of Operations: Upgrade• This assumes minimal additional VMware Solutions. Always refer to the Update Sequence KB article located

here. If you use adjacent VMware solutions – it will likely alter the order below.

• SSO to PSC

• vCenter Server

• vCenter Post-Upgrade Tasks (link)

• Upgrade vSphere Update Manager (VUM) if applicable

• Upgrade ESXi host(s) – via VUM, Scripting or Manually

• Reconnect to vCenter and apply vSphere license post upgrade.

• Upgrade VM hardware version and VMware Tools

vCenter / PSC Upgrade: Windows (v5.0) to Windows (v6.0) • You are not using SSO in v5.0, so you get to

pick the deployment option (embedded or external)

• Remember! If you want to use enhanced linked mode, replicate PSC’s amongst one another, have multiple sites…..use external!

vCenter / PSC Upgrade: Windows (v5.1 / 5.5) to Windows (v6.0) • The major factor here is whether your legacy vCenter /

SSO is distributed or embedded.

• If SSO is external to vCenter – the upgrade will result in (1) vCenter server and (1) external PSC

• If SSO is embedded with vCenter – the upgrade will result in (1) vCenter / PSC machine in the same installation.

• If your result is an embedded vCenter / PSC installation post-upgrade – you CAN change this topology after the fact.

• Instructions for repointing vCenter in an embedded PSC deployment to external PSC deployment are here

38

5.1 / 5.5 With Embedded SSO to 6.0 Supported External PSC Topology

1. Migrate to 5.1/5.5 SSO Domain in an external deployment

2. If using Linked Mode – break that link

3. Re-register vSphere Web Client(s) and Inventory Service, for each vCenter, to the new SSO instance

4. Re-register each vCenter itself to the new SSO instance

5. Remove embedded SSO installation on each machine

6. Upgrade external SSO to external PSC for each

7. Upgrade vCenter to 6.0 on each

8. KB here

vCenter Upgrade: VCSA to VCSA

• You must be running VCSA 5.1U3 or later – hard requirement

• Understand your existing deployment model.

• Is SSO embedded or external?

• If you are external, you can only upgrade if you are at 5.5.x

• Do you use a load balancer in multi-instanced SSO?

• Once identified – use the decision tree

• This is a migration, not an in-place upgrade. A 6.0 VCSA instance will ride alongside your 5.x VCSA. All settings will migrate and the old VCSA will be decommissioned.

• Remember: If you upgrade VCSA w/ embedded SSO, the upgrade will result in a VCSA w/ an embedded PSC

vCenter Upgrade: Windows (v5.5.x) to VCSA (v6.0)• Available from 6.0 U2m release only!

• From vCenter 5.5 (Windows) to v6

• Any databases from supported 5.5 vCenter will be migrated into vPostgres appliance database

• If VUM is co-located on the 5.5 vCenter install….it will need to be moved to a dedicated server before upgrade.

• Configuration, inventory, and alarm data will be migrated automatically, historical and performance data (stats, tasks, events) are optional

• There will be some downtime for this – this old vCenter gets powered down!

• No changes are actually made to the source vCenter – this allows for easy rollback in the event of problems. The VCSA assumes the IP, hostname, UUID, etc of the source vCenter.

• 3rd party extensions are migrated – but you MAY have to re-register them.

• Sample migration walkthrough video located here!

vCenter Upgrade: Know before you go….Mixed-Version Transition

• This is pointed at those who have multiple vCenter instances and will have a period of time where one vCenter server will be at 6.x while others are at 5.5. You cannot run 5.1 & 6.0 in a mixed state, as 5.1 vCenter cannot communicate appropriately with a 6.0 PSC

• Note – This configuration is NOT supported in production environments. It is recommended only during transitions. In short – don’t have a lengthy transition if you can help it.

• Linked Mode no longer functions.

• vCenter Server 5.5 instances continue to operate with the upgraded Platform Services Controller as they did before the upgrade without any problems or required reconfiguration.

• In a vCenter Server mixed-version 5.5 and 6.0 environment, a vSphere Web Client 6.0 instance shows vCenter Server 5.5 instances.

• vSphere Web Client 5.5 shows vCenter 5.5 Server instances only, not 6.0 instances

• You cannot add a new vCenter 5.5 to a PSC 6.0 environment! This is a major reason to minimize your mixed-version transition window

• A useful video on mixed version transitions can be found here.

Some final thoughts….

42

Questions?

43

Thank you!

44

Useful Links • Upgrading from vSphere 5.x to vSphere 6.0 Best Practices (2130664)  -

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2130664

• Update sequence for vSphere 6.0 and its compatible VMware products (2109760) - https://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=2109760

• vSphere 6.0 Documentation Center - http://pubs.vmware.com/vsphere-60/index.jsp#com.vmware.vsphere.doc/GUID-1B959D6B-41CA-4E23-A7DB-E9165D5A0E80.html

• Lifecycle Support Policies - https://www.vmware.com/support/policies.html#lifecycle-table

• Product Lifecycle Matrix - http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/support/product-lifecycle-matrix.pdf

• Configuring PSC 6.0 High Availability for vSphere 6.0 using vCenter Server 6.0 Appliance - https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2113315

• FAQ: VMware Platform Services Controller in vSphere 6.0 (2113115) - https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2113115

• vSphere 6 Feature Walkthroughs - https://featurewalkthrough.vmware.com/#!/vsphere-6-0