© 2014 VMware Inc. All rights reserved.

VMware Virtual SANIt changes everything

Duncan Epping

Principal Architect – R&D

Virtual SAN Technical Walkthrough

1 Introduction

2 Requirements

3 Architecture

4 Configuration Walkthrough

5 Virtual Machine Provisioning Operations

2

VMware Virtual SAN

• Storage scale out architecture built into the hypervisor

• Aggregates locally attached storage from each ESXi host in a cluster

• Dynamic capacity and performance scalability

• Flash optimized storage solution

• Fully integrated with vSphereand interoperable:

• vMotion, DRS, HA, VDP, VR …

• VM-centric data operations

3

vSphere + Virtual SAN

Hard disksHard disksSSD SSD Hard disks

SSD

…

Virtual SAN Shared Datastore

• Hypervisor-Converged storage platform

Storage Policy-Based

Management

VSAN Shared

Datastore

Simplifies and Automates Storage Management

4

Per VM storage service levels from a single self-tuning datastore

Capacity

Performance

Availability

Per VM Storage Policies

Policies set based on application needs

vSphere + VSAN

SLAs

Software automates control of service levels

No overprovisioning

Less resources, less time

Easy to change

Today

5. Consume from pre-allocated bin

4. Select appropriate bin

3. Expose pre-allocated bins

2. Pre-allocate static bins

1. Pre-define storage configurations

1. Define storage policy

2. Apply policy at VM creation

VSAN

VSAN

Shared

Datastore

Resource and data service are

automatically provisioned and

maintained

✖ Overprovisioning (better safe than sorry!)

✖ Wasted resources, wasted time

✖ Frequent Data Migrations

Virtual SAN Puts The App In Charge

Simpler and automated storage management through application centric approach

5

6

Management Clusters

Use Cases

Backup and DR

Target

DMZ / Isolated

Tier 2 / Tier 3

Test / Dev / Staging

Private cloud

Virtual Desktop

ROBO

VDI

Site A Site B

vSphereVSAN

VMware Virtual SANHardware Requirements

Two Ways to Build a Virtual SAN Node

8

Completely Hardware Independent

1. Virtual SAN Ready Node

…with multiple options available at GA + 30

Preconfigured server ready to use Virtual SAN…

2. Build Your Own

…using the Virtual SAN Compatibility Guide*

Choose individual components …

Flash: SSD or PCIe

SAS/NL-SAS/ SATA HDDs

Any Server on vSphere Hardware Compatibility List

HBA/RAID Controller

Hardware Requirements

9

Any Server on the VMware Compatibility Guide

• SSD, HDD, and Storage Controllers must be listed on the VMware Compatibility Guide for VSAN http://vmwa.re/vsanhcl

1Gb/10Gb NIC

SAS/SATA Controllers (RAID Controllers must work in “pass-through” or RAID0” mode

SAS/SATA/PCIe

SSD

SAS/NL-SAS/SATA

HDD

At least 1

of each

ESXi Boot device: 4GB to 8GB USB/SD

Flash Based Devices

In Virtual SAN ALL write operations always go directly to the Flash tier, and most reads will come from Flash

Flash based devices serve two purposes in Virtual SAN

1. Non-volatile Write Buffer (30%)

– Writes are acknowledged when they enter prepare stage on SSD.

– Reduces latency for writes

2. Read Cache (70%)

– Cache hits reduces read latency

– Cache miss – retrieve data from HDD

Choice of hardware is the #1 performance differentiator between Virtual SAN configurations.

10

Flash Based Devices

• VMware SSD Performance Classes

– Class A: 2,500-5,000 writes per second

– Class B: 5,000-10,000 writes per second

– Class C: 10,000-20,000 writes per second

– Class D: 20,000-30,000 writes per second

– Class E: 30,000+ writes per second

• Endurance

– 10 Drive Writes per Day (DWPD) for 5 years, and

– Random write endurance up to 3.5 PB on 8KB transfer size per NAND module, or 2.5 PB on 4KB transfer size per NAND module

11

Flash Capacity Sizing

The general recommendation for sizing Virtual SAN's flash capacity is to have 10% of the anticipated consumed storage capacity before the Number of Failures To Tolerate is considered

Total flash capacity percentage should be based on use case, capacity and performance requirements

– 10% is a general recommendation, could be too much or it may not be enough

Measurement Requirements Values

Projected VM space usage 50GB

Projected number of VMs 100

Total projected space consumption per VM (50% of 50GB) x 100 = 2,500 GB = 2.5 TB

Target flash capacity percentage 10%

Total flash capacity required 2.5TB x .10 = 250 GB

12

Magnetic Disks (HDD)

• SAS/NL-SAS/SATA HDDs supported

– 7200 RPM for capacity

– 10000 RPM for performance

– 15000 RPM for additional performance

• NL SAS will provide higher HDD controller queue depth at same drive rotational speed and similar price point

– NL SAS recommended if choosing between SATA and NL SAS

• When doing the design, take “overhead”, “slack space” and “failures to tolerate” in to account!

13

Storage Controllers

• SAS/SATA Storage Controllers

– Pass-through or “RAID-0” mode supported

– Also sometimes referred to as “Virtual SAN SAS” or “Virtual SAN SATA”

• Performance using RAID-0 mode is controller dependent

– Check with your vendor for SSD performance behind a RAID-controller

– Disable controller cache

• Storage Controller Queue Depth matters

– Higher storage controller queue depth will increase performance

• Validate number of drives supported for each controller

14

Network

• 1Gb / 10Gb supported

– 10Gb shared with NIOC for QoS will support most environments

– If 1GB then recommend dedicated links for Virtual SAN

• Jumbo Frames will provide nominal performance increase

– Enable for greenfield deployments, could lower CPU overhead

• Virtual SAN supports both VSS & VDS

– NIOC requires VDS

– Nexus 1000v – Should work but hasn't been fully tested

• Network bandwidth performance has more impact on host evacuation, rebuild times than on workload performance

15

ESXi Boot Devices

What installation device to use:

– Depends on amount of host memory

– Up to 512 GB

– Use of SD/USB devices or magnetic disks as the ESXi boot supported

– 512 GB or greater

– Only supported when using a magnetic disk or solid stated disk as the ESXi boot device

16

VMware Virtual SANTechnical Characteristics and Architecture

Technical Characteristics

Virtual SAN is a cluster level feature similar to:

– vSphere DRS

– vSphere HA

– Virtual SAN

Deployed, configured and manage from vCenter through the vSphere Web Client (ONLY!).

– Radically simple

• Configure VMkernel interface for Virtual SAN

• Enable Virtual SAN by clicking Turn On

18

Virtual SAN Implementation Requirements

• Virtual SAN requires:

– Minimum of 3 hosts in a cluster configuration

– All 3 host must contribute storage

• vSphere 5.5 U1 or later

– Maximum of 32 hosts

– Locally attached disks

• Magnetic disks (HDD)

• Flash-based devices (SSD)

– Network connectivity

• 1GB Ethernet

• 10GB Ethernet (preferred)

19

VSAN 10GbE netw ork

esxi-01 esxi-02 esxi-03

Virtual SAN and HA/DRS Cluster

Virtual SAN Constructs and Artifacts

New Virtual SAN constructs, artifacts and terminologies:

• Disk Groups

• VSAN Datastore

• Objects

• Components

• Virtual SAN Network

20

Virtual SAN Disk Groups

• Virtual SAN uses the concept of disk groups to pool together flash devices and magnetic disks as single management constructs

• Disk groups are composed of at least 1 flash device and 1-7 magnetic disks

– Flash devices are use for performance (Read cache + Write buffer)

– Magnetic disks are used for storage capacity

– Disk groups cannot be created without a flash device

21

disk group disk group disk group disk group disk group

Each host: 5 disk groups max. Each disk group: 1 SSD + 1 - 7 HDDs

Virtual SAN Datastore

• Virtual SAN is an object store solution that is presented to vSphere as a file system

• The object store mounts the VMFS volumes from all hosts in a cluster and presents them as a single shared datastore

– Only members of the cluster can access the Virtual SAN datastore

– Not all hosts need to contribute storage, but its recommended

22

vsanDatastore

esxi-01

disk group disk group disk group disk group

Single VSAN datastore per cluster

esxi-02 esxi-03 esxi-04

Virtual SAN Objects

• Virtual SAN manages data in the form of flexible data containers called objects. Virtual machine files are referred to as objects

• Virtual machines files are referred to as objects

– There are four different types of virtual machine objects:

• VM Home

• VM swap

• VMDK

• Snapshots

• Virtual machine objects are split into multiple components based on performance and availabilityrequirements defined in VM Storage profile

23

vsanDatastore

Virtual SAN Components

• Virtual SAN components are chunks of objects distributed across multiple hosts in a cluster in order to tolerate simultaneous failures and meet performance requirements

• Virtual SAN utilizes a Distributed RAID architecture to distribute data across the cluster

• Components are distributed with the use of two main techniques:

– Striping (RAID-0)

– Mirroring (RAID-1)

• Number of component replicas andcopies created is based on the object policy definition

24

raid-1

Mirror Copy Mirror Copy

ESXi Host ESXi Host

stripe-1b

stripe-1a

stripe-2b

stripe-2a

raid-0raid-0

Virtual SAN Network

• New Virtual SAN traffic VMkernel interface

– Dedicated for Virtual SAN intra-cluster communication and data replication

• Supports both Standard and Distributed vSwitch

– Leverage NIOC for QoS in shared scenarios

• NIC teaming – used for availability and not for bandwidth aggregation

• Layer 2 Multicast must be enabled on physical switches

– Much easier to manage and implement than Layer 3 Multicast

25

Management Virtual Machines vMotion Virtual SAN

Distributed Switch

20 shares 30 shares 50 shares 100 shares

vmnic0 vmnic1

vmk1 vmk2vmk0

VMware Virtual SANConfiguration Walkthrough

Configure Network

27

• Configure the new dedicated Virtual SAN network

– vSphere Web Client network template configuration feature.

Enable Virtual SAN

• One click configuration

– Virtual SAN configured in Automatic mode, all empty local disks are claimed by Virtual SAN for the creation of the distributed vsanDatastore

– Virtual SAN configured in Manual mode, the administrator must manually select disks to add the the distributed vsanDatastore by creating Disk Groups

28

Disk Management

• Each host in the cluster creates a single or multiple disk groups which contain a combination of HDDs, and SSDs

29

Virtual SAN Datastore

• A single Virtual SAN Datastore is created and mounted, using storage from all multiple hosts and disk groups in the cluster

• Virtual SAN Datastore is automatically presented to all hosts in the cluster

• Virtual SAN Datastore enforces thin-provisioning storage allocation by default

30

Virtual SAN Capabilities

• Virtual SAN currently surfaces five unique storage capabilities to vCenter

31

Number of Failures to Tolerate

• Number of failures to tolerate

– Defines the number of hosts, disk or network failures a storage object can tolerate. For “n” failures tolerated, “n+1” copies of the object are created and “2n+1” host contributing storage are required

32

vsan network

vmdkvmdk witness

esxi-01 esxi-02 esxi-03 esxi-04

~50% of I/O ~50% of I/O

Virtual SAN Policy: “Number of failures to tolerate = 1”

raid-1

Number of Disk Stripes Per Object

• Number of disk stripes per object

– The number of HDDs across which each replica of a storage object is distributed. Higher values may result in better performance.

33

vsan network

stripe-2b witness

esxi-01 esxi-02 esxi-03 esxi-04

stripe-1b

stripe-1a stripe-2a

raid-0raid-0

VSAN Policy: “Number of failures to tolerate = 1” + “Stripe Width =2”

raid-1

Virtual SAN Storage Capabilities

• Force provisioning

– if yes, the object will be provisioned even is the policy specified in the storage policy is not satisfiable with the resources currently available.

• Flash read cache reservation (%)

– Flash capacity reserved as read cache for the storage object. Specified as a percentage of logical size of the object.

• Object space reservation (%)

– Percentage of the logical size of the storage object that will be reserved (thick provisioned) upon VM provisioning. The rest of the storage object is thin provisioned.

34

Virtual SAN I/O flow – Write Acknowledgement

vsan network

vmdkvmdk

esxi-01 esxi-02 esxi-03 esxi-04

VSAN mirrors write IOs to all active mirrors, these are acknowledged when they hit the flash buffer!

witness

Destaging to HDD is done independently between hosts.

raid-1

VSAN IO flow – Reads

vsan network

witness

esxi-01 esxi-02 esxi-03

Read Cache

Write Buffer

Magnetic Disks

Read Cache

Write Buffer

Magnetic Disks

VSAN track of where IO resides and reads from where located

Read block 1 and 2.Block 1 is owned by

esxi-01 and block 2 by esxi-03.

vmdk vmdk

1MB(1)

1MB(2)

VMware Virtual SANVirtual Machine Provisioning Operations

Virtual Machine Provisioning Operations

• All VM provisioning operation include access to VM Storage Policies

38

Virtual Machine Provisioning Operations

• If the Virtual SAN Datastore understands the capabilities in the VMStorage Policy, it will be displayed as a matching resource

39

Virtual Machine Provisioning Operations

– If the VSAN Datastore can satisfy the VM Storage Policy, the VM Summary tab will display the VM as compliant

– If not, due to failures, or the force provisioning capability, the VM will be shown as non-compliant

40

Virtual Machine Policy Management

• Modify VM performance, capacity, and availability requirements withoutdowntime

41

Maintenance Mode – planned downtime

3 Maintenance mode options:

Ensure accessibility

Full data migration

No data migration

VMware Virtual SANSummarizing

44

Installs in two clicks

Managed from vSphere Client

Policy-based management

Self-tuning and elastic

Deep integration with VMware stack

Radically Simple

Embedded in vSphere kernel

Flash-accelerated

Matches the VDI density of all flash array

Best price/performance

100 kazillion IOps

High Performance Lower TCO

Eliminates large upfront investments (CAPEX)

Grow-as-you-go (OPEX)

Flexible choice of industry standard hardware

Does not require specialized skills

Virtual SAN Key Benefits

Don’t forget… ;-)

45