Cluster Shared Volumes Reborn in Windows Server 2012Subhasish BhattacharyaProgram ManagerMicrosoft Corporation

WSV430

Steve WienfeldTMEA Microsoft LeadNetApp Corporation

AgendaCluster Shared Volumes (CSV) in Windows Server 2012

Overview

What’s changed in Windows Server 2012

New architecture

Performance enhancements

Improved Backup of volumes

What is Cluster Shared Volumes (CSV)

Clustered file system in Windows Server 2012Layer of abstraction above NTFS

All cluster nodes can read/write to the CSV volume

LUN ownership by node abstracted from application

Applications failover without drive ownership changes

No dismounting and remounting of volumesFaster failover times (less downtime)

CSV Helps Address Management ComplexityChallenges managing large numbers of LUN’s

Manageability• Multi-path• Masking several LUN’s

Flexibility• LUN - smallest unit of

failover

Capacity• Poor SAN space

utilization

Scalability• Complexity with drive

letters

Cluster Shared Volumes as you know it TodayWindows Server 2008 R2

First introduced in Windows Server 2008 R2

Only supported Hyper-V workloadFocused v1 release targeted at enabling Hyper-V

Implemented as file system mini-filter Intercepted and routed I/O

CSV Motivations for Windows Server 2012

Expand CSV to more workloadsFile Server in addition to Hyper-V

Improve Backup

Improved performanceDirect I/O for more scenarios

Support for Spaces storage virtualization

Multi-subnet support

New CSV ArchitectureWhat it delivers

Improved interoperability with file system mini-filter driversBetter interoperability

Anti-virus softwareBackup Software

Application consistent distributed backups

Removed external authentication dependencies Improved performance and resiliencyNo longer Active Directory dependencies

Support for memory mapped files

Allows volume encryptionBitLocker encrypted volumes

Integrated with new File System featuresSupport for Offloaded Data Transfer (ODX)Spot-fixing integrated to do online correction

Under the hood

CSV Architecture

Cluster Shared Volumes (CSV)I/O synchronization overview

Metadata

Shared LUN

Shared StorageVHD VHD VHD

Read/Write

Simultaneous read/write access on all

Cluster Nodes

Server side metadata

synchronization - Avoids I/O

interruptions

When do Metadata Updates Occur

Virtual MachineCreation/deletionPower on/offMobility (live/storage migration)Extending dynamic VHDRenaming VHD

Backup - Snapshot creation

Key TakeawaysMetadata updates - small operations, infrequent for VMsParallel metadata updates - non-disruptive for applications

Disk

Volume Manager

NTFS

CSV File System Filter

VMShare

Server / SMB

Node 2

Disk

VMShare

MUP/RDBSS/

SMB

Node 1

VMShare

MUP/RDBSS/

SMB

Node 3

Direct I/O

CSV VolumeMg

r

CSV Proxy File

System

Storage Connection Broken or not present

Coordination Node

CSV VolumeMg

r

CSV Proxy File

System

CSV VolumeMg

r

CSV Proxy File

System

CSVFS

SAN

LBFO/RDMA

Simplified SetupConfiguring a CSV Disk

Failover Cluster Manager Storage view integration“Cluster Shared Volumes” container removed

CSV integrated into Failover Cluster core featureNo longer needs to be explicitly enabled

Simple right-click to add to CSVThat’s it!

Single NamespaceConsistent view across the cluster

Single consistent file name spaceFiles have same name and path on any cluster node

Volumes exposed under “ClusterStorage” root directoryVolumeX directory name can be renamed

CSV NamespaceMount Points

Used custom reparse points in Win2008 R2

Win2012 uses standard Mount Points

Delivers better interoperability with:Performance CountersSystem Center Operations ManagerMonitoring free space on CSV volumesBetter interoperability with backup software

CSV Proxy File System

CSV enabled volumes now appear as “CSVFS”NTFS file system under the covers

Enables applications to be CSV awareEnsures compatibility

demo

Subhasish Bhattacharya Program ManagerClustering and High Availability

Setting up Clustered Shared Volumes

Resiliency

How CSV Enables Even Higher Availability

Fault Tolerant Application HandlesCSV Resiliency

CSV provides I/O fault toleranceTransparently handles node, network, and HBA failures

CSVFS virtualizes file handles to applicationsVolume Paused - I/O queuedReopens “true” files handles and remaps the “virtual” handlesVolume Resumed – I/O completed

Failover is transparent to application!

VHD

I/O Connectivity Fault Tolerance

VM running on Node 2 is unaffected

Coordination Node

SAN Connectivity

Failure

I/O Redirected via network

VM’s can then be live migrated to another node with zero client downtime

VHD

Node Fault Tolerance

Volume relocates to a healthy node

Brief queuing of I/O while volume

ownership is changed

Node Failure VM running

on Node 2 is

unaffected

Coordination Node

New Coordinator

Node

VHD

Network Fault Tolerance

Volume mounted on Node 1 Network Path

Connectivity Failure

Metadata Updates

Rerouted to redundant network

Fault-Tolerant TCP connections make a path failure seamless

VM running on Node 2 is

unaffected

demo

Subhasish Bhattacharya Program ManagerClustering and High Availability

Cluster Shared Volume Resiliency

Continuously Available Scale out File ServerFlexible storage choices for the private cloud

Cluster platform for a continuously available scale out file serverCluster-wide client access point Consistent cluster-wide file server configurationCSV cluster-wide file system

Zero client downtime failover – both planned and unplanned downtime

Cluster Shared Volumes

Single Logical Server (\\Foo\Share)

Accessing VHDs over SMB Hyper-V Cluster

File Server Cluster

Single File System Namespace

Maximized File System AvailabilityFrom Hours to Seconds …

Disk scanning process separated from repair process

Online volume scanning Volume offline only to repairBased on number of errors to fix, not size of volume

Zero offline time with CSVChkdsk/Spotfix integrated

100

Millio

n File

s

200

Millio

n File

s

300

Millio

n File

s0

50

100

150

200

250

300

350

400

Windows Server 2008 R2

Windows Server 2012M

inu

tes

<3 sec of downtime

Performance

Improved CSV Performance

Cluster Shared Volumes CachingImproved CSV I/O Performance

Windows Cache Manager integrationBuffered read and write I/O’s cached like traditional NTFS

CSV Block CacheRead-Only cache for un-buffered I/O

I/O which is excluded from Cache Manager

Distributed cache consistent across cluster

Huge value for Pooled VM VDI scenarios

512 MB recommended value

Disabled by default

No downtime to modify

demo

Subhasish Bhattacharya Program ManagerClustering and High Availability

Cluster Shared Volume Block Cache in action

Redirected I/O Less OftenCSV Optimizations

Direct I/O for more scenariosDelivers faster I/O performance and lower network overheadDirect I/O for all types of file opensBuffered Reads and WritesBetter VM creation and copy performance

New algorithm for I/O redirection detectionOpportunistic Locking as distributed locking mechanism

• Block level I/O performance parityDirect I/O• Remote file system (SMB) performance

parityRedirected I/O

Block Level I/O RedirectionHigh Performance fault condition I/O redirection

2x performance over File System redirection

Disk

Volume Manager

NTFS

CSV File System Filter

VMShare

Server / SMB

Node 1

VMShare

MUP/RDBSS/

SMB

Node 2

Storage Connection Broken or not present

Coordination Node

CSV VolumeMg

r

CSV Proxy File

System

CSV VolumeMg

r

CSV Proxy File

System

CSVFS

LBFO/RDMA

Avoids traversing file system stack

twice

Redirected I/O ModesMultiple levels of CSV I/O redirection

• Redirection on a per file basis• Opening file for shared access

File level redirected

• Volume redirection at top of CSV pseudo-file system stack• Manually placing CSV in redirected mode• Snapshot creation

File System redirection

• Volume redirection at bottom of CSV pseudo-file system stack directly to bottom on coordinator• Storage connectivity to volume lost

Block redirection

Node 1 Node 2 Node 3

CSVFSMUP/ RDBSS/ RDR/SMB

MetadataOr

RedirectedIO

CSV Volume Manager

Direct IO

Volume Mounted on

Node 2

CSVFS MUP/ RDBSS/ RDR/SMB CSVFS

NTFS

SMB Server

MetadataOr

RedirectedIOMetadata

OrRedirected

IO

Direct IO

CsvFlt

VolumeCSV Volume

ManagerCSV Volume

Manager

Direct IO

MetadataOr

RedirectedIO

Direct IO

DirectIO

Block LevelRedirected

IO

Block LevelRedirected

IO

Physical Disk Physical Disk

IODirect IO

NTFS

Csv Namespace

Filter

SystemVolume

Block LevelRedirected

IO

Block LevelRedirected

IO

Application Application Application

SMB 3.0 Performance ImprovementsInherit gains for CSV redirection performanceImproved Performance of refactored SMB 3.0 client (98%)

Network transport optimizationsTCP/IP – SMB multi-channel & NIC Teaming, TCP offloads, DC-TCPRDMA – Lowest network CPU overhead (cycles/byte)

SMB SMB + DAS0

500

1000

1500

2000

2500

3000

3500

4000

4500

1175 (27%)

4270 (98%) 4315 (100%)

SQL TPC-C transactions/s

High Speed CSV I/O RedirectionSMB 3.0 integration

SMB multi-channelTraffic streamed across multiple networksImproved I/O performance in redirected mode

SMB Direct (SMB 3.0 over RDMA)

CSV Streaming I/O Across Multiple

Networks

Coordinator Node

VHD

10.10.10.X

20.20.20.X

VM with I/O being

redirected

Performance Improvements – RecapCSV Redirected mode enhancement summary

I/O redirection needed less

often

CSV Block Level Redirection

SMB multi-channelDirect (RDMA)

Improved SMB performance ~2% off block

Many Pieces Come Together for Radical Improvement

Deployment

CSV Deployment Considerations

Planning VM Density Per CSV VolumeDeployment considerations

How many VMs per CSV volume? No CSV volume restrictions VMFS limitations do not apply to CSVCSV volume Metadata updates orchestrated server side and parallelized

How many IOPS can your storage array handle?

Asymmetric Storage Configurations Deployment considerations

Tradeoff Between Storage Connectivity and Network Bandwidth

Asymmetric node – Redirected I/O modeSMB multi-channel – I/O streamed in parallelPrivate interfaces used for cluster traffic if availableRecommendation: Distribute CSV coordinator nodes across cluster

Backup

CSV Backup

CSV Backup Key Wins

Distributed Snapshots

• Distributed app consistent snapshot creation across cluster

Non-disruptive backups

• CSV volume ownership does not change during backup

Parallel Backups

• On same or different• CSV volumes• Cluster nodes

Improved Interoperability

• Backup applications / requestors not required to be ‘CSV aware’

• With filter driversImproved I/O performance

• Direct I/O mode for software snapshots

NetApp

Alex Jauch Architect,  Microsoft Private Cloud NetApp

CSV Backup using NetApp SnapManager

NetApp SnapManager® for Hyper-V

Policy-based Backup and Restore of Hyper-V VMs

Protects VMs running on multiple Hyper-V clusters and/or nodes

Single pane of glass for multi-host Hyper-V backups

VMs grouped into “Datasets” for ease of backup administration

Data-protection policies applied to datasets

CSV Backups in Windows Server 2012

Distributed application consistent VM backups Uses New ‘VSS CSV Writer’ and ‘VSS CSV Provider’

CSV value for Hyper-V backups Initializes VSS backup once on ‘backup node’VSS providers only called on ‘backup node’Faster backups, Fewer snapshots

Backup ArchitectureSMHV MMC Snap InPowerShell CLI

SMHV Service (VSS Requester)

VSS Service

Data ONTAP Hardware provider

VM2

VM1VM3

VM4

Hyper-V writer

CSV Providers

CSV writer

VSS Requester (no participation)

VSS Service

Hardware provider (no participation)

Hyper-V writer

CSV Providers

CSV writer

Non Requestor Nodes

Data ONTAP®

VHD VHD VHD

CSV

Single Windows Volume

iSCSI, FCP, FCoE

“Host1”Demosvr-01

“Host2”Demosvr-02

Backup Node

Backup Demo

CSV Backup using NetApp SnapManagerfor Hyper-VSteve WienfeldTMEA Microsoft LeadNetApp Corporation

SummaryKey Takeaways

CSV significantly enhanced in Windows Server 2012Support for more workloadsHigh performanceName a concern, it’s gone!CSV is a core infrastructure to enable your private cloud

INFRASTRUCTUREa s a S E RV I C E

Related Content

Find Us Later at the “Availability” Booth

WSV324 Building a Highly Available Failover Cluster Solution with Windows Server 2012 from the Ground UP

VIR301 Windows Server 2012 Hyper-V Storage

VIR306 Hyper-V over SMB: Remote File Storage Support in Windows Server 2012 Hyper-V

VIR304 Building Flexible Hyper-V Environments Windows Server 2012 Hyper-V Live Migration and Live Storage Migration

VIR401 Hyper-V High-Availability and Mobility: Designing the Infrastructure for Your Private Cloud

VIR302 Enabling Disaster Recovery for Hyper-V Workloads Using Hyper-V Replica

WSV310 Windows Server 2012: Cluster-in-a-Box, RDMA, and More

Resources

http://blogs.msdn.com/b/clustering/http://

social.technet.microsoft.com/Forums/en/winserverClustering/

Failover Clustering Blog Failover Discussion Forum

SIA, WSV, and VIR Track Resources

DOWNLOAD Windows Server 2012 Release Candidate

microsoft.com/windowsserver

#TEWSV430 DOWNLOAD Microsoft System Center 2012 Evaluation

microsoft.com/systemcenterHands-On Labs

Talk to our Experts at the TLC

Resources

Connect. Share. Discuss.

http://europe.msteched.com

Learning

Microsoft Certification & Training Resources

www.microsoft.com/learning

TechNet

Resources for IT Professionals

http://microsoft.com/technet

Resources for Developers

http://microsoft.com/msdn

Evaluations

http://europe.msteched.com/sessions

Submit your evals online

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS

PRESENTATION.