Architecting Fibre Channel HA Solutions

Architecting Fibre Channel HA Solutions

Rick Jooss

[email protected]

2NetApp Confidential -- Do Not Distribute

Agenda

CFModes

Single System Imagine

Multipathing

Host Clustering

Storage System Backend HA

Q&A


Agenda

CFModes

Single System Image

Multipathing

Host Clustering


Q&A


CFMODE – Cluster Failover Mode

What is CFMODE?– FCP Setting– Determines behavior of FC Target Ports, particularly

during a CFO event

Why is there more than one CFMODE?– Original CFMODE (standby) did not work for all host

types (HP-UX, AIX)– Original CFMODE did not work with the 270C

because it only has a single FC port


Available Paths- Standby Mode

Switch/Fabric 1 Switch/Fabric 2

Host

LUNs LUNs

0c 0d 0c 0d0a 0b 0a 0bHA Configuration

Controller 1 Controller 2

Solid Blue are paths to the LUNs being served by Controller 1

Dashed Purple are paths to the LUNs being served by Controller 2




Path Access (Switch Failure) – Standby Mode


Host

LUNs LUNs

Solid and Blue are paths to the LUNs being served by Controller 1

Dashed and Purple are paths to the LUNs being served by Controller 2

Switch/Fabric 1 will experience a failureMP layer works around the failure




Controller 2 Takes over all operations

Path Access (CFO event)- Standby Mode


Host

LUNs LUNs

Conntroller 1 will experience a failure






Filer Head 2 Takes over all operationsMP layer is not involved in switchover

Path Access (CFO event)- Standby Mode


Host

LUNs LUNs

Controller 1 will experience a failure

WWN1 WWN2 WWN3 WWN4 WWN5 WWN6 WWN7 WWN8




Available Paths - Partner Mode


Host

LUNs LUNs






Available Paths - Partner Mode – FAS3000 Default Configuration


Host

LUNs LUNs



0c 0d 0c 0dHA Configuration



Available Paths - Dual Fabric

0c_0


Host

HA Configuration




LUNsLUNs

0c_00c_2 0c_2


Agenda

CFModes


Multipathing

Host Clustering


Q&A


What is the single system image cfmode?

Universal cfmode– Works on all HA storage systems– Works on all switches

Presents the HA configuration as a single target

All LUNs are visible on all controller ports

All hosts require multipathing software


0c 0c 0d0dHA Configuration


Available Paths - Single System Image – Single Card


Host

LUNs LUNs






Path Access (Switch Failure) - Single System Image – Single Card


Host

LUNs LUNs

Solid and Blue are paths to the LUNs being served by Head 1

Dashed and Purple are paths to the LUNs being served by Head 2

MP layer works around the failureSwitch/Fabric 1 will experience a failure




Path Access (CFO event) - Single System Image – Single Card


Host

LUNs LUNs



Controller 2 takes over all operations MP layer works around the failure

Controller 1 will experience a failure


Available Paths - Single System Image – Single Port


Host

LUNs LUNs



0d 0dHA Configuration



Available Paths - Single System Image – Single Port

Host

LUNs LUNs



0d 0dHA Configuration


Loop ModeLoop Mode


Why SSI mode?

Works in all configurations

Makes us look more like other SAN vendors

Reduces port burn without using FC Loop– Fully redundant config requires only 1 “wire” per

controller, instead of 2.

Simpler wiring, no a/b port distinctions and no requirement to run the same cables from each controller to the same switch.


Management changes

Unified LUN mapping address space across the HA configuration.– Controller prevents these conflicts by checking with

the partner controller.

If the controller interconnect is down, some operations are disabled by default– Igroup add, lun map, lun online, igroup set ostype


SSI Roadmap

Introduced in ONTAP 7.1

Refer to FCP host compatibility matrix http://now.netapp.com/NOW/knowledge/docs/san/fcp_iscsi_config/index.shtml for specific host support

http://now.netapp.com/NOW/knowledge/docs/san/fcp_iscsi_config/index.shtml

http://now.netapp.com/NOW/knowledge/docs/san/fcp_iscsi_config/index.shtml


Agenda

CFModes


Multipathing

Host Clustering


Q&A


Multipathing

Multipathing provides multiple paths from the host to the external storage device

Provides High-Availability – Protects against path failures– Ensures high availability of applications and data by

eliminating single points of failure

Provides Improved Performance – Increases potential performance by utilizing multiple

paths




Multipathing


Host

LUNs LUNs




A/P (active passive) policy – Single LUN


LUNs LUNs

Hosts




A/P (active passive) policy – No Round Robining


Hosts

LUN1 LUN3LUN2 LUN4




A/P (active passive) policy - Round Robining


Hosts

LUN1 LUN3LUN2 LUN4


A/P (active/passive)

Active/Passive Configuration– 1 active path to a single LUN

• Performance to a LUN is limited by that paths capability (HBA, switch, target port)

– Possible to round robin multiple LUNs across multiple paths

– All other paths to the LUN are passive– On failover

• Primary paths are tried first• Secondary paths are used if no primary paths are

available


A/A (Active active) policy (cfmode = standby)


Hosts

LUNs LUNs




A/A (active/active)

Host accessing data from a single LUN across multiple paths simultaneously– Typically used for load balancing

• Round Robin• Least Queue Depth• Weighted

– On failure I/Os are sent down remaining available paths




A/A/A (asymmetric active active)


Host

LUNs LUNs


A/A/A (asymmetric active active)

Distinguishes between primary and secondary paths

Does active/active across primary paths only

Only uses secondary paths when no primary are available


NetApp’s Multipathing Strategy

2 pronged strategy– Support for “native” solutions

• What most customers rightly feel best about– Support for host and storage independent

solution• VERITAS• Allows common solution across various

server as well as storage variants


Multipathing For Windows

Windows MPIO– Uses the Microsoft standard infrastructure– A/P Policy– Automatically chooses primary paths for failover

before trying proxy ones– In standby the LUNS are automatically round

robined across all paths MPIO

Partner/SSI cfmode A/P

Standby cfmode A/P

Dual Fabric cfmode A/P


MultiPathing For Solaris

DMP 4.0 MPxIO

Partner/SSI cfmode A/A/A A/P

Standby cfmode A/A N/A

Dual Fabric cfmode A/P A/P


MultiPathing For Solaris

VERITAS DMP 4.0– NetApp ASL 4.0– Supports A/P, A/A, & A/A/A (Active Passive

Concurrent)

SUN Native MPxIO– Not supported with standby cfmode– Supports A/P – Can be A/A but required manual failback– Manual configuration required– Round Robining of the LUNs possible– Sometimes called

• Traffic Manager • Leadville Stack


MultiPathing For Linux

Qlogic– A/P Policy– Manually configured– Round Robining of LUNs is possible

DCM– Linux native solution

Qlogic DM

Partner/SSI cfmode A/P A/A/A

Standby cfmode A/P A/A



MultiPathing For AIX

DMP 4.0 SANpath MPIO

Partner/SSI cfmode A/A/A A/A/A A/A/A

Standby cfmode N/A N/A NA

Dual Fabric cfmode A/P A/P A/P


MultiPathing For AIX

SANpath– A/A/A– Automatically chooses primary paths for failover

before trying proxy ones– Special policy for SCSI-2 reservation

– Required for host clustering HACMP– Can only use A/P

VERITAS DMP 4.0– Only supports A/A/A

IBM MPIO– IBM native solution with NetApp PCM


Multipathing for HP-UX

Partner/SSI cfmode A/P A/P

Standby cfmode N/A N/A


PVLinks DMP 3.5


Multipathing for HP-UX

PVlinks/LVM– A/P policy– Single active path per LUN, user controlled– Ordering for remaining paths for failover– ntap_config_paths

• NETAPP script to define path ordering based on filer path types: primary, proxy

• automatically round robin primary paths among all LUNS

– Supports both FCP and iSCSI paths

VERITAS DMP 3.5– A/P Policy


Multipathing for VMware

VMware– A/P Policy– Manually configured– Round Robining of LUNs possible

VMware


Standby cfmode A/P



Multipathing for Netware

Novell– A/P Policy– Manually configured– Round Robining of LUNs possible

Novell


Standby cfmode A/P



Fibre Channel SAN Host Support

Partner/SSI cfmode

Windows“NTAP DSM”

Standby cfmode

Solaris “DMP”

Dual Fabric cfmode

Linux: Qlogic“Failover Mode”

A/P A/P

VMwareMultipathing

Solaris“MPxIO”

A/A/A A/A A/P

A/P

A/P A/P A/P

A/P A/P A/P

A/P N/A A/P

AIX“SANpath”

A/A/A N/A A/P

HP-UX“PVLinks”

Novell

A/P N/A A/P

A/P A/P A/P


Agenda

CFModes


Multipathing

Host Clustering


Q&A


Host Clustering & Storage

LUNs need to be made visible to host simultaneously

Some Host Clustering solutions require SCSI reservations to avoid to split brain

0c

Switch/Fabric 1

Host 1

LUNs

Controller 10b 0d

Controller 2

Controller 1 Active Shelf(s)

Controller 2 Active Shelf(s)

Switch/Fabric 2

0a

0d0b 0a0c

Host 2


Host Clustering for Microsoft

Microsoft Cluster– SnapDrive is integrated to help configuration– WIN2K3 allows single HBA for both boot device &

shared storage– Cannot grow LUN online in cluster

• SnapDrive ability to very quickly grow a LUN minimizes the pain caused by this


Host Clustering for VERITAS

VCS– By default does not us I/O fencing to protect against

split brain– I/O fencing requires SCSI-3 reservations– 7.0.3 will have SCSI-3 reservations that are

compatible with VERITAS– Does not do failover on FC links


Host Clustering for HP-UX

ServiceGuard– 1 to 3 node clusters using SCSI-2 locks as arbitrator

to avoid split brain– Does not do failover in dead FC links


Host Clustering for AIX

HACMP– Uses SCSI-2 locks as arbitrator to avoid split brain

• “setsp –b2” to enable locks with SANpath• SCSI-2 locks to active/active are mutually

exclusive


Fibre Channel SAN Host Support

Host ClusterOS Vendor Multipath File SystemHBA

Native SANpath HACMP

Volume Mgr

LVM

QLogic QLogic Novell Clusters NSS

Emulex MPIO MSCS MMC NTFS

Emulex Veritas DMP Veritas VCS Veritas VxVM Veritas VxFS

JFS/2Raw

NativeHP PVLInksVeritas DMP

MC ServiceGuardVeritas VCS

LVMVeritas VxVM

JFS/ HFSRaw

Veritas VxFS

QLogic QLogic Oracle 9i, 10g RACext3ext2

Reiser

QLogic QLogic Oracle 9i, 10g RACext3ext2

Reiser

EmulexQLogic

VMWare MSCSVirtualCenter (VMotion)

VMware VMFS 2.xRaw

Shared Storage


Agenda

CFModes


Multipathing

Host Clustering


Q&A


Enables Dual Path HA

Key Benefits Full storage hardware

redundancy in HA systems

Prevent cluster failover events due to many storage issues.

Complements CFO for improved HA and resiliency

Key Benefits Full storage hardware

redundancy in HA systems

Prevent cluster failover events due to many storage issues.

Complements CFO for improved HA and resiliency

Loop 1 Loop 2 Loop 3 Loop 4

X

Protect Against Cable Pulls Or

Breaks

X Protect Against Single HBA Failure

X

Protect Against Storage Controller

(eg. ESH2) Hot Swap


Switched Back-End

Dual Active Paths for HA Environments– Reduces the number of HA failovers– Improve overall HA performance– Data ONTAP tries to balance load across paths

SyncMirror– SyncMirror requires 100% disk overhead– Proper configuration survives all single failures


Agenda

CFModes


Multipathing

Host Clustering


Q&A?

Documents

Architecting Fibre Channel HA Solutions