Upload
jonav71
View
245
Download
0
Embed Size (px)
Citation preview
8/10/2019 VMWare + EMC storage.pdf
1/102
EMC Corporation
Corporate Headquarters:
Hopkinton, MA 01748 -91031-508 -435 -1000
www.emc.com
EMC Host Connectivity Guide
for VMware ESX Server v2.x
P/N 300-002-304REV A02
8/10/2019 VMWare + EMC storage.pdf
2/102
EMC Host Connectivity Guide for VMware ESX Server v2.xii
Copyright 2001-2005 EMC Corporation. All rights reserved.
Updated March 2005
EMC believes the information in this publication is accurate as of its publication date. Theinformation is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED "AS IS." EMC CORPORATIONMAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THEINFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIEDWARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires anapplicable software license.
Trademark Information
8/10/2019 VMWare + EMC storage.pdf
3/102
EMC Host Connectivity Guide for VMware ESX Server v2.x iii
Preface ............................................................................................................................ ix
Chapter 1 Introduction
Purpose of this Document.............................................................. 1-2Related Documentation .................................................................. 1-2VMware ESX Server v2.x Features................................................ 1-3
VMotion ..................................................................................... 1-3Raw Device Mappings ............................................................. 1-3
Chapter 2 Installation Information for VMware ESX Server v2.xwith Symmetrix and CLARiiON Arrays
Overview........................................................................................... 2-2VMware ESX Server Options.................................................. 2-4VMkernel Parameters .............................................................. 2-8
Driver Configuration in VMware ESX Server v2.x................... 2-23Rescanning the SCSI Bus .............................................................. 2-26Persistent Binding.......................................................................... 2-30LUN Spanning................................................................................ 2-34
Setting Up and Installing Virtual Machines............................... 2-37Additional Notes............................................................................ 2-45
Chapter 3 VMware ESX Server v2.x and Symmetrix Storage ArrayFibre Channel Environment
VMware/Symmetrix Environment............................................... 3-2Host and Hardware Connectivity.......................................... 3-2
Symmetrix Connectivity.......................................................... 3-2Booting the VMware ESX Server v2.5 from the Symmetrix ...... 3-6
Contents
8/10/2019 VMWare + EMC storage.pdf
4/102
EMC Host Connectivity Guide for VMware ESX Server v2.xiv
Contents
Fabric Zoning ................................................................................... 3-8
Required Storage System Configuration...................................... 3-8Useful VMware ESX Server Utilities and Functions .................. 3-9Addressing Symmetrix Devices .................................................. 3-10
Fabric Addressing .................................................................. 3-10SCSI-3 FCP Addressing ......................................................... 3-11
Chapter 4 VMware ESX Server v2.x and CLARiiON Storage ArrayFibre Channel Environment
VMware/CLARiiON Environment .............................................. 4-2Host and Hardware Connectivity.......................................... 4-2CLARiiON Connectivity ......................................................... 4-2Booting the VMware ESX Server v2.5 from CLARiiONStorage Arrays .......................................................................... 4-5Fabric Zoning............................................................................ 4-7Required Storage System Configuration .............................. 4-7
Useful VMware ESX Server Utilities and Functions ........... 4-7Failover Functionality in VMware ESX Server v2.x for Attachto CLARiiON Storage Arrays ........................................................ 4-9Manual Registration of VMware ESX Server v2.x for Attachto CLARiiON Storage Arrays ...................................................... 4-13Adding the VMware ESX Server Host to a Storage Group..... 4-20Manual Trespass on CLARiiON Arrays to Recover theOriginal Path .................................................................................. 4-21
Index................................................................................................................................ i-1
8/10/2019 VMWare + EMC storage.pdf
5/102
EMC Host Connectivity Guide for VMware ESX Server v2.x v
2-1 VMware ESX Server Options ...................................................................... 2-42-2 Startup Profile ............................................................................................... 2-52-3 Security Settings ........................................................................................... 2-72-4 VMkernel Parameter Listing ....................................................................... 2-9
2-5 Storage Management > Disks and LUNs Tab ........................................ 2-372-6 Storage Management > Disks and LUNs > Core Dump Partition ...... 2-382-7 Access Mode ................................................................................................ 2-392-8 Add Virtual Machine Dialog .................................................................... 2-402-9 Status Monitor > Virtual Machines .......................................................... 2-402-10 VMware Remote Console > Connect to Virtual Machines ................... 2-412-11 Configuration Editor, Hardware, Connection ....................................... 2-433-1 One Host, Two Switches, and One Symmetrix Array ............................ 3-4
3-2 One Host, Two Switches, and Four Symmetrix Arrays .......................... 3-54-1 One Host, Two Switches, and One CLARiiON Array ............................ 4-34-2 One Host, Two Switches, and Four CLARiiON Arrays ......................... 4-44-3 Connectivity Status dialog ........................................................................ 4-15
Figures
8/10/2019 VMWare + EMC storage.pdf
6/102
vi EMC Host Connectivity Guide for VMware ESX Server v2.x
Figures
8/10/2019 VMWare + EMC storage.pdf
7/102
EMC Host Connectivity Guide for VMware ESX Server v2.x vii
2-1 VMkernel Parameters, Descriptions, and Default Values .................... 2-123-1 Symmetrix SCSI-3 Addressing Modes .................................................... 3-124-1 Useful VMware ESX Server Utilities and Functions ............................... 4-7
Tables
8/10/2019 VMWare + EMC storage.pdf
8/102
EMC Host Connectivity Guide for VMware ESX Server v2.x
Tables
viii
8/10/2019 VMWare + EMC storage.pdf
9/102
EMC Host Connectivity Guide for VMware ESX Server v2.x ix
Preface
As part of an effort to improve and enhance the performance and capabilitiesof its product line, EMC from time to time releases revisions of its hardwareand software. Therefore, some functions described in this document may not
be supported by all revisions of the software or hardware currently in use.For the most up-to-date information on product features, refer to your
product release notes.
If a product does not function properly or does not function as described inthis document, please contact your EMC representative.
This guide describes the features and setup procedures for VMware
ESX Server host interfaces to EMC Symmetrix and CLARiiONStorage Array systems.
Audience This guide is intended for use by storage administrators, systemprogrammers, or operators who are involved in acquiring, managing,or operating Symmetrix, CLARiiON, and host devices.
Readers of this guide are expected to be familiar with the followingtopics:
Symmetrix or CLARiiON system operation
VMware ESX Server operating environment
8/10/2019 VMWare + EMC storage.pdf
10/102
x EMC Host Connectivity Guide for VMware ESX Server v2.x
Preface
Organization Here is an overview of where information is located in this guide.
Chapter 1, Introduction This document is meant to assist in theinstallation and configuration of VMware ESX Server v2.x attached toEMC CLARiiON and Symmetrix storage systems.
Chapter 2, Installation Information for VMware ESX Server v2.x withSymmetrix and CLARiiON Arrays This chapter covers installationinformation for the VMware ESX Server v2.x with Symmetrix andCLARiiON arrays.
Chapter 3, VMware ESX Server v2.x and Symmetrix Storage Array FibreChannel Environment This chapter lists Symmetrix Fibre Channelsupport information specific to the VMware ESX Server environment.
Chapter 4, VMware ESX Server v2.x and CLARiiON Storage Array FibreChannel Environment This chapter lists CLARiiON Fibre Channelsupport information specific to the VMware ESX Server environment.
8/10/2019 VMWare + EMC storage.pdf
11/102
EMC Host Connectivity Guide for VMware ESX Server v2.x xi
Preface
Conventions Used inThis Guide
EMC uses the following conventions for notes and cautions.
A note presents information that is important, but not hazard-related.
CAUTION
A caution contains information essential to avoid damage to thesystem or equipment. The caution may apply to hardware orsoftware.
Typographical ConventionsEMC uses the following type style conventions in this guide:
AVANTGARDE Keystrokes
Palatino,bold
Dialog box, button, icon, and menu items in text Selections you can make from the user interface,
including buttons, icons, options, and fieldnames
Palatino,italic
New terms or unique word usage in text Command line arguments when used in text Book titles
Courier,italic
Arguments used in examples of command linesyntax.
Courier System prompts and displays and specificfilenames or complete paths. For example:
working root directory [/user/emc]:
c:\Program Files\EMC\Symapi\db
Courier,
bold
User entry. For example:
symmpoll -p
Options in command line syntax
8/10/2019 VMWare + EMC storage.pdf
12/102
xii EMC Host Connectivity Guide for VMware ESX Server v2.x
Preface
Where to Get Help For questions about technical support, call your local sales office orservice provider.
If you have a valid EMC service contract, contact EMC CustomerService at:
Follow the voice menu prompts to open a service call and select theapplicable product support.
If you are located outside the North America, call the nearest EMCoffice for technical assistance.
Sales and Customer
Service Contacts
For the list of EMC sales locations, please access the EMC home page
at:http://www.emc.com/contact/
For additional information on the EMC products and servicesavailable to customers and partners, refer to the EMC Powerlink Website at:
http://powerlink.emc.com
Your Comments Your suggestions will help us continue to improve the accuracy,organization, and overall quality of the user publications. Please senda message to [email protected] your opinions ofthis guide.
United States: (800) 782-4362 (SVC-4EMC)
Canada: (800) 543-4782 (543-4SVC)
Worldwide: (508) 497-7901
8/10/2019 VMWare + EMC storage.pdf
13/102
Introduction 1-1
1Invisible Body Tag
This document is meant to assist in the installation and configuration of
VMware ESX Server v2.x attached to EMC CLARiiON and Symmetrix
storage systems.
Purpose of this Document ................................................................1-2 Related Documentation.....................................................................1-2 VMware ESX Server v2.x Features ..................................................1-3
Introduction
8/10/2019 VMWare + EMC storage.pdf
14/102
1
1-2 EMC Host Connectivity Guide for VMware ESX Server v2.x
Introduction
Purpose of this Document
This document is meant to assist in the installation and configuration of
VMware ESX Server v2.x attached to EMCCLARiiONand Symmetrix
storage systems.
Related Documentation
For the EMC Support Matrix (ESM)for supported server and HBAcombinations, go to:
http://powerlink.EMC.com/
For Navisphereinformation, refer to the Navisphere ManagerAdministrators Guide.
For VMware specific documentation, such as the VMware ESX Server
Release Notes, ESX Server Administration Guide, and the ESX ServerInstallation Guide, go to:
http://www.vmware.com/support/
8/10/2019 VMWare + EMC storage.pdf
15/102
1
VMware ESX Server v2.x Features 1-3
Introduction
VMware ESX Server v2.x Features
VMware ESX Server is similar to Linux, but is its own operatingsystem. VMware GSX Server is an application that runs on top of astandard Linux or Windows operating system.
VMware ESX Server consists of virtualization software that providesserver consolidation and allows several instances of operatingsystems running as virtual machines on one physical machine.
Currently, VMware ESX Server installs and operates as a 32-bitoperating system on Intel-based and Opteron-based servers. Theoperating systems qualified to run as virtual machines include 32-bitversions of Linux, Windows, and NetWare.
VMotion VMotion is part of VMware Virtual Center, a systems managementand provisioning tool used by system administrators. VMotiontechnology allows the migration of virtual machines from onephysical server to another physical server. For instance, VMotioncould be used to migrate virtual machines from one server to anotherso that maintenance may be performed on the original server. Thisallows the operating systems running in the virtual machinepartitions to continue running their workloads.
Raw Device
Mappings
VMware ESX Server v2.5 introduced a new technology called raw
device mapping. Essentially, raw device mapping is a SCSIpass-through technology that allows Virtual Machines to pass SCSIcommands directly to the physical hardware.
Note that using raw disk mappings in Shared mode with Fibre ChannelHBAs is not supported. The HBAs must be allocated solely to the VirtualMachines in order to use raw disks.
8/10/2019 VMWare + EMC storage.pdf
16/102
1
1-4 EMC Host Connectivity Guide for VMware ESX Server v2.x
Introduction
8/10/2019 VMWare + EMC storage.pdf
17/102
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays 2-1
2Invisible Bo
This chapter covers installation information for the VMware ESXServer v2.x with Symmetrix and CLARiiON arrays.
Overview.............................................................................................2-2 Driver Configuration in VMware ESX Server v2.x.....................2-23 Rescanning the SCSI Bus.................................................................2-26 Persistent Binding ............................................................................2-30 LUN Spanning..................................................................................2-34 Setting Up and Installing Virtual Machines.................................2-37 Additional Notes..............................................................................2-45
Installation Information for
VMware ESX Server v2.x withSymmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
18/102
2
2-2 EMC Host Connectivity Guide for VMware ESX Server v2.x
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
Overview
The fibre channel HBA driver functions as a device driver layerbelow the standard VMware SCSI adapter driver. The Fibre Channelinterface is therefore transparent to the VMware disk administrationsystem.
Note:VMware ESX Server is NOT Linux and, therefore, requires its ownEMC-qualified drivers, not the standard EMC-qualified Linux drivers.
Install VMware ESX Server from the CD. The VMware ESX Serverinstallation is based on a modified Red Hat v7.2 v2.4.9 kernel.However, VMware ESX Server is NOT Linux.
VMware recommends partitioning the internal hard drive into fourpartitions. For example:
Note that a swap partition is required by both the service console and theVMkernel. The reference to the swap partition in this example is for theservice console. The swap partition and core dump partitions for theVMkernel can be created using the GUI after the initial installation of theservice console OS.
As stated in the Preface, EMC does not recommend fabric boot of theVMware ESX Server itself over the fabric, but we do support fabric boot with
VMware ESX Server v2.5. The Virtual Machines are expected to boot over thefabric using the QLogic or Emulex HBAs.
Refer to the ESM for specifics on which operating systems aresupported as Virtual Machines.
You may customize the installation according to your server and theamount of memory and hard disk space you have available.
/boot 50 MB primary
swap 256 MB to 1 GB primary
/ 2.5 GB primary
/vmimages 4 GB
8/10/2019 VMWare + EMC storage.pdf
19/102
2
Overview 2-3
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
The following are a few recommendations for the installation:
Use static IP addresses. Set the hardware clock when prompted.
Create at least one user account other than root.
Note that emacs, samba, and NFS are not enabled by default in theConsole OS.
Reboot the system after completing the installation.
For VMware ESX Server v2.1.x installations, when the systemreboots, you will be prompted with two options in the LILO bootprompt:
linux-up,linux
Assuming the system on which you are installing has multiple CPUs,the default boot image will be linux. This is expected andacceptable to use.
For VMware ESX Server v2.5 installations, when the system reboots,you are prompted with three options in the LILO boot prompt:
esx, linux-up,linux
The default boot image for VMware ESX Server v2.5 is esx.
You can now log in to the VMware server using IE or Netscape with
the host name or IP address. This allows you access to the VMwarehost using the Management User Interface (MUI).
After you log into the host, the first dialog is the VMware ESX ServerStatus Monitordialog.
The Status Monitordialog provides a summary reporting of thesystem CPU and processor usage by the Virtual Machines as well as avisual status report of the Virtual Machines themselves.
8/10/2019 VMWare + EMC storage.pdf
20/102
2
2-4 EMC Host Connectivity Guide for VMware ESX Server v2.x
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
VMware ESX Server Options
Select the Options tab. The Optionsdialog provides you with a listof actions that can be performed to configure or modify the VMwareESX Server, as shown in Figure 2-1.
Figure 2-1 VMware ESX Server Options
From the Optionstab, select the Startup Profilemenu. The StartupProfile creates the VMware ESX Server boot configuration and allows
you to specify whether the server resources should be allocated to theService Console, the Virtual Machines, or shared between the two.
Fibre Channel and SCSI HBAs can be allocated for use with theVirtual Machines only or can be shared between the Virtual Machinesand Service Console. In the case of booting from the fabric, the FCHBAs must be shared so that the Service Console and the VirtualMachines can access the boot device.
Networking cards cannot be shared between the Service Console andthe Virtual Machines so they must be allocated either to the Service
8/10/2019 VMWare + EMC storage.pdf
21/102
2
Overview 2-5
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
Console or to the Virtual Machines. Therefore, a minimum of twoNICs must be installed in the system.
For an example of the Startup Profiledialog, see Figure 2-2.
Figure 2-2 Startup Profile
After the changes appropriate to your system are made, save theconfiguration by clicking OK.
After saving the configuration changes, reboot the host so that thechanges can take effect.
8/10/2019 VMWare + EMC storage.pdf
22/102
2
2-6 EMC Host Connectivity Guide for VMware ESX Server v2.x
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
After the VMware host reboots, proceed with the VMkernelconfiguration.
The Network Connectionsmust be set up according to thenetwork interface cards (NICs) being used and the customerenvironment. Note that the server must have at least two NICs.Because NICs may not be shared, one must be allocated to theConsole and at least one must be allocated to the VirtualMachines.
It is not necessary to reboot the VMware ESX Server into the esxkernelin order for network configuration changes to take effect.
The Security Settingsselected are entirely dependent upon thecustomers desires. VMware recommends that customers use theHigh Security setting so all connections into ESX will beencrypted. If FTP and Telnet are required, the Medium Securitysetting will need to be selected. The Security Settingdialog is
shown in Figure 2-3 on page 2-7
8/10/2019 VMWare + EMC storage.pdf
23/102
2
Overview 2-7
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
Figure 2-3 Security Settings
Swap Configuration creates, modifies, and activates swap filesthat allow your virtual machines to use more memory than isphysically available. Whether or not you want to enable a swapfile depends on the amount of memory in the server. It isrecommended that a swap file be created so that the virtualmachines will be able to use more memory than is actuallyavailable in the server.
When booting from an internal hard drive, it is recommended that theswap file resides on the internal device, rather than a SAN-attacheddevice.
8/10/2019 VMWare + EMC storage.pdf
24/102
2
2-8 EMC Host Connectivity Guide for VMware ESX Server v2.x
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
If required, the SNMP agent is enabled in the SNMPConfigurationsection.
The Advanced Settingssection allows you to modify theVMkernel parameters. Currently, EMC tests with and supportsthe majority of the default VMkernel parameters. The tablebelow shows the parameters, their descriptions, and the VMwareand EMC default settings. It is highly recommended that theEMC default parameters are used when attached to EMC StorageArrays.
VMkernel Parameters
A number of parameters define the behavior of the VMkernel.Currently, EMC recommends that the majority of the parametervalues remain the same as the VMware defaults. The exception atthis time is the Disk.MaxLunparameter.
These values can be viewed and modified using the MUI and in thesubdirectories within the /proc/vmware/configdirectory.
To view the listing of directories in the /proc/vmware/configdirectory, change to the directory and perform a listing:
cd /proc/vmware/config
ls la
dr-xr-xr-x 2 root root 0 Jan 18 06:07 Cpudr-xr-xr-x 2 root root 0 Jan 18 06:07 Disk
dr-xr-xr-x 2 root root 0 Jan 18 06:07 FileSystemdr-xr-xr-x 2 root root 0 Jan 18 06:07 Irqdr-xr-xr-x 2 root root 0 Jan 18 06:07 Memdr-xr-xr-x 2 root root 0 Jan 18 06:07 Migratedr-xr-xr-x 2 root root 0 Jan 18 06:07 Miscdr-xr-xr-x 2 root root 0 Jan 18 06:07 Netdr-xr-xr-x 2 root root 0 Jan 18 06:07 Numadr-xr-xr-x 2 root root 0 Jan 18 06:07 Scsi
These directories provide the types of parameters that are listed ineach. For instance, the /proc/vmware/config/Diskdirectorycontains the parameters pertaining to the Disk subsystem. Anychanges made to the parameters in this directory will be specific toDisks. For example, Disk.MaxLUNis part of the/proc/vmware/configfile located in the Disk directory and theparameter itself is theMaxLUNsetting.
If desired, changes to the values of these parameters made byperformed using the MUI. Log into the server via the MUI and selectthe Optionstab. In the line for Advanced Settings | Service Console
8/10/2019 VMWare + EMC storage.pdf
25/102
2
Overview 2-9
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
Settings, select Advanced Settings. You will then be presented with awindow that contains the parameters and their values and
descriptions of the parameters.
The parameters are listed in alphabetical order. An example of theMUI dialog listing the beginning of the parameter list is shown inFigure 2-4.
Figure 2-4 VMkernel Parameter Listing
The values of the VMkernel parameters also can be set at thecommand line. This is done by echoing the /proc filesystem. Asmentioned previously, the parameters are ultimately listed in the/proc/vmware/configdirectory. The following exampledemonstrates the steps to view and change the value of a particularsetting.
I t ll ti I f ti f VM ESX S 2 ith S t i d CLARiiON A
8/10/2019 VMWare + EMC storage.pdf
26/102
2
2-10 EMC Host Connectivity Guide for VMware ESX Server v2.x
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
In this particular example, the LogMultiPathparameter is beingenabled.
[root@l82bi199 /]# cd /proc/vmware/config/Scsi[root@l82bi199 Scsi]# lsConflictRetries LogAborts LogMultiPath PassthroughLocking[root@l82bi199 Scsi]# cat LogMultiPathLogMultiPath (Log path state changes) [default = 0]: 0[root@l82bi199 Scsi]# echo 1 >> LogMultiPath[root@l82bi199 Scsi]# cat LogMultiPathLogMultiPath (Log path state changes) [default = 0]: 1
If the parameter is changed by echoing the/procat thecommand line, this change will not persist through a reboot.
Although it is not recommended, if you prefer to use thecommand line, you may modify the /etc/rc.localfile so thatthe changes you have via echoing /procat the command line willbe run upon a reboot.
The /etc/rc.localfile is that script that contains your
initialization parameters and is run after all of the standardsystem init scripts. It is recommended that you append yourmodifications to the end of the file.
For example, see the end of the /etc/rc.localfile for theaddition of the /proc modifications:
[root@l82bi199 root]# more /etc/rc.local
#!/bin/sh## This script will be executed *after* all the other init scripts.# You can put your own initialization stuff in here if you don't# want to do the full Sys V style init stuff.
# BEGINNING_OF_VMWARE_RC_DOT_LOCALif ( uname -a | grep -q vmnix ); thenR="VMware ESX Server 2.5.0"else
R="Linux"fiarch=$(uname -m)a="a"case "_$arch" in _a*) a="an";; _i*) a="an";;esac
NUMPROC=`egrep -c "^cpu[0-9]+" /proc/stat`if [ "$NUMPROC" -gt "1" ]; then
Installation Information for VMware ESX Server v2 x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
27/102
2
Overview 2-11
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
SMP="$NUMPROC-processor " if [ "$NUMPROC" = "8" -o "$NUMPROC" = "11" ]; then
a="an" else a="a" fifi
# This will overwrite /etc/issue at every boot. So, make any changesyou
# want to make to /etc/issue here or you will lose them when you reboot.echo "" > /etc/issue
echo "$R" >> /etc/issueecho "Kernel $(uname -r) on $a $SMP$(uname -m)" >> /etc/issue
cp -f /etc/issue /etc/issue.netecho >> /etc/issue
#Echo the /proc filesystem to enable the#multipath logging. This will ensure that#the change persists through reboots.
echo 1 >> /proc/vmware/config/Scsi/LogMultiPath
# END_OF_VMWARE_RC_DOT_LOCAL
By default, the maximum number of LUNs scanned per disk array byESX v2.x is 8. Depending upon the EMC storage array configurationand the devices that are being allocated to the VMware host, it isrecommended that the following parameter be changed in the
VMkernel configuration to accommodate the LUNs allocated to thehost.
For example:
Disk.MaxLun == 128(The default is 8.)
If the Symmetrix or CLARiiON Storage Group has LUNs allocated tothe VMkernel addressable as greater than eight, then this parameter
must be increased. The DiskMaxLunparameter can be as large as 255.However, the maximum number of LUNs that can be managed by asingle VMware ESX Server is 128. The VMware ESX Server willsimply stop scanning the SCSI bus once it has seen 128 actual LUNs.
Please note that increasing the Disk.MaxLunparameter may slow down theSCSI scanning time.
In VMware ESX Server v2.1.x, the VMkernel is configured withsupport for sparse LUNs by default. The Disk.SupportSparseLun
Installation Information for VMware ESX Server v2 x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
28/102
2
2-12 EMC Host Connectivity Guide for VMware ESX Server v2.x
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
parameter allows the VMkernel to scan past non-existent LUNs, thatis, allows LUN skipping. When attaching to EMC storage arrays, it is
strongly recommended that the Disk.SupportSparseLunparameterbe set at the default of 1.
The following table lists the VMkernel parameters with theircorresponding descriptions and the VMware default and EMCrecommended values.
Table 2-1 VMkernel Parameters, Descriptions, and Default Values
VMkernel Parameter and DescriptionVMwareDefaultValue
EMCRecommended
Value
Cpu.BoundLagQuanta
Number of global quanta before bound lag [1-100]
8 8
Cpu.CellMigratePeriodMilliseconds between opportunities to migrate across cells 1000 1000
Cpu.ConsoleMinCpu
Min percentage of CPU 0 to dedicate to console [0-100]
8 8
Cpu.ConsoleOSWarpPeriod
Period in milliseconds [0-100]
20 20
Cpu.CreditAgePeriod
Period in milliseconds [500-10000]
3000 3000
Cpu.IdlePackageRebalancePeriod
Usec between chances to rebalance idle packages (0 to disable, 100000 max)
541 541
Cpu.MachineClearThreshold
Machine clears per million cycles to trigger quarantine
100 100
Cpu.MigratePenalty
Penalty in milliseconds [0-2000]
100 100
Cpu.MigratePeriod
Milliseconds between opportunities to migrate across CPUs
20 20
Cpu.PreemptPenalty
Penalty in milliseconds [0-2000]
10 10
Cpu.Quantum
Quantum in milliseconds [1-1000]
50 50
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
29/102
2
Overview 2-13
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
Cpu.RunnerMovePeriod
Milliseconds between opportunities to move currently-running vcpu
200 200
Cpu.SharesPerVcpuHigh
Shares per vcpu for high CPU priority [100-10000]
2000 2000
Cpu.SharesPerVcpuLow
Shares per vcpu for low CPU priority [100-10000]
500 500
Cpu.SharesPerVcpuNormal
Shares per vcpu for normal/default CPU priority [100-10000]
1000 1000
Cpu.SkewSampleThreshold
Number of skew samples allowed before co-deschedule (0 to disable skew checking)
3 3
Cpu.SkewSampleUsecInterval between vsmp skew tests
500 500
Disk.DelayOnBusy
Delay in milliseconds for completion of commands with a BUSY status
400 400
Disk.MaskLUNs
LUNs to mask from kernel. Format: ::
[empty] [empty]
Disk.MaxLUNMaximum number of LUNs per target that we scan for
8 128Default: 8
Disk.MaxResetLatency
Delay in milliseconds before logging warnings and spawning new reset worlds if a reset isoverdue or taking too long
2000 2000
Disk.MaxVCNotReadyTime
The number of seconds a VC path can report NOT READY status before being declared dead
60 60
Disk.PathEvalTime
The number of seconds between FC path evaluations
300 300
Disk.ResetLatency
Delay in milliseconds between reset thread wake-ups
1000 1000
Disk.ResetMaxRetries
Max number of bus reset retries (0=infinite)
0 0
Table 2-1 VMkernel Parameters, Descriptions, and Default Values (continued)
VMkernel Parameter and Description VMwareDefaultValue
EMCRecommendedValue
Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
30/102
2
2-14 EMC Host Connectivity Guide for VMware ESX Server v2.x
y y
Disk.ResetOnFailover
Issue a SCSI reset when failing over to an alternate HBA
0 0
Disk.ResetOverdueLogPeriod
Delay in seconds between logs of overdue reset
60 60
Disk.ResetPeriod
Delay in seconds between bus resets retries
30 30
Disk.ResetThreadExpires
Life in seconds of an inactive reset handle thread
1800 1800
Disk.ResetThreadMax
Max number of reset handler threads
16 16
Disk.ResetThreadMinMin number of reset handler threads
1 1
Disk.RetryUnitAttention
Retry all SCSI commands that return a unit attention error
1 1
Disk.SANDevicesWithAPFailover
SAN devices with Active/Passive Path Failover. Format: ::...
[empty] [empty]
Disk.SchedNumReqOutstanding
Number of outstanding commands to a target with competing worlds
16 16
Disk.SchedQControlSeqReqs
Number of consecutive requests from a VM required to raise the outstanding commands tomax
128 128
Disk.SchedQControlVMSwitches
Number of switches between commands issued by different VMs required to reduceoutstanding commands to CONFIG_DISK_CIF
6 6
Disk.SchedQuantum
Number of consecutive requests from one World
8 8
Disk.SectorMaxDiff
Distance in sectors at which disk BW sched affinity stops
2000 2000
Disk.SharesHigh
Shares for high disk priority [100-10000]
2000 2000
Table 2-1 VMkernel Parameters, Descriptions, and Default Values (continued)
VMkernel Parameter and Description VMwareDefaultValue
EMCRecommendedValue
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
31/102
2
Overview 2-15
Disk.SharesLow
Shares for low disk priority [100-10000]
500 500
Disk.SharesNormal
Shares for normal/default disk priority [100-10000]
1000 1000
Disk.SupportSparseLUNSupport for sparse LUNs if set to one
1 1
Disk.UseDeviceReset
Use device reset (instead of bus reset) to reset a SCSI device
1 1
Disk.UseLunReset
Use LUN reset (instead of device/bus reset) to reset a SCSI device
1 1
Disk.UseReportLUNUse the REPORT LUN command to speed up scanning for devices
1 1
FileSystem.LockRetries
Maximum number of retries when encountering file system lock
15 15
FileSystem.RemoteFSEnabled
Enable remote VMFS support
0 0
Irq.BestVcpuRouting 1 1
Irq.IRQMaxLoadPct
Maximum percentage of a CPU's resources that should be devoted to interrupts
70 70
Irq.IRQRebalancePeriod
Time in ms between attempts to rebalance interrupts
500 500
Irq.RoutingPolicy
Policy for interrupt routing, 1 for idle-routing, 2 for random, 0 to disable moving IRQs
1 1
Mem.AdmitHeapMin
Free heap space required to power on VM, in KB [256-10240]
1024 1024
Mem.AllocHighThreshold
Threshold (in MB) at which we start allocating memory above 4GB
768 768
Mem.BalancePeriod
Period in seconds [1-120], 0 to disable
15 15
Table 2-1 VMkernel Parameters, Descriptions, and Default Values (continued)
VMkernel Parameter and Description VMwareDefaultValue
EMCRecommendedValue
8/10/2019 VMWare + EMC storage.pdf
32/102
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
33/102
2
Overview 2-17
Mem.SwapFileOnSAN
Enables swap files on the SAN [0-1]
1 1
Mem.SwapIORetry
Retry timeout for swap I/O in seconds [1-2678000], 0 to disable timeout
0 0
Mem.SwapSharedSet the max ref count of a swappable shared page [0-100]
2 2
Migrate.Enabled
Enable hot migration support
0 0
Migrate.MemChksum
Checksum VM's memory while migrating
0 0
Migrate.MinReservationReserve a percentage of a CPU for use by migration helper worlds[0, 100]
30 30
Migrate.NetTimeout
Timeout for migration network operations
10 10
Migrate.PageInProgress
Time in seconds after which a pagein will be killed if there is no progress[5-1800]
15 15
Migrate.PageInTimeoutTime in seconds to wait for pagein to finish [10-1800]
180 180
Migrate.PreCopyLeftMB
Maximum modified memory left over after pre-copy is done
16 16
Migrate.PreCopyMinProgressMB
Minimum reduction in modified memory after a pre-copy iteration
16 16
Migrate.TSMasterPseudo-synchronize clocks for migration to/from this machine[0, 1]
0 0
Misc.BlueScreenTimeout
Timeout in seconds, 0 is no timeout
0 0
Misc.ConsoleShareXfer
Transfer shares to console on RPC calls
1 1
Misc.CosCorefile
VMFS to write the Service Console core file to [must be symbolic filesystem name]
[empty] [empty]
Table 2-1 VMkernel Parameters, Descriptions, and Default Values (continued)
VMkernel Parameter and Description VMwareDefaultValue
EMCRecommendedValue
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
34/102
2
2-18 EMC Host Connectivity Guide for VMware ESX Server v2.x
Misc.DebugMemEnable
Enable memory debugging, 0 to disable
0 0
Misc.EnableHighDMA
Enable DMA above 4GB
0 0
Misc.KVMapGuardUncachedUse guard pages around uncached kvmap mappings, [0-1]
1 1
Misc.LogToFile 1 1
Misc.LogToSerial 1 1
Misc.LogWldPrefix 0 0
Misc.MemAdmitMapEntriesMin
Free KVMap entries required to power on VM, [0-100]
30 30
Misc.MemMapEntriesLow
Point at which to start conserving KVMap entries, [0-1024]
200 200
Misc.MinimalPanic 0 0
Misc.ProcVerbose
Option unused
[empty] [empty]
Misc.PsodOnCosPanic
PSOD VMkernel on Console OS panic / oops [0-1]
0 0
Misc.SerialBaudRate 115200 115200
Misc.SerialPort 1 1
Misc.TimerHardPeriod
Hard timer interrupt period in microseconds
1000 1000
Misc.TimerMinGuestPeriod
Minimum period for guest timer callbacks in microseconds
100 100
Misc.VmkperfPerWorld
Should performance counters be maintained per-world [0-1]
0 0
Net.ClusterHaltCheck
1 to check for clustered tx/rx packets on halt
1 1
Table 2-1 VMkernel Parameters, Descriptions, and Default Values (continued)
VMkernel Parameter and Description VMwareDefaultValue
EMCRecommendedValue
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
35/102
2
Overview 2-19
Net.CopiesBeforeRemap
Copies before remapping, 0 to disable
10 10
Net.DisableWatchdog
Disable network watchdog timeout handler
0 0
Net.MaxMallocPacketsMaximum number of packets that can be allocated via the memory allocator
256 256
Net.MaxPrivateSKBs 256 256
Net.MaxRecvPackets 64 64
Net.NotifySwitch
Broadcasts an arp request on net handle enable [0 = disabled, 1 = enabled]
1 1
Net.PktsDeferXmitIntIfKeeping
Defer xmit completion interrupts if this many packets still pending
6 6
Net.PktsPerXmitInterrupt
Packets to transmit before raising completion interrupt
5 5
Net.ProcVerbose
Verbose procfs output for networking, 0 to disable
0 0
Net.RXClusterCPUSamplePeriod
CPU %idle sample period (msec) [1-10000]
1000 1000
Net.RXClusterDelayOff
Weighting factor for deactivating clustering
10 10
Net.RXClusterDelayOn
Weighting factor for activating clustering
2 2
Net.RXClusterDelayTDec
Decrement timer freq this often
1000 1000
Net.RXClusterDelayTInc
Increment timer freq this often
0 0
Net.RXClusterTMaxFreq
Max timer freq as a power of two [1-13]
10 10
Table 2-1 VMkernel Parameters, Descriptions, and Default Values (continued)
VMkernel Parameter and Description VMwareDefaultValue
EMCRecommendedValue
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
36/102
2
2-20 EMC Host Connectivity Guide for VMware ESX Server v2.x
Net.RXClusterTMigrateDelay
Timer CPU migration intervals [1-100]
5 5
Net.RXClusterTMigrateThresh
Timer CPU migration threshold (delta %idle)
20 20
Net.RXClusterTMinFreqMin time freq as a power of two [1-13]
6 6
Net.RXClusterThreshOff
Interrupts/sec to deactivate clustering
2000 2000
Net.RXClusterThreshOn
Interrupts/sec to activate clustering
4000 4000
Net.RXClusterTimerCPUWhich cpu to run the timer on, 0 for migratory timer
0 0
Net.RecvClusterDynAdjRate 50 50
Net.RecvClusterDynQueueMax 20 20
Net.RecvClusterOffCount 30 30
Net.RecvClusterOnCount 40 40
NetRecvClusterQueueMax 10 10
Net.RecvClusterTimeoutCount 1 1
Net.RecvClusterTimeoutMS 10 10
Net.RecvCopyLength 200 200
Net.SwitchFailoverBeaconEtherType
NIC Teaming switch failover beacon Ether Type [0x6000 - 0x9000]
36864 36864
Net.SwitchFailoverBeaconInterval
NIC Teaming switch failover beacon interval [1-60]
1 1
Net.SwitchFailoverThreshold
NIC Teaming switch failover threshold, a positive number to enable switch failover policy
2 2
Net.TCPIPLog
TCP/IP logging, 0 to disable
0 0
Table 2-1 VMkernel Parameters, Descriptions, and Default Values (continued)
VMkernel Parameter and DescriptionVMwareDefaultValue
EMCRecommended
Value
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
37/102
Overview 2-21
Net.TcpSegmentationOffload
Allow TSO, 0 to disable
1 1
Net.VlanTrunking
1 to enable 802.1Q VLAN Tagging by VMkernel
1 1
Net.XmitClusterOffCount 30 30
Net.XmitClusterOnCount 40 40
Net.XmitClusterQueueMax 10 10
Net.XmitClusterTimeoutCount 1 1
Net.XmitClusterTimeoutMS 10 10
Net.XmitInterruptIfStoppedRaise an interrupt if a xmit packet is returned and the guest has stopped transmitting
0 0
Net.XmitInterruptTimeout
Timeout in milliseconds to check for completed transmits
10 10
Net.XmitMinWinZeroCopyLen
For windows guests the minimum size of packets to try to avoid the copy
256 256
Net.ZeroSpeedLinkDownZero speed means link down, 1 to enable it
0 0
Numa.AutoMemAffinity
1 to set mem affinity automatically based on cpu affinity, 0 to disable
1 1
Numa.Debug
Level of NUMA scheduler debugging information to print [0-4]
0 0
Numa.MigImbalanceThresholdMinimum deviation in owed ms between nodes, per second, to trigger migration
75 75
Numa.PageMig
1 to permit NUMASched to manipulate page migration, 0 to disallow it
1 1
Numa.Rebalance
1 to use NUMASched rebalancer, 0 to disallow it
1 1
Numa.RebalancePeriodFrequency of NUMA node rebalancing, in milliseconds 2000 2000
Table 2-1 VMkernel Parameters, Descriptions, and Default Values (continued)
VMkernel Parameter and DescriptionVMwareDefaultValue
EMCRecommended
Value
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
38/102
2-22 EMC Host Connectivity Guide for VMware ESX Server v2.x
It is not necessary to reboot the VMware ESX Server in order for anychanges to the parameter values to take effect.
Scsi.ConflictRetries
Maximum number of retries when encountering reservation conflict
4 4
Scsi.LogAborts
Log timing of aborted or reset I/Os
0 0
Scsi.LogMultiPathLog path state changes
0 0
Scsi.PassthroughLocking 1 1
Table 2-1 VMkernel Parameters, Descriptions, and Default Values (continued)
VMkernel Parameter and DescriptionVMwareDefaultValue
EMCRecommended
Value
8/10/2019 VMWare + EMC storage.pdf
39/102
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
40/102
2-24 EMC Host Connectivity Guide for VMware ESX Server v2.x
The following VMnix kernel images are defined on your system:
Boot image configuration: vmnix Image file: /boot/vmlinuz-2.4.9-vmnix2 Memory: 384M Service Console devices: Ethernet controller: BROADCOM Corporation NetXtreme BCM5703 Gigabit Ethernet
(rev 02) RAID storage controller: Compaq Computer Corporation Smart Array 5i/532 (rev
01) (shared) VM devices: Ethernet controller: BROADCOM Corporation NetXtreme BCM5703 Gigabit Ethernet
(rev 02) Fiber storage controller: QLogic Corp QLA231x/2340 (rev 02) Ethernet controller: Intel Corporation 82557 [Ethernet Pro 100] (rev 08) RAID storage controller: Compaq Computer Corporation Smart Array 5i/532 (rev
01) (shared)
Type in the name of the boot image configuration you wish to configureor type "new" to create a new image [vmnix]:
Selected configuration was 'vmnix'.
Configuring your setup. Enter in the options that you want. Defaults have been provided.
Name of this configuration [vmnix]:
Now allocating memory to the Service Console. The memory that is not allocatedto the Service Console will be allocated to the VMs. Be sure to indicate anumber smaller than you actually have on your system or you may not be able to
run any VMs.
Your system is reporting approximately 2,559 MB of memory.Note: you cannot allocate more than 800 MB to the Service Console.
If you plan to run 4 virtual machines or fewer, 128 MB should suffice.Set this to 192 MB for 8 virtual machines, 272 MB for 16 virtual
machines, 384 MB for 32 virtual machines, or 512 MB for more than32 virtual machines.
Amount of memory (in megs) to allocate to the Service Console [384]:Enable hyperthreading? (Only works with 8 or less physical processors) [y]:
Now to divide the SCSI controllers and network adapters. In this process,you will select whether to give the previous devices to the Service Console orthe virtual machines that you will be running. In order to ensure optimalperformance for your virtual machines, we suggest you allocate as many devicesas possible to the virtual machines, leaving only the ones necessary toconfigure and maintain the Service Console.
Type 'c' to allocate to the Service Console or 'v' to allocate to the virtual
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
41/102
Driver Configuration in VMware ESX Server v2.x 2-25
machines, or 's' to share between both.
RAID storage controller Compaq Computer Corporation Smart Array 5i/532 (rev 01)running on bus 1 device
3 function 0 (cciss.o) [vmhba4] [s]:Ethernet controller BROADCOM Corporation NetXtreme BCM5703 Gigabit Ethernet (rev
02) running on bus 2device 1 function 0 (bcm5700.o) [] [c]:Ethernet controller BROADCOM Corporation NetXtreme BCM5703 Gigabit Ethernet (rev
02) running on bus 2device 2 function 0 (bcm5700.o) [vmnic0] [v]:Fiber storage controller QLogic Corp QLA231x/2340 (rev 02) running on bus 6 device
1 (qla2300_604.o) [vmhba2] [v]:Ethernet controller Intel Corporation 82557 [Ethernet Pro 100] (rev 08) running
on bus 6 device 2 function0 (e100.o) [vmnic1] [v]:
Configuration complete.Commit changes (y/n)? [y]:
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
42/102
2-26 EMC Host Connectivity Guide for VMware ESX Server v2.x
Rescanning the SCSI Bus
For use in the VMkernel itself, rather than using the standard Linux/procfile system to obtain information on the HBAs, refer to the/proc/vmware/scsidirectory. This directory contains entries foreach HBA in the system allocated to the Virtual Machines. Each HBAis denoted byvmhbaN(where Nindicates the file for each adapterowned by the Virtual Machines).
You may confirm that the QLogic or Emulex driver is loaded in the
VMkernel with the vmkload_mod command. For example, thefollowing command may be issued on this host to list the QLogicdriver:
root@l82bi199 /]# vmkload_mod -lName R/O Addr Length R/W Addr Length ID Loadedvmklinux 0x4dd000 0xf000 0x12414a0 0x53000 1 Yesnfshaper 0x4ec000 0x1000 0x1297f28 0x1000 2 Yesqla2300_604 0x4ed000 0x19000 0x1298f30 0x22000 3 Yes
bond 0x506000 0x2000 0x150e1d8 0x2000 4 Yes [Assuming that the devices have been allocated to the VMware hostand that the VMkernel has been configured appropriately, theStorage Managementdialog in the MUI reports the devices that areallocated to the host.
You may also view the device listing at the command line. This maybe done with the following commands:
1.more /proc/scsi/qla2300/N
where Nis the sequential value of each QLogic HBA installed inthe system, beginning with the number after the last host adapternumber entry in the file. (The first host adapter number entrybegins with zero.) This is the same as it would be for standardLinux.
2.ls la /proc/vmware/scsi/vmhbaN
where Nindicates the file for each adapter owned by theVMkernel and, subsequently, the Virtual Machines
After the host is added to the Storage Group, the SCSI bus needs to berescanned.
8/10/2019 VMWare + EMC storage.pdf
43/102
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
44/102
2-28 EMC Host Connectivity Guide for VMware ESX Server v2.x
For example:
vmkfstools -s vmhba0
If you are using the MUI, then go to the Storage Managementdialog and click on the Rescan SANoption to rescan the SCSIbus.
5. For either QLogic or Emulex HBAs, you may also remove andreinsert the driver from the VMkernel to initiate a rescan of theSCSI bus.
For QLogic:vmkload_mod lvmkload_mod u qla2300_604vmkload_mod /usr/lib/vmware/vmkmod/qla2300_604.o vmhba
For Emulex:
vmkload_mod lvmkload_mod u lpfcddvmkload_mod /usr/lib/vmware/vmkmod/lpfcdd.o vmhba
It is recommended in the case of using either the QLogic or Emulex HBAsthat the Rescan SANoption in the MUI be used to rescan the SCSI bus.
For VMware ESX Server v2.5:
1. It is recommended that the MUI be used to rescan the SCSI bus.Simply go to the Storage Managementdialog, and click on theRescan SANoption to rescan the SCSI bus. This process calls thecos-rescan.sh.
2. If the command line is preferred, you may use thecos-rescan.shscript to initiate the rescan.
For example:
[root@l82bi199 /]# cos-rescan.sh vmhba2
Rescanning vmhba2...done.On scsi1, removing:On scsi1, adding:
3. You may reboot the host, but since this is disruptive, it is notrecommended.
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
45/102
Rescanning the SCSI Bus 2-29
4. If booting from an internal hard drive rather than from the SAN,you may also remove and reinsert the driver from the VMkernel
to initiate a rescan of the SCSI bus. This is pertinent for bothQLogic and Emulex HBAs.
For QLogic:
vmkload_mod lvmkload_mod u qla2300_604vmkload_mod /usr/lib/vmware/vmkmod/qla2300_604.o vmhba
For Emulex:
vmkload_mod lvmkload_mod u lpfcddvmkload_mod /usr/lib/vmware/vmkmod/lpfcdd.o vmhba
In VMware ESX Server v2.5, the vmkfstoolscommand should not beused to rescan the SCSI bus; otherwise, the Service Console will notsee or report the changes made.
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
46/102
2-30 EMC Host Connectivity Guide for VMware ESX Server v2.x
Persistent Binding
EMC highly recommends that persistent binding be enabled inVMware ESX Server v2.x. When persistent binding is enabled, thetarget IDs are assigned to specific storage array devices. Thisassignment persists through reboots so the association remains thesame. This is particularly helpful when using raw devices.
Persistent binding may be enabled or disabled using the MUI or thecommand line.
If using the MUI, select the Optionstab, and then select the StorageManagementtab. Under Storage Management, there are threeoptions: Disks and LUNs, Failover Paths, and Adapter Bindings.To view the current information for your system, select the AdapterBindingsoption.
An example of the output from the Adapter Bindingsdialog isbelow:
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
T i d h i bi di f h d li
8/10/2019 VMWare + EMC storage.pdf
47/102
Persistent Binding 2-31
To view and reset the persistent binding from the command line,follow the steps below.
1. Log in as root.
2. To view the current persistent binding entries for the HBAs in thesystem, look at the pbindingsfile.
An example of what the pbindingsfile may look like is asfollows:
[root@l82bi199 /]# more /etc/vmware/pbindings6.1.1.qla2300_607 = "scsi-qla1-tgt-0-di-0-port=50060160082012bb;scsi-qla1-tgt-1-di-0-port=50060168082012bb;scsi-qla1-tgt-2-di-0-port=5006016010601270;scsi-qla1-tgt-3-di-0-port=5006016810601270;"6.1.0.qla2300_607 = "scsi-qla0-tgt-0-di-0-port=50060168082012bb;scsi-qla0-tgt-1-di-0-port=50060160082012bb;scsi-qla0-tgt-2-di-0-port=5006016010601270;scsi-qla0-tgt-3-di-0-port=5006016810601270;"
3. To remove the existing persistent binding settings, either thepbindingsfile may be deleted or the pbind.plscript may be run.
Thepbind.pl
is located in the/usr/sbin
directory and may berun as root from any directory location.
To delete the file, use the following command:
rm f /etc/vmware/pbindings
To remove the settings using the pbind.plscript, use the Dswitch. This switch deletes the persistent binding information forall of the adapters within the system.
pbind.pl -D
For example:
[root@l82bi199 /]# pbind.pl -DDeleting ALL bindings
4. Reboot the host.
5. After the system reboots successfully, log in and again run thepbind.plscript. This time use the A switch which will add thepersistent binding information for all of the adapters within thesystem.
pbind.pl A
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
F l
8/10/2019 VMWare + EMC storage.pdf
48/102
2-32 EMC Host Connectivity Guide for VMware ESX Server v2.x
For example:
[root@l82bi199 /]# pbind.pl -A
+-----------------------------------------+| Setting up the following bindings... |+--------+---------+----------------------+| Host | Target | World Wide Name || 01 | 000 | 50060160082012bb || 01 | 001 | 50060168082012bb || 01 | 002 | 5006016010601270 || 01 | 003 | 5006016810601270 || 00 | 000 | 50060168082012bb || 00 | 001 | 50060160082012bb || 00 | 002 | 5006016010601270 || 00 | 003 | 5006016810601270 |+-----------------------------------------+
6. Edit the vmwareinitialization script. This script is run when thesystem is booting up and starts and manages the services neededby the VMkernel.
vi /etc/init.d/vmware
7. Search for the line containing the textsave_san_persistent_bindings. The code should be locatedapproximately around line 896 in the file.
896 # Save the SAN persistent bindings for the next boot897 save_san_persistent_bindings
8. Comment out the line save_san_persistent_bindings.
This is only a temporary change. When this line is commentedout, the adapter to target bindings will not be saved andtherefore, will not persist through reboots.
896 # Save the SAN persistent bindings for the next boot897 #save_san_persistent_bindings
9. Save the file and reboot the host again.
10. Once the system is rebooted, the persistent binding line will needto be uncommented.
Change to the /etc/init.d/vmwaredirectory again:
vi /etc/init.d/vmware
Uncomment the line for save_san_persistent_bindingsandsave the file.
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
11 Now run wwpn pl to associate the targets HBA WWPN and
8/10/2019 VMWare + EMC storage.pdf
49/102
Persistent Binding 2-33
11. Now runwwpn.plto associate the targets, HBA WWPN, andvmhba.
[root@l82bi199 /]# wwpn.pl -vWWPN 1.02 Copyright VMware 2003Display WW port names and VMHBA information for fibre channel adaptersFor each vmhba here are the corresponding Qlogic and Emulex WW Port NamesAdapter WWPN PCI (decimal)vmhba1: 210000e08b0910a7 (Qlogic) 6:1:0 /proc/scsi/qla2300/0vmhba1:0: 50060168082012bb scsi-qla0-port-0=0000000000000000:50060168082012bb;vmhba1:1: 50060160082012bb scsi-qla0-port-1=0000000000000000:50060160082012bb;vmhba1:2: 5006016010601270 scsi-qla0-port-2=0000000000000000:5006016010601270;
vmhba1:3: 5006016810601270 scsi-qla0-port-3=0000000000000000:5006016810601270;vmhba2: 210100e08b2910a7 (Qlogic) 6:1:1 /proc/scsi/qla2300/1vmhba2:0: 50060160082012bb scsi-qla1-port-0=0000000000000000:50060160082012bb;vmhba2:1: 50060168082012bb scsi-qla1-port-1=0000000000000000:50060168082012bb;vmhba2:2: 5006016010601270 scsi-qla1-port-2=0000000000000000:5006016010601270;vmhba2:3: 5006016810601270 scsi-qla1-port-3=0000000000000000:5006016810601270;
12. The persistent binding is now reset to the most recentinformation.
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
50/102
2-34 EMC Host Connectivity Guide for VMware ESX Server v2.x
LUN Spanning
When a volume is spanned, the volume is extended to includemultiple VMFS-2 disks or partitions. A physical extent is defined bythe VMFS-2 disks or partitions that compose a logical spannedVMFS-2 volume.
There are some restrictions to using spanned volumes. They are:
Spanning volumes is supported only with VMFS-2 volumes, notwith VMFS-1 volumes.
The maximum number of files may not be changed.
A spanned volume may not be removed.
Data on the physical extents are lost when they are added to theVMFS-2 spanned volume. Therefore, it is strongly recommendedthat only newly created partitions or newly added devices areused for spanning volumes.
A VMFS-2 volume may span up to a maximum of 32 LUNs.
To span a volume using the MUI, select the Optionstab then selectStorage Management. Select the Disks and LUNscategory. In theDisks and LUNscategory, you are provided with a list of the devicesthat are assigned to the host. Select the one that you want to expand.The configuration window for the device you chose will be shown onyour screen and will provide you with the specifics on the selected
device plus the devices that may be added to the selected device tocreate a spanned VMFS-2 volume.
See the figure that follows for an example using the devicevmhba1:2:13.
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
51/102
LUN Spanning 2-35
After you have spanned a VMFS-2 volume, you may view thechanges immediately because the /vmfsdirectory is updateddynamically. The command vmkfstoolsshould be used to list theinformation on the particular device and to provide the spanning andphysical extent information.
For example:
[root@l82bi199 /]# cd /vmfs[root@l82bi199 /]# vmhba1:0:0:1 vmhba1:2:11:1 vmhba1:2:13:1 vmhba1:2:3:1[root@l82bi199 /]# vmkfstools --list vmhba1:2:13:1
Name: 1270data13 (public)Capacity 2826683392 (2679 file blocks * 1048576), 1701838848 (1623 blocks) avail
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
Permission Uid Gid Attr Bytes (Blocks) Last Modified Filename
8/10/2019 VMWare + EMC storage.pdf
52/102
2-36 EMC Host Connectivity Guide for VMware ESX Server v2.x
rw------- 0 0 disk 1107296256 ( 1056) Aug 26 06:28 w2k3_data4.dsk
[root@l82bi199 /]# vmkfstools -P vmhba1:2:13:1vmhba1:2:13:1 is a VMFS-2.11 volume spanning 4 physical extents.
Volume label (if any): 1270data13UUID (if any): 409bb858-be8577b8-ccf1-0002b3248a69Physical Extents: vmhba1:2:13:1 vmhba1:2:5:1 vmhba1:2:6:1 vmhba1:2:7:1
To view the same volume information from the MUI, go to theOptionstab and select Storage Management. Select the Disks andLUNscategory and scroll to view the VMFS-2 volume that wasspanned. Please see the reference to Disk vmhba1:2:13in the figurethat follows for an example.
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
S tti U d I t lli Vi t l M hi
8/10/2019 VMWare + EMC storage.pdf
53/102
Setting Up and Installing Virtual Machines 2-37
Setting Up and Installing Virtual Machines
The devices to be used for the Virtual Machines with VMFS areformatted in the Storage Management > Disks and LUNs dialog.
When the Create Volumesoption is selected, you are prompted toselect the method with which the devices will be created.
The Typicaloption formats the entire device.
The Customoption allows you to manually create one or more
partitions according to the size required.Select the Typicalformatting option, as shown in Figure 2-5. If yoursystem has not been configured with a Core Dump Partition, thenyou will be prompted to create one.
Figure 2-5 Storage Management > Disks and LUNs Tab
When booting from an internal hard drive, it is recommended thatthe core dump partition be created on the hard disk internal to thesystem, not a SAN attached device. If booting from a SAN device,
then the core dump partition must be created on a SAN device aswell.
An example of the dialog providing this prompt is in Figure 2-6 onpage 2-38.
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
54/102
2-38 EMC Host Connectivity Guide for VMware ESX Server v2.x
Figure 2-6 Storage Management > Disks and LUNs > Core Dump Partition
Select Yes,and then select Finish.
After the partition has been created, it should be formatted withVMFS. You also can provide it with a Volume Label, or alias, that islisted as the VMFS Name.
For VMFS file systems, you can specify whether the file system isPublic(accessible by multiple physical servers) or Shared(accessibleby multiple physical servers, and can be used simultaneously byvirtual machines on these servers).
The accessibility should be set to the default which is Public in almostall cases. The accessibility should be set to Shared if the VMFS willhold disks to be used for MSCS clustering.
An example of the accessibility setting window is shown inFigure 2-7.
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
55/102
Setting Up and Installing Virtual Machines 2-39
Figure 2-7 Access Mode
After the devices are formatted with the VMFS file system, the VirtualMachines can be configured and installed.
In the main screen of the VMware ESX Server, go to the StatusMonitortab and select Add Virtual Machine. The Wizard will guideyou through the Virtual Machine configuration.
An example of the Virtual Machine Configurationdialog for aWindows 2000 virtual machine is in Figure 2-8 on page 2-40.
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
8/10/2019 VMWare + EMC storage.pdf
56/102
2-40 EMC Host Connectivity Guide for VMware ESX Server v2.x
Figure 2-8 Add Virtual Machine Dialog
In this dialog, you can allocate a name for the virtual machine and alocation for the configuration file.
The name allocated in the Virtual Machinesdialog will be displayedin the Status Monitordialog as shown in Figure 2-9as well as listedin the Remote Console listing:
Figure 2-9 Status Monitor > Virtual Machines
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
After the Virtual Machines are configured, the OS must be installedon that Virtual Machine To access a virtual machine the VMware
8/10/2019 VMWare + EMC storage.pdf
57/102
Setting Up and Installing Virtual Machines 2-41
on that Virtual Machine. To access a virtual machine, the VMwareRemote Console software needs to be installed on either a Windowsor Linux machine.
The Remote Console software is available to download from theStatus Monitordialog, as shown in Figure 2-9 on page 2-40, whichprovides a listing of the Virtual Machines configured on the VMwareServer. The Remote Console software is available as executable forWindows NT/Windows 2000 and as an RPM and a tar archive forLinux. Select and download the appropriate application for your
operating system.
After you have installed the VMware Remote Console software onyour operating system, start the application and log onto theVMware ESX Server.
Figure 2-10is an example of what the dialog will look like.
Figure 2-10 VMware Remote Console > Connect to Virtual Machines
Select a Virtual Machine, and select the Power Onbutton. This willstart up the virtual machine.
8/10/2019 VMWare + EMC storage.pdf
58/102
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
The figure below is an example of what the console will look likewhen using the Editoption.
8/10/2019 VMWare + EMC storage.pdf
59/102
Setting Up and Installing Virtual Machines 2-43
g p
Select the option to Use physical drive,and connect the CD-ROMdevice if an actual CD is to be used for the virtual machine OSinstallation.
If an ISO image is to be used for the OS installation on the virtualmachine, select Use ISO image, and browse to the location of the ISOimage. In Figure 2-11, an ISO image is selected for installation.
Figure 2-11 Configuration Editor, Hardware, Connection
After the installation has completed, the Virtual Machine must berebooted to be ready for use.
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
The following is an example of the Virtual Machine Overviewdialogwith multiple Virtual Machines installed:
8/10/2019 VMWare + EMC storage.pdf
60/102
2-44 EMC Host Connectivity Guide for VMware ESX Server v2.x
p
2Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays
Additional Notes
8/10/2019 VMWare + EMC storage.pdf
61/102
Additional Notes 2-45
Currently, EMC software will function on neither the ServiceConsole nor the Virtual Machines as the currently-releasedversions of the CLARiiON-based software applications do notinclude support for VMware ESX Server.
Currently, VMware ESX Server v2.x does not support accessingSnapView, MirrorView, or SAN Copy LUNs.
Again, please note that PowerPathis not available for the
VMkernel. The failover supported with VMware ESX Server v2.xand EMC Storage Arrays is native to the VMware ESX Serverv2.x itself. The native failover functionality performs failoveronly, not load balancing.
The VMware ESX Server host needs to be registered manually onthe CLARiiON array as the Naviagent is not supported forVMware ESX Server.
Neither the Naviagent nor the NaviCLI are supported forVMware ESX Server at this time.
It is recommended that Virtual Machines running on VMwareESX Server v2.x boot from the array.
For the Virtual Machines, it is recommended, but not required,that the boot LUNs be in a CLARiiON RAID group that isseparate from that RAID group for the data volumes.
Prior to VMware ESX Server v2.5, the VMware ESX Server itselfwas not supported booting from the array. VMware ESX v2.5adds support for booting from the SAN for the ESX Server itself.
When booting VMware ESX Server v2.5.0 from an EMC storagearray, it is recommended that the boot LUN be in a RAID groupseparate from the data volumes.
When booting the VMware ESX Server from an internal drive, the
VMware Core Dump and Swap Files should be created oninternal drives rather than SAN attached devices.
It is highly recommended that the vmware-tools be installed oneach Virtual Machine. The use of the vmware-tools will save onresources and provide better memory management. Thevmwaretools will also provide a VGA device driver and aheartbeat mechanism to communicate with the VMkernel.
8/10/2019 VMWare + EMC storage.pdf
62/102
3
VMware ESX Server v2.x andSymmetrix Storage Array
8/10/2019 VMWare + EMC storage.pdf
63/102
VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment 3-1
This chapter lists Symmetrix Fibre Channel support informationspecific to the VMware ESX Server environment.
VMware/Symmetrix Environment.................................................3-2
Booting the VMware ESX Server v2.5 from the Symmetrix ........3-6 Fabric Zoning......................................................................................3-8 Required Storage System Configuration ........................................3-8 Useful VMware ESX Server Utilities and Functions.....................3-9 Addressing Symmetrix Devices.....................................................3-10
Symmetrix Storage ArrayFibre Channel Environment
3VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment
VMware/Symmetrix Environment
8/10/2019 VMWare + EMC storage.pdf
64/102
3-2 EMC Host Connectivity Guide for VMware ESX Server v2.x
This section lists some Symmetrix Fibre Channel support informationspecific to the VMware ESX Server environment.
Host and Hardware Connectivity
Refer to the EMC Support Matrix or contact your EMC representativefor the latest information on qualified hosts, host bus adapters, and
connectivity equipment.
EMC does not support mixing HBAs from different vendors in the same host.
Symmetrix Connectivity
The Symmetrix system is configured by an EMC Customer Engineer
via the Symmetrix service processor.
The EMC Customer Engineer (CE) should contact the EMCConfiguration Specialist for updated online information. Thisinformation is necessary to configure the Symmetrix system tosupport the customers host environment.
When attaching a VMware ESX Server host to a Symmetrix, the Linuxdefault FA settings should be used. They are as follows:
SCP3 SCSI-3
PTOP: Point to Point
VCM: Volume Logix
C: Common Serial Number
EAN: Enable Auto Negotiation
Volume Logix must be enabled on the Symmetrix.After the EMC CE has assigned target IDs and LUNs and brought theSymmetrix channel and disk directors online, reboot the networkoperating systems, and go into the configuration program.
All qualified HBAs are listed in the EMC Support Matrix.
3VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment
Note that the VMware ESX Server installer will recognize LUNs25MB or less as management LUNs. This includes any gatekeepersassigned to the VMware host via Solutions Enabler
8/10/2019 VMWare + EMC storage.pdf
65/102
VMware/Symmetrix Environment 3-3
assigned to the VMware host via Solutions Enabler.
It is recommended that each VMware ESX Server host have amaximum of 8 paths to Symmetrix Storage Arrays. A path is definedby one HBA port that is zoned to one SP port.
With a recommended maximum of 8 paths from a VMware ESXServer to Symmetrix Storage Arrays, it is recommended that eachVMware ESX Server host be zoned to a maximum of 4 storage arrays.
A couple of possible configuration scenarios are described in Example1on page 3-4and Example 2on page 3-5.
3VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment
Example 1 In this example as shown in Figure 3-1, one host with two HBAs isattached to one Symmetrix array using two separate switches. TwoFA ports on each of two FAs within the array are being used. HBA1 is
8/10/2019 VMWare + EMC storage.pdf
66/102
3-4 EMC Host Connectivity Guide for VMware ESX Server v2.x
po s o e c o o s e y e e g se szoned to 15D0 and to 16C1. HBA2 is zoned to 15D1 and to 16C0. Thezones should be composed of a single initiator and a single target sothey would be created with one HBA and on FA port. Two HBAswith two paths each to the array totals four paths and if using singleinitiator/single target zoning, there are 4 zones.
In this particular example, two switches are used. Using only oneswitch is supported, but such a configuration would lack
redundancy. Preferably, a minimum of two switches should be usedto add another level of redundancy. Alternatively, for additionalredundancy, two separate fabrics could be utilized.
Figure 3-1 One Host, Two Switches, and One Symmetrix Array
MirroredFA 15D Symm 1
d0 d1 c0 c1
Server
H
B
A
0
H
B
A
1
MirroredFA 16C Symm 1
d0 d1 c0 c1
3VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment
Example 2 In this example as shown in Figure 3-2, one host with two HBAs isattached using a two-switch fabric to four Symmetrx arrays FA ports.In this configuration, the zones are created with one HBA and one FA
8/10/2019 VMWare + EMC storage.pdf
67/102
VMware/Symmetrix Environment 3-5
g
port. That is,
HBA1 is zoned to one 15D0 port on each of the four arrays.
HBA2 is zoned to one 16C1 port on each of the four arrays.
Figure 3-2 One Host, Two Switches, and Four Symmetrix Arrays
All qualified HBAs are listed in the EMC Support Matrix.
When assigning Symmetrix LUNs to a VMware ESX Server host, theLUNs should be assigned to the host across both FAs since theSymmetrix is an active/active array.
M i rr o r edF A 1 5 D S y m m 4
d 0 d 1 c 0 c 1
S e r v e r
H
B
A
0
H
B
A
1
M i r ro r e dF A 1 5 D S y m m 3
d 0 d 1 c 0 c 1
M i r ro r e dF A 1 5 D S y m m 1
d 0 d 1 c 0 c 1M i r ro r e d
F A 1 5 D S y m m 2
d 0 d 1 c 0 c 1
M i r ro r e dF A 1 6 C S y m m 2
d 0 d 1 c 0 c 1
M i r ro r e dF A 1 6 C S y m m 1
d 0 d 1 c 0 c 1
M i r ro r e dF A 1 6 C S y m m 3
d 0 d 1 c 0 c 1
M i r ro r e dF A 1 6 C S y m m 4
d 0 d 1 c 0 c 1
3VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment
Booting the VMware ESX Server v2.5 from the Symmetrix
8/10/2019 VMWare + EMC storage.pdf
68/102
3-6 EMC Host Connectivity Guide for VMware ESX Server v2.x
VMware ESX Server v2.5 hosts have been qualified for booting fromSymmetrix devices interfaced through Fibre Channel as specified inthe EMC Support Matrix.
EMC does not recommend booting VMware ESX Server from theSymmetrix system unless the host is using VMwares native failoverfunctionality.
Note that if VMwares native failover functionality is not used and it isnecessary to use a Symmetrix device as a boot disk, you should shut downthe Virtual Machines and the ESX Server during any maintenance proceduresthat might cause the boot disk to become unavailable to the host.
The Symmetrix device that is to contain the Master Boot Record(MBR) for the host must have a lower logical unit number (LUN)than any other device visible to the host. This device must be
mapped as /dev/sdaby the Linux operating system for the boot tosucceed from the device.
Note that this includes the case of the Symmetrix Volume Logixdatabase device. The Volume Logix device is write-protected so theinstaller will fail to write the MBR to this device.
To force the installer to avoid an attempt to write to this device,EMC recommends masking the LUN. The administrative host
must first initialize the Volume Logix database. The LUN maythen be masked by modifying the active configuration file toenable the fba flag2to restrict access to the Volume Logixdatabase.
An alternative to masking the VCM DB, is to map the VolumeLogix database device so that it is the highest LUN presented tothe host However, please be aware that the LUN number shouldnot be higher than 254 (FE) if it is to be used by Solutions Enabler
or Control Center running on a Microsoft host.
The EMC-recommended method is to use LUN masking.
When attaching the VMware host to the Symmetrix storage array, usethe adapter in the lowest-numbered PCI slot in the server.
To keep the configuration and installation simple, it is recommended
that only that HBA be cabled to the array. Ensure that the boot BIOS
3VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment
or firmware has been applied to your HBA and that the boot BIOShas been enabled on the HBA to be used for boot.
8/10/2019 VMWare + EMC storage.pdf
69/102
Booting the VMware ESX Server v2.5 from the Symmetrix 3-7
Prior to the installation on a Symmetrix LUN, the Linux host HBAmust have successfully logged into the array. Using SolutionsEnabler from another host, at least one LUN must be assigned to thehost.
During the installation procedure, it is recommended, but notrequired, that only one LUN be allocated to the host for ease of use.After the installation has completed, additional LUNs can beassigned to the host.
When performing the installation, you will need to manually typebootfromsanor bootfromsan-textat the installation prompt.
By selecting this option, the HBAs will automatically be set to beshared between the Service Console and the Virtual Machines.Remember that this parameter should not be changed when creatingyour VMware ESX Server configuration.
Cautions
CAUTION
If VMware loses all paths to the array for a long enough period, thedisks disappear from the system. A hard reboot is required tobring the system back to a usable state.
Any of these events could crash a system booting from a Symmetrixstorage array:
Lost connection to the Symmetrix system (pulled or damagedcable connection).
Symmetrix service and upgrade procedures, such as on-lineSymmetrix microcode upgrades and/or configuration changes.
Symmetrix director failures, including failed lasers.
Symmetrix system power failure.
Storage area network service/upgrade procedures such asfirmware upgrades or hardware replacements.
3VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment
Restrictions Emulex HBAs are not supported when booting from the fabric;only QLogic HBAs are supported.
R di k i i d h b i f h
8/10/2019 VMWare + EMC storage.pdf
70/102
3-8 EMC Host Connectivity Guide for VMware ESX Server v2.x
Raw disk mapping is not supported when booting from thefabric.
Only FC-SW environments are supported when booting from theSAN.
Fabric Zoning
When using VMware ESX Server hosts in a fabric environment, thezoning must be set up as single initiator and single target zoning. Asingle initiator/single target zone is composed of one HBA and oneSymmetrix storage array port.
Multi-initiator zones are not supported in a VMware ESX Server fabricenvironment.
Required Storage System Configuration
Symmetrix system configuration is performed by an EMC CustomerEngineer through the Symmetrix service processor.
Using Volume Logix to mask LUNs is highly recommended.
To verify that the VMware ESX Server host can see all of theSymmetrix target devices, configure the host as described in theremainder of the chapter.
3VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment
Useful VMware ESX Server Utilities and Functions
8/10/2019 VMWare + EMC storage.pdf
71/102
Useful VMware ESX Server Utilities and Functions 3-9
Utility/Function Description
fdisk Command used to create and manipulate partition tables.
grep Command used to search through a file or files to find specific text.
vmkpcidivy Command used to create a boot configuration for the VMkernel.
vmkfstools Command used to create and manipulate files on LUNs owned by the VMware ESX Server host.
vmkload_mod Command used to view, load, remove driver modules in the VMkernel.
vmkmultipath Command used to display and set the configuration on a particular device and the paths attachedto that device.
vm-support Command used to gather information about the VMware ESX Server itself and virtual machines toassist in debugging issues or to obtain performance information for the virtual machines.
vmkdump Command used to manage the VMkernels dump partition.
3VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment
Addressing Symmetrix Devices
8/10/2019 VMWare + EMC storage.pdf
72/102
3-10 EMC Host Connectivity Guide for VMware ESX Server v2.x
Fabric Addressing
Each port on a device attached to a fabric is assigned a unique 64-bitidentifier called a World Wide Port Name (WWPN). These names arefactory-set on the HBAs in the hosts, and are generated on the FibreChannel directors in the Symmetrix system.
For comparison to Ethernet terminology, an HBA is analogous to a NIC card,and a WWPN to a MAC address.
The ANSI standard also defines a World Wide Node Name (WWNN), butthis name has not been consistently defined by the industry.
When an N_Port (host server or storage device) connects to the fabric,
a login process occurs between the N_Port and the F_Port on thefabric switch. During this process, the devices agree on suchoperating parameters as class of service, flow control rules, and fabricaddressing. The N_Ports fabric address is assigned by the switchand sent to the N_Port. This value becomes the source ID (SID) onthe N_Port's outbound frames and the destination ID (DID) on theN_Port's inbound frames.
The physical address is a pair of numbers that identify the switch andport, in the format s,p, where s is a domain ID and p is a valueassociated to a physical port in the domain. The physical address ofthe N_Port can change when a link is moved from one switch port toanother switch port. The WWPN of the N_Port, however, does notchange. A Name Server in the switch maintains a table of alllogged-in devices, so an N_Port may adjust automatically to changesin the fabric address by keying off the WWPN.
The highest level of login that occurs is the process login. This is usedto establish connectivity between the upper-level protocols on thenodes. An example is the login process that occurs at the SCSI FCPlevel between the HBA and the Symmetrix system.
3VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment
SCSI-3 FCP Addressing
The Symmetrix director extracts the SCSI Command DescriptorBlocks (CDB) from the frames received through the Fibre Channel
8/10/2019 VMWare + EMC storage.pdf
73/102
Addressing Symmetrix Devices 3-11
Blocks (CDB) from the frames received through the Fibre Channellink. Standard SCSI-3 protocol is used to determine the addressingmode and to address specific devices.
The Symmetrix supports three addressing methods based on asingle-layer hierarchy as defined by the SCSI-3 Controller Commands(SCC):
Peripheral Device Addressing Logical Unit Addressing
Volume Set Addressing
All three methods use the first two bytes (0 and 1) of the 8-byte LUNaddressing structure. The remaining six bytes are set to 0s.
For Logical Unit and Volume Set addressing, the Symmetrix portidentifies itself as an Array Controller in response to a hosts Inquirycommand sent to LUN 00. This identification is done by returningthe byte 0x0C in the Peripheral Device Type field of the returneddata for Inquiry. If the Symmetrix system returns the byte 0x00 in thefirst byte of the returned data for Inquiry, the Symmetrix system isidentified as a direct access device.
Upon identifying the Symmetrix system as an array controller device,the host should issue a SCSI-3 Report LUNS command (0xA0), in
order to discover the LUNs.
3VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment
The three addressing modes, contrasted in Table 3-1, SymmetrixSCSI-3 Addressing Modes, differ in the addressing schema (Target ID,LUN, and Virtual Bus) and number of addressabledevices.
8/10/2019 VMWare + EMC storage.pdf
74/102
3-12 EMC Host Connectivity Guide for VMware ESX Server v2.x
The addressing modes are provided to allow flexibility in interfacing withvarious hosts. In all three cases the received address is converted to theinternal Symmetrix addressing structure. Volume Set addressing is thedefault for Symmetrix systems. Select the addressing mode that isappropriate to your host.
Table 3-1 Symmetrix SCSI-3 Addressing Modes
Addressing Mode CodeaABit
VBit
Response toInquiry
LUNDiscovery
Method
PossibleAddresses
MaximumLogical