hpux-scsi

8/6/2019 hpux-scsi

http://slidepdf.com/reader/full/hpux-scsi 1/8

Page 1

HP-UX

Technical

Maximum LUNconfiguration andconsiderations for

HP-UX

October 2005

U.S.A.

©2005 Hewlett-Packard Company

v 11.23

A Whitepaper onMaximum LUN supporton HP-UX

8/6/2019 hpux-scsi


ii

Table of Contents1. Abstract ...................................................................................................................................................... 1 2. Definitions.................................................................................................................................................... 1 3. Supported LUN Maximums on HP-UX ..................................................................................................... 2

3.1 11i v1 Maximums.................................................................................................................................. 3 3.2 11i v2 Maximums.................................................................................................................................. 3

4. Device Maximums..................................................................................................................................... 3 5. Considerations ........................................................................................................................................... 4

5.1 Server considerations .......................................................................................................................... 4 5.2 Kernel Tunables .................................................................................................................................... 4 5.3 Volume Manager considerations ..................................................................................................... 4 5.4 File System considerations .................................................................................................................. 4 5.5 Scan/Startup/Shutdown times........................................................................................................... 4 5.6 Topology considerations .................................................................................................................... 5

6. Roadmap / Futures ................................................................................................................................... 5

7. More Information ....................................................................................................................................... 5

8/6/2019 hpux-scsi


Maximum LUN Configuration and Considerations for HP-UX

Page 1

1. Abstract

Mass storage requirements have increased dramatically over the last few years. High-end HP-UXcustomers now require many terabytes of storage and thousands of storage devices (LUNs) to beaccessible from a single HP-UX server. This paper highlights the increases in the maximum number of LUNs supported with HP-UX servers and at component levels such as per-HBA (Host Bus Adapter) 1.The nature of these maximums will be defined, along with recommendations or restrictions that applywhen operating at or near these maximums, and expectations for the future and what to do if you need toexceed existing supported levels.

2. Definitions

The primary mass storage device protocol used in HP-UX and in the computer industry in general is theSCSI device protocol, which is used across parallel SCSI, Fibre Channel, and iSCSI in HP-UX today, andwill be used across other links such as SAS in the near future. The SCSI protocol defines the term LU(Logical Unit) to refer to an independently controllable device or to an independently controllable piece

of storage in a storage array device. The term LUN (Logical Unit Number) refers to an addressingidentifier of the LU, but it is also used to refer to the device or piece of storage, or even a hardware pathto the device. In this paper, to distinguish between the actual device and a path to that device, we'll usethe term "LUN device" to mean the actual piece of storage, and the term "LUN path" to mean a specifichardware path to the piece of storage. There can be multiple LUN paths to a given LUN device.

On an HP-UX system, a “LUN path” can be opened, accessed and closed via I/O or control operations.For user access a LUN path typically has at least one device file associated with it. Volume Managers or other multi-pathing modules running in HP-UX may abstract which path is being accessed for a givenLUN device, in which case an open of such an abstracted LUN path provides a mechanism for access tothe LUN device independently of any specific path.

Two cases of abstracted LUN paths currently used on HP-UX occur in alternate path failover functionality such as LVM’s PVLinks and in dynamic-multi-pathing functionality such as SecurePath.“Dynamic-multi-pathing” refers to the capability to determine the various paths to a given LUN device atruntime and to dynamically load-balance I/Os across those paths. As a result, all LUN paths that areaccessed via a dynamic-multi-pathing module are considered load-bearing and hence “active” in thedefinitions below. Dynamic-multi-pathing modules tend to include the alternate path failover functionality as well, but the key characteristic of such modules important for this paper is that theyincrease the number of “active” paths that exist on the system.

LVM’s PVLinks provides a mechanism for abstracting multiple paths (up to 8 paths per LUN device) for purposes of failover. Only one of the paths is a load-bearing path at any point in time.

For purposes of LUN count maximums we’ll define the following terms:

• Active LUN device : A LUN device with at least one Active LUN path.• Active LUN path : A load-bearing path to a LUN device.• Open LUN path : An "Active LUN path" or a path that is open with only occasional I/O to the LUN.• Visible LUN path : An "Open LUN path" or a path that is typically not open but is visible to ioscan.

1 The term HBA (Host Bus Adapter) in this paper is used to refer to a mass storage adapter (e.g., SCSI or FibreChannel) that attaches to a system I/O bus such as PCI.

8/6/2019 hpux-scsi



Page 2

Each of these four terms will be used in defining various supported LUN counts, as described in the nextsection below.

Note that all PVLinks paths to a given LUN device are considered “Open” but only one of the paths isactive at any point in time. Hence if a system has 2000 LUN devices attached to it with 2 paths to eachdevice for a total of 4000 LUN paths, all of which are PVLink pairs, then the number of Active LUN

paths would be 2000 and the number of Open and Visible LUN paths would be 4000.

On the other hand, if a system has 2000 attached LUN devices with 2 paths to each device, all of whichare accessed through a dynamic-multi-pathing module the number of active LUN paths would be 4000.

3. Supported LUN Maximums on HP-UX

The maximum supported LUNs on HP-UX releases are tested configurations. Customers may contacttheir HP support personnel to discuss support of higher LUN levels than those indicated below.

As of the writing of this paper, HP-UX supports maximum LUN configurations as defined in thefollowing table.

Table 1 - Max Supported LUNs

HP-UX Releases

Features10.20 11.00 11i v1* 11i v1**/11i v2

Active LUN devices 768 2400 4096 8192 Active LUN paths 768 2400 4096 8192Open/Visible LUN paths 1536 4800 8192 16384LUNs/HBA (non-HA) 1 512 512 512 1536LUNs/HBA (HA) 1 320 320 512 1536LVM PVs 2 65280 65280 65280 65280

* HP-UX 11i v1 prior to the December 2003 Operating Environment (OE)

** 11i v1, the December 2003 OE or later. See section 3.1 below for additional details.1 LUNs/HBA (HA and non-HA) indicates the maximum number of Visible LUN paths supported

through a single HBA port . In this context “HA” (High Availability) refers to usage of the HBA in anenvironment in which the MC-ServiceGuard or ServiceGuard-OPS (Oracle Parallel Server) products areused.

2 LVM PVs refers to the maximum number of LVM physical volumes (PVs) that can be configured on asystem. A PV corresponds to a LUN device.

These maximums may change over time as larger configurations are tested and supported.

Here are some examples using the “HP-UX 11i v1 prior to the December 2003 OE” column in Table 1:

a) 4096 LUN devices, with 2 paths to each device, all controlled by LVM’s PVLinks:⇒ 4096 Active LUN paths, 8192 Open and Visible LUN paths

b) 4100 LUN devices, with 2 paths to each device, and 4090 of them controlled by LVM’s PVLinks andthe other 10 LUNs only used occasionally:⇒ 4090 Active LUN paths, 8180 Open LUN paths, and 8200 Visible LUN paths

8/6/2019 hpux-scsi



Page 3

c) 2048 LUN devices, with 4 paths to each device, all controlled by LVM’s PVLinks:⇒ 2048 Active LUN paths, 8192 Open and Visible LUN paths

d) 1024 LUN devices, with 4 paths to each device, all accessed via dynamic-multi-pathing modules:⇒ 4096 Active LUN paths, 4096 Open and Visible LUN paths

e) 512 LUN devices, with 16 paths to each device, 8 of which are accessed via dynamic-multi-pathingmodules (the other 8 per LUN device not being configured for normal use):⇒ 4096 Active LUN paths, 8192 Open and Visible LUN paths

3.1 11i v1 Maximums

The high LUN maximums shown in the right-most column of Table 1, are supported on 11i v1 with theDecember 2003 OE or later. Key patches/driver levels required include:

FC-FCD driver version B.11.11.02 or later (FibrChanl-01 bundle) FC-Tachyon-TL driver version B.11.11.09 or later (FibrChanl-00 bundle) FC Mass Storage patch PHKL_28984 or later

LVM patch PHKL_26743 or later

The following patches, which provide additional benefits in scan times and I/O responsiveness, are alsorecommended on large LUN configurations:

PHKL_32002 and PHKL_32005: I/O responsiveness in large LUN configurations mtIOscan11i bundle 2: boot-time improvements in large LUN configurations PHKL_30515: improves system performance in large LUN configuration

Each of these patches will have some benefit in smaller LUN configurations but will provide moresignificant benefit in large LUN configurations. See section 6.5 for additional patch recommendations.

3.2 11i v2 Maximums The high LUN maximums shown in the right-most column of Table 1 are supported with the initialrelease of 11i v2 in May 2003 without the need for any patches. However, the September 2004 OE of 11iv2 or later is required to support PA-RISC HP 9000 servers and is recommended in general for large LUNconfigurations. See section 6.5 for additional patch recommendations.

4. Device Maximums

In addition to the system considerations specified above, the LUN configuration of the storage deviceitself needs to be planned both internally and with respect to the configuration/topology. Make sure youconsider all of the possible factors across the storage device, adapter, and the system in your LUN count

determinations. Review the storage device documentation for details on device-specific configurations.

2 The mtIOscan11i bundle provides performance improvements in 11i v1 with respect to the initial ioscan during boot. It was delivered in the December 2004 Software Pack, and can be downloaded from http://software.hp.com ;search for “mtIOscan11i”. See also the Product Note at http://www.hp.com/softwarereleases/releases-media2/latest/osoe/11i/12_04/12_04.pdf .

8/6/2019 hpux-scsi



Page 4

5. Considerations

Whenever a system is configured, the number of LUNs must be properly balanced with respect to thevarious I/O links and buses and with the configuration of volume manager volumes and volume groups,the size of memory, the number of CPUs, the corresponding application workload, etc. The details of system and mass storage configuration and tuning to support large LUN configurations are beyond thescope of this paper, but a few general guidelines and specific considerations applicable to large LUNconfigurations are outlined here.

5.1 Server considerations

Generally speaking the larger LUN levels are expected to only be connected to the larger HP-UX serverssuch as the rp7420/rp8420, rx7620/rx8620 and Superdome systems. Smaller servers can share in the useof large numbers of LUNs in a cluster, but due to memory and slot limitations would generally not beable to directly support large numbers of LUNs.

5.2 Kernel Tunables

As the number of LUNs is increased, the “maxfiles” and “maxfiles_lim” kernel tunables will typicallyalso need to be increased. Contact HP support personnel for help in tuning your specific configuration.

5.3 Volume Manager considerations

The LVM (Logical Volume Manager) product in HP-UX currently allows for a maximum of 256 VGs(volume groups) with up to 255 PVs (physical volumes) per VG. Therefore, given the original 11i v1LUN device maximums specified in Table 1, you could configure a system with 32 VGs, each of whichcontain 128 PVs, for a total of 4096 PVs (where a PV is equivalent to a LUN device). Or for a morewell-balanced configuration you could spread the PVs across a larger number of VGs: for example, aconfiguration with 64 VGs with 64 PVs/VG for 4096 total PVs.

It should be noted that the “maxvgs” tunable, which defaults to 10, restricts the number of PVs per VG.The maxvgs tunable needs to be increased to allow larger configurations.

For planning purposes please keep in mind that the total amount of disk space that a volume group cancontain is set when the VG is created, based on the extent size and the maximum number of extentsspecified at VG creation. Therefore, to ease future expansion, the options specified at VG creation should

be carefully considered.

5.4 File System considerations

The use of HFS file systems with large numbers of LUNs is not recommended due to recovery times; please use the VxFS file system with large LUN levels.

5.5 Scan/Startup/Shutdown times

As the number of LUNs increase, the time it takes to discover LUNs via ioscan or other device queryutilities increases. Because ioscans are done during boot, this can also impact boot time. Similarly, it canimpact application startup times, for applications that do an ioscan or similar general device discovery.These discovery times will be significantly reduced with future HP-UX releases.

8/6/2019 hpux-scsi



Page 5

Various patches are available for HP-UX 11.00, 11i v1, and 11i v2 releases which, when applied together reduce the ioscan time significantly. The following table lists these patches. Apply these patches (or

patches which supersede) together for reducing ioscan time.

Table 2 : ioscan performance improvement patches

HP-UX 11.00 PHCO_27818 PHKL_24165 PHKL_24187HP-UX 11i v1 PHCO_33136 PHKL_24163 PHKL_25165 PHKL_30257HP-UX 11i v2 PHKL_31501

In addition, the mtIOscan11i bundle, referenced in section 3.1, provides performance improvements withrespect to the initial ioscan that occurs during boot.

5.6 Topology considerations

The use of Fibre Channel arbitrated loop topologies can result in long latencies during initial device openswhen using more than 512 LUNs per HBA. To avoid this problem it is recommended to move to FibreChannel fabric (switched) topologies.

6. Roadmap / Futures

To meet future customer needs, the maximum LUNs supported by HP-UX will continue to increase. TheHP-UX development team is also working on significantly decreasing the overheads associated withmanaging, scanning, and displaying large numbers of LUNs to provide streamlined management of largestorage attachments.

7. More Information

Visit http://docs.hp.com/ web site for information on HP's Fibre Channel and SCSI host bus adapters. Go towww.hp.com/go/serverconnectivity and click on StorageWorks support matrices for information on StorageArea Network (SAN) connectivity options for HP-UX.

8/6/2019 hpux-scsi



Page 6

Legal Notices

The information in this document is subject to change without notice.Hewlett-Packard makes no warranty of any kind with regard to this white paper, including but not limited to the implied warranties ofmerchantability and fitness for a particular purpose. Hewlett-Packard shall not be held liable for errors contained herein or direct, indirect,special, incidental or consequential damages in connection with the furnishing, performance, or use of this material.

©Copyright 2005 Hewlett-Packard Company, all rights reserved.

HP-UX® is a registered trademark of the Hewlett-Packard Corporation. All other trademarks and registered trademarks are the property ofthe respective corporations.

Reproduction, adaptation, or translation of this document without prior written permission is prohibited, except as allowed under theco ri ht laws.

Documents

hpux-scsi