HPE Reference Architecture for VMware Cloud Foundation on

u Reference Architecture

HPE REFERENCE ARCHITECTURE FOR VMWARE CLOUD FOUNDATION ON HPE PROLIANT DL SERVERS Version 4.2.0

Reference Architecture

CONTENTS Executive summary .............................................................................................................................................................................................................................................................................................................. 3

Solution overview ................................................................................................................................................................................................................................................................................................................... 4

Solution components .......................................................................................................................................................................................................................................................................................................... 6

Hardware ................................................................................................................................................................................................................................................................................................................................. 6

Software ................................................................................................................................................................................................................................................................................................................................... 9

Design and configuration guidance ................................................................................................................................................................................................................................................................... 13

VMware Cloud Foundation installation on HPE ProLiant DL Server .................................................................................................................................................................................. 20

HPE storage solutions for VMware Cloud Foundation workload domain....................................................................................................................................................................... 20

Deploy vSphere Lifecycle Manager Baselines workload domain on HPE ProLiant DL Servers ................................................................................................................. 29

Deploy vSphere Lifecycle Manager Baselines workload domain ...................................................................................................................................................................................... 29

Deploy NSX-T Edge Cluster ............................................................................................................................................................................................................................................................................... 29

Deploy Kubernetes solution ............................................................................................................................................................................................................................................................................... 30

Deploy vSphere Lifecycle Manager images workload domain on HPE ProLiant DL Server ......................................................................................................................... 31

Patching and upgrading in VMware Cloud Foundation .................................................................................................................................................................................................................. 36

HPE Storage Integration for VMware vCenter ......................................................................................................................................................................................................................................... 37

HPE ProLiant DL Server firmware upgrade using HPE OneView for VMware vRealize Orchestrator ................................................................................................ 44

Managing and monitoring large scale HPE ProLiant DL Server based infrastructure for VMware Cloud Foundation ........................................................ 49

Consolidated architecture deployment of VMware Cloud Foundation ............................................................................................................................................................................. 65

Summary ..................................................................................................................................................................................................................................................................................................................................... 66

Appendix A: Bill of materials ..................................................................................................................................................................................................................................................................................... 66

Known issues .......................................................................................................................................................................................................................................................................................................................... 68

URLs for firmware, software, and documentation ................................................................................................................................................................................................................................. 68

Rack and power links ................................................................................................................................................................................................................................................................................................ 68

Network links .................................................................................................................................................................................................................................................................................................................... 68

Storage links ...................................................................................................................................................................................................................................................................................................................... 68

Server links ......................................................................................................................................................................................................................................................................................................................... 68

Software links ................................................................................................................................................................................................................................................................................................................... 68

VMware links .................................................................................................................................................................................................................................................................................................................... 69

Resources and additional links ............................................................................................................................................................................................................................................................................... 70

Reference Architecture Page 3

EXECUTIVE SUMMARY Businesses face challenges to turn ideas into services faster, respond quicker to new customer demands, and innovate better by building new services with technology to stay competitive. To meet these business demands, IT is increasingly adopting new cloud technologies, to replace expensive hardware with a software-defined model. Enterprises need an ideal Hybrid IT model that supports both traditional and cloud-native applications. Therefore, businesses are moving to digital transformation and software-defined data center (SDDC) solutions to support this shift.

Hewlett Packard Enterprise and VMware® collaborate to help customers accelerate the journey to the hybrid cloud and bring the promise of the software-defined data center to life. The combination of HPE ProLiant DL Server Infrastructure and VMware SDDC solution dramatically improves business outcomes as well as overall value for our customers. HPE ProLiant DL Servers combined with VMware Cloud Foundation™ (VCF) delivers a simplified and more secure private cloud - that is flexible, easy to deploy, seamless to manage, and simple to operate. For enterprise customers looking to accelerate their journey to hybrid cloud, HPE ProLiant DL Servers with VMware Cloud Foundation is the right solution to support and run all your enterprise apps—both traditional and containerized—in a cloud environment.

This Reference Architecture provides guidance for deploying and managing VMware Cloud Foundation on HPE ProLiant DL Servers and deployment of two types of VCF workload domain i.e., Kubernetes-based vSphere Lifecycle Manager Baselines workload domain and vSphere Lifecycle Manager Images workload domain. This paper also describes the Lifecycle management of HPE ProLiant DL Server firmware using HPE OneView for VMware vRealize® Orchestrator™ based firmware update workflow and VMware Lifecycle Manager. VMware Validated Design (VVD) is a family of solutions for data center designs that span compute, storage, networking, and management, serving as a blueprint for your SDDC implementation. VVD version 6.2 can be used with VCF 4.2 documentation and HPE's Reference Architecture, to build VCF 4.2 as a private cloud on HPE ProLiant DL3x0 servers.

Application virtual network (AVN) is a software-defined overlay network that provides many benefits in SDDC such as simplified data mobility, improved security, and disaster recovery procedure. The use of AVN in VCF 4.2 is optional. Aruba CX 8325 switches which support up to 100Gb per port, high-performance server connectivity, and the capability to handle virtual environment are used as TORs to configure BGP Uplinks for AVN. This Reference Architecture also covers using the HPE Primera and HPE Nimble as principal storage as well as supplementary storage for the VCF workload domain.

Benefits include:

• Combines HPE ProLiant DL Servers with VCF from a deployment and lifecycle perspective, in a cost-effective and simplified management for faster time to value

• An easy-to-operate Virtual Infrastructure (VI) traditional data center in a VCF workload domain

• Ability to begin with a small infrastructure footprint and expand on-demand to quickly meet changing business requirements

• Remediate drivers and firmware upgrade in a single maintenance window along with ESXi update for a vLCM-based workload domain through the Hardware Support Manager (HSM) service integrated into HPE OneView for VMware vCenter® (OV4vC) integration

• An easy and automated way to upgrade HPE ProLiant DL Server firmware using VMware vRealize® Orchestrator™ workflow for baseline workload domain

• Guidance to attach and configure HPE Primera and HPE Nimble as principal storage and supplementary storage for VCF VI workload domain

• Support for consolidated architecture deployment of VCF

Target audience: This document is intended for IT decision-makers as well as architects, system engineers, and system administrators who want to understand enterprise-ready private cloud solutions using the HPE ProLiant DL Servers and VMware Cloud Foundation. The reader should have a solid understanding and familiarity with VMware Cloud Foundation, VMware vRealize Orchestrator, Enterprise Networking, Storage, and HPE ProLiant DL Servers. For assistance with automated deployment of HPE ProLiant DL Servers with VMware Cloud Foundation, contact your HPE representative.

Document purpose: The purpose of this document is to demonstrate an example use case of enterprise-ready private cloud solution by combining the value of VCF and HPE ProLiant DL Servers Infrastructure that is flexible and easy to deploy.

This Reference Architecture describes solution testing performed in March 2021.


SOLUTION OVERVIEW This Reference Architecture demonstrates best practices for customers building a cloud solution in an enterprise data center and deploying business-critical applications in an automated manner. The solution design is based on VMware Cloud Foundation on HPE ProLiant DL Servers. VMware Cloud Foundation provides a unified SDDC platform comprising VMware vSphere® Hypervisor, VMware Virtual SAN™ Storage (vSAN), and VMware NSX-T Data Center™ networking. This Reference Architecture demonstrates the following components for VMware Cloud Foundation:

• HPE Servers - HPE ProLiant DL360 Gen10 as management domain hosts and HPE ProLiant DL380 Gen10 with HPE SN1600Q 32Gb FC cards as workload domain hosts.

• HPE Storage – HPE Primera, and HPE Nimble act as principal storage and supplementary storage to a VI -workload domain on HPE ProLiant DL Server. This storage can be configured either as Virtual Machine File System (VMFS) on FC or Virtual Volume (VVols) in the VCF VI workload domain.

• HPE OneView - HPE OneView management appliance deploys and maintains infrastructure faster, simplifies IT operations, and increases productivity.

• HPE OneView for vRealize Orchestrator – HPE OneView for vRealize Orchestrator workflow updates the HPE ProLiant compute firmware.

• HPE OneView for VMware vCenter – HPE OneView for VMware vCenter (OV4vC performs remediation on the servers via vLCM in addition to additional HPE hardware management and monitoring tasks.

• HPE Storage Integration Pack for VMware vCenter - Enables the vSphere administrator to easily manage array storage, datastores, and virtual machines by providing the ability to clearly view and directly manage the HPE Primera and HPE Nimble from within the vCenter console.

• VMware Kubernetes for VCF workload domain - VMware Kubernetes acts as a cloud-native platform for VCF workload domain on HPE ProLiant DL Server.

• vSphere Lifecycle Manager - vLCM, a vCenter service, integrated with VCF helps to create and remediate cluster images for centralized and simplified lifecycle management of ESXi hosts including firmware.

• VMware Cloud Foundation lifecycle management – Leverages VCF updates for HPE ProLiant DL Servers.


Figure 1 shows the physical rack layout showcasing the solution components validated in this Reference Architecture.

FIGURE 1. Physical rack layout showcasing the solution components

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

Front view

AB

AB

AB

AB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

HDD1TB

Nimble Storage

HPE Nimble Storage Adaptive Flash Array

HPE SAN Switch SN6600B 32Gb

40 51 62 73 128 139 1410 1511 3632 3733 3834 3935 4440 4541 4642 4743

48 49 50 51 56 57 58 59

52 53 54 55 60 61 62 632016 2117 2218 2319 2824 2925 3026 3127

1 3

0 2

HPE SN6600BFC Switch

40 51 62 73 128 139 1410 1511 3632 3733 3834 3935 4440 4541 4642 4743

48 49 50 51 56 57 58 59

52 53 54 55 60 61 62 632016 2117 2218 2319 2824 2925 3026 3127

1 3

0 2


HPE Primera Storage

Drive Bay ID:

Box 1 Box 2

Box 3

1 2 3 41 2 3 41 2 3 4

UID

ProLiantDL380Gen10

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

Drive Bay ID:

Box 1 Box 2

Box 3

1 2 3 41 2 3 41 2 3 4

UID

ProLiantDL380Gen10

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

Drive Bay ID:

Box 1 Box 2

Box 3

1 2 3 41 2 3 41 2 3 4

UID

ProLiantDL380Gen10

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

Drive Bay ID:

Box 1 Box 2

Box 3

1 2 3 41 2 3 41 2 3 4

UID

ProLiantDL380Gen10

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

SAS600 GB

15K

HPE ProLiant DL380 Gen10

HPE Aruba 6300M Switch (OOBM)

HPE Aruba 8325 Switch (ToR)


Workload Domain

Green=100Gbps,Yellow=40/26/10GbpsQSFP28 323130292827262524232221121110987654321 20191413

FlexFabric5945 Switch

JQ077A

Green=10Gbps, Yellow=1Gbps 15 161 2 17 18 31 32 33 34 47 48SFP+ Green=40Gbps, Yellow=10GbpsQSFP

51

52

49

50

SYS

HP 5900Series SwitchJC772A

Green=10Gbps, Yellow=1Gbps 15 161 2 17 18 31 32 33 34 47 48SFP+ Green=40Gbps, Yellow=10GbpsQSFP

51

52

49

50

SYS

HP 5900Series SwitchJC772A

Green=100Gbps,Yellow=40/26/10GbpsQSFP28 323130292827262524232221121110987654321 20191413

FlexFabric5945 Switch

JQ077A

External Storage for VMware Cloud

Foundation workload domain

UID

iLO

Drive Bay ID:

108642

531 ProLiantDL360Gen10

UID

iLO

Drive Bay ID:

108642


UID

iLO

Drive Bay ID:

108642


UID

iLO

Drive Bay ID:

108642


Management Domain


SOLUTION COMPONENTS The VCF on HPE ProLiant DL solution is tested with the following hardware and software components. For additional component details, see the VCF 4.2 Release notes at https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/rn/VMware-Cloud-Foundation-42-Release-Notes.html.

For detailed firmware and software compatibility matrix for VMware Cloud Foundation 4.2 on HPE ProLiant DL Servers, see https://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50003639enw.

Hardware Table 1 shows the hardware components used in this solution.

TABLE 1. HPE hardware components

Components Qty Description


(Management domain node)

4 • 2x Intel® Xeon® Gold 6132 CPU @ 2.60 Ghz, 14/14 cores

• 384 GB RAM

• HPE Eth 10/25 gb 2p 631flr-sfp28 Adapter

• HPE Smart Array Controller p408i-a sr Gen10

• 1.9 TB x 4 SSD drive for vSAN


(Workload domain node)

4 • 2x Intel Xeon Silver 6126 CPU @ 2.60 Ghz, 12/12 cores

• 128 GB RAM

• HPE Eth 10/25 Gb 2p 640flr-sfp28 Adapter

• HPE SN1600Q 32Gb 2p FC HBA

• HPE Smart Array Controller p816i-a sr Gen10

• 10 TB x 10 SSD drive for vSAN

Aruba 8325 Switch 2 ToR – 32C 32-port 100G QSFP+/QSFP28

Aruba 6300M Switch 2 OOBM - 48-port 1GbE and 4-port SFP56

HPE Primera 600 Storage 1 A630 2-node Controller

HPE Nimble Storage 1 HF20 Adaptive Array

NOTE Both HPE ProLiant DL380 Gen10 and HPE ProLiant DL360 Gen10 servers were used in the solution build. However, you can deploy this solution on either server type. The management domain can only be deployed on vSAN ReadyNodes with vSAN as principal storage, however, vSAN is optional for workload domain nodes. Use either Fabric-Attached (FC) or Direct-Attached (DA) connected shared LUNs from either HPE Nimble or Primera with VMFS datastore as principal storage for workload domain. For additional details on vSAN ReadyNodes, see the VMware Compatibility Guide (VCG) for vSAN and Cloud Foundation VMware Cloud Foundation Planning and Preparation Guide.

The following server hardware components were used for validation in Hewlett Packard Enterprise labs. For additional details on vSAN ReadyNodes, see VMware Compatibility Guide (VCG) for vSAN and Cloud Foundation VMware Cloud Foundation Planning and Preparation Guide.

HPE ProLiant DL380 Gen10 server The HPE ProLiant DL380 Gen10 platform offers flexibility for end-user computing workload. With a choice of CPUs offering a balance between core counts and core frequencies, very large memory footprints, and disk options the HPE ProLiant DL380 Gen10 platform is an ideal choice for supporting demanding workloads. For this use case, each server used in this solution was equipped with 128 GB RAM and dual Intel Xeon Silver 2.6GHz 12 Core CPUs.

https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/rn/VMware-Cloud-Foundation-42-Release-Notes.html

https://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50003639enw

https://www.vmware.com/resources/compatibility/search.php?deviceCategory=vsan


https://docs.vmware.com/en/VMware-Cloud-Foundation/4.0/vcf-40-ovdeploy-guide.pdf





Figure 2 shows the front view of the HPE ProLiant DL380 Gen10 server.

FIGURE 2. HPE ProLiant DL380 Gen10 server

HPE ProLiant DL360 Gen10 server The HPE ProLiant DL360 Gen10 is a performance-driven dense 1U server that is ideal for many use cases, including end-user computing scenarios with pass-through graphics requirements. For this use case, each server was equipped with 384 GB RAM and dual Intel Xeon Gold 2.6 GHz 14 Core CPUs. The four (4) HPE ProLiant DL360 Gen10 servers deployed in this solution are management domain compute nodes.

Figure 3 shows the HPE ProLiant DL360 Gen10 server.

FIGURE 3. HPE ProLiant DL360 Gen10 server

The following networking hardware components were used for testing this solution.

Aruba 6300M Switch The Aruba CX 6300 Switch Series is a modern, flexible, and intelligent family of stackable switches ideal for enterprise network access, aggregation, core, and data center top of rack (ToR) deployment. Created for game-changing operational efficiency with built-in security and resiliency, the 6300 switches provide the foundation for a high-performance network supporting IoT, mobile, and cloud applications.

Figure 4 shows the front view of the Aruba 6300M Switch.

FIGURE 4. Aruba 6300M Switch

Aruba 8325 Switch Series The Aruba 8325 Switch Series offers a flexible and innovative approach to address the application, security, and scalability demands of the mobile, cloud, and IoT era. These switches serve the needs of the next-generation core and aggregation layer, as well as emerging data center requirements at the Top of Rack (ToR) and End of Row (EoR). They provide over 6.4Tbps of capacity, with line-rate Gigabit Ethernet interfaces including 1Gbps, 10Gbps, 25Gbps, 40Gbps, and 100Gbps. The 8325 series includes industry-leading line rate ports 1/10/25GbE (SFP/SFP+/SFP28) and 40/100GbE (QSFP+/QSFP28) with connectivity in a compact 1U form factor. These switches offer a fantastic investment for customers wanting to migrate from older 1GbE/10GbE to faster 25GbE, or 10GbE/40GbE to 100GbE ports.


Figure 5 shows the front view of the Aruba 8325 32Y8C Switch.

PS2

Reset

PS1

Fan

UID

484746454443424140393837363534333231302928272625242322212019181716151413121110987654321 5049 5251 5453 5655

L4L3L2L1 L4L4L3L2L1L4L3L2L1 L4L4L3L2L1

53

54

55

56

49

50

51

52

Mgmt Console

8325

USB Console

FIGURE 5. Aruba 8325 32Y8C Switch

HPE StoreFabric SN6600B Fibre Channel Switch The HPE StoreFabric SN6600B Fibre Channel Switch meets the demands of hyper-scale virtualization, larger cloud infrastructure, and growing flash-based storage environment by delivering market-leading 32Gb Fibre Channel technology and capabilities. It provides a high-density building block for increased scalability, designed to support growth, demanding workload, and data center consolidation in small to large-scale enterprise infrastructure. It is built for maximum flexibility, scalability, and ease of use. Organizations can scale from 24 to 128 ports, all in an efficient 1U or 2U package. It also provides a simplified deployment process and a point-and-click user interface, making it both powerful and easy to use. With the SN66x0B Switch, organizations gain the best of both worlds: high-performance access to industry-leading storage technology and "pay-as-you-grow" scalability to support an evolving storage environment.

Figure 6 shows the HPE StoreFabric SN6600B Fibre Channel Switch.

FIGURE 6. HPE StoreFabric SN6600B Fibre Channel Switch

If you are using a fabric-attached Fibre Channel topology, a minimum of two (2) SAN switches are required. If more than two switches are needed in the solution, switches of the same vendor/type must be added in pairs, so that there is always an even number of switches. It is important to ensure that you select the correct model switch for your future growth plan. Select a bigger switch with only the ports that require a license to start if you plan on expanding your HPE ProLiant DL Server solution in the future.

HPE Nimble Storage HF20 Adaptive Flash Array The HPE Nimble Storage HF20 Adaptive Flash Array is designed for both principal and secondary flash workload. It is a hybrid flash array for mixed, principal workload, where cost-efficient flash performance is important. It serves as a secondary flash array for backup and disaster recovery (DR) while allowing you to put your backup data to work.

Figure 7 shows the HPE Nimble Storage.

FIGURE 7. HPE Nimble Storage

40 51 62 73 128 139 1410 1511 3632 3733 3834 3935 4440 4541 4642 4743

48 49 50 51 56 57 58 59

52 53 54 55 60 61 62 632016 2117 2218 2319 2824 2925 3026 3127

1 3

0 2


Nimble Storage


HPE Primera 600 Storage HPE Primera storage system features a high-speed, full-mesh passive interconnect that joins multiple controller nodes (the high-performance data movement engines of the HPE Primera architecture) to form an all-active cluster. This low-latency interconnect allows tight coordination among the controller nodes and a simplified software model.

HPE Primera storage redefines mission-critical storage for tier-0 applications. Designed for NVMe and Storage Class Memory, HPE Primera delivers remarkable simplicity, app-aware resiliency for mission-critical workload, and intelligent storage that anticipates and prevents issues across the infrastructure stack.

HPE Primera delivers on the promise of intelligent storage, advanced data services, and simplicity for your mission-critical applications. With a services-centric OS that sets up in minutes, HPE Primera also upgrades seamlessly to minimize risk and be transparent to applications.

Figure 8 shows the HPE Primera A670 Storage.

FIGURE 8. HPE Primera A670 Storage

Software Table 2, 3, and 4 lists the software components used in this Reference Architecture.

TABLE 2. HPE software and firmware components

Component Version

HPE VMware Upgrade Pack ( VUP )

Note: HPE VUP can be used as an SPP to install drivers and

firmware on HPE ProLiant Servers

1.2(A)

P42413_001_VUP12A-SPP-VUP12A.2021_0106.20

Note: Apply VUP on DL Server hardware before deploying ESXi

HPE Nimble OS 5.2.1.500

HPE SSMC

Note: Used with Primera

3.7.1.1

HPE Primera OS 4.2.0

HPE B-series SAN Fabric OS (FOS) 9.0.0b

HPE OneView Global Dashboard 2.20

Aruba 6300M Switch 10.06

Aruba 8325 Switch ( JL627A ) 1.0.06

HPE G2 Metered and Switches PDUs 2.0.0.K


TABLE 3. VMware software components

Component Version

Cloud Builder VM 4.2.0.0 (Build 17559673)

SDDC Manager 4..2.0.0 (Build 17559673)

VMware vCenter Server® 7.0 (Appliance) 7.0 Update 1c (Build 17327517)

HPE OEM Customized vSphere Distribution 7.0 – ProLiant VMware-ESXi-7.0.1-17325551-HPE-701.0.0.10.6.3.9-Jan2021.iso (Build: 17325551)

VMware Patch bundle for 7.0 Update 1d VMware-ESXi-7.0U1d-17551050-depot.zip

Note: Apply the update 17551050 on ESXi before deploying the SDDC domain

VMware vSAN™ 7.0 Update 1d (Build: 17551050)

VMware NSX-T Data Center 3.1.0 (Build: 17107167)

VMware vRealize® Suite Lifecycle Manager 8.2 Patch 2

VMware vRealize Orchestrator 8.2

vRealize Operations Manager™ 8.2

vRealize Automation™ 8.2

vRealize Log Insight™ 8.2

TABLE 4. HPE Solution Integration Software for VMware

HPE Connector Version

HPE OneView ( Virtual Appliance ) 6.0

HPE OneView Global Dashboard 2.3

HPE OneView for VMware vRealize Orchestrator (vRO) 1.4

HPE Storage Integration Pack for VMware vCenter (SIPVC) 10.2

HPE OneView for VMware vCenter plug-in 10.2

Integrated Smart Update Tools (iSUT)

vSphere 7.0

2.7.0

cp045947.zip

For more information on VCF 4.2 software and firmware, refer to the firmware and software compatibility matrix for VMware Cloud Foundation at http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50003639enw.

VMware Cloud Foundation VMware Cloud Foundation (VCF) is the industry’s most advanced enterprise-ready hybrid cloud platform providing a complete set of software-defined services for compute, storage, networking, security, and cloud management to run enterprise apps whether it is traditional or containerized. VCF drastically simplifies data center operations by deploying a standardized and validated architecture with built-in lifecycle automation of the cloud stack. It orchestrates, provisions, and deploys an SDDC platform by integrating VMware vSphere, vSAN, and NSX into a full-stack HCI solution that delivers enterprise-ready cloud infrastructure.

http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50003639enw


Figure 9 shows the VMware Cloud Foundation components.

FIGURE 9. VMware Cloud Foundation components

VMware Cloud Foundation components The core components for VMware Cloud Foundation are as follows.

Cloud Foundation Builder VM The Cloud Foundation Builder VM is a one-time use VM that deploys and configures the management domain and transfers inventory and control to SDDC Manager. During the deployment process, the Cloud Foundation Builder VM validates network information provided in the deployment parameter spreadsheet such as DNS, network (VLANs IP Address, MTUs), and credentials. After the management domain is up and the SDDC Manager is running, the Cloud Foundation Builder VM must be powered off and archived.

Table 5 shows the Cloud Foundation Builder VM resource requirements.

TABLE 5. Cloud Foundation Builder VM resource requirements

Components Requirements

CPU 4 vCPUs

Memory 4GB

Storage 150GB

SDDC Manager SDDC Manager manages the bring-up of the Cloud Foundation system, creates and manages workload domains, and performs lifecycle management to ensure the software components remain up to date. SDDC Manager also monitors the logical and physical resources of VCF. It allows data center administrators to configure the additional hosts and racks into a logical pool of resources and thus multiple racks can be managed as a single VCF System. SDDC Manager controls these processes by using workflows. Each workflow comprises a series of tasks that are executed by the SDDC Manager.

VMware vCenter Server VMware vCenter Server provides management of a VMware virtualized environment with one or more ESXi hosts. SDDC Manager deploys one VMware vCenter Server per workload domain. By default, all vCenter Servers are configured in enhanced linked mode.

VMware vSphere (ESXi) ESXi is a type 1 hypervisor used to implement virtualization on bare-metal systems. ESXi provides compute virtualization within the software-defined data center and it is a foundational building block for implementing a private cloud.


VMware vSAN VMware vSAN aggregates local or direct-attached data storage devices to create a single storage pool shared across all hosts in the vSAN cluster. vSAN eliminates the need for external shared storage, simplifies storage configuration, and virtual machine provisioning.

VMware NSX-T VMware NSX-T is designed to address application framework and architecture that have heterogeneous endpoints and technology stacks. In addition to vSphere, this environment may include other hypervisors, containers, bare metal, and public clouds. NSX-T allows IT and development teams to choose the technologies best suited for their applications. NSX-T is also designed for management, operations, and consumption by development organizations besides used by IT.

VMware vRealize Orchestrator vRealize Orchestrator is a development and process-automation platform that provides an extensive library of workflow and a workflow engine. It simplifies the automation of complex IT tasks. vRealize Orchestrator is included as part of vSphere, vRealize Automation entitlements, or both. vRealize Automation-focused functionality within vRealize Orchestrator is only available as a part of vRealize Automation entitlement (standalone or vRealize Suite Advanced/Enterprise license keys).

VMware vRealize Portfolio

VMware Cloud Foundation also has the following optional components for which separate licenses are needed:

• VMware vRealize Operations Manager: vRealize Operations Manager delivers intelligent operations management with application-to-storage visibility across physical, virtual, and cloud infrastructure. Using policy-based automation, operation teams automate key processes and improve IT efficiency. This is an optional component.

• VMware vRealize Automation: vRealize Automation is a cloud automation tool that accelerates the delivery of IT services through automation and pre-defined policies, providing a high level of agility and flexibility for developers while enabling IT teams to maintain frictionless governance and control. This is an optional component.

HPE OneView for VMware vRealize Operations HPE OneView for VMware vRealize Operations provides integrated and highly automated performance, capacity, configuration compliance, and cost management tools to the vRealize Operations custom GUI. The plug-in seamlessly integrates the manageability features of HPE ProLiant DL Server with the VMware analytics engine that analyzes what is normal and then applies that baseline to a dynamic server environment.

When the HPE OneView for VMware vRealize Operations is installed, the custom HPE OneView dashboard is added to the vRealize Operation custom GUI. The HPE OneView dashboard allows you to monitor resources in a vRealize environment. The attributes that can be monitored include resource health, power, temperature (server and enclosure), and system alerts. The analytics engine allows proactive monitoring of the HPE OneView resource environment and indicates the state of the resources. If a problem occurs, an alert is triggered and displayed. The analytics engine also provides proactive prediction, which can determine the point in the future when a resource will reach a predefined critical level.

HPE OneView for VMware vRealize Orchestrator HPE OneView for VMware vRealize Orchestrator helps customers automate complex IT tasks in an extensible and repeatable manner. It provides a predefined collection of Hewlett Packard Enterprise tasks and workflow that can be used in vRealize Orchestrator (VRO) with easy-to-use, drag and drop access to the automation of HPE OneView managed hardware deployment, firmware updates, and other life cycle tasks. HPE OneView for VMware vRealize Orchestrator allows the advanced management features of HPE OneView to be incorporated into a larger IT workflow. HPE OneView workflow and actions can also be integrated into VMware vRealize Automation via vRealize Orchestrator.

HPE Storage Integration for VMware vCenter HPE Storage Integration Pack for VMware vCenter is a single, integrated plugin application for VMware vCenter management. It enables the vSphere administrator to quickly obtain context-aware information about HPE Storage in their VMware vSphere data center directly from within vCenter. This application enables the vSphere administrator to easily manage storage, datastores, and virtual machines by providing the ability to clearly view and directly manage the HPE Infrastructure from within the vCenter console.

HPE OneView for VMware vCenter HPE OneView for VMware vCenter is a VMware vCenter plugin that provides server hardware management capabilities, including comprehensive monitoring, firmware update, vSphere/ESXi image deployment, remote control, end-to-end monitoring for Virtual Connect, and power optimization for HPE servers in the VMware environment.


VMware vSphere Lifecycle Manager VMware vSphere Lifecycle Manager is the next version of Update Manager that enables centralized, automated patch and version management for VMware vSphere. VMware vSphere Lifecycle Manager provides the functionality to upgrade and patch ESXi. VMware vSphere Lifecycle Manager along with HPE OneView hardware support Manager (HSM) plug-in can also perform server firmware, drivers, and software updates in the same maintenance window as the ESXi server operating system updates. HPE OneView HSM is integrated inside HPE OneView for VMware vCenter.

DESIGN AND CONFIGURATION GUIDANCE The following section describes an example of the high-level steps necessary for deploying VCF on HPE ProLiant Gen10 servers. The sections are grouped into Infrastructure, management domain, and workload domain roles.

Infrastructure bring up

1. As per the cloud builder parameter sheet, create VLANs on the ToR switch.

Table 8 shows the networks and their VLAN ID.

TABLE 8. VLAN table

Networks VLAN IDs

VCF Management Domain Management VLAN 1611

VCF Management Domain vMotion VLAN 1612

VCF Management Domain vSAN VLAN 1613

VCF Management Domain Host Overlay 1614

VCF Management Domain Edge Overlay 1615

VCF NSX-T Edge Uplink 1 2711

VCF NSX-T Edge Uplink 2 2712

VCF Workload Domain Management VLAN 1616

VCF Workload Domain vMotion VLAN 1617

VCF Workload Domain vSAN VLAN 1618

VCF Workload Domain Host Overlay 1619

VCF Workload Domain Edge Overlay 1620

VCF NSX-T Edge Uplink 1 Workload 2721

VCF NSX-T Edge Uplink 2 Workload 2722

2. Deploy ESXi on all Management and workload domain server nodes.

3. Deploy HPE OneView Appliance on one of the HPE ProLiant DL Servers chosen for the management domain.

4. Configure the HPE OneView Appliance.

5. Import HPE ProLiant DL management and workload servers to the appliance using the server iLO IP address and credentials.

6. Create a Server Profile Template (SPT) for Management domain nodes.

7. Create and assign Server Profiles for the management domain.

8. Create an SPT for workload domain nodes.

9. Create and assign Server Profiles for workload domain nodes.

10. Configure VLAN ID, IP Address, DNS server IP, and FQDN for all the ESXi nodes.

SDDC Manager (Management Domain bring up)

1. Download the VCF parameter sheet from VMware downloads.


2. Deploy Cloud Builder VM on one of the HPE ProLiant DL Servers chosen for the management domain. The VCF parameter sheet .xls file can also be downloaded from the Cloud Builder VM.

3. Fill and upload the VCF parameter sheet in the CBVM for the VCF management domain which deploys the SDDC Manager, vCenter Server, NSX, vSAN.

4. After the SDDC manager bring-up is successful, verify vCenter and SDDC manager has no errors.

5. Power off and delete the CBVM.

With VCF 4.0.1, during the VCF workload domain deployment wizard, we are presented with two Lifecycle Manager options: Enable vSphere Lifecycle Manager Baselines and Enable vSphere Lifecycle Manager Images. Choose any one of the options. Select the first option if you want to install the baseline-based workload domain. Select the second option if you want to install the vLCM-based workload domain. Please note that if you choose vLCM based workload domain, you will not be able to deploy Kubernetes.

Baseline based Workload domain bring-up

1. Create a Network pool for the workload domain.

2. Commission workload domain hosts.

3. Create Baseline (Kubernetes) based workload domain.

4. Create edge cluster for workload domain.

5. Deploy Kubernetes solution on the workload domain.

vLCM Images based workload domain bring up

1. Create a Network pool for the workload domain.

2. Commission workload domain hosts.

3. Deploy Ov4VC on management vCenter and enable vLCM functionality to perform ESXi and firmware upgrades.

4. Deploy vLCM Images based (Non-Kubernetes) workload domain.

Deploy HPE Plug-ins for added functionality

1. Deploy HPE OneView for VMware vRealize Orchestrator (ov4VRO).

2. Deploy HPE Storage Integration Pack for VMware vCenter (SIP4VC).

Aruba CX 8325 for VMware Cloud Foundation Aruba 8325 Switches should be connected and configured for Virtual Switching Extension (VSX). Aruba’s VSX has been designed from the ground to provide industry-leading performance and high availability with much-needed simplicity. This is accomplished through the resiliency of AOS-CX, a modern network operating system that performs continuous state synchronization.

All the VLANs required for VCF deployment should be created on the top-of-rack (ToR) switches or allowed from customer network data center uplinks.


Figure 10 shows the network layout of Aruba Switches.

FIGURE 10. Network layout with Aruba switches

Application Virtual Networks (AVNs) VMware provides an option to disable Application Virtual Networks (AVNs) during bring up. AVNs deploy vRealize Suite components on NSX overlay networks and it is recommended you use this option during bring up. If you disable AVN during bring-up, vRealize Suite components should be deployed to a VLAN-backed distributed port group.

HPE Aruba 6300M

HPE Aruba 8325

LAG

P1 P2

VSF Links

HPE ProLiant DL

HPE Eth 10/25Gb 2p 640FLR-SFP28 Adptr

PS2

Reset

PS1

Fan

UID

484746454443424140393837363534333231302928272625242322212019181716151413121110987654321 5049 5251 5453 5655


53

54

55

56

49

50

51

52

Mgmt Console

8325

USB Console

HPE Aruba 6300M

Aruba 6300M VSF Fabric

PS2

Reset

PS1

Fan

UID

484746454443424140393837363534333231302928272625242322212019181716151413121110987654321 5049 5251 5453 5655


53

54

55

56

49

50

51

52

Mgmt Console

8325

USB Console

HPE Aruba 8325

Keep Alive

Keep Alive

VSX Links

ISL

LAG

iLO

Man

agem

ent


Figure 11 highlights the fields in the VCF deployment parameter sheet when ‘the field ‘Do you want to deploy and configure Application Virtual Networks’ is selected as No.

FIGURE 11. VCF deployment parameter sheet with ‘Application Virtual Networks’ disabled

Figure 12 highlights the fields in the VCF deployment parameter sheet when the field ‘Do you want to deploy and configure Application Virtual Networks’ is selected as Yes.

FIGURE 12. VCF deployment parameter sheet with ‘Application Virtual Networks’ enabled


NOTE BGP configuration must cover both the management and the workload domains. The Cloud builder sheet will only cover the management domain.

If the field ‘Do you want to deploy and configure Application Virtual Networks’ is enabled, then BGP configuration becomes necessary on the top of rack network switches to deploy VCF 4.2.0.0. Figures 13 and 14 highlight the fields corresponding to BGP parameters in the VCF deployment parameter sheet. This is required for setting up the network and routing configuration required for BGP on Aruba 8325 switches.

Figure 13 shows the BGP VLANs in the VCF deployment parameter sheet.

FIGURE 13. BGP VLANs as shown in the VCF deployment parameter sheet

vSphere Distributed Switch Profile as shown in Figure 13 allows deploying an SDDC with a custom VDS design. VCF 4.2 provides three (3) vSphere Distributed Switch profiles that address the limitation of previous VCF versions using only two physical NICs to perform the host bring up. vDS profiles in VCF 4.x performs host bring up with two or four pNICs and creates up to two vSphere Distributed Switches to isolate the VMkernel traffic. Figure 13 shows vSphere Distributed Profile chosen as Profile-1 out of the supported three VDS profile types.

As per the above VCF 4.2 parameter sheet, two additional VLANs need to be configured as ‘Uplinks’ for VMware NSX-T in VCF Management Domain. These uplinks will represent the NSX Edge VM uplinks to the Physical ToR switch for North-South communication. Also with VCF 4.2, static IP pool is supported for NSX-T Host Overlay Network along with DHCP. Users can opt for either dynamic or static allocation of IP for the NSX-T host overlay network. Figure 13 shows static IP pool enabled for NSX-T host overlay.

Before deploying the Cloud Builder VM for VCF 4.2.0.0 deployment, ensure to configure Aruba switches with required VLANs and BGP peering. An example configuration is shown in the following command list.

NOTE This example configuration also includes the configuration setup for a workload domain that is not documented in the cloud builder parameter sheet.


1. Configure VLANs required for VMware Cloud Foundation deployment with the following commands.

8325# configure terminal 8325(config)# vlan 1611 8325(config-vlan-1611)# interface vlan 1611 8325(config-if-vlan)# description vcf_mgmt 8325(config-if-vlan)# ip mtu 1500 8325(config-if-vlan)# ip address 172.16.11.200/24 8325(config-if-vlan)# exit 8325(config)# vlan 1612 8325(config-vlan-1612)# interface vlan 1612 8325(config-if-vlan)# description vcf_vMotion 8325(config-if-vlan)# ip mtu 9000 8325(config-if-vlan)# ip address 172.16.12.200/24 8325(config-if-vlan)# exit 8325(config)# vlan 1613 8325(config-vlan-1613)# interface vlan 1613 8325(config-if-vlan)# description vcf_vSAN 8325(config-if-vlan)# ip mtu 9000 8325(config-if-vlan)# ip address 172.16.13.200/24 8325(config-if-vlan)# exit 8325(config)# vlan 1614 8325(config-vlan-1614)# interface vlan 1614 8325(config-if-vlan)# description vcf_nsxt_host_overlay 8325(config-if-vlan)# ip mtu 9000 8325(config-if-vlan)# ip address 172.16.14.200/24 8325(config-if-vlan)# exit 8325(config)# vlan 1615 8325(config-vlan-1615)# interface vlan 1615 8325(config-if-vlan)# description vcf_nsxt_edge_overlay 8325(config-if-vlan)# ip mtu 9000 8325(config-if-vlan)# ip address 172.16.15.200/24 8325(config-if-vlan)# exit 8325(config)# vlan 2711 8325(config-vlan-2711)# interface vlan 2711 8325(config-if-vlan)# description vcf_nsxt_edge_uplink1 8325(config-if-vlan)# ip mtu 9000 8325(config-if-vlan)# ip address 172.27.11.200/24 8325(config-if-vlan)# exit 8325(config)# vlan 2712 8325(config-vlan-2712)# interface vlan 2712 8325(config-if-vlan)# description vcf_nsxt_edge_uplink2 8325(config-if-vlan)# ip mtu 9000 8325(config-if-vlan)# ip address 172.27.12.200/24 8325(config-if-vlan)# exit 8325(config)# vlan 1616 8325(config-vlan-1616)# interface vlan 1616 8325(config-if-vlan)# description vcf_mgmt_wld 8325(config-if-vlan)# ip mtu 1500 8325(config-if-vlan)# ip address 172.16.16.200/24 8325(config-if-vlan)# exit 8325(config)# vlan 1617 8325(config-vlan-1617)# interface vlan 1617 8325(config-if-vlan)# description vcf_vMotion_wld 8325(config-if-vlan)# ip mtu 9000 8325(config-if-vlan)# ip address 172.16.17.200/24 8325(config-if-vlan)# exit 8325(config)# vlan 1618 8325(config-vlan-1618)# interface vlan 1618


8325(config-if-vlan)# description vcf_vSAN_wld 8325(config-if-vlan)# ip mtu 9000 8325(config-if-vlan)# ip address 172.16.18.200/24 8325(config-if-vlan)# exit 8325(config)# vlan 1619 8325(config-vlan-1619)# interface vlan 1619 8325(config-if-vlan)# description vcf_nsxt_host_overlay_wld 8325(config-if-vlan)# ip mtu 9000 8325(config-if-vlan)# ip address 172.16.19.200/24 8325(config-if-vlan)# exit 8325(config)# vlan 1620 8325(config-vlan-1620)# interface vlan 1620 8325(config-if-vlan)# description vcf_nsxt_edge_overlay_wld 8325(config-if-vlan)# ip mtu 9000 8325(config-if-vlan)# ip address 172.16.20.200/24 8325(config-if-vlan)# exit 8325(config)# vlan 2721 8325(config-vlan-2721)# interface vlan 2721 8325(config-if-vlan)# description vcf_nsxt_edge_uplink1_wld 8325(config-if-vlan)# ip mtu 9000 8325(config-if-vlan)# ip address 172.27.21.200/24 8325(config-if-vlan)# exit 8325(config)# vlan 2722 8325(config-vlan-2722)# interface vlan 2722 8325(config-if-vlan)# description vcf_nsxt_edge_uplink2_wld 8325(config-if-vlan)# ip mtu 9000 8325(config-if-vlan)# ip address 172.27.22.200/24 8325(config-if-vlan)# exit

2. For application virtual networks in VCF 4.2, you need to configure BGP on the HPE Aruba switches. In the VCF parameter sheet, the BGP Autonomous System ID of the Aruba switch is mentioned as 65001. The edge cluster deployed as part of NSX-T deployment in the VCF management domain has the BGP Autonomous System ID as 65008. Hence, both the BGP Autonomous Systems need to establish peering for North-South communication.

3. Configure the NSX Edge uplinks to BGP peer with ToR Autonomous System ID 65001 as shown in the following BGP configuration.

BGP Configuration for VCF Management Domain NSX-T edge cluster peering:

#8325(config)#router bgp 65001 #8325(config)#bgp router-id 1.1.1.1 #All current BGP sessions in VRF default will be restarted #Do you want to continue(y/n)?y #8325(config-bgp)#neighbor 172.27.11.24 as-number 65008 #8325(config-bgp)#neighbor 172.27.11.24 password plaintext <Password> #8325(config-bgp)#neighbor 172.27.11.34 as-number 65008 #8325(config-bgp)#neighbor 172.27.11.34 password plaintext <Password> #8325(config-bgp)#neighbor 172.27.12.24 as-number 65008 #8325(config-bgp)#neighbor 172.27.12.24 password plaintext <Password> #8325(config-bgp)#neighbor 172.27.12.34 as-number 65008 #8325(config-bgp)#neighbor 172.27.12.34 password plaintext <Password> #8325(config-bgp)#address-family ipv4 unicast #8325(config-bgp-ipv4-uc)#network 172.16.0.0/20 #8325(config-bgp-ipv4-uc)#network 172.27.11.0/24 #8325(config-bgp-ipv4-uc)#network 172.27.12.0/24 #8325(config-bgp-ipv4-uc)#neighbor 172.27.11.24 activate #8325(config-bgp-ipv4-uc)#neighbor 172.27.11.34 activate #8325(config-bgp-ipv4-uc)#neighbor 172.27.12.24 activate #8325(config-bgp-ipv4-uc)#neighbor 172.27.12.34 activate #exit-address-family


4. The edge cluster deployed as part of NSX-T deployment in the VCF workload domain has the BGP Autonomous System ID as 65009. To establish BGP peering between the autonomous systems for North-South communication, configure the VCF workload domain NSX Edge uplinks to BGP peer with ToR Autonomous System ID 65001 as shown in the following BGP configuration.

BGP Configuration for VCF workload domain NSX-T edge cluster peering:

#8325(config)#router bgp 65001 #8325(config)#bgp router-id 1.1.1.1 #All current BGP sessions in VRF default will be restarted #Do you want to continue(y/n)?y #8325(config-bgp)#neighbor 172.27.21.24 as-number 65009 #8325(config-bgp)#neighbor 172.27.21.24 password plaintext <Password> #8325(config-bgp)#neighbor 172.27.21.34 as-number 65009 #8325(config-bgp)#neighbor 172.27.21.34 password plaintext <Password> #8325(config-bgp)#neighbor 172.27.22.24 as-number 65009 #8325(config-bgp)#neighbor 172.27.22.24 password plaintext <Password> #8325(config-bgp)#neighbor 172.27.22.34 as-number 65009 #8325(config-bgp)#neighbor 172.27.22.34 password plaintext <Password> #8325(config-bgp)#address-family ipv4 unicast #8325(config-bgp-ipv4-uc)#network 172.16.16.0/24 #8325(config-bgp-ipv4-uc)#network 172.27.21.0/24 #8325(config-bgp-ipv4-uc)#network 172.27.22.0/24 #8325(config-bgp-ipv4-uc)#neighbor 172.27.21.24 activate #8325(config-bgp-ipv4-uc)#neighbor 172.27.21.34 activate #8325(config-bgp-ipv4-uc)#neighbor 172.27.22.24 activate #8325(config-bgp-ipv4-uc)#neighbor 172.27.22.34 activate #exit-address-family

VMWARE CLOUD FOUNDATION INSTALLATION ON HPE PROLIANT DL SERVER Initiate VMware Cloud Foundation deployment across the four management nodes via VMware Cloud Builder VM and the VCF deployment parameter sheet. For more details, see the VMware Cloud Foundation Deployment Guide.

After the VMware Cloud Foundation is deployed successfully, log in to the SDDC Manager via browser https://<SDDC_Manager_IP>.

HPE STORAGE SOLUTIONS FOR VMWARE CLOUD FOUNDATION WORKLOAD DOMAIN VMware Cloud Foundation supports block storage through IP-based (iSCSI) external storage or FC-based external storage enabling the addition of principal and supplementary storage options for virtual infrastructure (VI) workload domain. The data stores or virtual volume provided via external storage are connected either through iSCSI or FC protocol and managed independently through the workload domain vCenter instance or storage GUI.

This section describes the deployment of HPE Primera and HPE Nimble Storage as principal storage and supplementary storage to a VMware Cloud Foundation 4.2.0.0 VI workload domain on the HPE ProLiant DL Server hardware platform.

HPE storage choices for Virtual Infrastructure workload domain without vSAN VCF supports workload domain deployment on block storage. A workload domain deployed on block storage is termed to be utilizing “VMFS on FC” or “VVols” as principal storage. The term supplementary storage refers to the next storage tier that is managed out of band either through storage-specific GUI or vCenter plugin.

Table 6 represents the choice of options tested as part of this Reference Architecture.

TABLE 6. List of HPE Storage systems validated for VCF workload domain without vSAN

HPE Storage systems for VCF Workload Domain Principal Storage Supplementary Storage

HPE Nimble Fabric-Attach

HPE Nimble Direct-Attach

HPE Nimble iSCSI NA

https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/com.vmware.vcf.ovdeploy.doc_42/GUID-F2DCF1B2-4EF6-444E-80BA-8F529A6D0725.html


HPE Storage systems for VCF Workload Domain Principal Storage Supplementary Storage

HPE Primera Fabric-Attach

HPE Primera Direct-Attach

HPE storage choices for Virtual Infrastructure workload domain with vSAN (HPE ProLiant DL380 Gen10 Internal Storage) If you choose to deploy the workload domain on vSAN, the following options are available as supplementary storage to the workload domain and are managed out of band either through storage-specific GUI or vCenter plugin.

TABLE 7. List of HPE Storage systems validated for VCF workload domain with vSAN as primary storage

vSAN as Principal Storage (Internal storage) Supplementary Storage

HPE Nimble Fabric-Attach

HPE Nimble Direct-Attach

HPE Nimble iSCSI

HPE Primera Fabric-Attach

HPE Primera Direct-Attach

NOTE For the management domain, vSAN primary storage is the only storage choice.

HPE Primera storage for Virtual Infrastructure workload domain HPE Primera is an intelligent storage technology in the form of all-flash or flash/spinning media converged architecture. It provides principal storage as well as supplementary storage options for the Virtual Infrastructure (VI) workload domain. The volume created from HPE Primera storage, connected to an HPE ProLiant DL380 Gen10 chassis through Fibre Channel (FC) connections, can be used to create ‘VMFS on FC’ data stores or ‘virtual volumes’ (VVols) for VI workload domain installation. Other volumes created from HPE Primera storage can be presented as supplementary storage options for the workload domain. To enable VVols as primary storage for the workload domain, the container should be created on HPE Primera and added to the storage details section in SDDC manager. This container will be leveraged during the workload domain deployment as the primary storage.

VMware Cloud Foundation workload domain on HPE ProLiant DL380 Gen10 can have HPE Primera integrated as principal storage and as supplementary storage through Fibre Channel (FC) protocol in either of the following configurations:

• HPE Primera Fabric-Attach Fibre Channel (FC) topology using SAN Switches and HPE SN1600Q 32Gb 2p FC HBA

• HPE Primera Direct-Attach Fibre Channel (FC) topology using HPE SN1600Q 32Gb 2p FC HBA


Figure 14 shows the block diagram for HPE Primera storage options for the VI workload domain.

FIGURE 14. HPE Primera storage options for VMware Cloud Foundation v4.2.0.0 workload domain

HPE Primera 4 Node Storage

HPE ProLiant DL Server

Workload domain 1Management domain

vSAN storage

Mgmt/Workload

NSX Managers

Mgmt vCenter server

Workload vCenter server

Edge VMs VM VM VM

HPE Primera Storage ArrayHPE Primera Storage Array

vSAN storage HPE Primera Storage Array

Workload domain 2

Edge VMs VM VM VM

Principal Storage

Supplementary Storage

Edge VMs


HPE Primera storage Fabric-Attach Fibre Channel topology using SAN Switches and HPE HPE SN1600Q 32Gb 2p FC HBA Figure 15 shows a cabling example of Fabric-Attached SAN networking using HPE StoreFabric SN6600B 32Gb Fibre Channel switches, HPE ProLiant DL380 Gen10 with HPE SN1600Q 32Gb 2p FC HBA, and HPE Primera A630. Hewlett Packard Enterprise recommends cabling odd-numbered Primera ports to one switch and even-numbered ports to the partner switch. At the minimum, two ports per component must be used to enable continuous server operation during the failure of a switch, Fibre cable, or server Fibre port and enable continuous operation during a firmware update to SAN switches.

FIGURE 15. Example for fabric-attach SAN network topology with HPE Primera storage

UID

iLO

1 44 1

2

1

3

5

4

6 TOP

PORT

2

PORT

1

TX RX TX RX

PCIe

FC

HPE SN1600Q 32Gb 2p FC HBA PORT-1

HPE StoreFabric SN6600B Fibre Channel Switch

HPE ProLiant DL380 Gen10 Compute Nodes

SAN BSAN A

HPE Primera630

ServiceMGMTSlot-3

12Gb SASDP-2DP-1

Slot-0

10GbE

Port-1 Port-1 UID

10GbE

Slot-1 Slot-2

Slot-4

HPE Primera630

ServiceMGMTSlot-3

12Gb SASDP-2DP-1

Slot-0

10GbE

Port-1 Port-1 UID

10GbE

Slot-1 Slot-2

Slot-4

UID

501X2(16)

800WPCBM

UID

501X2(16)

800WPCBM

1 2 3 4

UID

32GbFC

1 2 3 4

UID

32GbFC

1 2 3 4

UID

32GbFC

1 2 3 4

UID

32GbFC

40 51 62 73 128 139 1410 1511 3632 3733 3834 3935 4440 4541 4642 4743

48 49 50 51 56 57 58 59

52 53 54 55 60 61 62 632016 2117 2218 2319 2824 2925 3026 3127

1 3

0 2


40 51 62 73 128 139 1410 1511 3632 3733 3834 3935 4440 4541 4642 4743

48 49 50 51 56 57 58 59

52 53 54 55 60 61 62 632016 2117 2218 2319 2824 2925 3026 3127

1 3

0 2


SAN ASAN B

HPE Primera A6302N Controller



HPE Primera storage Direct-Attach (Flat SAN) Fibre Channel topology using HPE SN1600Q 32Gb 2p FC HBA Figure 16 shows the cabling example of HPE Primera storage connected directly to HPE ProLiant DL380 Gen10 Server. In this solution, an HPE Primera is connected directly to port 1 and port 2 of HPE SN1600Q 32Gb 2p FC HBA located on the HPE ProLiant DL380 Gen10 Server.

FIGURE 16. Example of direct-attach SAN network topology with HPE Primera storage

For more information on the deployment of HPE Primera as principal storage to VMware Cloud Foundation workload domain, see https://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50002347enw.

UID

iLO

1 44 1

2

1

3

5

4

6 TOP

PORT

2

PORT

1

TX RX TX RX

PCIe

FC



HPE Primera630

ServiceMGMTSlot-3

12Gb SASDP-2DP-1

Slot-0

10GbE

Port-1 Port-1 UID

10GbE

Slot-1 Slot-2

Slot-4

HPE Primera630

ServiceMGMTSlot-3

12Gb SASDP-2DP-1

Slot-0

10GbE

Port-1 Port-1 UID

10GbE

Slot-1 Slot-2

Slot-4

UID

501X2(16)

800WPCBM

UID

501X2(16)

800WPCBM

1 2 3 4

UID

32GbFC

1 2 3 4

UID

32GbFC

1 2 3 4

UID

32GbFC

1 2 3 4

UID

32GbFC

SAN ASAN B

HPE Primera A6302N Controller




HPE Nimble Storage for VMware Cloud Foundation v4.2.0 workload domain HPE Nimble Storage is an intelligent, self-managing storage technology in the form of all-flash and hybrid-flash. It provides principal storage as well as supplementary storage options for the VMware Cloud Foundation (VCF) 4.2.0 workload domain. The volume created from HPE Nimble Storage, connected to HPE ProLiant DL380 Gen10 Server through Fabric-Attached Fibre Channel (FC) connection or Direct-Attached Fibre Channel (FC) connection, can be used for VMFS on FC datastore or VVols to install VCF workload domain. Volume created from HPE Nimble Storage can be presented as principal or supplementary storage options for the VCF workload load domain. To enable VVols as primary storage for the workload domain, the container should be created on HPE Nimble and added to the storage section within the SDDC manager. This container will be leveraged during the workload domain deployment as the primary storage.

Figure 17 shows the block diagram of HPE Nimble Storage options for VMware Cloud Foundation.

• HPE Nimble Fabric-Attach Fibre Channel (FC) topology using SAN Switches and HPE SN1600Q 32Gb 2p FC HBA

• HPE Nimble Direct-Attach topology using HPE SN1600Q 32Gb 2p FC HBA

• HPE Nimble Storage iSCSI topology using Aruba switches and HPE Eth 10/25Gb 2p 640FLR-SFP28 Adapter

FIGURE 17. HPE Nimble Storage options for VMware Cloud Foundation 4.2.0 workload domain

HPE Nimble Storage

Workload domain 1Management domain

vSAN storage

VM VM VM

HPE Nimble Storage ArrayHPE Nimble Storage Array

vSAN storage HPE Nimble Storage Array

Workload domain 2

VM VM VMPrincipal Storage

Supplementary Storage

Nimble Storage

Mgmt vCenter server

Mgmt/Workload

NSX Managers

Workload vCenter server

Edge VMs

Edge VMs

Edge VMs



HPE Nimble Storage Fabric-Attach Fibre Channel topology using SAN switches and HPE HPE SN1600Q 32Gb 2p FC HBA The HPE Nimble Storage 2x 32Gb Fibre Channel 2-port adapter kit in the HPE Nimble Storage HF20 Base Array was leveraged for FC connectivity. The HPE SN1600Q 32Gb 2p FC HBA mounted on the HPE ProLiant DL380 Gen10 Server was connected to two HPE StoreFabric SN6600B Fibre Channel Switches as a Storage Area Network (SAN) Switch.

Figure 18 shows the cabling example of HPE Nimble Storage to SAN fabrics. All the odd-numbered Fibre Channel ports in 2 x 32Gb Fibre Channel 2-port adapter of the HPE Nimble Storage is connected to one SAN fabric and the even-numbered ports to the second SAN fabric. This configuration ensures continued server operation during the failure of a switch, Fibre cable, or server Fibre port and enables continued operation during a firmware update to the SAN fabrics.

FIGURE 18. Example of HPE Nimble Storage Fabric-Attach Fibre Channel (FC) topology

IN

OUT

IN

OUT

PORT

0

PORT

1

T X R X T X R X

FC16

PORT

0

PORT

1

T X R X T X R X

FC16

HPE Nimble Storage HF20 Adaptive Array

HPE StoreFabric SN6600B

Fibre Channel Switch

PORT

0

PORT

1

T X R X T X R X

FC16

PORT

0

PORT

1

T X R X T X R X

FC16

40 51 62 73 128 139 1410 1511 3632 3733 3834 3935 4440 4541 4642 4743

48 49 50 51 56 57 58 59

52 53 54 55 60 61 62 632016 2117 2218 2319 2824 2925 3026 3127

1 3

0 2


40 51 62 73 128 139 1410 1511 3632 3733 3834 3935 4440 4541 4642 4743

48 49 50 51 56 57 58 59

52 53 54 55 60 61 62 632016 2117 2218 2319 2824 2925 3026 3127

1 3

0 2


SAN BSAN A

SAN ASAN B

UID

iLO

1 44 1

2

1

3

5

4

6 TOP

PORT

2

PORT

1

TX RX TX RX

PCIe

FC





HPE Nimble Storage Direct-Attach topology using HPE HPE SN1600Q 32Gb 2p FC HBA Figure 19 shows the cabling example of HPE Nimble Storage connected directly to the HPE SN1600Q 32Gb 2p FC HBA mounted on HPE ProLiant DL380 Gen10 Server.

FIGURE 19. Example of HPE Nimble Storage Direct-Attach Fibre Channel (FC) topology

IN

OUT

IN

OUT

PORT

0

PORT

1

T X R X T X R X

FC16

PORT

0

PORT

1

T X R X T X R X

FC16


PORT

0

PORT

1

T X R X T X R X

FC16

PORT

0

PORT

1

T X R X T X R X

FC16

SAN ASAN B

UID

iLO

1 44 1

2

1

3

5

4

6 TOP

PORT

2

PORT

1

TX RX TX RX

PCIe

FC





HPE Nimble Storage iSCSI topology using Aruba Switches and HPE Eth 10/25Gb 2p 640FLR-SFP28 Adapter Figure 20 shows the cabling diagram of HPE Nimble Storage (iSCSI) with HPE ProLiant DL380 Gen10 Server. The cabling shows that the Nimble storage is connected to HPE ProLiant DL380 Gen10 Server through eth0a and eth0b on each HPE Nimble Storage controller via the HPE Eth 10/25Gb 2p 640FLR-SFP28 Adapter and Aruba 8325 switch. HPE Nimble Storage is configured to use two different networks, a management network of speed 1Gbps and a data network of speed 10Gbps.

FIGURE 20. Example of HPE Nimble Storage (iSCSI) with HPE ProLiant DL Servers

IN

OUT

IN

OUT


Aruba 8325 Switch

4847343332311817161521 Green=10Gbps,Yellow=1Gbps SFP+ QSFP28 Green=100Gbps,Yellow=40/10Gbps 5449 4847343332311817161521 Green=10Gbps,Yellow=1Gbps SFP+ QSFP28 Green=100Gbps,Yellow=40/10Gbps 5449

GRN=

10G

ACT/

LNK

B

GRN=

10G

ACT/

LNK

A

GRN=

10G

ACT/

LNK

B

GRN=

10G

ACT/

LNK

A

UID

iLO

1 44 1

2

1

3

5

4

6 TOP

HPE Eth 10/25Gb 2p 640FLR-SFP28 Adapter Port 1


HPE Eth 10/25Gb 2p 640FLR-SFP28 Adapter Port 2

P2 P1ACT LINK ACTLINK


HPE OneView is used to provision, manage, and administer the HPE Nimble Storage. For more information on the deployment of HPE Nimble Storage as supplementary storage to VMware Cloud Foundation workload domain, see http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50001414enw. Although the referred document is for VCF 3.9.1 the HPE Nimble configuration steps are still applicable for VCF 4.2.0.

See the HPE Nimble Storage documentation to understand the best practices of configuring HPE Nimble Storage at https://infosight.nimblestorage.com/InfoSight/app#documentation (requires HPE Nimble InfoSight login).

DEPLOY VSPHERE LIFECYCLE MANAGER BASELINES WORKLOAD DOMAIN ON HPE PROLIANT DL SERVERS This section illustrates the deployment of vSphere Lifecycle Manager Baselines workload Domain, NSX-T edge cluster, and Kubernetes as the platform for cloud-native applications. VMware Cloud Foundation by default provides an SDDC architecture by virtualizing compute, storage, and network. Using the SDDC Manager, VMware Kubernetes is deployed as a platform on the VMware Cloud Foundation NSX-T workload domain for organizations to deploy cloud-native applications. To support the deployment of Kubernetes the workload domain should be of type ‘Baselines’.

HPE ProLiant DL380 Gen10 comfortably maps all its native components to the VMware SDDC model thus making it an ideal fit for VMware SDDC deployments. HPE ProLiant DL380 Gen10 internal storage hosts software-defined VMware vSAN storage. HPE Eth 10/25Gb 2p 640FLR-SFP28 Adapter and HPE SN1600Q 32Gb 2p FC HBA enable the ProLiant servers to connect to the network and storage.

The following are the deployment steps to configure VMware Cloud Foundation NSX-T workload domain and VMware Kubernetes on HPE ProLiant DL Servers:

1. Deploy and configure a vSphere Lifecycle Manager Baselines workload domain on HPE ProLiant DL Servers.

2. Deploy and configure an NSX-T Edge cluster on the vSphere Lifecycle Manager Baselines workload domain for north-south communication.

3. Deploy and configure the Kubernetes solution on vSphere Lifecycle Manager Baselines workload domain deployed earlier with the NSX-T edge cluster.

Deploy vSphere Lifecycle Manager Baselines workload domain After the VCF management domain deployment is successful, the SDDC Manager is used to bring up the VCF workload domain. Before starting the deployment of the VCF workload domain, ESXi should be installed on HPE ProLiant DL Servers followed by commissioning of hosts onto SDDC manager.

A minimum of three servers is required to form a VCF workload domain and the workload domain storage can be either vSAN based, NFS based, or Fibre Channel based. During the workload domain deployment, the selected vSphere Lifecycle Manager type determines whether the workload domain supports Kubernetes solution or native firmware upgrades.

To deploy the Kubernetes cluster, the workload domain must be deployed with the option ‘vSphere Lifecycle Manager Baselines’. Once vSphere Lifecycle Manager Baselines is enabled, it is not possible to use vSphere Lifecycle Manager (vLCM) cluster images for firmware upgrade.

Deploy NSX-T Edge Cluster VMware NSX-T is designed to address application frameworks and architecture that have heterogeneous endpoints and technology stacks. In addition to vSphere, this environment may include other hypervisors, containers, bare metal, and public clouds. NSX-T allows IT and development teams to choose the technologies best suited for their applications. NSX-T is also designed for management, operations, and consumption by development organizations besides used by IT.


https://infosight.nimblestorage.com/InfoSight/app#documentation


Figure 21 shows the VMware NSX-T architecture.

FIGURE 21. VMware NSX-T architecture

An NSX-T Edge Cluster is a logical grouping of NSX-T Edge nodes. These NSX-T Edge nodes run on a vSphere cluster and provide north-south routing and network services for the management and VI workload. NSX-T data center supports a 2-tier routing model. The top tier is the tier0 logical router. Northbound, the tier-0 logical router connects to one or more physical routers or layer 3 switches and serves as a gateway to the physical infrastructure. Southbound, the tier-0 logical router connects to one or more tier-1 logical routers or directly to one or more logical switches. The bottom tier is the tier-1 logical router. Northbound, the tier-1 logical router connects to a tier-0 logical router. Southbound connects to one or more logical switches, also called segments in the NSX-T data center.

Before deploying the NSX-T edge cluster for the workload domain, the top of rack switches should be configured to BGP peer with NSX-T edge cluster. BGP Autonomous System ID should be configured on the Aruba 8325 Switches and BGP Autonomous System ID configured on NSX-T edge nodes are deployed as part of the VCF workload domain. This is done to establish BGP peering for North-South communication. To support Kubernetes, the use case should be chosen as ‘Workload Management’ when deploying edge cluster.

Deploy Kubernetes solution The vSphere with Kubernetes transforms vSphere to a platform for running Kubernetes workload natively on the hypervisor layer. When enabled on a vSphere cluster, vSphere with Kubernetes provides the capability to run Kubernetes workload directly on ESXi hosts and to create upstream Kubernetes clusters within dedicated resource pools.

VMware Kubernetes is a production-grade, highly-available container runtime that operates on vSphere and public clouds. With built-in intelligence and integration, VMware Kubernetes ties all these open-source and commercial modules together, delivering a simple-to-use product for customers, ensuring the customers have the most efficient Kubernetes deployment and management experience possible. With workload management, it is possible to deploy and operate the compute, networking, and storage infrastructure for vSphere with Kubernetes.



Figure 22 shows the VMware Kubernetes architecture.

FIGURE 22. VMware Kubernetes architecture

For more information on the deployment of vSphere Lifecycle Manager Baselines workload domain, NSX-T edge cluster, and VMware Kubernetes, see the VCF 4.0.1 documentation at http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50003451enw. However, the steps to deploy workload domain, NSX-T edge cluster, and VMware Kubernetes remain the same.

NOTE The NSX-T edge cluster section applies to both the types of VCF workload domain i.e., Kubernetes-based vSphere Lifecycle Manager Baselines workload domain and vSphere Lifecycle Manager Images workload domain.

DEPLOY VSPHERE LIFECYCLE MANAGER IMAGES WORKLOAD DOMAIN ON HPE PROLIANT DL SERVER This section illustrates the deployment of the vSphere Lifecycle Manager Images workload domain on HPE ProLiant DL380 Gen10 Server. During the deployment of the VCF workload domain, the Enable vSphere Lifecycle Manager Images option is selected to support the deployment of the vSphere Lifecycle Manager Images-based workload domain.

VMware ESXi on HPE ProLiant DL Server

VMware ESXi on HPE ProLiant DL Server



Figure 23 shows the selection option for vLCM based workload domain deployment.

FIGURE 23. vLCM based workload domain selection option during VCF workload domain deployment

With vLCM based workload domain, firmware and driver updates are done using HPE OneView for VMware vCenter and VMware vSphere Lifecycle Manager. VMware vSphere Lifecycle Manager provides the functionality to upgrade and patch ESXi. VMware vSphere Lifecycle Manager along with HPE OneView hardware support Manager (HSM) plug-in can also perform server firmware and software updates in the same maintenance window as the ESXi server Operating System updates. HPE OneView HSM is integrated inside HPE OneView for VMware vCenter.

vLCM Cluster image A vLCM desired image definition is a specification of the software, components, vendor add-on, and firmware to be applied on all hosts in vSphere clusters in VMware Cloud Foundation workload domains. Maintaining all hosts in the cluster using a single image assists in achieving a homogenous cluster environment with software and firmware updates in a single maintenance window reducing overall downtime.


vLCM Cluster image components for HPE ProLiant DL Server-based workload domain as shown in Figure 24 consists of the following files as mentioned in Table 8. It is important to note the versions detailed are relevant to VCF 4.2.0.0 and will change depending upon the VCF build versions. For detailed Software and firmware versions, see the software and firmware matrix documentation at https://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50003639enw.

FIGURE 24. vLCM Cluster Image components

TABLE 8. vLCM image components

Name Image file

VMware Base Image VMware-ESXi-7.0.1-17325551-HPE-701.0.0.10.6.3.9-Jan2021.iso

HPE Add-On HPE-701.0.0.10.6.3.9-Jan2021-Addon-depot.zip

VMware Upgrade Pack P42413_001_VUP12A-SPP-VUP12A.2021_0106.20.iso

NOTE Initial imaging of HPE ProLiant DL Servers should only be done using HPE ProLiant DL380 Gen10 Server Custom ISO image for ESXi 7.0u1c as in “VMware-ESXi-7.0.1-17325551-HPE-701.0.0.10.6.3.9-Jan2021.iso” and updated with the patch “VMware-ESXi-7.0U1d-17551050-depot.zip”.

Workflow for deploying vLCM based workload domain and remediating the cluster The workflow starts with the installation of the HPE OneView for vCenter appliance in the management domain of the VMware Cloud Foundation environment and uploading HPE Support Pack for ProLiant to HPE OneView. A cluster image without Support Pack for ProLiant has to be created next which can be used to create the vLCM based workload domain. A single cluster image can be used to create multiple vLCM based workload domains if there is a need to have all the vLCM workload domains at the same desired software specification level. Once vLCM based workload domain is created, it can be remediated after adding the HPE Service Pack for ProLiant to the vLCM based workload domain cluster image.



Figure 25 shows the approach for remediating the VMware Cloud Foundation workload domain cluster with the vLCM desired cluster image.

FIGURE 25. Flow diagram for vLCM based VCF workload domain remediation

Following are the steps to create a vLCM based workload domain and remediate the workload domain cluster with the desired cluster image.

1. Deploy HPE OneView for VMware vCenter in the management domain

2. Populate the vSphere Lifecycle Manager depot.

3. Create a Cluster Image without HPE Support Pack for ProLiant on an empty cluster from the management domain vCenter Server.

4. Import the cluster image into SDDC Manager.

5. Configure Network Pool.

6. Commission Host for vLCM based Workload creation.

7. Create a vLCM based Workload Domain.

8. Enable vSAN performance service.

9. Update Skyline Health.

10. Generate the HPE OneView for VMware vCenter certificate and publish to VCF workload domain vCenter.

11. Add workload domain vCenter to HPE OneView for vCenter

12. Configure the workload domain vCenter Certificate in the HPE OneView.

13. Register HPE OneView IP within HPE OneView for vCenter plug-in.

14. Configure the HPE OneView Hardware Support Manager (HSM) plug-in for VCF workload domain VMware vSphere Lifecycle Manager.

15. Pre-check and remediate the VCF workload domain cluster with the new image created after adding the HPE Support Pack for ProLiant.

Deploying HPE OneView for VMware vCenter in VMware Cloud Foundation Management domain The following steps explain the deployment of HPE OneView for vCenter 10.1.

1. HPE OneView for vCenter 10.1 appliance .ova can be downloaded from https://myenterpriselicense.hpe.com/cwp-ui/evaluation/ONEV_VCENTER/10.x/null (Sign-in credentials using HPE Passport account is required).

2. In the list of possible downloadable files, select “HPE_OneView_for_VMware_vCenter_10.1_October_2020_Z7550-03607.zip” and download it.

3. Extract the zip file to folder as “HPE_OneView_for_VMware_vCenter_10.1_October_2020_Z7550-03607”.

4. Log in to vCenter Server in the VMware Cloud Foundation management domain and right-click the cluster and select “Deploy OVF template” to start the deployment wizard for deploying the appliance.

https://myenterpriselicense.hpe.com/cwp-ui/evaluation/ONEV_VCENTER/10.x/null

https://myenterpriselicense.hpe.com/cwp-ui/evaluation/ONEV_VCENTER/10.x/null


Table 9 shows the details of each screen and the action needed to be performed.

TABLE 9. Deployment wizard actions for each screen

Installation Action Needed

Select an OVF template Select “local files” and point to folder HPE_OneView_for_VMware_vCenter_10.1_October_2020_Z7550-03607” extracted. Select the “OV4VC-10.1.0.15.ova” within the folder to deploy

Select a name and folder Provide the HPE OneView for vCenter Virtual Machine name

Select a compute resource Select the VCF management domain cluster as a destination compute resource for this virtual machine

Review details Check details of the appliance

License agreement Accept the license agreement

Select Storage Select VM storage policy as “vSAN default Storage policy”. Select VCF management domain vSAN storage in the list of storage

Select network HPE OneView for VMware vCenter allows you to configure up to three networks. At least one network needs to be configured during deployment. Configure “Network 1” to VCF management network during deployment. You may configure additional networks for redundancy or if the storage network is on a private network and vCenter on a public network. Configure additional networks from the Administrator Console post-deployment as needed. See Figure 33 for the networking configuration

Customize template Provide IP Address, Subnet Mask, Default gateway, DNS server, and fully qualified domain name for the “Network Settings”

Ready to Complete Review the details for the installation and click Finish to start the installation

Figure 26 shows the network configuration for HPE OneView for VMware vCenter appliance.

FIGURE 26. Network configuration for HPE OneView for VMware vCenter appliance


See the HPE OneView for VMware vCenter 10.1 Installation Guide for further details about installations of HPE OneView for VMware vCenter.

5. Once the HPE OneView for vCenter appliance is deployed, power on the appliance and connect to the Administrator console using the appliance’s fully qualified domain name (FQDN) or IP as https://<<ApplianceHostname/IPaddress>>.

6. Click Setup. Enter a “New Password” and “Confirm password” and click “OK”.

To know more about each of these steps for vLCM based workload domain deployment on HPE ProLiant DL Servers, see https://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50002714enw&#.

NOTE Customers with VMware vCenter 7.0u1c/7.0u1d releases might see errors while validating HPE VUPs in vLCM. VMware has released KB article (83305) to address this issue. https://kb.vmware.com/s/article/83305.

PATCHING AND UPGRADING IN VMWARE CLOUD FOUNDATION SDDC Manager internally has Lifecycle Management (LCM) enabled, which performs automated updates on VMware Cloud Foundation components such as SDDC Manager and its internal services and VMware components such as vCenter Server, VMware ESXi™, vRealize Suite, NSX-T, and VMware vRealize Suite Lifecycle Manager™. SDDC Manager is configured to communicate with the VMware software repository, if the SDDC Manager VM has internet access and the VMware depot credentials are valid. The high-level update workflow is as follows:

1. Receive notification of update availability.

2. Download the update bundle.

3. Select update targets and schedule the update.

However, this section intends to help administrators to understand how to perform VMware Cloud Foundation upgrade when SDDC Manager does not have access to the internet.

Offline VMware Cloud Foundation update This section intends to describe how to upgrade your VMware Cloud Foundation system if the SDDC Manager VM does not have internet access. The document goes into detail about how to use the Bundle Transfer utility to manually download the bundles from the VMware depot on your local computer with internet access and then upload them to an SDDC Manager VM and update your VMware Cloud Foundation system.

When to perform an offline update VMware Cloud Foundation update needs to be performed only after verifying if the underlying HPE Infrastructure including drivers and firmware are compatible with the version that is going to be installed. See the HPE ProLiant DL Server firmware and software compatibility matrix for the VMware Cloud Foundation guide at https://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50003639enw to check if the VCF version is listed as compatible along with the drivers and firmware.

ESXi Cluster-Level and Parallel upgrades Enables customers to update the ESXi software on multiple clusters in the management domain or a workload domain in parallel. Parallel upgrades reduce the overall time required to upgrade the VCF environment.

Bundle types Upgrade bundle An upgrade bundle contains bits to update the appropriate Cloud Foundation software components in your management domain or VI workload domain. In most cases, an upgrade bundle must be applied to the management domain before it can be applied to the workload domain.

Some upgrade bundles are cumulative. With a cumulative upgrade bundle, you can directly upgrade the appropriate software in your workload domain to the version contained in the cumulative bundle rather than applying sequential upgrades to reach the target version. Cumulative bundles are available only for the vCenter Server and ESXi.

https://support.hpe.com/hpesc/public/docDisplay?docId=a00106900en_us&docLocale=en_US

https://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50002714enw&

https://kb.vmware.com/s/article/83305



NOTE You can apply a cumulative bundle to a workload domain only if the target release in the bundle is lower than or at the same version as the management domain. If the cumulative bundle is available for both the management domain and VI workload domain, you must apply it to the management domain before applying it to the VI workload domain.

Install bundle VMware Cloud Foundation includes the following install bundles:

• VI workload domain install bundle is used to deploy new installation of the software components instead of the versions in your original Cloud Foundation installation. It includes software bits for vCenter Server and NSX for vSphere.

• Individual install bundles for vRealize products are used for deploying vRealize components.

• NSX-T install bundle is used for deploying an NSX-T based VI workload domain.

• Horizon 7 install bundle is used for creating a Horizon domain.

For more information on upgrading VMware Cloud Foundation, see https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/vcf-42-lifecycle/GUID-1D002991-ECBF-4F30-8C03-0ABC8A6C48B9.html.

HPE STORAGE INTEGRATION FOR VMWARE VCENTER HPE Storage Integration Pack for VMware vCenter (SIP4VC) is a single, integrated plug-in application for VMware vCenter management. It enables the vSphere administrator to quickly obtain context-aware information about HPE Storage in their VMware vSphere data center directly from within vCenter. This application enables the vSphere administrator to easily manage storage, datastores, and virtual machines. The application provides the ability to clearly view and directly manage the HPE Infrastructure from within the vCenter console, thus increasing the productivity of the VMware administrator and also ensuring the quality of service.

HPE Storage Integration Pack for VMware vCenter offers the following storage hardware management capabilities:

• Provides storage configuration and status information for mapping VMs, datastores, and hosts to LUNs on Hewlett Packard Enterprise Storage systems

• Supports provisioning on HPE Primera and HPE MSA Storage systems. Supported provisioning tasks include creating, expanding, or deleting a datastore, and creating a VM

• Displays view-only information for the HPE StoreOnce Backup systems where applicable

Installing the software 1. Download the OVA file from My HPE Software Center at https://myenterpriselicense.hpe.com/cwp-ui/free-software/HPESIPVC.

2. Deploy the OVA file in VMware vCenter using the available assigned IP addresses defined in DNS, subnet mask, and gateway details.

3. Power ON the OVA.

https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/vcf-42-lifecycle/GUID-1D002991-ECBF-4F30-8C03-0ABC8A6C48B9.html

https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/vcf-42-lifecycle/GUID-1D002991-ECBF-4F30-8C03-0ABC8A6C48B9.html

https://myenterpriselicense.hpe.com/cwp-ui/free-software/HPESIPVC


Storage Integration Pack for VMware vCenter setup 1. Launch the Administrator Console of HPE Storage Integration Pack for VMware vCenter using the appliance hostname or IP from

https://<ApplianceHostname>.

FIGURE 27. HPE Storage Integration Pack for VMware vCenter Console

Adding a vCenter To see the installed plug-in, you need to include a vCenter.

1. Select HPE Storage Integration Pack for VMware vCenter > MANAGERS > vCenters.

2. Click the + Add vCenter button in the left pane.

3. Enter the following details:

a. Name: Enter the vCenter hostname or IPv4 address

b. Username: Enter the vCenter username

c. Password: Enter the vCenter password

4. Click Add.


5. To add multiple vCenters, click Add +.

FIGURE 28. Adding vCenter to HPE Storage Integration Pack for VMware vCenter


Create VVols datastore using SIP4VC 1. Log in to vSphere Client.

2. From the menu drop-down, select HPE Storage > Datastores > VVol.

FIGURE 29. VMware vSphere Client with HPE Storage

3. Click + to create vVols Datastore.


4. On the create VVols Datastore page, select a cluster or host.

FIGURE 30. Select a cluster or host for VVol Datastore

5. Click Next.


6. On the Select Storage page, click New Storage Container and select the required storage system.

FIGURE 31. Select Storage system for VVol Datastore

7. Click Next.


8. On the Specify names page, enter the Datastore Name.

FIGURE 32. Specify the name of VVol Datastore

9. Click Next.


10. On the Summary page, review the details.

FIGURE 33. Summary of VVol Datastore

11. Click FINISH.

HPE PROLIANT DL SERVER FIRMWARE UPGRADE USING HPE ONEVIEW FOR VMWARE VREALIZE ORCHESTRATOR

NOTE The section applies to vSphere Lifecycle Manager Baselines Workload Domain. This type of workload domain uses a VMware vSphere Update Manager (VUM), and vRO workflows can be used to update the firmware.

HPE OneView for VMware vRealize Orchestrator (vRO) helps customers automate complex IT tasks in an extensible and repeatable manner. It provides a predefined collection of HPE OneView tasks and workflows that can be used in vRO with easy-to-use, drag and drop access to the automation of HPE OneView managed hardware deployment, firmware updates, and other life cycle tasks. HPE OneView for VMware vRO allows the advanced management features of HPE OneView to be incorporated into larger IT workflows. HPE OneView workflows and actions can also be integrated with VMware vRealize Automation™ using vRO.


Among the available workflows, IT administrators can use the “Update Cluster Firmware” workflow to update the HPE ProLiant DL Server firmware through HPE OneView and VUP for regular IT operation and maintenance tasks. This section describes the process of updating the compute firmware using the “HPE OneView for VMware vRO” workflow. The steps for updating the HPE ProLiant DL Server firmware using VMware vRealize Orchestrator are as follows:

1. Verify HPE SUT is installed on each of the HPE ProLiant DL Servers. HPE SUT is part of the HPE OEM custom image and is installed by default.

2. Configure VMware vRealize Orchestrator (vRO).

3. Install and configure the HPE OneView for the vRO plugin.

4. Upload SPPs/VUP on the HPE OneView appliance.

5. Configure the vRO clients for the workflows.

HPE OneView for vRealize Orchestrator can be downloaded from My HPE Software center at https://myenterpriselicense.hpe.com/. See http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50000681enw for more detailed information.

We tested the “Update Cluster Firmware” workflow in the Hewlett Packard Enterprise Engineering lab to update the HPE ProLiant DL Servers with VMware Cloud Foundation installed. The following section provides details about the test environment and the process.

Prerequisites • VMware vRealize Orchestrator is installed in the environment

• VMware vRealize Orchestrator vSphere vCenter plug-in is configured with VMware vCenter(s)

• At least one instance of HPE OneView is installed and configured

• An HPE OneView account is created and to be used by HPE OneView Management Instance in VMware vRealize Orchestrator

• VMware vRealize Orchestrator servers have network access to HPE OneView

• VMware Cloud Foundation software is installed and configured on HPE ProLiant DL Servers

• HPE OneView Server Profile template is configured for baseline VMware Upgrade Pack (VUP)

• The vRO workflow will be used to update the new VUP on the HPE ProLiant DL Server available and configure them as baselines for the Server Profile template

Configure HPE iSUT on each of the HPE ProLiant DL Server nodes HPE iSUT is already installed when HPE custom image is used to build ESXi OS on HPE ProLiant DL Server nodes. Set the SUT mode to “Autodeploy” using the following command:

#sut –set mode=Autodeploy

Testing the workflow In the Hewlett Packard Enterprise Engineering lab environment, the following VMware Upgrade Pack (VUP) was downloaded. Table 10 lists the firmware used in testing.

TABLE 10. Firmware used in testing

Name Version Size

1.2(A) SPP-VUP12A.2020_1204.19.iso 2020_1204.19 5.66GB

The firmware of the HPE ProLiant DL Servers was updated to firmware version “VUP1.2a” using the workflow “Update cluster Firmware” present in the OV4VRO workflow.

https://myenterpriselicense.hpe.com/



The following information was necessary to start the “Update cluster Firmware” workflow:

• vCenter Cluster – The vCenter Cluster on which the HPE ProLiant DL Server host firmware would be upgraded is shown in Figure 34.

FIGURE 34. Choose the vCenter Cluster in the vRO client


• HPE OneView Firmware Bundle –The version to which the firmware update would happen is shown in Figure 35.

FIGURE 35. HPE OneView Firmware Bundle selection

6. Once the workflow is started, it validates the firmware version and HPE OneView server and starts the update process.

FIGURE 36. OneView window showing new firmware baseline being applied


7. Once the task is completed successfully, on the workflow logs tab, verify that Task completed without errors appears.

FIGURE 37. Workflow logs


8. Once the firmware update is complete, the iLO firmware update state shows as Completed.

FIGURE 38. HPE ProLiant DL380 Gen10 iLO showing the new firmware applied successfully

NOTE Certain firmware components need a manual reboot for the complete installation of the firmware.

MANAGING AND MONITORING LARGE SCALE HPE PROLIANT DL SERVER BASED INFRASTRUCTURE FOR VMWARE CLOUD FOUNDATION In an environment where multiple instances of VCF are run on multiple HPE ProLiant DL Server Infrastructure in geographically spread data centers, management and monitoring become a challenging task for cloud administrators. HPE OneView Global Dashboard and VMware Multi-Instance Management feature together provide a unified view of all resources managed by HPE OneView and VCF SDDC Manager. The HPE OneView Global Dashboard and VMware Multi-Instance Management feature enable administrators to monitor the component health, simplify operations, optimize resources, and increase productivity.

HPE OneView Global Dashboard overview The HPE OneView Global Dashboard provides customers with a unified view of the health of their HPE ProLiant DL Servers, profiles, and HPE Nimble storage systems deployed in multiple data centers across the globe. HPE OneView Global Dashboard supports up to 75 HPE OneView appliances and 20,000 servers which enables IT staff to troubleshoot alerts and view core inventory data.

The HPE OneView Global Dashboard provides the following functionalities:

• Aggregates critical activities from multiple appliances into a single feed to quickly identify issues occurring on monitored hardware for prompt resolution.

• Enables granular reporting to view inventory and verify firmware versions so that equipment meets the organization’s compliance standards.

• Identifies duplicate resources promptly in your environment based on their serial number and World Wide Identifier (WWID).

• Enables IT staff to quickly find devices across data centers by their attributes in seconds enabling faster decision-making by using Smart Search, a real search feature.


• Enables drill down to the problem resources and remediation using an easy filter mechanism.

• Restricts system access to authorized users only using Role-based access control (RBAC)

HPE OneView Global Dashboard deployment The HPE OneView Global Dashboard can be downloaded from My HPE Software Center and deployed as an Open Virtual Appliance (OVA) using vCenter Server. The high-level steps are listed as follows. For detailed steps, see the HPE OneView Global Dashboard user guide at https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=emr_na-a00082863en_us.

1. Download the OVA file to a local directory.

2. Start the vSphere client and log into vCenter.

3. From the menu, select File > Deploy OVF Template to launch the Deploy OVF Template wizard.

4. On the Source screen, specify the following:

a. The path of the OVA file

b. Verify the information on the OVF Template Details screen

c. On the Name and Location screen, enter a name and location for the deployed template that is unique within the inventory folder and then select the inventory location

d. On the Host/Cluster screen, select the host or cluster on which you want to run the deployed template

e. On the Storage screen, select a storage destination for the VM files

f. On the Disk Format screen, select the format for storing the virtual disks. Thick Provisioning is highly recommended.

g. On the Network Mapping screen, select a network that has connectivity to your management LAN

h. On the Ready to complete screen, verify your selection. Use the Back button to make changes, if necessary. Click Finish to complete the template and start the deployment. When the deployment is complete, the VM is available for use

i. Before powering on the VM, configure the VM to synchronize time with the hypervisor host.

5. On the Summary tab, click Power on. Click the Console tab to watch the appliance start.

Configuring the HPE OneView Global Dashboard After the HPE OneView Global Dashboard is deployed as a virtual machine, configure it as follows:

1. Using a VMware console or web browser, access the HPE OneView Global Dashboard.

2. On the HPE OneView Global Dashboard EULA page, review the end-user agreement, and click Agree to continue. If you click Disagree, the installation process is canceled.

https://myenterpriselicense.hpe.com/cwp-ui/free-software/Z7550-63371

https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=emr_na-a00082863en_us


3. On the Login screen, log in as an administrator on the local domain to configure HPE OneView Global Dashboard for the first time.

FIGURE 39. OneView Global Dashboard login window

4. Add HPE OneView appliance.


5. On the Appliances page, select the menu icon and click Add.

FIGURE 40. Add HPE OneView Appliance to Global Dashboard


6. Specify the IP address and password of the remote appliance and click Add.

FIGURE 41. Specify the IP address for the remote HPE OneView appliance


7. On the Certificate page, click Trust.

FIGURE 42. Certificate page


8. The HPE OneView appliance gets connected and added to the HPE OneView Global Dashboard.

FIGURE 43. HPE OneView appliance added to the HPE OneView Global Dashboard


9. On the HPE OneView Global Dashboard, you can now see the discovered appliances, enclosures, storage, servers, and alerts. For example, clicking the appliances will display the HPE ProLiant DL Servers OneView Appliance connected to the HPE OneView Global Dashboard.

FIGURE 44. Dashboard showing discovered resources

VMware Cloud Foundation Multi-Instance Management With VCF 4.2, VMware introduced the Multi-Instance management feature which is a single pane of glass to manage and monitor multiple cloud foundation instances. This feature enables cloud admins to monitor VCF instances deployed in data centers spread across multiple geographies. Cloud administrators can use this data for capacity planning (compute, network, and storage) and maintain compliance with IT security regulations. Multi-Instance works by grouping individual instances into a federation. The instance from which the admin initiates the creation of the federation is designated as the controller and other instances that join the federation are referred to as general members. A controller can invite other instances to become members of the federation and promote a general member to a controller. A minimum of three controllers is required for high availability. All the federation members will publish their local data to a common message bus. Using a global dashboard, all the federated members can get an aggregated view of the resources and health status of the entire federation and drill down further. For more information on the Multi-Instance Management, see https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/vcf-42-introducing/GUID-26F1DF9F-AEF4-48C8-9F27-ECC904798DFC.html.

https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/vcf-42-introducing/GUID-26F1DF9F-AEF4-48C8-9F27-ECC904798DFC.html

https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/vcf-42-introducing/GUID-26F1DF9F-AEF4-48C8-9F27-ECC904798DFC.html


Create federation 1. On the SDDC Manager, click the Multi-Instance Management icon.

FIGURE 45. VCF SDDC Manager Dashboard

2. Click Create a Federation.

FIGURE 46. Multi-Instance Management page


3. On the Create Federation page, fill in the required details, and click Create.

FIGURE 47. Create Federation page


4. It takes a few minutes to create the federation.

FIGURE 48. Federation is created

5. Once the federation is created, you can invite members by clicking the Invite Members button.

FIGURE 49. Click Invite Member


6. On the Invite Member page, enter the fully qualified domain name (FQDN) of the instance you are inviting to join the federation and click Next.

FIGURE 50. Invitation sent to members to join the federation


7. On the High Availability page, if you want to make this member a controller, click the Designate this member as a controller checkbox.

FIGURE 51. High Availability page


8. On the Instructions for a new member page, click Copy Info and send it to the member you want to invite to the federation.

FIGURE 52. Instruction for a new member


9. The invited member can then use the URL to join the federation.

FIGURE 53. Join the federation


10. On the Federation window, you can see both instances. The green status shows that both instances are communicating.

FIGURE 54. Federation status showing both the data centers are live

11. On further drill down, you can see the number of management/workload domains, resource (CPU, memory, and storage), and software update details.

FIGURE 55. Shows the status of the resource utilization


CONSOLIDATED ARCHITECTURE DEPLOYMENT OF VMWARE CLOUD FOUNDATION VMware Cloud Foundation can be deployed as either a standard architecture model or a consolidated architecture model. In the standard architecture model, there is a dedicated management domain that hosts infrastructure virtual machines and at least one workload domain to host user workloads. It also requires a minimum of seven (7) servers, four (4) for the management domain, and three (3) for the workload domain. In the consolidated architecture model, the management and customer workload VMs are part of the same domain or cluster and it leverages vSphere resource pools to provide isolation between management and user workloads.

The consolidated architecture model targets smaller Cloud Foundation deployments, special use cases, and can be deployed starting with four (4) hosts or higher. Management and user workload VMs run together in the same vSphere cluster and the environment is managed from a single vCenter. vSphere resource pools provide isolation between the management and user workload.

Figure 56 shows the VMware Cloud Foundation consolidated architecture deployed on HPE ProLiant DL Servers.

FIGURE 56. VMware Cloud Foundation consolidated architecture deployed on HPE ProLiant DL Servers

4 x HPE ProLiant DL Server


NOTE VCF consolidated deployment can support up to 64 hosts (to the limits of vCenter). However, this solution is tested with four to six hosts. HPE automation also enables four to six hosts for consolidated architecture (targeting Proof-of-Concept scenario and use cases for ROBO, SMB deployments).

For more information on VMware Cloud Foundation consolidated architecture, see https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/vcf-42-introducing/GUID-61453C12-3BB8-4C2A-A895-A1A805931BB2.html.

SUMMARY Hewlett Packard Enterprise and VMware deliver the highly scalable VCF software-defined solution on HPE ProLiant DL Servers. The VCF solution can be effectively and rapidly deployed on HPE ProLiant DL Servers. Using VMware vLCM and vRO, this solution supports effective and reliable firmware life cycle management. vLCM can help the organization to maintain compliance by easily detecting firmware and drivers drift across the ESXi hosts in the cluster and swiftly remediating them. The following use cases were validated with VCF 4.2 on HPE ProLiant DL Servers:

• VCF management domain creation and deployment of SDDC Manager

• VCF workload domain commissioning and deployment through SDDC Manager

• Deployment of Kubernetes on workload domain

• Upgrade of firmware on the HPE ProLiant DL Servers using VMware vRO and vLCM Management of HPE storage from vSphere using HPE storage plugins

APPENDIX A: BILL OF MATERIALS Table A1 lists the hardware utilized for testing and developing the Reference Architecture.

NOTE Part numbers are at the time of publication and subject to change. The bill of materials does not include complete support options or complete rack and power requirements. For questions regarding ordering, consult with your HPE Reseller or HPE Sales Representative for more details. hpe.com/us/en/services/consulting.html.

TABLE A1. Bill of materials

Product Qty Product description

Rack and power

P9K10A 1 HPE 42U 600mmx1200mm G2 Enterprise Shock Rack

P9K40A 001 1 HPE Factory Express Base Racking Service

H6J85A 1 HPE Rack Hardware Kit

120672-B21 1 HPE Rack Ballast Kit

BW932A 1 HPE 600mm Rack Stabilizer Kit

BW932A#B01 1 HPE 600mm Rack include with Complete System Stabilizer Kit

P9S21A 2 HPE G2 Metered/Switched 3Ph 14.4kVA/CS8365C 40A/208V Outlets (12) C13 (12) C19/Vertical NA/JP PDU

Aruba Switches

JL762A 2 Aruba 6300M 48-port 1GbE and 4-port SFP56 Power-to-Port 2 Fan Trays 1 PSU Bundle

JL762A ABA 2 Aruba 6300M 48G Pwr2Prt 2F 1PS Bdl US en

J9283D 4 Aruba 10G SFP+ to SFP+ 3m Direct Attach Copper Cable

R0M46A 2 Aruba 50G SFP56 to SFP56 0.65m Direct Attach Copper Cable

https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/vcf-42-introducing/GUID-61453C12-3BB8-4C2A-A895-A1A805931BB2.html

https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/vcf-42-introducing/GUID-61453C12-3BB8-4C2A-A895-A1A805931BB2.html

http://www.hpe.com/us/en/services/consulting.html


Product Qty Product description

J9583B 2 Aruba X414 1U Universal 4-post Rack Mount Kit

JL627A 2 Aruba 8325-32C 32-port 100G QSFP+/QSFP28 Back-to-Front 6 Fans and 2 Power Supply Bundle

JL627A ABA 2 Aruba 8325-32C 32-port 100G QSFP+/QSFP28 Back-to-Front 6 Fans and 2 Power Supply Bundle US en

721064-B21 6 HPE BladeSystem c-Class 40G QSFP+ to 4x10G SFP+ 3m Direct Attach Copper Splitter Cable

R0Z25A 4 Aruba 100G QSFP28 to QSFP28 1m Direct Attach Copper Cable

R0Z26A 8 Aruba 100G QSFP28 to QSFP28 5m Direct Attach Copper Cable

JL483B 2 Aruba X474 4-post Rack Kit

HPE ProLiant DL Servers

868704-B21 4 HPE ProLiant DL380 Gen10 Server

867959-B21 4 HPE ProLiant DL360 Gen10 Server

P02499-L21 8 Intel Xeon-Gold 5220 (2.2GHz/18-core/125W) FIO Processor Kit for HPE ProLiant DL380 Gen10

P19888-B21 8 HPE 1.92TB SATA 6G Mixed Use SFF SC SM883 SSD

P19888-B21 8 HPE 240GB SATA 6G Read Intensive M.2 2280 5300B SSD

P21125-B21 8 HPE 400GB SAS 12G Write Intensive SFF SC SS540 SSD

P9M76A 8 HPE SN1600Q 32Gb Dual Port Fibre Channel Host Bus Adapter

Q0F09A 8 HPE CN1300R 10/25Gb Dual Port Converged Network Adapter

804338-B21 8 HPE Smart Array P816i-a SR Gen10 (16 Internal Lanes/4GB Cache/SmartCache) 12G SAS Modular Controller

867334-B21 8 HPE Ethernet 10/25Gb 2-port FLR-SFP28 QL41401-A2G Converged Network Adapter

Storage

Q8H72A 1 HPE Nimble Storage HF20 Adaptive Dual Controller 10GBASE-T 2-port Configure-to-order Base Array

Q8B69B 1 HPE Nimble Storage HF20/20C Adaptive Array 42TB (21x2TB) FIO HDD Bundle

Q8G27B 1 HPE Nimble Storage NOS Default FIO Software

Q8J30A 1 HPE Nimble Storage HF20 Adaptive Array R2 5.76TB (6x960GB) FIO Cache Bundle

R4G78A 2 HPE Nimble Storage 2x32Gb 2-port Fibre Channel FIO Adapter Kit

Q8J27A 2 HPE Nimble Storage C13 to C14 250V 10Amp 1.8m Universal FIO Power Cord

R3P91A 1 HPE Nimble Storage AF/HF Array Standard Tracking

N9Z46A 1 HPE Primera 600 2-way Storage Base

N9Z55A 1 HPE Primera A630 2-node Controller

581817-B21 1 HPE Configurator Defined Build Instruction Option

R1P32A 1 HPE Primera 600 without Installation Service

R3R39A 6 HPE Primera 600 1.92TB SAS SFF (2.5in) FIPS Encrypted SSD

N9Z39A 2 HPE Primera 600 32Gb 4-port Fibre Channel Host Bus Adapter

716195-B21 2 HPE External 1.0m (3ft) Mini-SAS HD 4x to Mini-SAS HD 4x Cable

N9Z50A 1 HPE Primera 600 2U 24-disk SFF Drive Enclosure

R3R39A 6 HPE Primera 600 1.92TB SAS SFF (2.5in) FIPS Encrypted SSD

NOTE The above BOM contains US localization (ABA is for the US), Customers must choose localization options based on the deployment location.


KNOWN ISSUES • Host commissioning will fail with self-signed certificates. Refer to https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/vcf-42-

ovdeploy-guide.pdf and follow the steps in the Regenerate the Self-Signed certificate on All Hosts section to regenerate self-signed certificates for ESXi hosts.

• In the VMware vCenter with 7.0 U1c or 7.0 U1d version, you cannot synchronize the HPE OneView for VMware vCenter repository for the VMware vSphere Lifecycle Manager updates. To resolve this issue, refer to the VMware KB article https://kb.vmware.com/s/article/83305.

• Management and workload domains bring up will fail if servers have 1Gb NIC Cards. During bring-up VCF expects at a minimum two 10GbE NIC ports enabled as vmnic0 and vmnic1. Any other onboard 1Gbe NICs should be disabled in the server BIOS to allow for proper nic enumeration during the initial deployment. Failing to disable the 1Gb NIC cards on servers hosting the management and workload domain will result in bring-up failure. We have disabled the 1Gb NIC cards using the BIOS setting available in the Server Profile Template. You may re-enable the 1Gb NIC cards post VCF deployment.

• Host addition fails with an error “HOST must have only one standard switch”. VCF expects only one standard switch before the VSS to vDS migration occurs. Virtual NIC can be Disabled/Enabled through iLO RESTful interface and iLO web UI. After the domain is up, Virtual NIC can be re-enabled.

URLS FOR FIRMWARE, SOFTWARE, AND DOCUMENTATION Rack and power links • HPE Rack and Power Infrastructure, https://www.hpe.com/us/en/integrated-systems/rack-power-cooling.html

Network links • Networking documentation page, https://asp.arubanetworks.com/

• Aruba 6300 Switch documentation page, https://h20195.www2.hpe.com/v2/default.aspx?cc=ie&lc=en&oid=1012138130

• Aruba 8325 Switch documentation page, https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=a00065472en_us

Storage links • HPE SN6600B Fibre Channel switches, https://www.hpe.com/us/en/product-catalog/storage/storage-switches/pip.hpe-storefabric- sn6600b-

fibre-channel-switch.1009472119.html

• HPE Primera 600 Storage, https://buy.hpe.com/us/en/storage/disk-storage-systems/primera-storage/primera-storage/hpe-primera-600-storage/p/1011657880 storage.8737813.html

• HPE Nimble Storage, https://www.hpe.com/us/en/storage/nimble.html

Server links • HPE ProLiant DL servers, https://www.hpe.com/us/en/servers/proliant-dl-servers.html

• SPP Documentation, http://www.hpe.com/info/spp/documentation

Software links • HPE Primera, https://h20272.www2.hpe.com/SPOCK/Pages/spock2Html.aspx?htmlFile=sw_array_primera.html

– HPE Storage Documentation, http://www.hpe.com/info/storage/docs

• HPE B-series SAN Network Advisor, http://h20565.www2.hpe.com/hpsc/swd/public/readIndex?sp4ts.oid=5401278

• HPE OneView

– Software, https://myenterpriselicense.hpe.com/cwp-ui/free-software/Z7550-63180

– Documentation, http://www.hpe.com/info/oneview/docs



https://kb.vmware.com/s/article/83305

https://www.hpe.com/us/en/integrated-systems/rack-power-cooling.html

https://asp.arubanetworks.com/

https://h20195.www2.hpe.com/v2/default.aspx?cc=ie&lc=en&oid=1012138130

https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=a00065472en_us

https://www.hpe.com/us/en/product-catalog/storage/storage-switches/pip.hpe-storefabric-%20sn6600b-fibre-channel-switch.1009472119.html

https://www.hpe.com/us/en/product-catalog/storage/storage-switches/pip.hpe-storefabric-%20sn6600b-fibre-channel-switch.1009472119.html

https://buy.hpe.com/us/en/storage/disk-storage-systems/primera-storage/primera-storage/hpe-primera-600-storage/p/1011657880

https://buy.hpe.com/us/en/storage/disk-storage-systems/primera-storage/primera-storage/hpe-primera-600-storage/p/1011657880

https://www.hpe.com/us/en/product-catalog/storage/disk-storage/pip.hpe-3par-storeserv-8000-storage.8737813.html

https://www.hpe.com/us/en/storage/nimble.html

https://www.hpe.com/us/en/servers/proliant-dl-servers.html

http://www.hpe.com/info/spp/documentation

https://h20272.www2.hpe.com/SPOCK/Pages/spock2Html.aspx?htmlFile=sw_array_primera.html

http://www.hpe.com/info/storage/docs

http://h20565.www2.hpe.com/hpsc/swd/public/readIndex?sp4ts.oid=5401278

https://myenterpriselicense.hpe.com/cwp-ui/free-software/Z7550-63180

http://www.hpe.com/info/oneview/docs


• HPE OneView Partner Integrations, http://www.hpe.com/info/ovpartners

• HPE Recovery Manager Central (RMC)

– Software, https://myenterpriselicense.hpe.com/cwp-ui/free-software/HPERMC

– Product Information, http://www.hpe.com/storage/rmc/docs

• HPE VMware OEM Software Delivery Repository, http://vibsdepot.hpe.com

• HPE OEM customized VMware image downloads, https://www.hpe.com/us/en/servers/hpe-esxi.html

VMware links – VMware Cloud Foundation 4.2.0, https://docs.vmware.com/en/VMware-Cloud-Foundation/4.2/rn/VMware-Cloud-Foundation-42-Release-

Notes.html

– VMware Cloud Foundation Planning and Preparation Guide, https://docs.vmware.com/en/VMware-Cloud-Foundation/index.html

– VMware vRealize Orchestrator 8.2.0, https://docs.vmware.com/en/vRealize-Orchestrator/8.2/com.vmware.vrealize.orchestrator-install-config.doc/GUID-64F03876-2EAB-4DB3-A95D-89842425FF7A.html

– VMware product patch find by release name, https://www.vmware.com/patchmgr/findPatchByReleaseName.portal

http://www.hpe.com/info/ovpartners

https://myenterpriselicense.hpe.com/cwp-ui/free-software/HPERMC

http://www.hpe.com/storage/rmc/docs

http://vibsdepot.hpe.com/

https://www.hpe.com/us/en/servers/hpe-esxi.html



https://docs.vmware.com/en/VMware-Cloud-Foundation/index.html

https://docs.vmware.com/en/vRealize-Orchestrator/8.2/com.vmware.vrealize.orchestrator-install-config.doc/GUID-64F03876-2EAB-4DB3-A95D-89842425FF7A.html

https://docs.vmware.com/en/vRealize-Orchestrator/8.2/com.vmware.vrealize.orchestrator-install-config.doc/GUID-64F03876-2EAB-4DB3-A95D-89842425FF7A.html

https://www.vmware.com/patchmgr/findPatchByReleaseName.portal

Reference Architecture

© Copyright 2021 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.

VMware, VMware vSphere, VMware NSX, VMware vRealize Suite are registered trademarks of VMware, Inc. in the United States and/or other jurisdictions. VMware Cloud Foundation, VMware vCenter, VMware ESXi, VMware vSAN, VMware NSX-T Data Center. vRealize Operations Manager, vRealize Automation, vRealize Log Insight are the trademark of VMware, Inc. in the United States and/or other jurisdictions.

a50004095enw, version 1.0, April 2021

RESOURCES AND ADDITIONAL LINKS HPE Reference Architectures, https://www.hpe.com/docs/reference-architecture

HPE Servers, hpe.com/servers

HPE Storage, hpe.com/storage

HPE Networking, hpe.com/networking

HPE GreenLake Advisory and Professional Services, hpe.com/us/en/services/consulting.html

To help us improve our documents, please provide feedback at hpe.com/contact/feedback.

https://www.hpe.com/docs/reference-architecture

http://www.hpe.com/servers

http://www.hpe.com/storage

http://www.hpe.com/networking

http://www.hpe.com/us/en/services/consulting.html

http://www.hpe.com/contact/feedback

Documents

HPE Reference Architecture for VMware Cloud Foundation on