c02616817

V

PF

Virtua

Part number c02First edition Nov

l Con

2616817 vember 2010

nect F

FlexFabbric CCookboook

Legal notices Never leverage legal and notice information. For the most current requirements, refer to HP Standard 004-0 Legal Notices for Product Documentation, Promotional Materials, and Software and Media located at http://standards.inet.cpqcorp.net/smc/hpstd/hpstds.htm

http://standards.inet.cpqcorp.net/smc/hpstd/hpstds.htm

Purpose 3

Contents

Purpose ................................................................................................................................................. 5

Introduction to Virtual Connect FlexFabric ................................................................................................. 6

Tunneled VLAN and Mapped VLANS ..................................................................................................... 10

Single Domain/Enclosure Scenarios ....................................................................................................... 13 Overview .............................................................................................................................................. 13

Requirements .................................................................................................................................... 13

Scenario 1 – Simple vNet with Active/Standby Uplinks and FCoE – Windows 2008 R2 .............................. 14 Overview .............................................................................................................................................. 14

Requirements .................................................................................................................................... 14 Installation and configuration................................................................................................................... 16

Switch configuration .......................................................................................................................... 16 Review ................................................................................................................................................. 23

Results – Windows 2008 R2 Networking Examples ............................................................................... 24 Results – Windows 2008 R2 SAN Connectivity..................................................................................... 28

Summary .............................................................................................................................................. 29

Scenario 2 –Share Uplink Sets with Active/Active uplinks and 802.3ad (LACP) - Windows - Ethernet and FCoE .......................................................................................................................................................... 30

Overview .............................................................................................................................................. 30 Requirements .................................................................................................................................... 30

Installation and configuration................................................................................................................... 32 Switch configuration .......................................................................................................................... 32

Review ................................................................................................................................................. 42 Results – Windows 2008 R2 Networking Examples ............................................................................... 43 Results – Windows 2008 R2 SAN Connectivity..................................................................................... 47

Summary .............................................................................................................................................. 48

Scenario 3 – Share Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - Windows - Ethernet and FCoE Boot from SAN .................................................................................................................................... 49




Summary .............................................................................................................................................. 68

Scenario 4 – Share Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - Windows Hyper-V - Ethernet and FCoE SAN .................................................................................................................................... 69




Summary .............................................................................................................................................. 96

Purpose 4

Scenario 5 – Share Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - vSphere - Ethernet and FCoE SAN ................................................................................................................................................... 97



Review ............................................................................................................................................... 112 Results – vSphere Networking Examples ............................................................................................. 112 Results – vSphere SAN Connectivity .................................................................................................. 115

Summary ............................................................................................................................................ 116

Scenario 6 – Tunneled VLANs and Share Uplink Set with Active/Active Uplinks - vSphere Ethernet and FCoE SAN ................................................................................................................................................. 117

Overview ............................................................................................................................................ 117 Requirements .................................................................................................................................. 117

Installation and configuration................................................................................................................. 119 Switch configuration ........................................................................................................................ 119

Review ............................................................................................................................................... 130 Results – vSphere Networking Examples ............................................................................................. 131 Results – vSphere SAN Connectivity .................................................................................................. 134

Summary ............................................................................................................................................ 135

Appendix A – Scenario-based CISCO Command Line Reference ............................................................. 136

Appendix B – Scenario-based ProCurve Command Line Reference ........................................................... 139

Appendix C: Acronyms and abbreviations ............................................................................................ 142

Appendix D: Useful VC CLI Command sets ............................................................................................ 144

Purpose 5

Purpose

The purpose of this Virtual Connect FlexFabric Cookbook is to provide users of Virtual Connect with a better understanding of the concepts and steps required when integrating HP BladeSystem and Virtual Connect FlexFabric components into an existing network.

The scenarios in this Cookbook vary from simplistic to more complex while covering a range of typical building blocks to use when designing Virtual Connect FlexFabric solutions. Although these scenarios are shown individually, some scenarios could be combined to create a more complex and versatile Virtual Connect environment, however, keeping in mind the difference between mapped and tunneled VLANs, discussed later in this paper are mutually exclusive.

The scenarios as written are meant to be self-contained configurations and do not build on earlier scenarios, with this you may find some repetition or duplication of configuration across scenarios.

This is not meant to be a complete or detailed guide to Virtual Connect FlexFabric, but is intended to provide the reader with some valid examples of how Virtual Connect FlexFabric could be deployed within their environments. Many additional configurations or scenarios could also be implemented. Please refer to the following section for additional reference material on Virtual Connect, Flex-10 and FlexFabric.

Introduction to Virtual Connect FlexFabric 6

Introduction to Virtual Connect FlexFabric

Virtual Connect is an industry standard-based implementation of server-edge virtualization. It puts an abstraction layer between the servers and the external networks so the LAN and SAN see a pool of servers rather than individual servers (Figure 1). Once the LAN and SAN connections are physically made to the pool of servers, the server administrator uses Virtual Connect management tools (Virtual Connect Enterprise Manager or Virtual Connect Manager) to create an Interconnect modules connection profile for each server.

Virtual Connect FlexFabric is an extension to Virtual Connect Flex-10 which leverages the new Fibre Channel over Ethernet (FCoE) protocols. By leveraging FCoE for connectivity to existing Fibre Channel SAN networks, we can reduce the number of switch modules and HBAs required within the server blade. This in turn further reduces cost, complexity, power and administrative overhead.

It is assumed that the reader has experience with and is familiar with Virtual Connect Flex-10. For additional information on Virtual Connect and Flex-10, please review the documents below. The Chapter on Flex-10 in the Virtual Connect Ethernet Cookbook, listed below, will be an excellent primer for this paper.

Additional Virtual Connect Reference Material Link to HP Virtual Connect technology site, provides a great deal of reference information on Virtual connect, Flex-10 and FlexFabric.

http://h18000.www1.hp.com/products/blades/virtualconnect/

Link to HP Virtual Connect technology for the HP BladeSystem c-Class, 4th edition when available

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00814156/c00814156.pdf

Link to HP Virtual Connect for c-Class BladeSystem Setup and Installation Guide

http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c01732252/c01732252.pdf

Link to HP Flex-10 technology


Link to HP BladeSystem Network Reference Architecture - FlexFabric and VMware vSphere 4

http://h20195.www2.hp.com/v2/GetPDF.aspx/c02505638.pdf

Link to Virtual Connect User, Setup and CLI Guides

http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&lang=en&cc=us&docIndexId=64180&taskId=101&prodTypeId=3709945&prodSeriesId=3794423

HP Virtual Connect FlexFabric Solutions Recipe


http://h18000.www1.hp.com/products/blades/virtualconnect/




http://h20195.www2.hp.com/v2/GetPDF.aspx/c02505638.pdf

http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportM



Virtual Connect Ethernet Cookbook Virtual Connect can be used to support both Ethernet and Fibre Channel connections. The Virtual Connect Ethernet Cookbook is provided with basic Virtual Connect configurations for both 1Gb and 10Gb solutions, whereas the Virtual Connect FlexFabric Cookbook provides additional scenarios that leverage the new FCoE protocols. If you are new to Virtual Connect it is highly recommended that you also review the existing Virtual Connect Ethernet Cookbook

Virtual Connect Ethernet (with Flex-10) Channel Cookbook http://h20000.www2.hp.com/bc/docs/support/SupportManual/c01990371/c01990371.pdf

(www.hp.com/go/blades)

Virtual Connect Fibre Channel Cookbook Virtual Connect can be used to support both Ethernet and Fibre Channel connections; however, this guide is focused completely on the Ethernet configuration.

For Fibre Channel connectivity, please refer to the Virtual Connect Fibre Channel Cookbook http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c01702940/c01702940.pdf

(www.hp.com/go/blades)

Virtual Connect 3.15 Firmware Release Virtual Connect firmware version 3.15 is the initial release to provide Fibre Channel over Ethernet (FCoE) support within Virtual connect. FCoE support is provided within the FlexFabric Modules discussed in the document.

Please refer to the Virtual Connect 3.15 Release note for further details on new features and resolved issues.

NOTE: When VC 8Gb 20-port FC module or VC FlexFabric 10Gb/24-port module Fibre Channel uplink ports are configured to operate at 8Gb speed and connecting to HP B-series (Brocade) Fibre Channel SAN switches, the minimum supported version of the Brocade Fabric OS (FOS) is v6.3.1 and v6.4.x. In addition, a fill word on those switch ports must be configured with option “Mode 3” to prevent connectivity issues at 8Gb speed.

On HP B-series (Brocade) FC switches use command portCfgFillWord (portCfgFillWord <Port#> <Mode>) to configure this setting:

Although this setting only affects devices logged in at 8G, changing the mode is disruptive regardless of the speed the port is operating at. The setting is retained and applied any time an 8G device logs in. Upgrades to FOS v6.3.1 or v6.4 from prior releases supporting only modes 0 and 1 will not change the existing setting, but a switch or port reset to factory defaults with FOS v6.3.1 or v6.4 will be configured to Mode 0 by default. The default setting on new units may vary by vendor. Please use portcfgshow CLI to view the current portcfgfillword status for that port.

Mode Link Init/Fill Word

Mode 0 IDLE/IDLE

Mode 1 ARBF/ARBF

Mode 2 IDLE/ARBF

Mode 3 If ARBF/ARBF fails use IDLE/ARBF


http://www.hp.com/go/blades


http://www.hp.com/go/blades


Modes 2 and 3 are compliant with FC-FS-3 specifications (standards specify the IDLE/ARBF behavior of Mode 2 which is used by Mode 3 if ARBF/ARBF fails after 3 attempts). For most environments, Brocade recommends using Mode 3, as it provides more flexibility and compatibility with a wide range of devices. In the event that the default setting or Mode 3 does not work with a particular device, contact your switch vendor for further assistance. When connecting to Brocade SAN Switches at 8Gb, “portCfgFillWord” must be set to Mode 3 – If ARBF/ARBF fails use IDLE/ARBF. In order to use Mode 3, FOS v6.3.1 or v6.4.x or better is required. Virtual Connect FlexFabric Hardware Overview

Virtual Connect FlexFabric is the first Data Center Bridging (DCB) and Fibre Channel over Ethernet (FCoE) solution introduced into the HP BladeSystem portfolio. It provides 24 line rate ports, Full-Duplex 240Gbps bridging, single DCB-hop fabric. As shown in Image 1, there are 8 faceplate ports. Ports X1-X4 are SFP+ transceiver slots only; which can accept a 10Gb or 4Gb/8Gb SFP+ transceiver. Ports X5-X8 are SFP and SFP+ capable which can accept 1Gb or 10Gb Ethernet connections, and do not support Fibre Channel transceivers.

Virtual connect FlexFabric Uplink Port Mappings In it is important to note how the external uplink ports on the FlexFabric module are configure. The graphic below outlines the type and speed each port can be configured as. Ports X1 – X4; Can be configured as 10Gb Ethernet or Fibre Channel, FC speeds supported =

2Gb, 4Gb or 8Gb using 4Gb or 8Gb FC SFP modules, please refer to the FlexFabric Quick Spec for a list of supported SFP modules

Ports X5 – X8: Can be configured as 1Gb or 10Gb Ethernet Ports X7 – X8; Are also shared as internal cross connect Uplink ports X1-X4 support 0.5–5m length DAC as stacking or uplink Uplink Ports X5-X8 support 0.5–7m length DAC as stacking or uplink

Note; 5m DAC cables are supported on all ports with FlexFabric, in addition, 7m DAC cables are also supported on ports X5 through X8.

Note: The CX-4 port provided by the Virtual Connect Flex-10 and legacy modules has been depreciated.

Note: Even though the Virtual Connect FlexFabric module supports Stacking, stacking only applies to Ethernet traffic. FC uplinks cannot be consolidated, as it is not possible to stack the FC ports, nor provide a multi-hop DCB bridging fabric today.

Fig

Noto E Figcon

ure 1 - FlexF

te: Even thouEthernet traffic

ure 2 - FlexNnfigured as Eth

abric Module

ugh the Virtua. FC uplinks

NIC Connectiohernet, FCoE o

e port configu

al Connect Flecannot be co

ons – It is impor iSCSI. Phy

ration, speeds

exFabric modunsolidated, as

portant to noteysical Function

Introducti

s and types

ules support Ss it is not poss

e that Physicalns 1, 3 and 4

ion to Virtual C

Stacking, stacsible to stack

l Function two4 are assigned

Connect FlexFab

king only appthe FC ports.

o (pf2) can bed as Ethernet o

bric 9

plies

e only

Tunneled VLAN and Mapped VLANS 10

Tunneled VLAN and Mapped VLANS

Virtual Connect provides two Ethernet networks connection methods. Both of these connection types are discussed within the following scenarios.

Tunnel vs. Map VLAN tags setting It is important to note that the behavior of both vNets and Shared Uplink Sets is dependent on whether VLAN Tunnel or Map VLAN Tags is set. Server VLAN Tagging Support, as configured in the “Advanced Ethernet Settings” tab of Virtual Connect is a Domain wide configuration.

If Virtual Connect is set to Tunnel Mode, you can do the following; Create a Shared Uplink Set – which can support several VLANs up to the publish limits. These VLANs can be presented to a Server NIC, one at a time – No multiple VLANS supported,

frames are presented to the NIC untagged Create a vNet – which can support both TAGGED and UNTAGGED frames, if tagged the host

system will need to interpret those tags

If Virtual Connect is set to Map VLAN Tags Mode, you can do the following; Create a Shared Uplink Set – (the behavior of a SUS changes and now provides the ability to

connect multiple networks to a NIC) which can support several VLANs up to the publish limits. These VLANs can be presented to a Server NIC, as either a single Network (where VC will

remove the tags and present an untagged frame to the NIC), or as multiple Networks, where VC will present all frames with their VLAN tags, in which case the host system will need to interpret the tags (one network could be configured as untagged)

Create a vNet – (the behavior of a vNet also changes) a vNet can now only support UNTAGGED frames, which means a vNet could then only support ONE VLAN/network

vNet There are two modes of operation for the vNet. The first is a simple vNet that will pass untagged frames. The second is a vNet tunnel which will pass tagged frames for one or many VLANs.

The vNet is a simple network connection between one or many server NICs to one or many uplink ports

A vNet could be used to connect a single VLAN, no tagging, to one or many server NICs, if this network is part of a VLAN, by configuring the upstream switch port as an access or untagged port, by extension, any server connected to this vNet would reside in that VLAN, but would not need to be configured to interpret the VLAN tags.

A tunneled vNet will pass VLAN tagged frames, without the need to interpret or forward those frames based on the VLAN tag. Within a tunneled vNet the VLAN tag is completely ignored by Virtual Connect and the frame is forwarded to the appropriate connection (server NIC[s] or uplinks) depending on frame direction flow. In this case, the end server would need to be configured to interpret the VLAN tags. This could be a server with a local operating system, in which the network stack would need to be configured to understand which VLAN the server was in, or a virtualization host with a vSwitch supporting multiple VLANs.

The tunneled vNet has no limit to the number of VLANs it can support.


Benefits of a vNet If no VLAN support is required, support for a single specific VLAN being presented as untagged or many VLANs need to be presented to the server a vNet is a very simple network to configure and manage within Virtual Connect.

A vNet can be utilized in one of two ways, a simple vNet, used to pass untagged frames and a tunneled vNet. A tunneled vNet can be used to pass many VLANs without modifying the VLAN tags, functioning as a transparent VLAN Pass-Thru module.

Shared Uplink Set (SUS) The SUS provides the ability to support VLAN tagging and forward frames based on the VLAN tags of those frames. The SUS connects one or many server NICs to one or many uplink ports. A SUS would be configured for the specific VLANs it will support. If support for additional VLANs is required, those VLANs need to be configured within the SUS.

When connecting a server NIC to a network within a SUS, there are two choices provided. The key difference between these two options is the state in which the frame is passed to the server NIC;

1. Select a single network – which would be mapped to a specific VLAN.

If a single network is selected, the frames will be presented to the server NIC WITHOUT a VLAN tag. In this case the host operating system does not need to understand which VLAN it resides in. When the server transmits frames back to VC, those frames will not be tagged, however; Virtual Connect will add the VLAN tag and forward the frame onto the correct VLAN.

2. Select multiple networks – which would provide connectivity to several VLANs.

The Map VLAN Tags feature provides the ability to use a Shared Uplink Set to present multiple networks to a single NIC. If you select Multiple Networks when assigning a Network to a server NIC, you will have the ability to configure multiple Networks (VLANS) on that server NIC. At this point VC tags ALL the packets presented to the NIC — unless the Native check box is selected for one of the networks, in which case packets from this network (VLAN) will be untagged, and any untagged packets leaving the server will be placed on this Network (VLAN).

With Mapped VLAN Tags, you can create a Shared Uplink Set that contains ALL the VLANs you want to present to your servers, then present only ONE network (the one associated with the VLAN we want the server NIC in) to the Windows, LINUX or the ESX Console NIC, then select Multiple Networks for the NIC connected to the ESX vSwitch and select ALL the networks that we want presented to the ESX host vSwitch. The vSwitch will then break out the VLANs and present them to the guests. Using Mapped VLAN Tags minimizes the number of uplinks required.

In order to utilize the Multiple Networks feature of Virtual Connect, the Map VLAN Tags feature, needs to be turned on under the Ethernet Settings/Advanced tab within the Virtual Connect manager or the Virtual Connect CLI.

Benefits of a SUS A Shared Uplink Set can be configure to support both tagged and un-tagged network traffic to a server NIC, which simplifies the overall configuration and minimizes the number of uplink cables required to support the network connections.


Virtual Connect VLAN Support SUS - Restrictions and limitations Shared Uplink Sets provide administrators with the ability to distribute VLANs into discrete and defined Ethernet Networks (vNet.) These vNets can then be mapped logically to a Server Profile Network Connection allowing only the required VLANs to be associated with the specific server NIC port. This also allows the flexibility to have various network connections for different physical Operating System instances (i.e. VMware ESX host and physical Windows host.)

The following Shared Uplink Set rules apply per domain: 320 VLANs per Virtual Connect Ethernet module. Note; Every VLAN on every uplink counts

towards the 320-VLAN limit. If a Shared Uplink Set is comprised of multiple uplinks *from the same module*, each VLAN on that Shared Uplink Set is counted multiple times towards 320 limit.

128 VLANs per Shared Uplink Set (64 VLAN Support is provided in VC firmware older than 2.30)

28 VLANs per Ethernet server profile connection regardless of physical vs. FlexNIC connections. On Flex-10 adapters, this amounts to 28 x 4 = 112 VLANs per physical 10Gb port. Note; only applies when in Mapped VLAN mode.

The above configuration rules apply only to a Shared Uplink set. If support for a larger numbers of VLANs is required, the VC domain can be set to Tunneled VLAN mode and a VLAN Tunnel con be configured to support a large number of VLANs.

Please see the Virtual Connect 3.15 Release Notes for future details.

S

O

R

Single

OverviThisSingBayenclrequthat Win

RequiremThisFlexin se10GChacon

Paircon

It is andexac700Inter

FiguFlex

e Dom

iew Cookbook w

gle HP BladeSs 1 and 2. Vlosures, whichuirements. Ea configurationndows and/o

ments Cookbook w

xFabric modulerver Bay 1. Gb FlexFabricannel over Ethnect to the Fle

r of managed nected to the

assumed that no VC Netwmples are pro00 enclosure wrconnect module

ure 3 - c700xFabric modul

main/E

will provide sevSystem c7000Virtual Connech provides a sach scenario wn and include r VMware vSp

will utilize a sines installed in The server wi adapter porternet (FCoE).

exFabric modu

Ethernet switcFlexFabric mo

t a Virtual Conworks, uplink seovided in Appwith Half Heighte bays 1& 2.

00 enclosure wes in Intercon

Enclos

veral configur0 enclosure wict FlexFabric asingle point ofwill provide a both GUI andphere will also

ngle HP Bladen I/O Bays 1 ill connect to ts. Each Flex Port 1 will cule in Bay 2.

ches, as well odules.

nnect Domainets or Server

pendix C. t G7 BladeSyst

with Half Heignect module b

ure Sc

ration scenariith two Virtualalso supports Mf managemenn overview ofd CLI (scriptedo be provided

eSystem c700and 2 and a the FlexFabric

xFabric Adaptonnect to the

as a pair of B

n has been creProfiles have

em servers and

ght G7 BladeSbays 1& 2

Singl

cenari

ios of Virtual Cl Connect FlexMulti-Enclosur

nt and can furf the configurad) methods. Wd.

00 enclosure w BL460c G7 hc models in Bater port suppo FlexFabric mo

Brocade Fibre

eated either thbeen created

d two Virtual Co

System servers

e Domain/Enc

os

Connect FlexFxFabric modure stacking, forther reduce caation, show hWhere possib

with TWO Virhalf height Blaays 1 and 2, orts Ethernet aodule in Bay

e Channel swi

hrough the GU. Virtual Con

onnect FlexFab

s and two Vir

losure Scenario

Fabric, using ules installed ior up to 4 able connectiow to comple

ble, examples

rtual Connect adeSystem Sethrough with nd iSCSI or F1 and Port 2

tches are

UI or a CLI scnnect scripting

bric modules in

rtual Connect

os 13

a n

vity ete for

rvers two

Fibre will

ript g

Scenario 1 – Simple vNet with Active/Standby Uplinks and FCoE – Windows 2008 R2 14

Scenario 1 – Simple vNet with Active/Standby Uplinks and FCoE – Windows 2008 R2

Overview This simple configuration uses the Virtual Connect vNet along with FCoE for SAN connectivity. When VLAN mapping is not required, the vNet is the simplest way to connect Virtual Connect to a network and server. In this scenario, the upstream network switch connects a network to a single port on each FlexFabric module. In addition, Fibre Channel uplinks will also be connected to the FlexFabric modules to connect to the existing Fibre Channel infrastructure.

No special upstream switch configuration is required as the switch is in the factory default configuration, typically configured as an Access or untagged port on either the default VLAN or a specific VLAN, either way, Virtual Connect does not receive VLAN tags.

When configuring Virtual Connect, we can provide several ways to implement network fail-over or redundancy. One option would be to connect TWO uplinks to a single vNet; those two uplinks would connect from different Virtual Connect modules within the enclosure and could then connect to the same or two different upstream switches, depending on your redundancy needs. An alternative would be to configure TWO separate vNets, each with a single, or multiple, uplink configured. Each option has its advantages and disadvantages. We will review the first option in this scenario.

In addition, several vNets can be configured to support the required networks to the servers within the BladeSystem enclosure. These networks could be used to separate the various network traffic types, such as iSCSI, backup, VMotion from production network traffic.

This scenario will also leverage the Fibre Channel over Ethernet (FCoE) capabilities of the FlexFabric modules. Each Fibre channel fabric will have one uplink connected to each of the FlexFabric modules.

Requirements This scenario will support both Ethernet and fibre channel connectivity. In order to implement this scenario, an HP BladeSystem c7000 enclosure with one or more server blades and TWO Virtual Connect FlexFabric modules, installed in I/O Bays 1& 2 are required. In addition, we will require ONE or TWO external Network switches. As Virtual Connect does not appear to the network as a switch and is transparent to the network, any standard managed switch will work with Virtual Connect. The Fibre Channel uplinks will connect to the existing FC SAN fabrics. The SAN switch ports will need to be configured to support NPIV logins. One uplink from each FlexFabric module will be connected the existing SAN fabrics.

Figuthe fswit

TheX1

ure 4 - Physfirst network stch.

SAN fabric ito SAN switc

Scenario 1 –

sical View; switch, a sing

s also connech/fabric B.

– Simple vNet w

Shows a single uplink from

cted as Modu

with Active/Sta

gle Ethernet upm Port X5 on M

le 1 port X1 t

ndby Uplinks a

plink from PorModule 2 to P

o SAN switch

and FCoE – Wi

rt X5 on Modort 1 on the s

h/fabric A an

ndows 2008 R

ule 1 to Port 1second netwo

d module 2 p

R2 15

1 on rk

port

I

S

V

FignetwEtheconthe

Installa

Switch cAppCiscapp

ThisConis NThe the sEdgallowThe

VC CLI coIn aa Cthe a

ure 5 - Logwork switch aernet uplinks annects to the e existing SAN

ation a

configurpendices A anco IOS or a Prpendices assum

The switch pVLAN or a sAs an alternVLANS, Virtnetwork, hoand the vNeneed to be c

scenario assnnect domain

NOT configure network switcswitch as an ae, it allows thws a fail-over SAN switch p

mmands ddition to theLI command sactive VCM.

Scenario 1 –

ical View; Snd a single uare connectedexisting SAN F Fabric B thro

and co

ation nd B provide aroCurve netwmes the followports are confspecific VLANnative, if the swtual Connect wwever; the Viet would needconfigured to umes the swit could be coned for VLAN Tch port shouldaccess device

he switch to pl link or newerports connecti

GUI many ofset. In order t Once logged

– Simple vNet w

Shows a singleplink from Pord to the same Fabric A throuough port X1 o

nfigura

a summary of ork infrastruct

wing informatiigured as AC

N and will be fwitch ports wewould forwardrtual Connect

d to be config interpret thosch port is configured for eth

Tunneling andd be configuree and not anolace the port ir connected ping to the Flex

f the configurato connect to Vd in, VC provi

with Active/Sta

e Ethernet uplrt X5 on Mod vNet, vNet-PRugh port X1 oon Module 2

ation

the commandture. The confion: CESS or untaforwarding unere configuredd those tagge domain wouured for VLANe VLAN tags.

nfigured as anher Tunneled the frames aed for Spannither switch. Binto a forwardport to come oxFabric modu

ation settings VC via a CLI, ides a CLI wit

ndby Uplinks a

link from Port ule 2 to the seROD. In add

on Module 1 ( (Bay 2).

ds required tofiguration info

gged ports, entagged framd as TRUNK p

ed frames to thld need to be

N Tunneling. . n Access port or Mapped Vre presented tng Tree EdgeBy configuringding state muconline much qle must be co

within VC ca open an SSHh help menus

and FCoE – Wi

X5 on Moduecond networition, SAN FaBay 1) and FC

o configure theormation prov

either presenties ports and forwhe host NICs e configured in The connecte

(not tagging),VLAN mode, hto Virtual Con

e as Virtual Cog the port as Sch quicker thauicker. nfigured to ac

n be also be H connection t. The Virtual

ndows 2008 R

le 1 on the firrk switch. Botabric FCoE_A CoE_B conne

e switch in eitided in the

ng the Defaul

warding multiconfigured fon “Tunnel Moed host would

, the Virtual however, the vnnect as untagonnect appeaSpanning Trean otherwise,

ccept NPIV lo

accomplishedto the IP addr Connect CLI g

R2 16

rst th cts to

ther a

lt

ple or this ode” then

vNet gged rs to e this

ogins.

d via ress of guide


also provides many useful examples. Throughout this scenario the CLI commands to configure VC for each setting are provided.

Configuring the VC module Physically connect Port 1 of Network switch 1 to Port X5 on the VC module in Bay 1. Physically connect Port 1 of the second Network switch to Port X5 of the VC module in Bay 2, if

you have only one network switch, connect VC port 1 (Bay 2) to an alternate port on the same switch. This will NOT create a network loop and Spanning Tree is not required

Physically connect Port X1 on the FlexFabric module (Bay 1) to switch port in SAN Fabric A Physically connect Port X1 on the FlexFabric module (Bay 2) to switch port in SAN Fabric B

Configuring Fast MAC Cache Failover When an uplink on a VC Ethernet Module that was previously in standby mode becomes active, it can take several minutes for external Ethernet switches to recognize that the c-Class server blades must now be reached on this newly active connection. Enabling Fast MAC Cache Failover forces Virtual Connect to transmit Ethernet packets on newly

active links, which enables the external Ethernet switches to identify the new connection (and update their MAC caches appropriately). This transmission sequence repeats a few times at the MAC refresh interval (five seconds is the recommended interval) and completes in about one minute.

Configuring the VC Module for VLAN Tunneling and Fast Mac Cache Fail-over via GUI (Ethernet settings)

Virtual Connect tunnel mode can be enabled to provide the ability to “tunnel” an unlimited number of tagged VLANs through a Virtual Connect vNet. This can be very using in a network environment that utilizes a large number of VLANs. Note; for this imply scenario, the Domain mode could be configure for either Map VLAN tags or Tunnel mode.

Enable Tunnel VLAN Tags within Virtual Connect On the Virtual Connect Manager screen, Left pane, click Ethernet Settings, Advanced Settings Select Tunnel VLAN Tags Select Fast MAC Cache Fail-over with a refresh of 5 Select Apply

Configuring the VC Module for VLAN Tunneling and Fast Mac Cache Fail-over via CLI (Ethernet settings)

The following command can be copied and pasted into an SSH based CLI session with Virtual Connect; # Set Advanced Ethernet Settings to "Tunnel VLAN Tags" and Enable Fast MAC cache fail-over set enet-vlan vlantagcontrol=Tunnel (click apply) Click the “Other” tab set mac-cache Enabled=True Refresh=5 (click apply)

Figu

Fig

ure 6 - Confi

ure 7 - Set Fa

Scenario 1 –

gure Tunnel M

ast MAC Cac

– Simple vNet w

Mode (under E

che (under Eth

with Active/Sta

Ethernet Settin

hernet Settings

ndby Uplinks a

ngs “Advance

s “Advanced

and FCoE – Wi

ed Settings)

Settings - Oth

ndows 2008 R

her)

R2 18


Defining a new vNet via GUI Create a vNet and name it “vNet-PROD” Login to Virtual Connect, if a Domain has not been created, create it now, but cancel out of the

configuration wizards after the domain has been created. On the Virtual Connect Manager screen, click Define, Ethernet Network to create a vNet Ether the Network Name of “vNet-PROD”

Note; Do NOT select any of the options (ie; Smart Link, Private Networks etc.) Select Add Port, then add the following ports;

Enclosure 1, Bay 1, Port X5

Enclosure 1, Bay 2, Port X5 Leave Connection Mode as Auto Select Apply

Note: By connecting TWO Uplinks from this vNet we have provided a redundant path to the network. As each uplink originates from a different VC module, one uplink will be Active and the second will be in Standby. This configuration provides the ability to lose an uplink cable, network switch or depending on how the NICs are configured at the server (teamed or un-teamed), even a VC module.

Note: Smart Link – In this configuration Smartlink should NOT be enabled. Smartlink is used to turn off downlink ports within Virtual Connect, if ALL available uplinks to a vNet are down. We will use Smartlink in a later scenario.

Defining a new vNet via CLI The following command(s) can be copied and pasted into an SSH based CLI session with Virtual Connect # Create the vNet "vNet-PROD" and configure uplinks as discussed above add Network vNet-PROD add uplinkport enc0:1:X5 Network=vNet-PROD speed=auto add uplinkport enc0:2:X5 Network=vNet-PROD speed=auto set network vNet-PROD SmartLink=Disabled

D

D

Figinfoas b

Defining aCrea

Crea

Defining SThe Con#CrAddAdd

ure 8 - Definormation. If thbelow

a new (FCate a Fabric a

On the VirtuEther the NeSelect Add P

Select Apply

ate a second On the VirtuEther the NeSelect Add P

Select Apply

SAN Fabri following comnnect reate the SANd fabric FCoE_d fabric FCoE_

Scenario 1 –

ne Ethernet Nhe connected

oE) SAN and name it “ual Connect Metwork Name Port, then add

Enclosure 1, y

Fabric and naual Connect Metwork Name Port, then add

Enclosure 1, y

ics via CLmmand(s) can

N Fabrics FCoE_A Bay=1 Po_B Bay=2 Por

– Simple vNet w

etwork (vNet-switch suppor

Fabric viaFCoE_A”

Manager scree of “FCoE_A”d the following

Bay 1, Port X

ame it “FCoE_Manager scree of “FCoE_B” d the following

Bay 2, Port X

I n be copied a

E_A and FCorts=1 rts=1

with Active/Sta

-PROD). Noterts LLDP, the c

a GUI

en, click Defin” g ports;

1

_B” en, click Defin g ports;

1

nd pasted into

E_B and conf

ndby Uplinks a

: The Port Staconnected to i

ne, SAN Fabr

ne, SAN Fabr

o an SSH bas

figure uplinks

and FCoE – Wi

tus and Connnformation sh

ic to create th

ic to create th

sed CLI sessio

as discussed

ndows 2008 R

nected to hould be displ

he first Fabric

he second Fab

on with Virtual

above

R2 20

layed

bric

l

DE

Figuport

Figuserv

Defining aEach server N

On Prio

NotNIC

ure 9 – FCoEt numbers on

ure 10 – FCovers powered

a Server PNIC will con

the Virtual CoCreate a serIn the NetwoIn the NetwoExpand the Do not confiIn the Assign

r to applying

te: you shouldCs 1&2 should

Scenario 1 –

E SAN fabricsthe right

oE SAN Serveon)

rofile withnect to a spe

onnect Managrver profile caork Port 1 droork Port 2 droFCoE Connecigure FC SANn the Profile to the profile, e

d now have ad be connecte

– Simple vNet w

s configured w

er Connection

h NIC andecific networ

ger screen, clialled “App-1” op down box,op down box,ctions box, fo

N or iSCSI Coo a Server Bansure that the

a server profiled to network

with Active/Sta

with a single 4

ns (After serve

d FCoE Cork.

ick Define, Se select “vNet- select “vNet-r Bay 1, selecnnection y, select Bay server in Bay

e assigned to vNet_PROD a

ndby Uplinks a

4Gb uplink pe

er profiles hav

onnections

erver Profile to

-PROD” -PROD” ct FCoE_A for

1 and apply y 1 is currently

Bay 1, with 2and FCoE SAN

and FCoE – Wi

er fabric. No

ve been create

s, via GUI

o create a Ser

r Bay 2, select

y OFF

2 Server NIC N fabrics FCo

ndows 2008 R

ote the bay an

ed, assigned

I

rver Profile

t FCoE_B

connections. oE_A and FCo

R2 21

nd

and

oE_B.

DDefining aThe Con# Caddset eset eset fset fassi Figu

Notthe LOM

a Server P following comnnect reate and Ass

d profile App-1enet-connectioenet-connectiofcoe-connectiofcoe-connectiogn profile Ap

ure 11 - Defin

te: The speed FCoE connect

M:2-b.

Scenario 1 –

rofile withmmand(s) can

sign Server Pr1 on App-1 1 pxon App-1 2 pxon App-1:1 Faon App-1:2 Fap-1 enc0:1

e Server Profile

of the NIC antions are assig

– Simple vNet w

h NIC andn be copied a

rofile App-1 to

xe=Enabled Nxe=Disabled Nabric=FCoE_Aabric=FCoE_B

e (App- 1)

nd SAN conngned to the tw

with Active/Sta

d FCoE Cond pasted into

o server bay 1

Network=vNeNetwork=vNeA SpeedType=B SpeedType=

nections, as wwo SAN fabric

ndby Uplinks a

onnectionso an SSH bas

1

et-PROD et-PROD =4Gb =4Gb

well as the MAcs created ea

and FCoE – Wi

s, via CLI sed CLI sessio

AC and WWNarlier and use

ndows 2008 R

on with Virtual

N. Also, note ports LOM:1-

R2 22

l

that -b and

R

Figu

Review

In thModoneFab Weto sguaenceac The Addincr

ure 12 - Serve

w his scenario wdule, by doin

e uplink will bebrics, utilizing

e created at seustain a link o

arantees that Alosure. Alternh vNet, provi

FCoE SAN fa

ditional uplinkrease perform

Scenario 1 –

er Profile View B

we have createg so we provie active and t a single uplin

erver profiles, or module failALL server to snatively, we cding an Activ

abric connect

ks could be adance and/or

– Simple vNet w

Bay 1

ed a simple vide redundanthe alternate unk each.

with two NICure and not loserver commuould have cre

ve/Active upli

s to each SAN

dded to either availability.

with Active/Sta

vNet utilizing ut connections uplink will be

Cs connected ose connectionications witheated a two vnk scenario.

N fabric over

r the San fabr

ndby Uplinks a

uplinks origin out of the Vir in standby. W

to the same von to the netwhin the enclosuNet and assig

a single uplin

rics or the Ethe

and FCoE – Wi

ating from thertual Connect We create tw

vNet, which pork; this confiure will remaigned uplinks f

nk per module

ernet network

ndows 2008 R

e each FlexFa domain, whe

wo FCoE SAN

rovides the abiguration alsoin inside the from one mod

e.

ks, which coul

R2 23

abric ere

bility o

dule to

d

RResults –We haveFCo

AlthG7 they

Figu3 Et

The YouNICwe serv

NotG6

Figu

Notcan

– Windo have successe created a se

oE connections

ough both Eth servers; eachy appear in th

ure 13 - Examhernet persona

following grau will also notiCs are currentdid not requir

ver would then

e: the BL465c and BL680c G

ure 14 - Wind

te: The NICs t go into Netw

Scenario 1 –

ows 200sfully configurerver profile tos created with

hernet and Fib capability (LAe server as in

mple of Emulex'alities and one F

aphics show aice that Windly configured re SAN connen have 8 NIC

c G7 and BL6G7 utilizes an

dows 2008 R2

that are not cowork Connecti

– Simple vNet w

08 R2 Ned FlexFabrico connect to thin the profile

bre channel cAN and SANndividual netw

s OneManageFCoE personali

a Windows 2dows believes within FlexFaectivity on this

C ports availab

685c G7 utilizn NC553i chi

Network Conn

onfigured withions for the W

with Active/Sta

Networkc with a simplethe vNet with .

onnectivity is N) is provided work and SAN

r Utility (formerity per port, as

008 R2 serve there are 6 N

abric, the extras server, the Fble to the OS.

ze a NC551i ipset.

nections (2 Con

hin VC will apWindows 2008

ndby Uplinks a

king Exae vNet and re TWO NICs a

provided by by a differen

N adapters.

rly known as H configured in t

er with TWO NICs within tha NICs are ofFCoE connecti.

chipset, wher

nnections Active

ppear with a 8 server and

and FCoE – Wi

amples edundant SANand the SAN

the CNA adat component o

BA Anywhere).the server profi

FlexNICs conhis server. Hoffline and couions could be

reas the BL46

e

red x as not cDisable any N

ndows 2008 R

N fabrics. Wfabrics using

apter used theof the adapte

. Note that therle.

figured at 6G

owever, only Tuld be disable deleted and

60c G7, BL62

connected. YoNICs that are

R2 24

We the

e in r,

re are

Gb. TWO d. If the

0c

ou not

currLOMadd

Figu

Figu

Not100200

rently in use. M:1 and threedresses to veri

ure 15 - Wind

ure 16 - Wind

e: In windows0MB incremen03 as a 2Gb N

Scenario 1 –

Windows asse on LOM:2. fy which NIC

dows 2008 R2

dows 2008 R2

s 2003 the Nnts above 1GbNIC. Window

– Simple vNet w

signs the NIC You may nee is which.

Extra Network

Network Conn

NIC speeds mab. IE: if a NICws 2008 doe

with Active/Sta

s as NIC 1-6,ed to refer to t

Connections –

nection Status

ay not be showC is configurees not have thi

ndby Uplinks a

, whereas threthe FlexFabric

– Disabled

wn accuratelyed for 2.5Gb is limitation.

and FCoE – Wi

ee of the NICc server profile

y when speed it will be disp

ndows 2008 R

s will reside oe for the NIC

ds are configuplayed in Win

R2 25

on MAC

red in ndows

Figuconf

The concoulfail,modwas

Figu

ure 17 - Wifigured two o

following granected to the ld be teamed Virtual Conn

dule failure, ths the active N

ure 18 - Both N

Scenario 1 –

ndows 2008 f the NICs an

aphics providenetwork, initia to provide Nect would fail

he server’s NICIC, NIC team

NICs for Profile

– Simple vNet w

R2, Device Md two FCoE H

es an exampleally each NICIC fail-over rel-over to the stC teaming soing would fai

e App-1are con

with Active/Sta

Manager, SIX HBAs.

e of a WindoC has its own edundancy. tandby uplinkftware would l-over to the s

nected to the n

ndby Uplinks a

NICs are sho

ows 2008 R2 TCP/IP addreIf an active up

k. In the even see one of thstandby NIC.

network through

and FCoE – Wi

own, however,

server with TWess, alternativeplink or netwot of a Virtual

he NICs go of

h vNet-PROD

ndows 2008 R

, we have on

WO NICs ely, both NICork switch weConnect FlexFffline, assumin

R2 26

ly

Cs re to Fabric ng it

NNIC TeamIf hinetwteamquiteYou1 to

The andstanthe s

Figu

Figu

Variyou

ming gher availabi

work configuraming. In this ee simple. Ho will note the

o Bay 2 for ex

following gra connected to

ndby uplinks, standby NIC.

ure 19 – Team

ure 20 - Both N

ious modes ca can select the

Scenario 1 –

lity is desiredations. Simplexample, we wever, if mult BAY#-Port# ixample.

aphics provideo the network.if a VC FlexFa

m both NICs, us

NICs for Profile

an be configue team and ed

– Simple vNet w

, NIC teamingy, open the Nhave only TWtiple NICs arendication with

e an example In the event abric module

sing the HP Net

e App-1are team

ured for NIC Tdit its properti

with Active/Sta

g in Virtual CoNIC teaming UWO NICs avaie available, enhin each NIC.

e of a Window of an Uplink were to fail, t

twork Configur

med and conne

Teaming, suchies. Typically

ndby Uplinks a

onnect works Utility and conilable, so selensure that the. You would

ws 2008 R2 s or switch failthe NIC team

ration Utility

ected to the net

h as NFT, TLB y, the default s

and FCoE – Wi

the same wanfigure the avecting NICs foe correct pair otypically TEA

server with TWure, VC will faing software w

twork through v

etc. Once thsettings can b

ndows 2008 R

ay as in standavailable NICs or teaming wiof NICs is teaM a NIC from

WO NICs teamail-over to thewould fail-ove

vNet-PROD

he Team is crebe used.

R2 27

ard for ll be

amed. m Bay

med e er to

eated,

R

Figu

Figu

Results –Figu

ure 21 - View

ure 22 - Both

– Windoure 23 - Durin

Scenario 1 –

– Network Co

NICs are team

ows 200ng server boot,

– Simple vNet w

onnections – NIC

ed and connec

08 R2 Syou can see th

with Active/Sta

C Team #1 – W

ct to the networ

SAN Coat Port 1 of the

ndby Uplinks a

Windows

rk with a comm

onnectivie FlexHBA is co

and FCoE – Wi

on IP Address

ity onnected to an

ndows 2008 R

MSA1000 SAN

R2 28

N LUN

S

Figu

SummaWe path

Whconinfratheirservactivconf

Addto aIf adadju

As aprofserv

ure 24 - Wind

ary presented a Vh SAN fabric

en VC profilenected througastructure thror own IP addrver could acceve at the timefigured for 4G

ditional NICs c total of 6 NICdditional or leusted.

additional serfiles, configurever bays.

Scenario 1 –

dows 2008 R2

Virtual Connefor storage co

App-1 is appgh each FlexFaough the 10Gress or as a paess the networ. Each NIC is

Gb of SAN ba

could be addCs, the NIC s

ess SAN band

vers are addee the NICs fo

– Simple vNet w

Disk Administr

ect Network sconnectivity.

plied to the seabric module b uplinks. Theair of TEAMErk through eiths configured fandwidth.

ed within Flexpeed can the

dwidth is requ

ed to the enclor vNet-PROD

with Active/Sta

ator. Note; tha

cenario by cre

erver in bay1 connected to ese NICs coulD NICs. Eitheher NIC or eitfor 6Gb of ne

xFabric, by sin be adjustedired, the spee

osure, simply and the SAN

ndby Uplinks a

at D: is the SAN

eating a simp

and is power “vNet-PROD”ld now be coner NIC could ther uplink caetwork bandw

mply powerind accordingly ed of the SAN

create additio fabrics and a

and FCoE – Wi

N attached volu

ple vNet, we i

red up, it has ”, which connnfigured as in be active. A

able, dependinwidth and eac

ng the server o to suit the ne

N connection c

onal profiles, apply them to

ndows 2008 R

ume

ncluded a du

one NIC nects to the nendividual NICsAs a result, thisng on which ih FCoE port is

off and addineds of each Ncan also be

or copy existo the appropri

R2 29

al

etwork s with s is s

ng up NIC.

ting ate

Scenario 2 –Shared Uplink Sets with Active/Active uplinks and 802.3ad (LACP) - Windows - Ethernet and FCoE 30

Scenario 2 –Shared Uplink Sets with Active/Active uplinks and 802.3ad (LACP) - Windows - Ethernet and FCoE

Overview This scenario will implement the Shared Uplink Set (SUS) to provide support for multiple VLANs. In this scenario, the upstream network switches connect a shared uplink set to two ports on each FlexFabric modules, LACP will be used to aggregate those links.

As multiple VLANs will be supported in this configuration, the upstream switch ports connecting to the FlexFabric modules will be configured to properly present those VLANs. In this scenario, the upstream switch ports will be configured for VLAN trunking/VLAN tagging.

When configuring Virtual Connect, we can provide several ways to implement network fail-over or redundancy. One option would be to connect TWO uplinks to a single virtual connect network; those two uplinks would connect from different Virtual Connect modules within the enclosure and could then connect to the same upstream switch or two different upstream switches, depending on your redundancy needs. An alternative would be to configure TWO separate Virtual Connect networks, each with a single, or multiple, uplinks configured. Each option has its advantages and disadvantages. We will review the second option in this scenario.

In addition, several Virtual Connect Networks can be configured to support the required networks to the servers within the BladeSystem enclosure. These networks could be used to separate the various network traffic types, such as iSCSI, backup, VMotion from production network traffic.

This scenario will also leverage the Fibre Channel over Ethernet (FCoE) capabilities of the FlexFabric modules. Each Fibre channel fabric will have two uplinks connected to each of the FlexFabric modules.

Requirements This scenario will support both Ethernet and fibre channel connectivity. In order to implement this scenario, an HP BladeSystem c7000 enclosure with one or more server blades and TWO Virtual Connect FlexFabric modules, installed in I/O Bays 1& 2 are required. In addition, we will require ONE or TWO external Network switches. As Virtual Connect does not appear to the network as a switch and is transparent to the network, any standard managed switch will work with Virtual Connect. The Fibre Channel uplinks will connect to the existing FC SAN fabrics. The SAN switch ports will need to be configured to support NPIV logins. Two uplinks from each FlexFabric module will be connected the existing SAN fabrics.

Scenario

Figand1 aupli

o 2 –Shared Up

ure 25 - Phyd 2 on the firstnd 2 on the sinks per fabric

plink Sets with A

ysical View; St network switecond networc, from ports X

Active/Active u

hows two Ethetch and two Erk switch. TheX1 and X2 on

uplinks and 80

ernet uplinks Ethernet uplinke SAN fabricsn module 1 to

2.3ad (LACP) -

from Ports X5ks from Ports Xs are also con Fabric A and

- Windows - Eth

5 and X6 on MX5 and X6 onnnected redund ports X1 and

hernet and FCo

Module 1 to Pn Module 2 tondantly, with Td X2 to Fabric

oE 31

orts 1 o Ports TWO c B.

I

S

V

Scenario

FigNICVLAVLAPortconthe

Installa

Switch cAppCiscapp

NotsamportThe the sEdgallowThe con

VC CLI coManordeloggVC

o 2 –Shared Up

ure 26 - LogCs 1 and 2 arAN_102-x throAN-Trunk-2 rests X5 and X6

nnects to the e existing SAN

ation a


The switch p105. All fracould be coUplink Set w

e: when addime FlexFabric ts will need to network switcswitch as an ae, it allows thws a fail-over SAN connectnecting to the

mmands ny of the confer to connect ged in, VC profor each settin

plink Sets with A

gical View; there connected tough VLAN_1spectively. Th on each FlexFexisting SAN F Fabric B thro

and co

ation nd B provide aroCurve netwmes the followports are confames will be fnfigured as (D

would be confng additionalmodule to the

o be configurech port shouldaccess device

he switch to pl link or newertion will be m

e FlexFabric m

figuration settito VC via a Covides a CLI wng will also be

Active/Active u

e server bladeto VLAN_10105-x, which a

he VLAN-TrunkFabric ModuleFabric A throuough port X1 o

nfigura


wing informatiigured as VLAorwarding to Default) untagigured and sel uplinks to the

e same switch,ed for LACP wd be configuree and not anolace the port ir connected p

made with redumodule must be

ings within VCCLI, open an Swith help mene provided.

uplinks and 80

e profile is con-x, NICs 3 anare part of theks are connece in Bays 1 augh port X1 oon Module 2

ation

the commandture. The confion: AN TRUNK po Virtual Conneged, if so, theet as “Default”e SUS, if the a, in order to e

within the sameed for Spannither switch. Binto a forwardport to come oundant connee configured t

C can also beSSH connectionus. Through

2.3ad (LACP) -

nfigured with nd 4 are conne Shared Uplicted, at 10Gbnd 2. In add

on Module 1 ( (Bay 2).


orts (tagging) ect with VLANen a correspo”. additional upensure all the e Link Aggregng Tree EdgeBy configuringding state muconline much qections to eachto accept NPI

e accomplisheon to the IP ad this scenario

- Windows - Eth

Four FlexNICnected to multink Sets, VLANb, to a networdition, SAN FaBay 1) and FC


to support VLN tags. Optioonding vNet w

links are connuplinks are ac

gation Group.e as Virtual Cog the port as Sch quicker thauicker. h Fabric. SANIV logins.

ed via a CLI coddress of the a the CLI comm

hernet and FCo

Cs and 2 FlexHtiple networksN-Trunk-1 andrk switch, throabric FCoE_ACoE_B conne


LANs 101 throonally, one VLwithin the Sha

necting from tctive, the swit. onnect appeaSpanning Trean otherwise,

N switch ports

ommand set. active VCM.

mands to confi

oE 32

HBAs.

d ough A

cts to

ther a

ough LAN red

he tch

rs to e this

s

In Once igure


Configuring the VC module Physically connect Port 1 of Network switch 1 to Port X5 on the VC module in Bay 1

Physically connect Port 1 of Network switch 2 to Port X6 on the VC module in Bay 1

Physically connect Port 1 of the second Network switch to Port X5 of the VC module in Bay 2

Physically connect Port 1 of Network switch 1 to Port X6 on the VC module in Bay 2, if you have only one network switch, connect VC port 1 (Bay 2) to an alternate port on the same switch. This will NOT create a network loop and Spanning Tree is not required

Physically connect Ports X1 and X2 on the FlexFabric module Bay 1 to switch port in SAN Fabric A

Physically connect Ports X1 and X2 on the FlexFabric module Bay 2 to switch port in SAN Fabric B



Configuring the VC Module for Map VLAN Tags and Fast Mac Cache Fail-over via GUI (Ethernet settings)

Note: Flex-10 has also provided additional controls when configuring VC for mapped VLAN tags (Multiple Networks over a single link) support. These features provide the ability to set a Custom or Preferred network speed value for each NIC. These are VC domain settings and when configured will limit the maximum configurable speed of a NIC. In order to enable this capability, Mapped VLAN Tags must be enabled.

Enable Map VLAN Tags within Virtual Connect On the Virtual Connect Manager screen, Left pane, click Ethernet Settings, Advanced Settings Select Map VLAN Tags (click apply) Optionally, select a preferred/Maximum link speed Select apply Select the Other tab Select Fast MAC Cache Fail-over with a refresh of 5 Select Apply

Configuring the VC Module for Map VLAN Tags and Fast Mac Cache Fail-over via CLI (Ethernet settings)

The following command(s) can be copied and pasted into an SSH based CLI session with Virtual Connect

# Set Advanced Ethernet Settings to "Map VLAN Tags" and set "Force server connections" to disabled set enet-vlan vlantagcontrol=map sharedservervlanid=false set mac-cache Enabled=True Refresh=5

Scenario

Noonlyor M Selefor netw # Seset e

Figu

Fig

o 2 –Shared Up

te: Do not sey. For the purMaximum Con

ecting the Set Maximum Linworks to 500M

et Advanced enet-vlan Pref

ure 27 - Ethern

ure 28 - Set

plink Sets with A

t a Preferred orpose of this snnection spee

a Custom valk Connection Mb and maxi

Ethernet SettinSpeedType=C

net Settings - M

t Fast MAC C

Active/Active u

or Max speedscenario, we weds. However

lue for Preferr Speed, Sets tmum connect

ngs to a PrefeCustom PrefSp

Map VLAN Tags

ache (under E

uplinks and 80

d for this scenwill not be cor, the CLI com

red Link Connthe respectiveion speed to 2

erred speed ofpeed=500 Ma

s (under Ether

Ethernet Settin

2.3ad (LACP) -

ario. This exnfiguring the

mmands are pr

ection Speed e speed for all2.5Gb.

f 500Mb andaxSpeedType

rnet Settings “

ngs “Advance

- Windows - Eth

ample is prov custom valuesrovided below

and/or Set al connections

d a Max Spee=Custom Max

“Advanced Se

ed Settings - O

hernet and FCo

vided for refers for Preferred

w for reference

a Customer va using multiple

ed of 2500MbxSpeed=2500

ettings)

Other)

oE 34

rence d Link e.

alue e

b 0


Defining a new Shared Uplink Set (VLAN-Trunk-1) Connect Port 1 of FlexFabric module 1 to Port 1 on switch 1 Create a SUS named “VLAN-Trunk-1” and connect it to FlexFabric Port X1 on Module 1 On the Virtual Connect Home page, select Define, Shared Uplink Set Insert Uplink Set Name as VLAN-Trunk-1 Select Add Port, then add the following port;


Enclosure 1, Bay 1, Port X6 Add Networks as follows (to add a network, right click on the grey bar under the “Associate

Networks (VLAN)” header, the select ADD;

VLAN101-1 = VLAN ID=101




VLAN105-1 = VLAN ID=105 Enable SmartLink on ALL networks Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set

Native to Enabled Click Apply




Add Networks as follows (to add a network, right click on the grey bar under the “Associate Networks (VLAN)” header, the select ADD;





VLAN105-2 = VLAN ID=105 Enable SmartLink on ALL networks Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set



Defining a new Shared Uplink Set via CLI The following script can be used to create the first Shared Uplink Set (VLAN-Trunk-1)


# Create Shared Uplink Set "VLAN-Trunk-1" and configure uplinks add uplinkset VLAN-Trunk-1 add uplinkport enc0:1:X5 Uplinkset=VLAN-Trunk-1 speed=auto add uplinkport enc0:1:X6 Uplinkset=VLAN-Trunk-1 speed=auto # Create Networks VLAN101-1 through VLAN104-1 for Shared Uplink Set "VLAN-Trunk-1" add network VLAN101-1 uplinkset=VLAN-Trunk-1 VLanID=101 Set Network VLAN101-1 SmartLink=Enabled add network VLAN102-1 uplinkset=VLAN-Trunk-1 VLanID=102 Set Network VLAN102-1 SmartLink=Enabled add network VLAN103-1 uplinkset=VLAN-Trunk-1 VLanID=103 Set Network VLAN103-1 SmartLink=Enabled add network VLAN104-1 uplinkset=VLAN-Trunk-1 VLanID=104 Set Network VLAN104-1 SmartLink=Enabled add network VLAN105-1 uplinkset=VLAN-Trunk-1 VLanID=105 Set Network VLAN105-1 SmartLink=Enabled

The following script can be used to create the Second Shared Uplink Set (VLAN-Trunk-2) # Create Shared Uplink Set "VLAN-Trunk-2" and configure uplinks add uplinkset VLAN-Trunk-2 add uplinkport enc0:2:X5 Uplinkset=VLAN-Trunk-2 speed=auto add uplinkport enc0:2:X6 Uplinkset=VLAN-Trunk-2 speed=auto # Create Networks VLAN101-2 through VLAN104-2 for Shared Uplink Set "VLAN-Trunk-2" add network VLAN101-2 uplinkset=VLAN-Trunk-2 VLanID=101 Set Network VLAN101-2 SmartLink=Enabled add network VLAN102-2 uplinkset=VLAN-Trunk-2 VLanID=102 Set Network VLAN102-2 SmartLink=Enabled add network VLAN103-2 uplinkset=VLAN-Trunk-2 VLanID=103 Set Network VLAN103-2 SmartLink=Enabled add network VLAN104-2 uplinkset=VLAN-Trunk-2 VLanID=104 Set Network VLAN104-2 SmartLink=Enabled add network VLAN105-2 uplinkset=VLAN-Trunk-2 VLanID=105 Set Network VLAN105-2 SmartLink=Enabled

Scenario

Figu

Notthe savaswit

Figu

o 2 –Shared Up

ure 29 - Define

te: The “Connswitch this upilable in all swtches and por

ure 30 - Active

plink Sets with A

e a Shared Upl

nected to” fielink is connecwitch productsts VC is conne

e / Active Shar

Active/Active u

link Set (VLAN-T

ld in the grapcted to. This is. This informected to.

red Uplink Set

uplinks and 80

Trunk-1) and ad

phic above disnformation is

mation can be

2.3ad (LACP) -

dd VLANs

splays the MA provided thro very helpful w

- Windows - Eth

AC address anough LLDP diswhen determi

hernet and FCo

nd port numbscover and is ning which

oE 37

er of not

D

Scenario

Figu

Sumorigsepprofwhicreawou

Figu

Defining aCrea

o 2 –Shared Up

ure 31 - Assoc

mmary Notginating from arate and redfiles, you will ch provides thated a single uld have prov

ure 32 - Exam



Select the A

Select Apply

plink Sets with A

ciated Network

te: In this scenthe opposite F

dundant conne see how the he ability to cSUS and assi

vided an Activ

mple of an Activ

oE) SAN and name it “ual Connect Metwork Name Port, then addEnclosure 1, Enclosure 1, dvanced SettiSelect Automay

Active/Active u

ks tab

nario we haveFlexFabric Moections out of NICs will conreate an Activgned both set

ve/Standby up

ve/Standby Sha

Fabric viaFCoE_A”

Manager scree of “FCoE_A”d the followingBay 1, Port XBay 1, Port X2ings atic Login Re-

uplinks and 80

e created two odules, by do the Virtual Co

nnect to VLANve / Active upts of these uplplink scenario

ard Uplink Set

a GUI

en, click Defin” g ports; 1 2

Distribution

2.3ad (LACP) -

independent oing so we proonnect domai

Ns accessed thplink scenariolink ports to th

o, example be

ne, SAN Fabr

- Windows - Eth

t Share Uplinkovide the abilin. When wehrough the opo. Alternativehe same SUS,elow.

ic to create th

hernet and FCo

k Sets (SUS), eity to create

e create the seposite VC moly, we could h, however, thi

he first Fabric

oE 38

each

erver odule, have s

D

Scenario

Crea

Defining SThe Con#CrAddAdd Figu

o 2 –Shared Up


Select the A

Select Apply


ure 33 - SAN

plink Sets with A

Fabric and naual Connect Metwork Name Port, then addEnclosure 1, Enclosure 1, dvanced SettiSelect Automay



N Login Distri

Active/Active u

ame it “FCoE_Manager scree of “FCoE_B” d the followingBay 2, Port XBay 2, Port X2ings atic Login Re-

I n be copied a

E_A and FCorts=1,2 LinkDi

rts=1,2 LinkDis

ibution Setting

uplinks and 80

_B” en, click Defin g ports; 1 2

Distribution

nd pasted into

E_B and confist=Auto st=Auto

g

2.3ad (LACP) -

ne, SAN Fabr

o an SSH bas

figure uplinks

- Windows - Eth

ic to create th

sed CLI sessio

as discussed

hernet and FCo

he second Fab

on with Virtual

above

oE 39

bric

l

D

Scenario

Figunum

Figuserv

Defining aWe wEach

Prio

NotNICFCo

o 2 –Shared Up

ure 34 – FCombers on the ri


a Server Pwill create a seserver NIC wOn the mainCreate a serIn the NetwoIn the NetwoExpand the Do not confiIn the Assignthey apply

r to applying

te: you shouldCs 1&2 shouldoE_A and FCo

plink Sets with A

oE SAN fabriight

oE SAN Serveon)

rofile erver profile will connect to n menu, selectrver profile caork Port 1 droork Port 2 droFCoE Connecigure FC SANn Profile to Se

the profile, e

d now have ad be connecteoE_B.

Active/Active u

cs configured

er Connection

with two servea specific nett Define, thenalled “App-1” op down box,op down box,ctions box, fo

N or iSCSI Coerver Bay box

nsure that the

a server profiled to networks

uplinks and 80

d with two 4G

ns (After serve

er NICs. twork. Server Profile select VLAN1 select VLAN1r Bay 1, selecnnection , locate the Se

server in Bay

e assigned to s VLAN101-1

2.3ad (LACP) -

Gb uplinks per

er profiles hav

e

101-1 101-2 ct FCoE_A for

elect Location

y 1 is currently

Bay 1, with 2 and VLAN10

- Windows - Eth

r fabric. Note

ve been create

r Bay 2, select

n drop down a

y OFF

2 Server NIC 01-2 and FCo

hernet and FCo

e the bay and

ed, assigned

t FCoE_B

and select Bay

connections. oE SAN fabric

oE 40

port

and

y 1,

cs

D

Scenario

Defining aThe Con

Figu

Notthe LOM

o 2 –Shared Up

a Server P following comnnect

# Create Seadd profile Aset enet-connset enet-connset fcoe-conset fcoe-conpoweroff seassign profi

ure 36 - Defin


M:2-b.

plink Sets with A

rofile via mmand(s) can

erver Profile AApp-1 nection App-1nection App-1nection App-1nection App-1rver 1 le App-1 enc0

e a Server Prof


Active/Active u

CLI n be copied a

App-1

1 1 pxe=Enab1 2 pxe=Disab1:1 Fabric=FC1:2 Fabric=FC

0:1

file (App-1)


uplinks and 80

nd pasted into

bled Network=bled NetworkCoE_A SpeedCoE_B SpeedT


2.3ad (LACP) -

o an SSH bas

=VLAN101-1 k=VLAN101-2dType=4Gb Type=4Gb


- Windows - Eth

sed CLI sessio

2


hernet and FCo

on with Virtual


oE 41

l

that -b and

R

Scenario

Figu

Review

In thUplreduwe acti WeVLAthe presVLA AddSAN The

o 2 –Shared Up

ure 37 - Serve

w his scenario winks originatinundant connehave also levve. We create

e created a seAN_101-2, whnetwork. VLAsented to the

AN, but taking

ditionally, FCoN Fabric FCo

FCoE SAN fa

plink Sets with A

er Profile View

we have createng from the eaections out of teraged LACP e two FCoE S

rver profile, which provides

AN101-1 and NIC(s) withou

g a different p

oE port 1 is coE_B, providin

abric connect

Active/Active u

Bay 1

ed Two Shareach FlexFabrithe Virtual Co to improved

SAN Fabrics.

with two NICs the ability to VLAN101-2 aut VLAN tags path out of the

onnected to Sg a multi-path

s to each SAN

uplinks and 80

ed Uplink Setsc Module con

onnect domainuplink perform

s connected to sustain a linkare configure(untagged), th

e enclosure.

AN fabric FChed connected

N fabric over

2.3ad (LACP) -

s (SUS), provinnect to each n. As multiplemance. In thi

o the same VLAk or module faed to support Vhese two NIC

CoE_A and FCd to the SAN.

a single uplin

- Windows - Eth

ding support SUS, by doine uplinks are us scenario, al

AN, VLAN_1ailure and not VLAN 101, fr

Cs are connect

CoE SAN port

nk per module

hernet and FCo

for 5 VLANs. ng so we provused for eachll uplinks will

01-1 and lose connectirames will be ted to the sam

t is connected

e.

oE 42

vide SUS, be

ion to me

d to

R

Scenario

Results –We haveFCo

AlthG7 they

Figu3 Et

The YouNICwe serv

NotG6

Figu

o 2 –Shared Up

– Windo have successe created a se

oE connections




ver would then


ure 39 - Wind

plink Sets with A

ows 200sfully configurerver profile tos created with





dows 2008 R2

Active/Active u

08 R2 Ned FlexFabrico connect to thin the profile




C ports availab


Network Conn

uplinks and 80

Networkc with a simplethe vNet with .


er Utility (formerity per port, as



ze a NC551i ipset.

nections (2 Con

2.3ad (LACP) -

king Exae vNet and re TWO NICs a


N adapters.



chipset, wher

nnections Active

- Windows - Eth

amples edundant SANand the SAN


BA Anywhere)the server profi


reas the BL46

e)

hernet and FCo

N fabrics. Wfabrics using


. Note that therle.

figured at 6Gowever, only Tuld be disable deleted and

60c G7, BL62

oE 43

We the

e in r,

re are

Gb. TWO d. If the

0c

Scenario

NotcancurrLOMadd

Figu

Figu

Not100200

o 2 –Shared Up

te: The NICs t go into Netwrently in use. M:1 and threedresses to veri

ure 40 - Wind

ure 41 - Wind


plink Sets with A

that are not cowork Connecti Windows asse on LOM:2. fy which NIC

dows 2008 R2

dows 2008 R2


Active/Active u

onfigured withions for the Wsigns the NIC You may nee is which.

Extra Network

Network Conn


uplinks and 80



Connections –

nection Status


2.3ad (LACP) -

ppear with a 8 server and , whereas threthe FlexFabric

– Disabled


- Windows - Eth

red x as not cDisable any Nee of the NICc server profile


hernet and FCo

connected. YoNICs that are s will reside oe for the NIC


oE 44

ou not on MAC

red in ndows

Scenario

Figuconf


Figu

o 2 –Shared Up




ure 43 - Both N

plink Sets with A




NICs for Profile

Active/Active u



e App-1are con

uplinks and 80

Manager, SIX HBAs.


nected to the n

2.3ad (LACP) -

NICs are sho



network through

- Windows - Eth

own, however,


he NICs go of

h vNet-PROD

hernet and FCo

, we have on


oE 45

ly


N

Scenario

NIC TeamIf hinetwteamquiteYou1 to

The andstanthe s

Figu

Figu

Variyou

Figu

o 2 –Shared Up

ming gher availabi


o Bay 2 for ex

following gra connected to

ndby uplinks, standby NIC.

ure 44 – Team

ure 45 - Both N


ure 46 - View

plink Sets with A


aphics provideo the network.if a VC FlexFa

m both NICs, us

NICs for Profile


– Network Co

Active/Active u


e an example In the event abric module

sing the HP Net

e App-1are team


onnections – NIC

uplinks and 80


e of a Window of an Uplink were to fail, t

twork Configur

med and conne


C Team #1 – W

2.3ad (LACP) -


ws 2008 R2 s or switch failthe NIC team

ration Utility

ected to the net


Windows

- Windows - Eth


server with TWure, VC will faing software w

twork through v


hernet and FCo


WO NICs teamail-over to thewould fail-ove

vNet-PROD


oE 46

ard for ll be

amed. m Bay

med e er to

eated,

R

Scenario

Figu

Results –Figu

o 2 –Shared Up

ure 47 - Both


plink Sets with A

NICs are team


Active/Active u

ed and connec


uplinks and 80

ct to the networ


2.3ad (LACP) -

rk with a comm


- Windows - Eth

on IP Address

ity onnected to an

hernet and FCo

MSA1000 SAN

oE 47

N LUN

S

Scenario

Figu

SummaWe SUSpath

WhconthroNICNICNICof n

Addto aIf adadju

As aprofserv

o 2 –Shared Up

ure 49 - Wind

ary presented a V

S is connectedh SAN fabric

en VC profilenected througugh FlexFabri

Cs could now Cs. Either NICC or either upletwork bandw


additional serfiles, configurever bays.

plink Sets with A

dows 2008 R2

Virtual Conned with TWO afor storage co

App-1 is appgh FlexFabric ic module 2 (cbe configured

C could be actink, dependinwidth and eac


ess SAN band

vers are addee the NICs fo

Active/Active u

Disk Administr

ect Network scactive uplinks, onnectivity.

plied to the semodule 1 (coconnected to d as individuative. As a resng on which Nch FCoE port


dwidth is requ

ed to the enclor vNet-PROD

uplinks and 80

ator. Note; tha

cenario by cre both SUS’ ca

erver in bay1 nnected to “VVLAN_101-2

al NICs with thsult, this serveNIC is active ais configured


osure, simply and the SAN

2.3ad (LACP) -

at D: is the SAN

eating two shan actively pa

and is powerVLAN_101-1”). Each NIC iheir own IP ader could accesat the time. E for 4Gb of S


create additio fabrics and a

- Windows - Eth

N attached volu

hared uplink sass traffic. We

red up, it has ), the second s configured ddress or as ass the networkEach NIC is coAN bandwid


N connection c

onal profiles, apply them to

hernet and FCo

ume

ets (SUS), eace included a d

one NIC NIC is conneat 6Gb. Thesa pair of TEAMk through eitheonfigured for th.


or copy existo the appropri

oE 48

ch dual

ected se MED er 6Gb

ng up NIC.

ting ate

Scenario 3 – Shared Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - Windows - Ethernet and FCoE Boot from SAN 49

Scenario 3 – Shared Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - Windows - Ethernet and FCoE Boot from SAN

Overview This scenario will implement the Shared Uplink Set (SUS) to provide support for multiple VLANs. In this scenario, the upstream network switches connect a shared uplink set to two ports on each FlexFabric modules, LACP will be used to aggregate those links. This scenario is identical to Scenario 2, however, scenario 3 also provides the steps to configure a windows 2008 R2 server to Boot to SAN using the FC0E connections provided by FlexFabric. When using Virtual Connect/FlexFabric in a Boot to SAN implementation, no custom or special HBA configuration is required. The HBA configuration is controlled by Virtual Connect and maintained as part of the server profile. Once the server profile has been configured and applied to the server bays, the controller will be configured on the next and subsequent boot. When we later configure the server profile, we will also configured the HBA boot parameters.






Scenario 3 –

Figand1 aupli

Shared Uplink

ure 50 - Phyd 2 on the firstnd 2 on the sinks per fabric

Set with Active

ysical View; St network switecond networc, from ports X

e/Active Uplink

hows two Ethetch and two Erk switch. TheX1 and X2 on

ks and 802.3ad

ernet uplinks Ethernet uplinke SAN fabricsn module 1 to

d (LACP) - Win

from Ports X5ks from Ports Xs are also con Fabric A and

dows - Ethernet

5 and X6 on MX5 and X6 onnnected redund ports X1 and

t and FCoE BooSAN

Module 1 to Pn Module 2 tondantly, with Td X2 to Fabric

ot from N 50

orts 1 o Ports TWO c B.

I

S

V

Scenario 3 –

FigFlexVLAnetwSANSANFCo

Installa

Switch cAppCiscapp

Notsamwill The con


Shared Uplink

ure 51 - LogxHBAs. NICs AN-Trunk-1 anwork switch, tN connectionsN Fabric FCooE_B connects

ation a


The switch pthrough 105VLAN couldShared Upli

e: when addime Flex-10 mo need to be co SAN connectnecting to the


Set with Active

gical View; the1 and 2 are cd VLAN-Trunkthrough Ports s are connecteE_A connectss to the existin

and co

ation nd B provide aroCurve netwmes the followports are conf5. All frames be configurenk Set would ng the additiodule to the saonfigured for tion will be m

e FlexFabric m


e/Active Uplink

e server bladeconnected to Vk-2 respectiveX5 and X6 oned through po to the existin

ng SAN Fabric

nfigura


wing informatiigured as VLA will be forwaed as (Default) be configuredonal uplinks toame switch, inLACP within t



ks and 802.3ad

e profile is conVLAN_101-x ly. The VLANn each FlexFaorts X1 and X2g SAN Fabricc B through p

ation

the commandture. The confion: AN TRUNK poarding to Virtu) untagged, if d and set as “o the SUS, if t order to ensuthe same Link undant connee configured t


d (LACP) - Win

nfigured with which are pa

N-Trunks are cabric Module 2 on each Flec A through pport X1 on Mo


orts to supporual Connect wf so, then a co“Default”. the additionalure all the upl Aggregation

ections to eachto accept NPI


dows - Ethernet

TWO FlexNIart of the Sharconnected, at in Bays 1 and

exFabric moduport X1 on Moodule 2 (Bay 2


rt tagging on Vwith VLAN tagorresponding

l uplinks are clinks are activ Group. h Fabric. SANIV logins.


t and FCoE BooSAN

Cs and 2 red Uplink Set10Gb, to a d 2. The FCoule. In additio

odule 1 (Bay 12).


VLANs 101 s. OptionallyvNet within th

connecting frove, the switch

N switch ports


mands to confi

ot from N 51

ts,

oE on, 1) and

ther a

y, one he

om the ports

s

In Once igure















# Set Advanced Ethernet Settings to "Map VLAN Tags" and set "Force server connections" to disabled

set enet-vlan vlantagcontrol=map sharedservervlanid=false

set mac-cache Enabled=True Refresh=5

Scenario 3 –


Figu

Fig

Shared Uplink




ure 52 - Ethern

ure 53 - Set

Set with Active




net Settings - M

t Fast MAC C

e/Active Uplink




Map VLAN Tags

ache (under E

ks and 802.3ad




s (under Ether

Ethernet Settin

d (LACP) - Win


mmands are pr



rnet Settings “

ngs “Advance

dows - Ethernet




“Advanced Se

ed Settings - O

t and FCoE BooSAN


w for reference



ettings)

Other)

ot from N 53

rence d Link e.

alue e

b 0


Defining a new Shared Uplink Set (VLAN-Trunk-1) Connect Ports X5 and X6 of FlexFabric module 1 to Port 1 on switch 1 Create a SUS named “VLAN-Trunk-1” and connect it to FlexFabric Ports X5 and X6 on Module 1 On the Virtual Connect Home page, select Define, Shared Uplink Set Insert Uplink Set Name as VLAN-Trunk-1 Select Add Port, then add the following port;


Enclosure 1, Bay 1, Port X6 Add Networks as follows (to add a network, right click on the grey bar under the “Associate

Networks (VLAN)” header, the select ADD;





VLAN105-1 = VLAN ID=105 Enable Smartlink on ALL networks Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set


Defining a new Shared Uplink Set (VLAN-Trunk-2) Connect Ports X5 and X6 of FlexFabric module 2 to Ports X5 and X5 on switch 2 Create a SUS named “VLAN-Trunk-2” and connect it to FlexFabric Port X1 on Module 2 On the Virtual Connect Home page, select Define, Shared Uplink Set Insert Uplink Set Name as VLAN-Trunk-2 Select Add Port, then add the following port;

Enclosure 1, Bay 2, Port X5 Enclosure 1, Bay 2, Port X6


VLAN101-2 = VLAN ID=101 VLAN102-2 = VLAN ID=102 VLAN103-2 = VLAN ID=103 VLAN104-2 = VLAN ID=104 VLAN105-2 = VLAN ID=105

Enable smartlink on ALL networks Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set




# Create Shared Uplink Set "VLAN-Trunk-1" and configure uplinks


add uplinkset VLAN-Trunk-1 add uplinkport enc0:1:X5 Uplinkset=VLAN-Trunk-1 speed=auto add uplinkport enc0:1:X6 Uplinkset=VLAN-Trunk-1 speed=auto # Create Networks VLAN101-1 through VLAN104-1 for Shared Uplink Set "VLAN-Trunk-1" add network VLAN101-1 uplinkset=VLAN-Trunk-1 VLanID=101 Set Network VLAN101-1 SmartLink=Enabled add network VLAN102-1 uplinkset=VLAN-Trunk-1 VLanID=102 Set Network VLAN102-1 SmartLink=Enabled add network VLAN103-1 uplinkset=VLAN-Trunk-1 VLanID=103 Set Network VLAN103-1 SmartLink=Enabled add network VLAN104-1 uplinkset=VLAN-Trunk-1 VLanID=104 Set Network VLAN104-1 SmartLink=Enabled add network VLAN105-1 uplinkset=VLAN-Trunk-1 VLanID=105 Set Network VLAN105-1 SmartLink=Enabled


Scenario 3 –

Figu

Notthe savaswit

Figu

Shared Uplink

ure 54 - Define


ure 55 - Active

Set with Active

e a Shared Upl


e / Active Shar

e/Active Uplink

link Set (VLAN-T


red Uplink Set

ks and 802.3ad

Trunk-1) and ad


mation can be

d (LACP) - Win

dd VLANs


dows - Ethernet


t and FCoE BooSAN


ot from N 56

er of not

D

Scenario 3 –

Figu

Summoriginand rewill sethe abSUS aan Ac

Figu

Defining aCrea

Shared Uplink

ure 56 - Assoc

mary Note:nating from theedundant conee how the Nbility to createand assigned ctive/Standby

ure 57 - Exam



Select the A

Select Apply

Set with Active

ciated Network

: In this scenae opposite Flennections out oICs will conne

e an Active / both sets of thy uplink scena

mple of an Activ


e/Active Uplink

ks tab

rio we have cexFabric Modof the Virtual Cect to VLANs aActive uplink hese uplink porio, example

ve/Standby Sha

Fabric viaFCoE_A”


ks and 802.3ad

created two inules, by doingConnect domaccessed thro scenario. Altorts to the sambelow.

ard Uplink Set

a GUI


Distribution

d (LACP) - Win

ndependent Sg so we proviain. When w

ough the oppoternatively, wme SUS, howe

ne, SAN Fabr

dows - Ethernet

hare Uplink Side the ability

we create the osite VC modu

we could have ever, this wou

ic to create th

t and FCoE BooSAN

Sets (SUS), eay to create sepserver profilesule, which pro created a sinuld have provi

he first Fabric

ot from N 57

ch parate s, you ovides ngle ided

D

Scenario 3 –

Crea

Defining SThe Con#CrAddAddFigu

Figuport

Shared Uplink


Select the A

Select Apply

SAN Fabri following comnnect reate the SANd fabric FCoE_d fabric FCoE_ure 58 - SAN

ure 59 – FCot numbers on

Set with Active



N Fabrics FCoE_A Bay=1 Po_B Bay=2 PorN Login Distri

oE SAN fabrithe right

e/Active Uplink


I n be copied a


rts=1,2 LinkDis

ibution Setting

cs configured

ks and 802.3ad


Distribution

nd pasted into


g

d with a single

d (LACP) - Win

ne, SAN Fabr

o an SSH bas

figure uplinks

e 4Gb uplink

dows - Ethernet

ic to create th

sed CLI sessio

as discussed

per fabric. N

t and FCoE BooSAN

he second Fab

on with Virtual

above

Note the bay a

ot from N 58

bric

l

and

D

Scenario 3 –

Figuserv

Defining aWe wEach

Prio

NotNICFCo

Shared Uplink


a Server Pwill create a seserver NIC wOn the mainCreate a serIn the NetwoIn the NetwoExpand the Select the “F

Sele Clic Clic Clic

“Se Clic Clic Clic

Do not confiIn the Assignthey apply

r to applying

te: you shouldCs 1&2 shouldoE_A and FCo

Set with Active

oE SAN Serveon)

rofile erver profile will connect to n menu, selectrver profile caork Port 1 droork Port 2 droFCoE ConnecFibre Channelect PORT 1 anck on the Targck on the LUNck on PORT 2econdary” ck on the Targck on the LUNck Apply igure FC SANn Profile to Se

the profile, e

d now have ad be connecteoE_B.

e/Active Uplink

er Connection

with two servea specific nett Define, thenalled “App-1_op down box,op down box,ctions box, fol Boot Paramend click on thget Port Name

N field and en and click on

get Port NameN field and en


nsure that the


ks and 802.3ad

ns (After serve

er NICs. twork. Server Profile

_BfS” select VLAN1 select VLAN1r Bay 1, seleceters” box unde drop down e field and enter the boot LU the drop dow

e field and enter the boot LU

nnection , locate the Se

server in Bay

e assigned to s VLAN101-1

d (LACP) - Win

er profiles hav

e

101-1 101-2 ct FCoE_A forder the FCoE under “SAN

nter the SAN cUN number, w

wn under “SA

nter the SAN cUN number, w

elect Location

y 1 is currently

Bay 1, with 2 and VLAN10

dows - Ethernet

ve been create

r Bay 2, selectconfiguration Boot” and secontroller ID which is typic

AN Boot” and

controller ID which is typic

n drop down a

y OFF

2 Server NIC 01-2 and FCo

t and FCoE BooSAN

ed, assigned

t FCoE_B box

elect “Primary”

cally 1 select

cally 1

and select Bay

connections. oE SAN fabric

ot from N 59

and

”

y 1,

cs

D

Scenario 3 –

Defining aThe Con

Figu

Notthe LOM

Shared Uplink


# Create Seadd profile Aset enet-connset enet-connset fcoe-conBootPort=50set fcoe-conBootPort=50poweroff seassign profiNote: youand subst

ure 61 - Defin


M:2-b.

Set with Active


erver Profile AApp-1 nection App-1nection App-1nection App-10:08:05:F3:0nection App-10:08:05:F3:0rver 1 le App-1 enc0

u will need ttitute the ad

e a Server Prof


e/Active Uplink

CLI n be copied a

App-1

1 1 pxe=Enab1 2 pxe=Disab1:1 Fabric=FC0:00:58:11 B1:2 Fabric=FC0:00:58:12 B

0:1 to locate theddresses ab

file (App-1), inc


ks and 802.3ad

nd pasted into

bled Network=bled NetworkCoE_A SpeedTBootLun=1 CoE_B SpeedTBootLun=1

he WWN forbove.

cluding Boot to


d (LACP) - Win

o an SSH bas

=VLAN101-1 k=VLAN101-2Type=4Gb Bo

Type=4Gb Bo

r the contro

SAN configura


dows - Ethernet

sed CLI sessio

2 ootPriority=Pr

ootPriority=Se

oller you are

ation


t and FCoE BooSAN

on with Virtual

imary

econdary

e booting to


ot from N 60

l

o

that -b and

Scenario 3 –

Figu

Figu

Shared Uplink

ure 62 - Conf

ure 63 - Serve

Set with Active

igure Boot from

er Profile View

e/Active Uplink

m SAN Controll

Bay 1

ks and 802.3ad

ler information

d (LACP) - Win

dows - Ethernet

t and FCoE BooSAN

ot from N 61


Review

In this scenario we have created Two Shared Uplink Sets (SUS), providing support for 5 VLANs. Uplinks originating from the each FlexFabric Module connect to each SUS, by doing so we provide redundant connections out of the Virtual Connect domain. As multiple uplinks are used for each SUS, we have also leveraged LACP to improved uplink performance. In this scenario, all uplinks will be active. We create two FCoE SAN Fabrics. We created a server profile, with two NICs connected to the same VLAN, VLAN_101-1 and VLAN_101-2, which provides the ability to sustain a link or module failure and not lose connection to the network. VLAN101-1 and VLAN101-2 are configured to support VLAN 101, frames will be presented to the NIC(s) without VLAN tags (untagged), these two NICs are connected to the same VLAN, but taking a different path out of the enclosure. Additionally, FCoE port 1 is connected to SAN fabric FCoE_A and FCoE SAN port is connected to SAN Fabric FCoE_B, providing a multi-pathed connected to the SAN. We configured the server profile for Boot from SAN, connected a SAN LUN to the server, installed and booted Windows 2008 R2 from the SAN LUN. The FCoE SAN fabric connects to each SAN fabric over a pair of uplinks per module. SAN logins are distributed across the multiple paths.

Results – Windows 2008 R2 Networking Examples We have successfully configured FlexFabric with a simple vNet and redundant SAN fabrics. We have created a server profile to connect to the vNet with TWO NICs and the SAN fabrics using the FCoE connections created within the profile.

Although both Ethernet and Fibre channel connectivity is provided by the CNA adapter used the in G7 servers; each capability (LAN and SAN) is provided by a different component of the adapter, they appear in the server as individual network and SAN adapters.

Scenario 3 –

Figu3 Et

The YouNICwe serv

NotG6

Figu

NotcancurrLOMadd

Shared Uplink



ver would then


ure 65 - Wind

te: The NICs t go into Netwrently in use. M:1 and threedresses to veri

Set with Active




dows 2008 R2

that are not cowork Connecti Windows asse on LOM:2. fy which NIC

e/Active Uplink



C ports availab


Network Conn

onfigured withions for the Wsigns the NIC You may nee is which.

ks and 802.3ad

r Utility (formerity per port, as



ze a NC551i ipset.

nections (2 Con



d (LACP) - Win



chipset, wher

nnections Active

ppear with a 8 server and , whereas threthe FlexFabric

dows - Ethernet

BA Anywhere).the server profi


reas the BL46

e)

red x as not cDisable any Nee of the NICc server profile

t and FCoE BooSAN

. Note that therle.

figured at 6Gowever, only Tuld be disable deleted and

60c G7, BL62

connected. YoNICs that are s will reside oe for the NIC

ot from N 63

re are

Gb. TWO d. If the

0c

ou not on MAC

Scenario 3 –

Figu

Figu

Not100200

Shared Uplink

ure 66 - Wind

ure 67 - Wind


Set with Active

dows 2008 R2

dows 2008 R2


e/Active Uplink

Extra Network

Network Conn


ks and 802.3ad

Connections –

nection Status


d (LACP) - Win

– Disabled


dows - Ethernet


t and FCoE BooSAN


ot from N 64

red in ndows

Scenario 3 –

Figuconf


Figu

Shared Uplink




ure 69 - Both N

Set with Active




NICs for Profile

e/Active Uplink



e App-1are con

ks and 802.3ad

Manager, SIX HBAs.


nected to the n

d (LACP) - Win

NICs are sho



network through

dows - Ethernet

own, however,


he NICs go of

h vNet-PROD

t and FCoE BooSAN

, we have on


ot from N 65

ly


N

Scenario 3 –

NIC TeamIf hinetwteamquiteYou1 to

The andconserv

Figu

Figupath

Shared Uplink

ming gher availabi


o Bay 2 for ex

following gra connected tonection to the

ver NIC port o

ure 70 – Team

ure 71 - Both Nh could be activ

Set with Active


aphics provideo the network. network; Sma

off, causing th

m both NICs, us

NICs for Profileve

e/Active Uplink


e an example In the event artLink will alte NIC teamin

sing the HP Net

e App-1are team

ks and 802.3ad


e of a Window of an Uplink ter the NIC teng software to

twork Configur

med and conne

d (LACP) - Win


ws 2008 R2 s or switch faileaming softwao fail-over to th

ration Utility

ected to the net

dows - Ethernet


server with TWure, the SUS ware to this evehe alternate N

twork through V

t and FCoE BooSAN


WO NICs teamwill lose

ent, by turningNIC.

VLAN-Trunk-x, e

ot from N 66

ard for ll be

amed. m Bay

med

g the

either

R

Scenario 3 –

Variyou

Figu

Figu

Results –Figu

Shared Uplink


ure 72 - View

ure 73 - Both


Set with Active


– Network Co

NICs are team


e/Active Uplink


onnections – NIC

ed and connec


ks and 802.3ad


C Team #1 – W

ct to the networ


d (LACP) - Win


Windows

rk with a comm


dows - Ethernet


on IP Address

ity onnected to an

t and FCoE BooSAN


MSA1000 SAN

ot from N 67

eated,

N LUN

S

Scenario 3 –

Figu

SummaWe SUSpath

WhconthroNICNICNICof n

Addto aIf adadju

As aprofapp

Shared Uplink

ure 75 - Wind

ary presented a V

S is connectedh FCoE SAN f

en VC profilenected througugh FlexFabri

Cs could now Cs. Either NICC or either upletwork bandw


additional serfiles, configure

propriate serve

Set with Active

dows 2008 R2

Virtual Conned with TWO afabric for stor

App-1 is appgh FlexFabric ic module 2 (cbe configured

C could be actink, dependinwidth and eac


ess SAN band

vers are addee the NICs foer bays.

e/Active Uplink

Disk Administr

ect Network scactive uplinks, age connectiv

plied to the semodule 1 (coconnected to d as individuative. As a resng on which Nch FCoE port


dwidth is requ

ed to the enclor the appropr

ks and 802.3ad

ator. Note; tha

cenario by cre both SUS’ cavity and boot

erver in bay1 nnected to “VVLAN_101-2

al NICs with thsult, this serveNIC is active ais configured


osure, simply riate VLAN, SA

d (LACP) - Win

at D: is the SAN

eating two shan actively pa to SAN.

and is powerVLAN_101-1”). Each NIC iheir own IP ader could accesat the time. E for 4Gb of S


create additioAN fabrics an

dows - Ethernet

N attached volu


red up, it has ), the second s configured ddress or as ass the networkEach NIC is coAN bandwid


N connection c

onal profiles, nd apply them

t and FCoE BooSAN

ume


one NIC NIC is conneat 6Gb. Thesa pair of TEAMk through eitheonfigured for th.


or copy existm to the

ot from N 68

ch dual

ected se MED er 6Gb

ng up NIC.

ting

Scenario 4 – Shared Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - Windows Hyper-V - Ethernet and FCoE SAN 69

Scenario 4 – Shared Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - Windows Hyper-V - Ethernet and FCoE SAN







Scenario 4 –

Fig1 aPortwithFab

– Shared Uplin

ure 76 - Phynd 2 on the fits 1 and 2 onh TWO uplinkbric B.

nk Set with Activ

ysical View; Sirst network sw the second n

ks per fabric,

ve/Active Uplin

Shows two Ethwitch and twonetwork switchfrom ports X1

nks and 802.3a

hernet uplinks o Ethernet uplih. . The SAN and X2 on m

ad (LACP) - Wi

from Ports X5inks from Port

N fabrics are amodule 1 to Fa

indows Hyper-V

5 and X6 on Mts X5 and X6 also connecteabric A and p

V - Ethernet andSAN

Module 1 to Pon Module 2

ed redundantlyports X1 and X

d FCoE N 70

Ports to y, X2 to

I

S

V

Scenario 4 –

FigFlexnetwandthroare FCocon

Installa

Switch cAppCiscapp

Notsamwill The con


– Shared Uplin

ure 77 - LogxHBAs. NICs works VLAN_d VLAN-Trunk-ough Ports X5 connected thoE_A connectsnnects to the e

ation a



e: when addime Flex-10 mo need to be co SAN connectnecting to the


nk Set with Activ

gical View; th1 and 2 are c102-x through-2 respectively and X6 on earough ports Xs to the existin

existing SAN F

and co

ation nd B provide aroCurve netwmes the followports are conf5. All frames be configurenk Set would ng the additiodule to the saonfigured for tion will be m

e FlexFabric m


ve/Active Uplin

e server bladeconnected to Vh VLAN_105-y. The VLAN-ach FlexFabri

X1 and X2 on ng SAN FabriFabric B throu

nfigura


wing informatiigured as VLA will be forwaed as (Default) be configuredonal uplinks toame switch, inLACP within t



nks and 802.3a

e profile is coVLAN_101-x,-x, which are -Trunks are coc Module in B each FlexFabic A through pugh port X1 on

ation

the commandture. The confion: AN TRUNK poarding to Virtu) untagged, if d and set as “o the SUS, if t order to ensuthe same Link undant connee configured t


ad (LACP) - Wi

onfigured with, NICs 3 and part of the Sh

onnected, at 1Bays 1 and 2bric module. port X1 on Mon Module 2 (B


orts to supporual Connect wf so, then a co“Default”. the additionalure all the upl Aggregation



indows Hyper-V

h Four FlexNIC 4 are connechared Uplink S0Gb, to a ne. The FCoE SIn addition, Sodule 1 (Bay Bay 2).



l uplinks are clinks are activ Group. h Fabric. SANIV logins.


V - Ethernet andSAN

Cs and 2 cted to multiplSets, VLAN-Tretwork switch,SAN connectioSAN Fabric 1) and FCoE_



connecting frove, the switch

N switch ports


mands to confi

d FCoE N 71

le runk-1 ons

_B

ther a

y, one he

om the ports

s

In Once igure
















Scenario 4 –


Figu

Fig

– Shared Uplin




ure 78 - Ethern

ure 79 - Set

nk Set with Activ




net Settings - M

t Fast MAC C

ve/Active Uplin




Map VLAN Tags

ache (under E

nks and 802.3a




s (under Ether

Ethernet Settin

ad (LACP) - Wi


mmands are pr



rnet Settings “

ngs “Advance

indows Hyper-V




“Advanced Se

ed Settings - O

V - Ethernet andSAN


w for reference



ettings)

Other)

d FCoE N 73

rence d Link e.

alue e

b 0






Enable SmartLink on ALL networks Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set






Enable SmartLink on ALL networks Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set







Scenario 4 –

Figu

Notthe savaswit

Figu

– Shared Uplin

ure 80 - Define


ure 81 - Active

nk Set with Activ

e a Shared Upl


e / Active Shar

ve/Active Uplin

link Set (VLAN-T


red Uplink Set

nks and 802.3a

Trunk-1) and ad


mation can be

ad (LACP) - Wi

dd VLANs


indows Hyper-V


V - Ethernet andSAN


d FCoE N 76

er of not

D

Scenario 4 –

Figu


Figu

Defining aCrea

– Shared Uplin

ure 82 - Assoc


ure 83 - Exam



Select the A

Select Apply

nk Set with Activ

ciated Network



vided an Activ

mple of an Activ


ve/Active Uplin

ks tab


ve/Standby up

ve/Standby Sha

Fabric viaFCoE_A”


nks and 802.3a



ard Uplink Set

a GUI


Distribution

ad (LACP) - Wi



o, example be

ne, SAN Fabr

indows Hyper-V


ic to create th

V - Ethernet andSAN



he first Fabric

d FCoE N 77

each

erver odule, have s

D

Scenario 4 –

Crea


Figuport

– Shared Uplin


Select the A

Select Apply


ure 84 - SAN

ure 85 – FCot numbers on

nk Set with Activ




N Login Distri

oE SAN fabrithe right

ve/Active Uplin


I n be copied a


rts=1,2 LinkDis

ibution Setting

cs configured

nks and 802.3a


Distribution

nd pasted into


g

d with a single

ad (LACP) - Wi

ne, SAN Fabr

o an SSH bas

figure uplinks

e 4Gb uplink

indows Hyper-V

ic to create th

sed CLI sessio

as discussed

per fabric. N

V - Ethernet andSAN

he second Fab

on with Virtual

above

Note the bay a

d FCoE N 78

bric

l

and

D

Scenario 4 –

Figuserv

Defining aWe wEach

Prio

NotNICVLA

– Shared Uplin


a Server Pwill create a seserver NIC wOn the mainCreate a serIn the NetwoSet the port In the NetwoSet the port Left click on (add two adIn the Netwo Configure f Leave the nIn the Netwo Configure f Leave the nExpand the Do not confiIn the Assignthey apply

r to applying

te: you shouldCs 1&2 shouldAN102-x throu

nk Set with Activ

oE SAN Serveon)

rofile erver profile will connect to n menu, selectrver profile caork Port 1 dro speed to Cusork Port 2 dro speed to Cus either of Port dditional network Port 3 drofor networks Vnetwork speedork Port 4 drofor networks Vnetwork speedFCoE Connecigure FC SANn Profile to Se

the profile, e

d now have ad be connecteugh VLAN105

ve/Active Uplin

er Connection

with two servea specific nett Define, thenalled “App-1” op down box,stom at 1Gb op down box,stom at 1Gb 1 or Port 1 in

work connectioop down box,VLAN102-1 thd as Auto op down box,VLAN102-1 thd as Auto ctions box, fo


nsure that the


5-x and FCoE

nks and 802.3a

ns (After serve

er NICs. twork. Server Profile select VLAN1

select VLAN1

n the Ethernet ons) select Multiphrough VLAN

select Multiphrough VLAN

r Bay 1, selecnnection , locate the Se

server in Bay

e assigned to s VLAN101-x, SAN fabrics

ad (LACP) - Wi

er profiles hav

e

101-1

101-2

t Connections

ple Networks 105-1

ple Networks 105-1

ct FCoE_A for

elect Location

y 1 is currently

Bay 1, with 2, NICs 3&4 sh FCoE_A and

indows Hyper-V

ve been create

box, and sele

r Bay 2, select

n drop down a

y OFF

2 Server NIC hould be conn FCoE_B.

V - Ethernet andSAN

ed, assigned

ect ADD netw

t FCoE_B

and select Bay

connections. nected to netw

d FCoE N 79

and

work

y 1,

works

D

Scenario 4 –

Defining aThe Con

Figu

Notthe LOM

– Shared Uplin


# Create Seadd profile Aset enet-conn1000 set enet-conn1000 add enet-coadd server-padd server-padd server-padd enet-coadd server-padd server-padd server-pset fcoe-conset fcoe-conpoweroff seassign profi

ure 87 - Defin


M:2-b.

nk Set with Activ


erver Profile AApp-1 nection App-1

nection App-1

nnection Appport-map App-port-map App-port-map App-nnection Appport-map App-port-map App-port-map App-nection App-1nection App-1rver 1 le App-1 enc0

e a Server Prof


ve/Active Uplin

CLI n be copied a

App-1

1 1 pxe=Enab

1 2 pxe=Disab

p-1 pxe=Disab-1:3 VLAN_10-1:3 VLAN_10-1:3 VLAN_10

p-1 pxe=Disab-1:4 VLAN_10-1:4 VLAN_10-1:4 VLAN_101:1 Fabric=FC1:2 Fabric=FC

0:1

file (App-1)


nks and 802.3a

nd pasted into

bled Network=

bled Network

bled 03-1 VLanId=04-1 VLanId=05-1 VLanId=

bled 03-2 VLanId=04-2 VLanId=05-2 VLanId=CoE_A SpeedCoE_B SpeedT


ad (LACP) - Wi

o an SSH bas

=VLAN101-1

k=VLAN101-2

=103 =104 =105

=103 =104 =105 dType=4Gb Type=4Gb


indows Hyper-V

sed CLI sessio

SpeedType=

2 SpeedType=


V - Ethernet andSAN

on with Virtual

Custom Spee

=Custom Spee


d FCoE N 80

l

d-

ed-

that -b and

Scenario 4 –

Figu

Figu

– Shared Uplin

ure 88 - Conf

ure 89 - Conf

nk Set with Activ

igure Customer

igure NIC Port

ve/Active Uplin

r NIC Speed - m

3 for Multiple

nks and 802.3a

move the slider

Networks - VLA

ad (LACP) - Wi

r left/right to co

AN102-1 throu

indows Hyper-V

onfigure NIC sp

gh VLAN105-1

V - Ethernet andSAN

peed.

1

d FCoE N 81

Scenario 4 –

Figu

– Shared Uplin

ure 90 - Conf

nk Set with Activ

igure NIC Port

ve/Active Uplin

4 for multiple N

nks and 802.3a

Networks - VLA

ad (LACP) - Wi

AN102-2 throug

indows Hyper-V

gh VLAN105-2

V - Ethernet andSAN

2

d FCoE N 82

Scenario 4 –

Figu

Figu

– Shared Uplin

ure 91 - Serve

ure 92 - By ho

nk Set with Activ

er Profile View B

overing over th

ve/Active Uplin

Bay 1

e “Multiple Ne

nks and 802.3a

tworks” page,

ad (LACP) - Wi

a listing of the

indows Hyper-V

connected Net

V - Ethernet andSAN

tworks is displa

d FCoE N 83

ayed.


Review In this scenario we have created Two Shared Uplink Sets (SUS), providing support for 5 VLANs. Uplinks originating from the each FlexFabric Module connect to each SUS, by doing so we provide redundant connections out of the Virtual Connect domain. As multiple uplinks are used for each SUS, we have also leveraged LACP to improved uplink performance. In this scenario, all uplinks will be active. We also create two FCoE SAN Fabrics. We created a server profile, with two NICs connected to the same VLAN, VLAN_101-1 and VLAN_101-2, which provides the ability to sustain a link or module failure and not lose connection to the network. VLAN101-1 and VLAN101-2 are configured to support VLAN 101, frames will be presented to the NIC(s) without VLAN tags (untagged), these two NICs are connected to the same VLAN, but taking a different path out of the enclosure.

Network Ports 3 and 4 were added, these NICs will be connected to “Multiple Networks” and each NIC will then be configured for networks VLAN102-x through VLAN105-x. As these networks are tagging, frames will be presented to the server with VLAN tags. NICs 3 and 4 will be teamed and connected to a Hyper-V virtual switch. VLAN tagged frames for these networks will be forwarded to the Virtual switch and then passed on to the appropriate Virtual Machine. Additionally, FCoE port 1 is connected to SAN fabric FCoE_A and FCoE SAN port is connected to SAN Fabric FCoE_B, providing a multi-pathed connected to the SAN. The FCoE SAN fabric connects to each SAN fabric over a pair of uplinks per module. SAN logins are distributed across the multiple paths.

Results – Windows 2008 R2 Networking Examples We have successfully configured FlexFabric with a simple vNet and redundant SAN fabrics. We have created a server profile to connect to the vNet with TWO NICs and the SAN fabrics using the FCoE connections created within the profile.


Scenario 4 –

Figu3 Et

The 1Gbservandcou

NotG6

Figu

Notcannot LOMfor t

– Shared Uplin


following grab and two at ver. Howeverd could be disld be deleted


ure 94 - Wind

te: The NICs t go into Netw currently in usM:1 (a,c &d)athe NIC MAC

nk Set with Activ


aphics show a5Gb. You wr, only four Nabled. If we and the serve

c G7 and BL6G7 utilize an

dows 2008 R2

that are not cowork Connectise. Windows

and three on LC addresses to

ve/Active Uplin


a Windows 2ill also notice ICs are curren did not requier would then

685c G7 utiliz NC553i chip

Network Conn

onfigured withions for the Ws assigns the NLOM:2 (a,c &o verify which

nks and 802.3a

er Utility (formerity per port, as

008 R2 serve that Windowntly configuredre SAN conne have 8 NIC

ze an NC551pset.

nections (4 Con


NICs as NIC &d). You may

NIC is which

ad (LACP) - Wi


er with FOUR ws believes thed within FlexFectivity on this ports availab

i chipset, whe

nnections Active

ppear with a 8 R2 server a 1-6, whereas need to referh.

indows Hyper-V

BA Anywhere)the server profi

FlexNICs conere are 6 NICFabric, the exts server, the F

ble to the OS.

ereas the BL4

e)

red x as not cnd disable an

s three of the Nr to the FlexFa

V - Ethernet andSAN

. Note that therle.

nfigured, two Cs within this tra NICs are oFCoE connecti

60c G7, BL6

connected. Yony NICs that aNICs will residabric server pr

d FCoE N 85

re are

at

offline ions

20c

ou are de on rofile

Scenario 4 –

Figu

Figu

Not100200

– Shared Uplin

ure 95 - Wind

ure 96 - Wind


nk Set with Activ

dows 2008 R2

dows 2008 R2


ve/Active Uplin

Network Conn

Network Conn


nks and 802.3a

nections (2 Con

nection Status


ad (LACP) - Wi

nnections Disab


indows Hyper-V

bled)


V - Ethernet andSAN


d FCoE N 86

red in ndows

Scenario 4 –

Figuconf

The to thprovnetwuplievenone stan

Figuconn

– Shared Uplin

ure 97 - Wifigured four o

following grahe network, invide NIC fail-owork (VLAN 1nk or networknt of a Virtual of the NICs g

ndby NIC.

ure 98 – Two nected through

nk Set with Activ

ndows 2008 of the NICs an

aphics providenitially each Nover redunda01) and one

k switch were Connect Flexgo offline, ass

NICs for Profile VLAN 104.

ve/Active Uplin

R2, Device Mnd two FCoE H

es an exampleNIC has its owncy. In this sc for the Virtua to fail, VirtuaxFabric modulsuming it was

e App-1are con

nks and 802.3a

Manager, SIX HBAs.

e of a Windown TCP/IP addcenario we wl guest netwol Connect wole failure, the the active NI

nnected to the n

ad (LACP) - Wi

NICs are sho

ows 2008 R2 dress, alternat

will create two rks (VLANs 1

ould fail-over to server’s NIC C, NIC teami

network through

indows Hyper-V

own, however,

server with fotively, NICs c teams, one fo02 through 1o the standby teaming softwing would fail

h VLAN-101 an

V - Ethernet andSAN

, we have on

our NICs connould be teamor the manag05). If an ac

y uplink. In thware would sel-over to the

nd two NICs ar

d FCoE N 87

ly

nected ed to ement

ctive e ee

re

N

Scenario 4 –

NIC TeamBefoversNICMAcrea

NICopeFOUfor awithNIC

The as B

The conthe off,

Notthe NCConHyp

Figuadde

– Shared Uplin

ming ore configurinsion of the HP C Names that C address of ating the team

C teaming in Vn the NIC tea

UR NICs availaccess to andhin each NIC.Cs for the first

second team Bay 1 Port 1/

following granected to the network; Sma causing the N

te: The HypeHP NIC teamU by openingnfiguration Utiper-V role has

ure 99 - Uninsed.

nk Set with Activ

g NIC teamin ProLiant Netwwere set earli the NIC and

m.

Virtual Connecaming Utility alable and will managemen Select the Nteam.

will be used 3 and Bay 2

aphics providenetwork. In t

artLink will alteNIC teaming s

r-V role must bing Utility (NC

g the property lity” and click been added.

stall the HP NC

ve/Active Uplin

ng, ensure thawork Teamingier. You can compare it to

ct works the saand configure create two tet of the local o

NICs that are i

to provide acPort 1/3 as th

e an examplehe event of aer the NIC teasoftware to fa

be added befCU) is installe of one of thek Uninstall. N

CU, reinstall the

nks and 802.3a

at the Hyper-Vg Software (vealso highlight

o the MAC ad

ame way as i the availableeams. The firsoperating sysndicated as B

ccess to the guhe NICs for th

e of a Windown Uplink or swaming softwaril-over to the a

fore the NIC td, prior to the existing Loca

NCU version 1

NCU (version

ad (LACP) - Wi

V role has beeersion 10.10)t each NIC andress shown

in standard nee NICs for teast team, usingstem. You wilBay 1 Port 1/

uest VMs. Sehe second tea

ws 2008 R2 switch failure, tre to this evenalternate NIC

teaming softwe Hyper-V roleal Area Conne10.10 or later

10.10 or later

indows Hyper-V

en added, then and run the und select propin the VC ma

etwork configming. In this

g the two 1Gbl note the BAY1 and Bay 2

lect the NICs am.

server with TWthe SUS will lont, by turning .

ware is installee being addedections, selectr should be in

r), after the Hyp

V - Ethernet andSAN

n install the lautility. Noticeperties to verifnager, before

gurations. Sim example, web NICs will beY#-Port# indic Port 1/1 as t

that are indic

WO NIC teamose connectiothe server NIC

ed or configurd, uninstall tht the “HP Netwstalled after th

per-V role has b

d FCoE N 88

atest e the fy the e

mply, e have e used cation he

cated

ms and on to C port

red. If e work he

been

Scenario 4 –

Figuand

Figunetw

Variyou the ttagg

– Shared Uplin

ure 100 – Tea the Bay Port #

ure 101 – Thework. Note that

ious modes ca can select theteam that willging pass-thro

nk Set with Activ

am both NICs, u on the right. Y

e four NICs cont each Team lev

an be configue team and ed be used for t

ough is require

ve/Active Uplin

using the HP NYou typically w

nfigured within tverages one N

ured for NIC Tdit its propertithe VM Guested.

nks and 802.3a

etwork Configuould team the s

the profile (AppIC from Bay 1

Teaming, suchies. Typicallys will need to

ad (LACP) - Wi

uration Utility –same port numb

p-1) are teamed and 1 NIC fro

h as NFT or TLy, the default so have Promisc

indows Hyper-V

– Note the NIC ber to the oppo

d in pairs and m Bay 2.

LB. Once thesettings can bcuous mode e

V - Ethernet andSAN

numbers on theosite bay numb

connected to th

e Team is creabe used. Howenabled, if VLA

d FCoE N 89

e Left ber.

he

ated, wever, AN

Scenario 4 –

Figu

Figube c

NotePromnot etaggsupp

– Shared Uplin

ure 102 - View

ure 103 - If Vconnected to the

e: VLAN Promismiscuous is to aenabled, VLANging, an Etherneporting many V

nk Set with Activ

w – Network C

LAN pass-throue Hyper-V virtua

scuous is a featllow VLAN tag

N tagged frameset connection nLANs, without t

ve/Active Uplin

Connections – N

ugh is required,al switch.

ture that was adged frames to s will be dropp

needed to be dethe need to def

nks and 802.3a

NIC Team #1 (m

, Promiscuous m

dded to this vepass through th

ped. In earlier vefined for eachfine each in the

ad (LACP) - Wi

management) T

mode must be e

rsion of the HPhe NIC teamingversions of the

h VLAN. VLANe NCU.

indows Hyper-V

Team #2 Hyper

enabled on the

P NCU. The pug software. If V HP NCU, in or promiscuous p

V - Ethernet andSAN

r-V Guests

NIC Team tha

urpose of VAL VLAN Promiscurder to support provides the ab

d FCoE N 90

t will

ous is VLAN

bility of

C

Scenario 4 –

FiguCon

ConfigurinOn verscreaManADD

Figu

CreaNetwVLAmannetwcrea

– Shared Uplin

ure 104 - Bothnection 8 is TEA

ng Hyper-Vthe Windows

sion 10.10 or ated. Open thnager. SelectD.

ure 105 - Cre

ate the Virtuawork”, select

ANs 102-105)nagement opework, a warniated.

nk Set with Activ

h teamed and cAM 1 and Con

V Virtual Ns 2008 R2 ser later has beehe Hyper-V mt New Virtual

eate a new virtu

l network for TEAM #2 as . Optionally,

erating systemng box will a

ve/Active Uplin

connect to the nnnection 9 is Te

Network rver, verify thaen installed ananager conso Network, in t

ual network (ste

the guest Virtu the network c, if this networ

m to share this ppear and the

nks and 802.3a

network, each ceam 2 which w

at the Hyper-Vnd both NIC tole in Windowthe create Virt

ep 1)

ual Machinesconnection (wrk will not be network adae network ada

ad (LACP) - Wi

configured for will connect to th

V role has beeteams as descw 2008 R2 anrtual Network

. Name the vwhich is config used to manapter. Click apapter may res

indows Hyper-V

DHCP and recehe Hyper-v Virtu

en added andcribed above nd click on Vi box, select “E

virtual networgured for VLANage the host, dpply to createset as the virtu

V - Ethernet andSAN

eived an IP Adual Switch

d that HP NCUhave been irtual NetworkExternal” and

rk as “VM N tagging onde-select Allowe the virtual ual network is

d FCoE N 91

dress.

U

k d click

w

s

C

Scenario 4 –

Figu

Creating aClicthe VNetwThisto in

– Shared Uplin

ure 106 - Cre

a Virtual Mk “NEW” in tVirtual Machiwork created system will thnstall an opera

nk Set with Activ

eate the virtual n

Machine athe Hyper-V mne settings bo earlier, click hen be connecating system i

ve/Active Uplin

network (step 2

and connemanager and sox, click on th on “enable vcted to VLAN in the virtual m

nks and 802.3a

2)

ecting to thselect Virtual e network adirtual LAN ide 102, click Omachine.

ad (LACP) - Wi

he virtual Machine to c

dapter and coentification” a

OK to apply th

indows Hyper-V

network create a new vnfigure it to u

and input the Ve settings. Th

V - Ethernet andSAN

virtual machinse the VM VLAN ID 102he next step w

d FCoE N 92

ne. In

2. will be

T

Scenario 4 –

Figu

Testing VMAfte105Sett

Figu

– Shared Uplin

ure 107 - Con

M connecter installing W5, by simply ctings page.

ure 108 - VM

nk Set with Activ

nfigure the virtu

tivity Windows 2008changing the V

M2 has received

ve/Active Uplin

ual machine to f

8 R2 onto theVM ID that is

d an IP address

nks and 802.3a

function on VLA

VM, we can configured on

on VLAN 102

ad (LACP) - Wi

AN 102

move the VMn the Network

2

indows Hyper-V

M from VLAN k Adapter wit

V - Ethernet andSAN

102 through thin the VM

d FCoE N 93

VLAN

Scenario 4 –

By scan

Figu

Figu

– Shared Uplin

simply changi change the V

ure 109 - Mov

ure 110 - VM

nk Set with Activ

ing the VLAN VLAN that this

ve VM2 to VLA

2 has now rece

ve/Active Uplin

ID indicated s VM connects

AN 104 by cha

eived an IP add

nks and 802.3a

in the Settingss to.

nging the VLAN

dress on VLAN

ad (LACP) - Wi

s of the Netw

N ID used for th

104.

indows Hyper-V

work Adapter f

his VM.

V - Ethernet andSAN

for this VM, w

d FCoE N 94

we

R

Scenario 4 –

Results –FiguLUN

Figu

– Shared Uplin

– Windoure 111 - Dur

ure 112 - Win

nk Set with Activ

ows 200ring server boot

ndows 2008 R

ve/Active Uplin

08 R2 St, you can see t

2 Disk Adminis

nks and 802.3a

SAN Cothat Port 1 of th

strator. Note; t

ad (LACP) - Wi

onnectivihe FlexHBA is c

that D: is the SA

indows Hyper-V

ity connected to an

AN attached vo

V - Ethernet andSAN

n MSA1000 SA

olume

d FCoE N 95

AN


Summary We presented a Virtual Connect Network and SAN scenario by creating two shared uplink sets (SUS), each SUS is connected with TWO active uplinks; both SUS’ can actively pass traffic. We included a dual path SAN fabric for storage connectivity.

When VC profile App-1 is applied to the server in bay1 and is powered up, it has one NIC connected through FlexFabric module 1 (connected to “VLAN_101-1”), the second NIC is connected through FlexFabric module 2 (connected to VLAN_101-2). Each NIC is configured at 1Gb. These NICs are teamed and will be used to manage the Hyper-v host. Either NIC or path could be active. The second pair of NICs are also teamed and the team is configured in promiscuous mode to support multiple tagged VLANs (102-105). Each of these NICs is configured for 5Gb. Either NIC could be active. As a result, this server could access the network through either NIC or either uplink, depending on which NIC is active at the time. This host is also configured for FCoE based SAN access and connects to a SAN LUN to store the Guest VMs. Each FCoE port is configured for 4Gb of SAN bandwidth.

Additional NICs could be added within FlexFabric, by simply powering the server off and adding up to a total of 6 NICs, the NIC speed can then be adjusted accordingly to suit the needs of each NIC. If additional or less SAN bandwidth is required, the speed of the SAN connection can also be adjusted. If the FCoE SAN connections are not required, these could be deleted, in which two additional NIC ports would then be made available to the host.

As additional servers are added to the enclosure, simply create additional profiles, or copy existing profiles, configure the NICs for LAN and SAN fabrics as required and apply them to the appropriate server bays and power the server on.

We then added the Hyper-V role to the server and created a VM guest. The guest was configured for VLAN 102, and then was later moved to VLAN 104, by simply changing the VLAN id as configured in the VM settings tab. Additional VLANs can be configured within the Shared Uplink Set and presented to the NIC team supporting the guests; those VLANs would then be made available to Hyper-V manager and any VM guests.

Scenario 5 – Shared Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - vSphere - Ethernet and FCoE SAN 97

Scenario 5 – Shared Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - vSphere - Ethernet and FCoE SAN







Scenario 5 –

Fig1 aPortwithFab

– Shared Uplin

ure 113 - Pnd 2 on the fits 1 and 2 onh TWO uplinkbric B.

k Set with Activ

hysical View; irst network sw the second n

ks per fabric,

ve/Active Uplin

Shows two Ewitch and twonetwork switchfrom ports X1

nks and 802.3a

Ethernet uplinko Ethernet uplih. . The SAN and X2 on m

ad (LACP) - vSp

ks from Ports Xinks from Port

N fabrics are amodule 1 to Fa

phere - Ethernet

X5 and X6 onts X5 and X6 also connecteabric A and p

t and FCoE SAN

n Module 1 toon Module 2

ed redundantlyports X1 and X

N 98

o Ports to y, X2 to

I

S

V

Scenario 5 –

FigNICNICof thconBayFlexporMo

Installa

Switch cAppCiscapp

NotsamportThe con


– Shared Uplin

ure 114 - LoCs 1 and 2 arC3 5 and 6 arhe Shared Up

nnected, at 10ys 1 and 2. TxFabric modut X1 on Modudule 2 (Bay 2

ation a



e: when addime FlexFabric ts will need to SAN connectnecting to the


k Set with Activ

ogical View; tre connected tre connected plink Sets, VLA0Gb, to a netwThe FCoE SANle. In additioule 1 (Bay 1) a2).

and co

ation nd B provide aroCurve netwmes the followports are conf5. All frames be configurenk Set would ng the additiomodule to the

o be configuretion will be m

e FlexFabric m


ve/Active Uplin

the server blato VLAN_101to multiple ne

AN-Trunk-1 anwork switch, tN connectionsn, SAN Fabriand FCoE_B c

nfigura


wing informatiigured as VLA will be forwaed as (Default) be configuredonal uplinks toe same switch,ed for LACP wmade with redumodule must be


nks and 802.3a

de profile is c-x, NICs 3 an

etworks VLANnd VLAN-Trunkhrough Ports X are connectec FCoE_A coconnects to th

ation

the commandture. The confion: AN TRUNK poarding to Virtu) untagged, if d and set as “o the SUS, if t, in order to e

within the sameundant connee configured t


ad (LACP) - vSp

configured wind 4 are conn_103-x througk-2 respectiveX5 and X6 oned through ponnects to the

he existing SA


orts to supporual Connect wf so, then a co“Default”. the additionalensure all the e Link Aggreg



phere - Ethernet

th SIX FlexNICnected to VLAgh VLAN_105ely. The VLANn each FlexFaorts X1 and X2existing SAN

AN Fabric B th



l uplinks are cuplinks are ac

gation Group.h Fabric. SANIV logins.


t and FCoE SAN

C and 2 FlexHN_102-x and5-x, which areN-Trunks are abric Module 2 on each Fabric A thro

hrough port X



connecting froctive, the swit. N switch ports


mands to confi

N 99

HBAs. d e part

in

ough 1 on

ther a

y, one he

om the tch

s

In Once igure
















Scenario 5 –


Figu

Fig

Shared Uplink




ure 115 - Ethe

ure 116 - S

Set with Active




ernet Settings - M

et Fast MAC

e/Active Uplink




Map VLAN Tag

Cache (under

ks and 802.3ad




gs (under Ethe

r Ethernet Sett

d (LACP) - vSph


mmands are pr



ernet Settings

tings “Advanc

here - Ethernet a




s “Advanced S

ced Settings -

and FCoE SAN


w for reference



Settings)

Other)

N 101

rence d Link e.

alue e

b 0


Defining a new Shared Uplink Set (VLAN-Trunk-1) Connect Ports X5 and X6 of FlexFabric module 1 to Ports and 2 on switch 1 Create a SUS named “VLAN-Trunk-1” and connect it to FlexFabric Port X1 on Module 1 On the Virtual Connect Home page, select Define, Shared Uplink Set Insert Uplink Set Name as VLAN-Trunk-1 Select Add Port, then add the following port;




Enable Smartlink on ALL networks Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set


Defining a new Shared Uplink Set (VLAN-Trunk-2) Connect Ports X5 and X6 of FlexFabric module 2 to Ports 1 and 2 on switch 2 Create a SUS named “VLAN-Trunk-2” and connect it to FlexFabric Port X1 on Module 2 On the Virtual Connect Home page, select Define, Shared Uplink Set Insert Uplink Set Name as VLAN-Trunk-2 Select Add Port, then add the following port;




Enable Smartlink on ALL networks Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set







Scenario 5 –

Figu

Notthe savaswit

Figu

Shared Uplink

ure 117 - Defi


ure 118 - Acti

Set with Active

ne a Shared U


ive / Active Sh

e/Active Uplink

plink Set (VLAN


ared Uplink Se

ks and 802.3ad

N-Trunk-1) and


mation can be

et

d (LACP) - vSph

add VLANs


here - Ethernet a


and FCoE SAN


N 104

er of not

D

Scenario 5 –

Figu


Figu

Defining aCrea

Shared Uplink

ure 119 - Asso


ure 120 - Exa



Select the A

Select Apply

Set with Active

ociated Netwo



vided an Activ

mple of an Act


e/Active Uplink

rks tab


ve/Standby up

tive/Standby S

Fabric viaFCoE_A”


ks and 802.3ad



hard Uplink Se

a GUI


Distribution

d (LACP) - vSph



o, example be

et

ne, SAN Fabr

here - Ethernet a


ic to create th

and FCoE SAN



he first Fabric

N 105

each

erver odule, have s

D

Scenario 5 –

Crea


Figunum

Shared Uplink


Select the A

Select Apply


ure 121 - SAN

ure 122 – FCobers on the rig

Set with Active




N Login Distribu

oE SAN fabricsht

e/Active Uplink


I n be copied a


rts=1,2 LinkDis

ution Setting

s configured wi

ks and 802.3ad


Distribution

nd pasted into


th a single 4Gb

d (LACP) - vSph

ne, SAN Fabr

o an SSH bas

figure uplinks

b uplink per fa

here - Ethernet a

ic to create th

sed CLI sessio

as discussed

bric. Note the

and FCoE SAN

he second Fab

on with Virtual

above

bay and port

N 106

bric

l

D

Scenario 5 –

Figupow

Defining aWe wEach

Prio

NotNICVLAthro

Shared Uplink

ure 123 – FCowered on)

a Server Pwill create a seserver NIC wOn the mainCreate a serIn the NetwoSet the port In the NetwoSet the port Left click on ADD networIn the NetwoSet the port In the NetwoSet the port In the Netwo Configure f Leave the nIn the Netwo Configure f Leave the nExpand the Do not confiIn the Assignthey apply

r to applying

te: you shouldCs 1&2 shouldAN102-x (whic

ugh VLAN10

Set with Active

oE SAN Server

rofile erver profile will connect to n menu, selectrver profile caork Port 1 dro speed to Cusork Port 2 dro speed to Cus either of Port rk (add four aork Port 3 dro speed to Cusork Port 4 dro speed to Cusork Port 5 drofor networks Vnetwork speedork Port 6 drofor networks Vnetwork speedFCoE Connecigure FC SANn Profile to Se

the profile, e

d now have ad be connectech will be use5-x and FCoE

e/Active Uplink

Connections (A

with two servea specific nett Define, thenalled “ESX-1” op down box,stom at 1Gb op down box,stom at 1Gb 1 or Port 2 in

additional netwop down box,stom at 1.5Gbop down box,stom at 1.5Gbop down box,VLAN103-1 thd as Auto op down box,VLAN103-1 thd as Auto ctions box, fo


nsure that the


ed for VMotionE SAN fabrics

ks and 802.3ad

After server pro

er NICs. twork. Server Profile

select VLAN1

select VLAN1

n the Ethernet work connecti select VLAN1

b select VLAN1

b select Multiphrough VLAN

select Multiphrough VLAN

r Bay 1, selecnnection , locate the Se

server in Bay

e assigned to s VLAN101-x,n) NICs 5&6 s FCoE_A and

d (LACP) - vSph

ofiles have been

e

101-1

101-2

t Connections ions) 102-1

102-2

ple Networks 105-1

ple Networks 105-1

ct FCoE_A for

elect Location

y 1 is currently

Bay 2, with 6, NICs 3&4 shshould be con

d FCoE_B.

here - Ethernet a

n created, assig

box, then rig

r Bay 2, select

n drop down a

y OFF

6 Server NIC hould be connnnected to ne

and FCoE SAN

gned and serve

ght click and s

t FCoE_B

and select Bay

connections. nected to tworks VLAN

N 107

ers

select

y 2,

103-x

D

Scenario 5 –

Defining aThe Con# Caddset eset eaddaddaddaddaddaddaddaddaddaddset fset fpowassi

Figu

Notthe LOM

Shared Uplink

a Server P following comnnect Create Server d profile ESX-1enet-connectioenet-connectio

d enet-connectd enet-connectd enet-connectd server-port-md server-port-md server-port-md enet-connectd server-port-md server-port-md server-port-mfcoe-connectiofcoe-connectio

weroff server 1ign profile ESX

ure 124 - Defi


M:2-b.

Set with Active


Profile ESX-1 w1 on ESX-1 1 pxon ESX-1 2 pxtion ESX-1 pxetion ESX-1 pxetion ESX-1 pxe

map ESX-1:5 Vmap ESX-1:5 Vmap ESX-1:5 Vtion ESX-1 pxe

map ESX-1:6 Vmap ESX-1:6 Vmap ESX-1:6 Von ESX-1:1 Faon ESX-1:2 Fa1 X-1 enc0:2

ine a Server Pro


e/Active Uplink

CLI n be copied a

with 6 NICs a

xe=Enabled Nxe=Disabled Ne=Disabled Ne=Disabled Ne=Disabled VLAN_103-1 VVLAN_104-1 VVLAN_105-1 Ve=Disabled VLAN_103-2 VVLAN_104-2 VVLAN_105-2 Vabric=FCoE_Aabric=FCoE_B

ofile (ESX-1)


ks and 802.3ad

nd pasted into

and 2 FCoE S

Network=VLANNetwork=VLA

Network=VLANNetwork=VLAN

VLanId=103 VLanId=104 VLanId=105

VLanId=103 VLanId=104 VLanId=105

A SpeedType=B SpeedType=


d (LACP) - vSph

o an SSH bas

SAN connectio

N101-1 SpeeAN101-2 SpeeN102-1 SpeeN102-2 Spee

=4Gb =4Gb


here - Ethernet a

sed CLI sessio

ons

edType=CustoedType=Custo

edType=CustoedType=Custo


and FCoE SAN

on with Virtual

om Speed-100om Speed-10

om Speed-150om Speed-150


N 108

l

00 00

00 00

that -b and

Scenario 5 –

Figu

Notthe and

Figu

Shared Uplink

ure 125 - Defi

te: The speedFCoE connec

d LOM:2-b.

ure 126 - Con

Set with Active

ine a Server Pro

of the NIC ations are assi

nfigure Custom

e/Active Uplink

ofile (ESX-1)


NIC Speed - m

ks and 802.3ad

nections, as wwo SAN fabri

move the slider

d (LACP) - vSph

well as the MAics created ea

left/right to co

here - Ethernet a


onfigure NIC sp

and FCoE SAN


peed.

N 109

e that -b

Scenario 5 –

Figu

Figu

Shared Uplink

ure 127 - Con

ure 128 - Con

Set with Active

nfigure NIC Po

nfigure NIC Po

e/Active Uplink

rt 3 for Multiple

rt 4 for multiple

ks and 802.3ad

e Networks - VL

e Networks - VL

d (LACP) - vSph

LAN103-1 thro

LAN102-2 thro

here - Ethernet a

ough VLAN105

ough VLAN105

and FCoE SAN

5-1

-2

N 110

Scenario 5 –

Figu

Figu

Shared Uplink

ure 129 - Serv

ure 130 - By

Set with Active

ver Profile View

hovering over t

e/Active Uplink

w Bay 1

the “Multiple N

ks and 802.3ad

Networks” page

d (LACP) - vSph

e, a listing of th

here - Ethernet a

he connected N

and FCoE SAN

Networks is disp

N 111

played.


Review In this scenario we have created Two Shared Uplink Sets (SUS), providing support for 5 VLANs. Uplinks originating from the each FlexFabric Module connect to each SUS, by doing so we provide redundant connections out of the Virtual Connect domain. As multiple uplinks are used for each SUS, we have also leveraged LACP to improved uplink performance. In this scenario, all uplinks will be active. We also create two FCoE SAN Fabrics. We created a server profile, with two NICs connected to the same VLAN, VLAN_101-1 and VLAN_101-2, which provides the ability to sustain a link or module failure and not lose connection to the network. VLAN101-1 and VLAN101-2 are configured to support VLAN 101, frames will be presented to the NIC(s) without VLAN tags (untagged), these two NICs are connected to the same VLAN, but taking a different path out of the enclosure.

Network Ports 3 and 4 were added, these NICs will be connected to “Multiple Networks” and each NIC will then be configured for networks VLAN102-x through VLAN105-x. As these networks are tagging, frames will be presented to the server with VLAN tags. NICs 3 and 4 will be connected to the same vSwitch, which will provide load balancing across both NICs, as well as fail-over capabilities. VLAN tagged frames for these networks will be forwarded to the vSwitch and then passed on to the appropriate Virtual Machine. Additionally, FCoE port 1 is connected to SAN fabric FCoE_A and FCoE SAN port is connected to SAN Fabric FCoE_B, providing a multi-pathed connected to the SAN. The FCoE SAN fabric connects to each SAN fabric over a pair of uplinks per module. SAN logins are distributed across the multiple paths.

Results – vSphere Networking Examples We have successfully configured FlexFabric with a Share Uplink Set, supporting 5 VLANs and redundant SAN fabrics. We have created a server profile to connect to the various vNet with SIX NICs and the SAN fabrics using the FCoE connections created within the profile.


The following graphics show an ESX 4.0 server with SIX FlexNICs configured, two at 1Gb (console), two at 1.5Gb (VMotion) and two at 3.5Gb (Guests port groups). If we did not require SAN connectivity on this server, the FCoE connections could be deleted and the server would then have 8 NIC ports available to the OS. In addition, if we did not want FCoE connectivity and instead wanted to leverage iSCSI, we could delete the FCoE connected and re-create those connects as iSCSI connections, with offload and optionally iSCSI boot.

Note: the BL465c G7 and BL685c G7 utilize a NC551i chipset, whereas the BL460c G7, BL620c G6 and BL680c G7 utilizes an NC553i chipset. This scenario leverages a BL465c G7 server.

Scenario 5 –

Figu

Figu

Shared Uplink

ure 131 – ESX

ure 132 - ESX

Set with Active

X 4.0 Network

X 4.0 networkin

e/Active Uplink

Connections

ng - three vSwit

ks and 802.3ad

tches configure

d (LACP) - vSph

ed. (Note the N

here - Ethernet a

NIC speeds)

and FCoE SAN

N 113

Scenario 5 –

Figu

Figu

Figu

Shared Uplink

ure 133 - Con

ure 134 - VM

ure 135 – VM

Set with Active

nfiguring the vS

M1 configured fo

M1 on VLAN 10

e/Active Uplink

Switch for multi

or VLAN 105

05

ks and 802.3ad

ple port groups

d (LACP) - vSph

s / VLANs

here - Ethernet aand FCoE SAN

N 114

R

Scenario 5 –

Results –FiguLUN

Shared Uplink

– vSpheure 136 - Dur

N

Set with Active

re SANring server boo

e/Active Uplink

Conneot, you can see

ks and 802.3ad

ctivity that Port 1 of th

d (LACP) - vSph

he FlexHBA is c

here - Ethernet a

connected to a

and FCoE SAN

n MSA1000 SA

N 115

AN

S

Scenario 5 –

Figuthe S

SummaWe SUSpath

WhthroFlexconwhicconfconfacceSAN

Addto aIf adadjuadd

As aprofserv

Shared Uplink

ure 137 – ESXSAN.

ary presented a V

S is connectedh SAN fabric

en VC profileugh FlexFabri

xFabric modulnected to the ch is configurfigured at 1.5figured to supess and conneN bandwidth.

ditional NICs c total of 6 NICdditional or leusted. If the Fditional NIC p

additional serfiles, configurever bays and p

Set with Active

X storage config

Virtual Conned with TWO afor storage co

ESX-1 is appic module 1 (ce 2 (connecteconsole vSwited for VMotio

5Gb. The lastpport VLANs 1ects to a SAN


ess SAN bandFCoE SAN coorts would the

vers are addee the NICs fopower the ser

e/Active Uplink

guration, the Sh

ect Network scactive uplinks; onnectivity.

plied to the serconnected to ed to VLAN_1tch. The secoon and is cont pair of NICs103 through 1

N LUN to store


dwidth is requnnections areen be made a

ed to the enclor LAN and SArver on.

ks and 802.3ad

hared Storage

cenario by cre both SUS’ ca

rver in bay2 a“VLAN_101-01-2). Each N

ond pair of Nnected to VLAs 5 and 6, are105. This hose the Guest VM

xFabric, by sin be adjustedired, the spee not required,

available to th

osure, simply AN fabrics as

d (LACP) - vSph

LUN is provide

eating two shan actively pa

and is powere1”), the seconNIC is configuICs are conne

AN102-x throue connected tost is also confMs. Each FCo

mply powerind accordingly ed of the SAN, these could he host.

create additio required and

here - Ethernet a

ed through the F


ed up, it has ond NIC is conured at 1Gb. ected to the seugh NICs 3 ao the third vSwigured for FCoE port is con


N connection cbe deleted, in

onal profiles, d apply them t

and FCoE SAN

FCoE connectio


one NIC connnnected throug These NICs econd vSwitchand 5 which awitch, which i

CoE based SAnfigured for 4G

off and addineds of each Ncan also be n which two

or copy existto the approp

N 116

ons to

ch dual

nected gh are h, are is N Gb of

ng up NIC.

ting priate

Scenario 6 – Tunneled VLANs and Shared Uplink Set with Active/Active Uplinks - vSphere Ethernet and FCoE SAN 117

Scenario 6 – Tunneled VLANs and Shared Uplink Set with Active/Active Uplinks - vSphere Ethernet and FCoE SAN

Overview This scenario will implement a Shared Uplink Set (SUS) to provide support for individual VLANs for management connectivity and a Tunneled vNet to support many VLANs. In this scenario, the upstream network switches connect a shared uplink set to one port on each FlexFabric module, a second switch port connects to each FlexFabric module to support the large number of VLANs.


When configuring Virtual Connect, we can provide several ways to implement network fail-over or redundancy. One option would be to connect TWO uplinks to a single virtual connect network; those two uplinks would connect from different Virtual Connect modules within the enclosure and could then connect to the same upstream switch or two different upstream switches, depending on your redundancy needs. An alternative would be to configure TWO separate Virtual Connect networks, each with a single, or multiple, uplinks configured. Each option has its advantages and disadvantages. We will review the first option in this scenario.




Scenario 6 –

Fig1 aPortVLAand

– Tunneled VLA

ure 138 - Pnd 2 on the fits 1 and 2 on

ANs. The SANd X2 on modu

ANs and Shared

hysical View; irst network sw the second n

N fabrics are le 1 to Fabric

d Uplink Set wit

Shows two Ewitch and twonetwork switchconnected red

c A and ports

th Active/Activ

Ethernet uplinko Ethernet uplih. The uplinksdundantly, wi X1 and X2 to

ve Uplinks - vSp

ks from Ports Xinks from Ports from each mith TWO uplino Fabric B.

phere Ethernet a

X5 and X6 onts X5 and X6 module will sunks per fabric

and FCoE SAN

n Module 1 toon Module 2 pport differen

c, from ports X

N 118

o Ports to nce X1

I

S

Scenario 6 –

FigNICare andto aSANFCoFCo

Installa

Switch cAppCiscapp

NotconactivThe con

– Tunneled VLA

ure 139 - LoCs 1 and 2 ar part of the Sh

d 6 are connea network switN connectionsoE_A connectsoE_B connects

ation a


The switch psupport VLAVirtual ConnThe switch psupport VLAVLAN tags. be tagged. upstream swVirtual Conn

e: when addinecting from tve, the switch SAN connectnecting to the

ANs and Shared

ogical View; tre connected thared Uplink

ected to VLANtch, through Ps are connectes to the existins to the existin

and co

ation nd B provide aroCurve netwmes the followports connectiANs 101 and nect with VLANports connecteANs 2000 thro This will formBy using a VL

witch ports andnect configurang the additiothe same Flex ports will neetion will be m

e FlexFabric m

d Uplink Set wit

the server blato VLAN_101Sets, VLAN-Tr_Many_x, wh

Ports X5 and Xed through pong SAN Fabring SAN Fabric

nfigura


wing informating to VLAN_T102 for manaN tags. ed to VLAN_Mough 3000. Am the VLAN tuLAN tunnel in d will be presation would beonal uplinks toxFabric moduled to be confi


th Active/Activ

de profile is c-x, NICs 3 anrunk-1 and VLhich are vNet X6 on each Florts X1 and X2ic A through pc B through p

ation

the commandture. The confion: Trunk_x are coagement and

Many_x are coAll frames wilunnel. When this fashion, asented to Virtue required to o a SUS or VLe to the sameigured for LACundant connee configured t

ve Uplinks - vSp

configured wind 4 are connLAN-Trunk-2 re Tunnels. All uexFabric Mod2 on each Fleports X1 and Xports X1 and X


onfigured as V VMotion. Al

onfigured as Vll be forwardi VLAN tunneliadditional VLA

ual Connect tr present theseLAN Tunnel, ie switch, in ordCP within the


phere Ethernet a

th SIX FlexNICnected to VLAespectively. Iuplinks are codule in Bays 1exFabric moduX2 on ModuleX2 on Module


VLAN TRUNKl frames will b

VLAN Trunk/tng to Virtual Cing is configuANs can be aransparently, ne additional Vf the additionder to ensure same Link Agh Fabric. SANIV logins.

and FCoE SAN

C and 2 FlexHN_102-x, whIn addition, Nonnected at 101 and 2. The ule. SAN Fabe 1 (Bay 1) ae 2 (Bay 2).


K/tagging porbe forwarding

tagging ports Connect with red all traffic added to the no changes toLANs to the sal uplinks are all the uplinkggregation GrN switch ports

N 119

HBAs. ich

NIC 5 0Gb, FCoE bric nd

ther a

rts to g to

to must

o the erver.

e ks are roup. s


VC CLI commands Many of the configuration settings within VC can also be accomplished via a CLI command set. In order to connect to VC via a CLI, open an SSH connection to the IP address of the active VCM. Once logged in, VC provides a CLI with help menus. Through this scenario the CLI commands to configure VC for each setting will also be provided.

Configuring the VC module Physically connect Port 1 of Network switch 1 to Port X5 on the VC module in Bay 1 Physically connect Port 1 of Network switch 2 to Port X6 on the VC module in Bay 1 Physically connect Port 1 of the second Network switch to Port X5 of the VC module in Bay 2 Physically connect Port 1 of Network switch 1 to Port X6 on the VC module in Bay 2, if you have

only one network switch, connect VC port 1 (Bay 2) to an alternate port on the same switch. This will NOT create a network loop and Spanning Tree is not required

Physically connect Ports X1 and X2 on the FlexFabric module in Bay 1 to switch port in SAN Fabric A

Physically connect Ports X1 and X2 on the FlexFabric module in Bay 2 to switch port in SAN Fabric B



Configuring the VC Module for Tunnel VLAN Tags and Fast Mac Cache Fail-over via GUI (Ethernet settings)


Enable Tunnel VLAN Tags within Virtual Connect On the Virtual Connect Manager screen, Left pane, click Ethernet Settings, Advanced Settings Select Tunnel VLAN Tags (click apply) Optionally, select a preferred/Maximum link speed Select apply Select the Other tab Select Fast MAC Cache Fail-over with a refresh of 5 Select Apply

Co

Scenario 6 –

Configurinover via C

The Con


Figu

– Tunneled VLA

ng the VC CLI (Ethern

following comnnect

# Set Advandisabled set enet-vlanset mac-cach




ure 140 - Ethe

ANs and Shared

Module fet settingsmmand(s) can

nced Ethernet

n vlantagcontrhe Enabled=T




ernet Settings –

d Uplink Set wit

for Tunnel s) n be copied a

Settings to "T

rol=tunnel True Refresh=5




Tunnel VLAN T

th Active/Activ

VLAN Ta

and pasted int

Tunnel VLAN T

5




Tags (under E

ve Uplinks - vSp

ags and Fa

to an SSH bas

Tags" and set


mmands are pr



Ethernet Settin

phere Ethernet a

ast Mac C

sed CLI sessio

t "Force serve




gs “Advanced

and FCoE SAN

Cache Fail

on with Virtua

er connections


w for reference



d Settings)

N 121

l-

l

s" to

rence d Link e.

alue e

b 0

D

D

Scenario 6 –

Fig

Defining aConneCreat

Defining aConneCreat

– Tunneled VLA

ure 141 - S

a new Shaect Port X5 of e a SUS name

On the Virtua

Insert Uplink S

Select Add Po

Add Network(VLAN)” head

Enable Smart

Optionally, ifEnabled

Click Apply

a new Shaect Port X5 of e a SUS name

On the VirtuInsert UplinkSelect Add P

Add NetwoNetworks (V

Enable SmaOptionally, Native to EnClick Apply

ANs and Shared

et Fast MAC

ared Uplin FlexFabric med “VLAN-Tru

al Connect Hom

Set Name as V

ort, then add thEnclosure 1, Ba

ks as follows (toder, the select AVLAN101-1 = VLAN102-1 =

tlink on ALL netw

one of the VLA

ared Uplin FlexFabric med “VLAN-Truual Connect Hk Set Name asPort, then addEnclosure 1, Ba

rks as followsVLAN)” headeVLAN101-2 = VLAN102-2 =

rtlink on ALL nif one of the V

nabled

d Uplink Set wit

Cache (under

nk Set (VLAmodule 1 to Pounk-1” and come page, select

VLAN-Trunk-1

he following poay 1, Port X5

o add a networADD; VLAN ID=101 VLAN ID=102

works

ANs is configur

nk Set (VLAmodule 2 to Pounk-2” and coHome page, ses VLAN-Trunkd the followingay 2, Port X5

(to add a neter, the select A VLAN ID=101 VLAN ID=102

networks VLANs is conf

th Active/Activ

r Ethernet Sett

AN-Trunk-ort 1 on switchnnect it to Fle Define, Shared

rt;

rk, right click on

ed as Default/u

AN-Trunk-2ort 1 on switchnnect it to Fleelect Define, Sk-2 g port;

twork, right cADD;

figured as De

ve Uplinks - vSp

tings “Advanc

1) h 1 exFabric Port Xd Uplink Set

n the grey bar

untagged, on t

2) h 2 exFabric Port XShared Uplink

lick on the gre

efault/untagge

phere Ethernet a

ced Settings -

X5 on Module

under the “Asso

hat VLAN only,

X5 on Modulek Set

ey bar under

ed, on that VL

and FCoE SAN

Other)

e 1

ociate Network

, set Native to

e 2

the “Associat

LAN only, set

N 122

ks

te

D

Scenario 6 –

Defining twThe fo

The foConne

The fo

Figu

Notthe savaswit

– Tunneled VLA

wo new Sollowing scrip

ollowing commect

# Create Sadd uplinkadd uplink # Create Nadd netwoSet Netwoadd netwoSet Netwo

ollowing scrip# Create Sadd uplinkadd uplink # Create Nadd netwoSet Netwoadd netwoSet Netwo

ure 142 - Defi


ANs and Shared

Shared Upt can be used

mand(s) can b

Shared Uplinkkset VLAN-Trunkport enc0:1:X

Networks VLAork VLAN101-rk VLAN101-

ork VLAN102-rk VLAN102-

t can be usedShared Uplinkkset VLAN-Trunkport enc0:2:X

Networks VLAork VLAN101-rk VLAN101-

ork VLAN102-rk VLAN102-

ne a Shared U


d Uplink Set wit

plink Sets vd to create the

be copied and

Set "VLAN-Trnk-1 X5 Uplinkset=

N101-1 throu-1 uplinkset=V1 SmartLink=E-1 uplinkset=V1 SmartLink=E

d to create the Set "VLAN-Trnk-2 X5 Uplinkset=

N101-2 throu-2 uplinkset=V2 SmartLink=E-2 uplinkset=V2 SmartLink=E

plink Set (VLAN


th Active/Activ

via CLI e first Shared U

d pasted into

runk-1" and c

VLAN-Trunk-1

ugh VLAN102VLAN-Trunk-1 Enabled VLAN-Trunk-1 Enabled

e Second Sharrunk-2" and c

VLAN-Trunk-2

ugh VLAN102VLAN-Trunk-2 Enabled VLAN-Trunk-2 Enabled

N-Trunk-1) and


mation can be

ve Uplinks - vSp

Uplink Set (VL

an SSH based

configure uplin

speed=auto

2-1 for Shared VLanID=101

VLanID=102

red Uplink Seconfigure uplin

2 speed=auto

2-2 for Shared VLanID=101

VLanID=102

add VLANs


phere Ethernet a

LAN-Trunk-1)

d CLI session

nks

d Uplink Set "

et (VLAN-Trunknks

d Uplink Set "


and FCoE SAN

with Virtual

VLAN-Trunk-1

k-2)

VLAN-Trunk-2


N 123

1"

2"

er of not

Scenario 6 –

Figu

Figu


Figu

– Tunneled VLA

ure 143 - Acti

ure 144 - Asso


ure 145 - Exa

ANs and Shared

ive / Active Sh

ociated Netwo



vided an Activ

mple of an Act

d Uplink Set wit

ared Uplink Se

rks tab


ve/Standby up

tive/Standby S

th Active/Activ

et



hard Uplink Se

ve Uplinks - vSp



o, example be

et

phere Ethernet a


and FCoE SAN



N 124

each

erver odule, have s


Defining two new vNets via GUI Create a vNet and name it “VLAN-Many-1” On the Virtual Connect Manager screen, click Define, Ethernet Network to create a vNet Enter the Network Name of “VLAN-Many-1”

Select Smart Link Select Enable VLAN Tunneling

Select Add Port, then add the following ports; Enclosure 1, Bay 1, Port X6

Leave Connection Mode as Auto Select Apply

Create a vNet and name it “VLAN-Many-2” On the Virtual Connect Manager screen, click Define, Ethernet Network to create a vNet Enter the Network Name of “VLAN-Many-2”

Select Smart Link Select Enable VLAN Tunneling

Select Add Port, then add the following ports; Enclosure 1, Bay 2, Port X6

Leave Connection Mode as Auto Select Apply

Note: By creating TWO vNets we have provided a redundant path to the network. As each uplink originates from a different VC module within each vNet, both uplinks will be active. This configuration provides the ability to lose an uplink cable, network switch or even a VC module.

Note: Smart Link – In this configuration Smartlink SHOULD be enabled. Smartlink is used to turn off downlink ports within Virtual Connect if ALL available uplinks to a vNet or SUS are down. In this scenario if an upstream switch or all cables to a vNet were to fail, VC would turn off the downlink ports connect to that vNet, which would then force the NIC Teaming software to fail-over to the alternate NIC.

Defining two new vNets via CLI The following command(s) can be copied and pasted into an SSH based CLI session with Virtual Connect

# Create the vNet "VLAN-Many" and configure uplinks as discussed above add Network VLAN-Many-1 add uplinkport enc0:1:X6 Network=VLAN-Many-1 speed=auto set network VLAN-Many-1 SmartLink=Enabled VLanTunnel=Enabled add Network VLAN-Many-2 add uplinkport enc0:2:X6 Network=VLAN-Many-2 speed=auto set network VLAN-Many-2 SmartLink=Enabled VLanTunnel=Enabled

D

D

Scenario 6 –

Figu

Defining aCre

Crea

Defining SThe Con#CrAddAdd

– Tunneled VLA

ure 146 - Def

a new (FCeate a Fabric


Select the A

Select Apply


Select the A

Select Apply


ANs and Shared

fine a vNet Tun

oE) SAN c and name iual Connect Metwork Name Port, then addEnclosure 1, BaEnclosure 1, Ba

dvanced SettiSelect Automat

y

Fabric and naual Connect Metwork Name Port, then addEnclosure 1, BaEnclosure 1, Ba

dvanced SettiSelect Automat

y



d Uplink Set wit

nnel (VLAN-Man

Fabric viait “FCoE_A”

Manager scree of “FCoE_A”d the followingay 1, Port X1 ay 1, Port X2

ings tic Login Re-Dis

ame it “FCoE_Manager scree of “FCoE_B” d the followingay 2, Port X1 ay 2, Port X2

ings tic Login Re-Dis

I n be copied a


rts=1,2 LinkDis

th Active/Activ

ny-1), VLANs w

a GUI en, click Defin” g ports;

tribution

_B” en, click Defin g ports;

tribution

nd pasted into


ve Uplinks - vSp

will be tunneled

ne, SAN Fabr

ne, SAN Fabr

o an SSH bas

figure uplinks

phere Ethernet a

ic to create th

ic to create th

sed CLI sessio

as discussed

and FCoE SAN

he first Fabric

he second Fab

on with Virtual

above

N 126

bric

l

Scenario 6 –

Figu

Figuport

Figuserv

– Tunneled VLA

ure 147 - SA

ure 148 – FCt numbers on

ure 149 – FCvers powered

ANs and Shared

AN Login Dist

CoE SAN fabthe right

CoE SAN Seron)

d Uplink Set wit

tribution Settin

brics configure

rver Connectio

th Active/Activ

ng

ed with a two

ons (After serv

ve Uplinks - vSp

4Gb uplinks

ver profiles ha

phere Ethernet a

per fabric. N

ave been crea

and FCoE SAN

Note the bay a

ated, assigned

N 127

and

d and


Defining a Server Profile We will create a server profile with two server NICs. Each server NIC will connect to a specific network. On the main menu, select Define, then Server Profile Create a server profile called “ESX-1” In the Network Port 1 drop down box, select VLAN101-1

Set the port speed to Custom at 1Gb In the Network Port 2 drop down box, select VLAN101-2

Set the port speed to Custom at 1Gb Left click on either of Port 1 or Port 2 in the Ethernet Connections box, then right click and select

ADD network (add four additional network connections) In the Network Port 3 drop down box, select VLAN102-1

Set the port speed to Custom at 1.5Gb In the Network Port 4 drop down box, select VLAN102-2

Set the port speed to Custom at 1.5Gb In the Network Port 5 drop down box, select VLAN-Many-1

Leave the network speed as Auto In the Network Port 6 drop down box, select VLAN-Many-2

Leave the network speed as Auto Expand the FCoE Connections box, for Bay 1, select FCoE_A for Bay 2, select FCoE_B Do not configure FC SAN or iSCSI Connection In the Assign Profile to Server Bay box, locate the Select Location drop down and select Bay 2,

they apply Prior to applying the profile, ensure that the server in Bay 1 is currently OFF

Note: You should now have a server profile assigned to Bay 2, with 6 Server NIC connections. NICs 1&2 should be connected to networks VLAN101-x, NICs 3&4 should be connected to VLAN102-x (which will be used for VMotion) NICs 5&6 should be connected to networks VLAN-Many-x and FCoE SAN fabrics FCoE_A and FCoE_B.

Defining a Server Profile via CLI The following command(s) can be copied and pasted into an SSH based CLI session with Virtual Connect # Create Server Profile ESX-1 with 6 NICs and 2 FCoE SAN connections add profile ESX-1 set enet-connection ESX-1 1 pxe=Enabled Network=VLAN101-1 SpeedType=Custom Speed=1000 set enet-connection ESX-1 2 pxe=Disabled Network=VLAN101-2 SpeedType=Custom Speed=1000 add enet-connection ESX-1 pxe=Disabled Network=VLAN102-1 SpeedType=Custom Speed=1500 add enet-connection ESX-1 pxe=Disabled Network=VLAN102-2 SpeedType=Custom Speed=1500 add enet-connection ESX-1 pxe=Disabled Network=VLAN-Many-1 SpeedType=Auto add enet-connection ESX-1 pxe=Disabled Network=VLAN-Many-2 SpeedType=Auto set fcoe-connection ESX-1:1 Fabric=FCoE_A SpeedType=4Gb set fcoe-connection ESX-1:2 Fabric=FCoE_B SpeedType=4Gb poweroff server 1 assign profile ESX-1 enc0:2

Scenario 6 –

Figu

Notthe LOMextra

Figu

– Tunneled VLA

ure 150 - Defi


M:2-b. If 4Gba bandwidth

ure 151 - Con

ANs and Shared

ine a Server Pro

of the NIC antions are assigb of FC bandwcan then be a

nfigure Custom

d Uplink Set wit

ofile (ESX-1)

nd SAN conngned to the twwidth is not reapplied to one

NIC Speed - m

th Active/Activ

nections, as wwo SAN fabricequired, the FCe of, or distrib

move the slider

ve Uplinks - vSp

well as the MAcs created eaCoE connectiobuted across,

left/right to co

phere Ethernet a

AC and WWNarlier and use on speed can the NIC conn

onfigure NIC sp

and FCoE SAN

N. Also, note ports LOM:1-n be reduced, nections.

peed.

N 129

that -b and that

R

Scenario 6 –

Figu

ReviewIn thUplreduWesup WeVLAthe presVLAcon

NetMaVLAacroforw

– Tunneled VLA

ure 152 - Serv

w his scenario winks originatinundant conne

e also create twport to conne

e created a seAN101-2, whinetwork. VLAsented to the

AN, but takingnect to VLAN

twork Ports 5 ny-2”. Netwo

AN. NICs 5 anoss both NICswarded to the

ANs and Shared

ver Profile View

we have createng from the eaections out of two FCoE SANction of many

rver profile, wich provides t

AN101-1 and NIC(s) withou

g a different p 102 (VMotio

and 6 were aorks VLAN-Mand 6 will be cs, as well as f vSwitch and

d Uplink Set wit

w Bay 2

ed Two Shareach FlexFabrithe Virtual CoN Fabrics. In y VLANs to Fle

with two NICshe ability to s VLAN101-2 aut VLAN tags path out of theon) through VL

added, these Nany-x are conconnected to tail-over capab then passed o

th Active/Activ

ed Uplink Setsc Module con

onnect domain addition, we exFabric.

s connected toustain a link oare configure(untagged), th

e enclosure. TLAN102-1 an

NICs will be cnected to switthe same vSwbilities. VLANon to the app

ve Uplinks - vSp

s (SUS), provinnect to each n. In this scena created TWO

o the same VLAor module fai

ed to support Vhese two NICThis is also trund VLAN102-2

connected to tch ports conf

witch, which wN tagged framropriate Virtu

phere Ethernet a

ding support SUS, by doinario, all uplinO vNets (redu

AN, VLAN10lure and not lVLAN 101, fr

Cs are connecte for NICs 3 2.

“VLAN-Manyfigured for a l

will provide loames for these nal Machine.

and FCoE SAN

for 2 VLANs. ng so we provks will be act

undant) Tunne

01-1 and ose connectiorames will be ted to the sam and 4, which

y-1 and VLAN-large number ad balancing networks will

N 130

vide ive.

els to

on to me h

- of be

R

Scenario 6 –

AddSAN The are

Results –We reduNIC

AlthG7 they

The twoconNICto lecon

NotG6

Figu

– Tunneled VLA

ditionally, FCoN Fabric FCo

FCoE SAN fa distributed ac

– vSphe have successundant SAN f

Cs and the SA


following gra at 1.5Gb (VMnectivity on th

C ports availaeverage iSCSnections, with


ure 153 – ESX

ANs and Shared

oE port 1 is coE_B, providin

abric connectcross the mult

re Netwsfully configurabrics. We hN fabrics usin


aphics show aMotion) and this server, the ble to the OSI, we could deh offload and


X 4.0 Network

d Uplink Set wit

onnected to Sg a multi-path

s to each SANiple paths.

working ed FlexFabrichave created ng the FCoE c


an ESX 4.0 setwo at 3.5Gb FCoE connec. In addition,elete the FCoE optionally iSC


Connections

th Active/Activ

AN fabric FChed connected

N fabric over

Examplc with a Sharea server profi

connections cr


erver with SIX (Guests port ctions could b, if we did noE connected aCSI boot.

ze a NC551i ipset. This sce

ve Uplinks - vSp

CoE_A and FCd to the SAN.

a pair of upli

les e Uplink Set, sle to connect reated within


N adapters.

FlexNICs con groups). If wbe deleted andt want FCoE cand re-create

chipset, wherenario leverag

phere Ethernet a

CoE SAN port

inks per modu

supporting ma to the various the profile.


nfigured, two we did not reqd the server wconnectivity a those connec

reas the BL46ges a BL465c

and FCoE SAN

t is connected

ule. SAN log

any VLANs ans vNet with SI


at 1Gb (consquire SAN would then havand instead wcts as iSCSI

60c G7, BL62c G7 server.

N 131

d to

ins

nd IX

e in r,

sole),

ve 8 anted

0c

Scenario 6 –

FiguNot

– Tunneled VLA

ure 154 - ESXte: add port gr

ANs and Shared

X 4.0 networkinroups for the ap

d Uplink Set wit

ng - three vSwitppropriate VLA

th Active/Activ

tches configureNs

ve Uplinks - vSp

ed. (Note the N

phere Ethernet a

NIC speeds)

and FCoE SAN

N 132

Scenario 6 –

Figu

Figu

– Tunneled VLA

ure 155 - Con

ure 156 - VM

ANs and Shared

nfiguring the vS

M1 configured fo

d Uplink Set wit

Switch for multi

or VLAN 2793

th Active/Activ

ple port groups

ve Uplinks - vSp

s / VLANs

phere Ethernet aand FCoE SAN

N 133

R

Scenario 6 –

Figu

Results –FiguLUN

– Tunneled VLA

ure 157 – VM

– vSpheure 158 - Dur

N

ANs and Shared

M1 on VLAN 27

re SANring server boo

d Uplink Set wit

793

Conneot, you can see

th Active/Activ

ctivity that Port 1 of th

ve Uplinks - vSp

he FlexHBA is c

phere Ethernet a

connected to a

and FCoE SAN

n MSA1000 SA

N 134

AN

S

Scenario 6 –

Figuthe S

SummaWe SUScreafor s

WhthroFlexconsecoandvSwon twoubaseconf

Addto aIf adadjuadd

As aprofserv

– Tunneled VLA

ure 159 – ESXSAN.

ary presented a V

S is connectedated two vNetstorage conne

en VC profileugh FlexFabri

xFabric modulnected to the ond vSwitch, w 4 which are

witch, which ishe network sw

uld then be creed SAN accefigured for 4G

ditional NICs c total of 6 NICdditional or leusted. If the Fditional NIC p

additional serfiles, configurever bays and p

ANs and Shared

X storage config

Virtual Conned with ONE act Tunnels, whiectivity.

ESX-1 is appic module 1 (ce 2 (connecteconsole vSwitwhich is confi configured ats configured towitch ports, Veated to supp

ess and conneGb of SAN ba


ess SAN bandFCoE SAN coorts would the

vers are addee the NICs fopower the ser

d Uplink Set wit

guration, the Sh

ect Network scctive uplink; bch will pass ta

plied to the serconnected to ed to VLAN_1tch for managigured for VMt 1.5Gb. Theo support manirtual connect

port connectivicts to a SAN andwidth.


dwidth is requnnections areen be made a

ed to the enclor LAN and SArver on.

th Active/Activ

hared Storage

cenario by creboth SUS’ canagged frames

rver in bay2 a“VLAN_101-01-2). Each N

gement. The sMotion and is ce last pair of Nny tagged VLAt will simply pity to each VLA LUN to store

xFabric, by sin be adjustedired, the spee not required,

available to th

osure, simply AN fabrics as

ve Uplinks - vSp

LUN is provide

eating two shn actively passs. We also in

and is powere1”), the seconNIC is configusecond pair oconnected to NICs 5 and 6ANs. As addass those VLA

LAN. This hos the Guest VM

mply powerind accordingly ed of the SAN, these could he host.

create additio required and

phere Ethernet a

ed through the F

hared uplink ss traffic. In adncluded a dua

ed up, it has ond NIC is conured at 1Gb. of NICs are co VLAN102-x t, are connect

ditional VLANsANs to the swst is also conf

Ms. Each FCo


N connection cbe deleted, in

onal profiles, d apply them t

and FCoE SAN

FCoE connectio

ets (SUS), eacddition, we al path SAN f

one NIC connnnected throug These NICs onnected to ththrough NICs ted to the thirds are configuritch. A port gigured for FC

oE port is

off and addineds of each Ncan also be n which two

or copy existto the approp

N 135

ons to

ch

fabric

nected gh are he 3 d red group oE

ng up NIC.

ting priate

Appendix A – Scenario-based CISCO Command Line Reference 136

Appendix A – Scenario-based CISCO Command Line Reference

All of the following commands in this appendix assume an unaltered factory default configuration before execution of the switch commands.

Scenario 1– Cisco IOS command line configuration (Simple vNet with Active/Standby Uplinks) Connect to the Cisco switch servicing the VC-Enet uplink ports and enter the following IOS commands.

NOTE: If two switches are being used, issue the same commands on the second switch.

Table 1 Cisco IOS command line configuration

Command Shortcut Description

>enable >en Privilege mode

#configure terminal #config t Configure via terminal

#interface gigabitethernet0/1 #int gi1/0/1 Focus on Port 1

#switchport mode access #sw mo ac Set port 1 for Single VLAN mode

#switchport access vlan 1 #sw ac vl 1 Allow Port 1 access to VLAN 1

#spanning-tree portfast trunk #sp portf tr Enable portfast on Port 1

#exit #exit Remove focus from Port 1

#show vlan brief #sh vl br Display all VLANs

#show interface gigabitethernet0/1 status

#sh int gi0/1 status Display the status of Port 1

#copy running-config startup-config (For permanent changes only)

#cop ru st Save the running configuration to NVRAM. Otherwise, the changes will be lost on the next reboot.


Scenarios 2 through 5 - Cisco IOS command line configuration (Shared Uplink Set with VLAN tagging and 802.3ad (LACP)) Connect to the Cisco switch servicing the VC-Enet uplink ports and enter the following IOS commands.


Table 2 Cisco IOS command line configuration (802.1Q, 802.3ad)




#interface gigabitethernet0/1 #int gi0/1 Focus on Port 1

#switchport trunk allowed vlan 101-105

#sw tr ac vl 101-105 Configure port for VLANs 101 through 105

#switchport mode trunk #sw mo tr Enable trunking on Port 1

#channel-protocol lacp #channel-p l Configure port 20 for 802.3ad LACP

#channel-group 10 mode active #channel-g 10 mo ac Enable channel group 10


#exit #ex Remove focus from Port 1



#sw tr ac vl 101-105 Configure Port 2 for VLANs 101 through 105


#channel-protocol lacp #channel-p l Configure Port 21 for 802.3ad LACP

#channel-group 10 mode active #channel-g 10 mo ac Enable channel group 10



#show lacp 10 internal #sh la 10 i Show the LACP group 10 configuration

#show etherchannel summary #sh eth sum Show the etherchannel configuration

#show interface port-channel10 trunk

#sh int port-channel 10 tr Show the port channel 10 trunk configuration




Scenarios 6 - Cisco IOS command line configuration (Shared Uplink Set with VLAN tagging) Connect to the Cisco switch servicing the VC-Enet uplink ports and enter the following IOS commands.


Table 3 Cisco IOS command line configuration (802.1Q)






#sw tr ac vl 101-102 Configure port for VLANs 101 through 102





#switchport trunk allowed vlan 2102,2113,2507,2793

#sw tr ac vl 2102,2113,2507,2973

Configure port for VLANs 2102,2113,2507,2793






Appendix B – Scenario-based ProCurve Command Line Reference 139

Appendix B – Scenario-based ProCurve Command Line Reference

Scenario – ProCurve command line configuration (Simple vNet with Active/Standby Uplinks) Connect to the ProCurve switch servicing the VC-Enet uplink ports and enter the following commands.


Table 4 ProCurve command line configuration (simple network)



#configure terminal #conf Configure in global mode

#span #span Enables spanning-tree (MSTP mode by default) Not required for Virtuakl

#vlan 1 untagged [Ethernet] 1 #vlan 1 untag 1 Allow VLAN 1 on Port 1 , and set Port 1 to untagged mode

#spanning-tree 1 admin-edge-port #span 1 admin-edge Set Port 1 to be an edge port (non bridging port). Note: port is set by default in “auto-edge” mode which automatically sets port to Edge if no BPDU are received after 3 sec.

#show interface brief 1 #sh int br 1 Display the status of Port 1

#show vlan ports 1 detail # show vlan ports 1 detail Displays the VLAN detail for Port 1

#show vlan 1 #sh vlan 1 Display VLAN 1 port information

#write memory (For permanent changes only)

#write mem Save the running configuration to NVRAM. Otherwise, the changes will be lost on the next reboot.


Scenarios 2 through 5 – ProCurve command line configuration (Shared Uplink Set with VLAN tagging and 802.3ad (LACP) Connect to the ProCurve switch servicing the VC-Enet uplink ports and enter the following commands.


Table 5 ProCurve command line configuration (tagged VLANs)




#span #span Enables spanning-tree (MSTP mode by default)

#trunk 1-2 trk1 lacp #trunk 1-2 trk1 lacp Configure LACP port-trunk 1 to include Ports 1 & 2

#vlan 101 tagged trk1 #vlan 101 tag trk1 Allow VLAN 101 on Ports 1 and 2. Trk1 (tagged)





#spanning-tree ethernet trk1 admin-edge-port

#span e trk1 admin-edge Set port 1 to be an edge port (non bridging port). Note: port is set by default in “auto-edge” mode which automatically sets port to Edge if no BPDU are received after 3 sec.

#show vlan 101 #sh vlan 101 Display VLAN 101




#show vlan ports trk1 detail # show vlan ports trk1 detail Displays the VLAN detail for Trunk 1

#show interface brief1-2 #sh int br1-2 Show Port 1-2 status




Scenario 6 – ProCurve command line configuration (Tunneled vNet and Shared Uplink Set with VLAN tagging) Connect to the ProCurve switch servicing the VC-Enet uplink ports and enter the following commands.


Table 6 ProCurve command line configuration (802.1Q)




#span #span Enables spanning-tree (MSTP mode by default)

#vlan 101 tagged 1 #vlan 101 tag 1 Allow VLAN 101 on Port 1 and set to tagged mode

#vlan 102 tagged 1 #vlan 102 tag 1 Add VLAN 102 on Port 1 and set to tagged mode

#vlan 103 tagged 2 #vlan 103 tag 2 Allow VLAN 103 on Port 2 and set to tagged mode



#vlan 1 #vlan 1 Set focus to VLAN 1

#no untagged 1 #no untagged 1 Disables VLAN 1 on Ports 1

#vlan 2 #vlan 2 Set focus to VLAN 2

#no untagged 2 #no untagged 2 Disables VLAN 1 on Ports 2

#exit #exit Exit VLAN 2





Appendix C: Acronyms and abbreviations 142

Appendix C: Acronyms and abbreviations

Term Definition

Auto Port Speed** Let VC automatically determine best Flex NIC speed

CLP String Flex-10 NIC settings written to the server hardware by VC/OA when the server is power off. Read by the server hardware upon power in.

Custom Port Speed** Manually set Flex NIC speed (up to Maximum value defined)

DCC** Dynamic Control Channel. Future method for VC to change Flex-10 NIC port settings on the fly (without power no/off)

EtherChannel* A Cisco proprietary technology that combines multiple NIC or switch ports for greater bandwidth, load balancing, and redundancy. The technology allows for bi-directional aggregated network traffic flow.

Flex NIC** One of four virtual NIC partitions available per Flex-10 Nic port. Each capable of being tuned from 100Mb to 10Gb

Flex-10 Nic Port** A physical 10Gb port that is capable of being partitioned into 4 Flex NICs

Flex HBA*** Physical function 2 or a FlexFabric CNA can act as eitheran Ethernet NIC, FCoE connection or iSCSI NIC with boot and iSCSI offload capabilities.

IEEE 802.1Q An industry standard protocol that enables multiple virtual networks to run on a single link/port in a secure fashion through the use of VLAN tagging.

IEEE 802.3ad An industry standard protocol that allows multiple links/ports to run in parallel, providing a virtual single link/port. The protocol provides greater bandwidth, load balancing, and redundancy.

LACP Link Aggregation Control Protocol (see IEEE802.3ad)

LOM LAN-on-Motherboard. Embedded network adapter on the system board

Maximum Link Connection Speed**

Maximum Flex NIC speed value assigned to vNet by the network administrator. Can NOT be manually overridden on the server profile.

Multiple Networks Link Speed Settings**

Global Preferred and Maximum Flex NIC speed values that override defined vNet values when multiple vNets are assigned to the same Flex NIC

MZ1 or MEZZ1; LOM Mezzanine Slot 1; (LOM) Lan Motherbard/systemboard NIC

Network Teaming Software A software that runs on a host, allowing multiple network interface ports to be combined to act as a single virtual port. The software provides greater bandwidth, load balancing, and redundancy.

pNIC** Physical NIC port. A Flex NIC is seen by VMware as a pNIC

Port Aggregation Combining ports to provide one or more of the following benefits: greater bandwidth, load balancing, and redundancy.

Port Aggregation Protocol (PAgP)* A Cisco proprietary protocol aids in the automatic creation of Fast EtherChannel links. PAgP packets are sent between Fast EtherChannel-capable ports to negotiate the forming of a channel.

Port Bonding A term typically used in the Unix/Linux world that is synonymous to NIC teaming in the Windows world.

Preferred Link Connection Speed** Preferred Flex NIC speed value assigned to a vNet by the network administrator.

Appendix C: Acronyms and abbreviations 143

Share Uplink Set (SUS) A set of Ethernet uplinks that are used together to provide improved throughput and availability to a group of associated Virtual Connect networks. Each associated Virtual Connect network is mapped to a specific VLAN on the external connection and appropriate VLAN tags are removed or added as Ethernet packets enter or leave the Virtual Connect domain.

Smart Link A feature that, when enabled, configures a Virtual Connect network so that if all external uplinks lose link to external switches, Virtual Connect will drop the Ethernet link on all local server blade Ethernet ports connected to that network.

Trunking (Cisco) 802.1Q VLAN tagging

Trunking (Industry) Combining ports to provide one or more of the following benefits: greater bandwidth, load balancing, and redundancy.

Trunking (Industry) Combining ports to provide one or more of the following benefits: greater bandwidth, load balancing, and redundancy.

VLAN A virtual network within a physical network.

VLAN Tagging Tagging/marking an Ethernet frame with an identity number representing a virtual network.

VLAN Trunking Protocol (VTP)* A Cisco proprietary protocol used for configuring and administering VLANs on Cisco network devices.

vNIC Virtual NIC port. A software-based NIC used by Virtualization Managers

vNet Virtual Connect Network used to connect server NICs to the external Network

*The feature is not supported by Virtual Connect

**This feature was added for Virtual Connect Flex-10

***This feature was added for Virtual Connect FlexFabric

Appendix D: Useful VC CLI Command sets 144

Appendix D: Useful VC CLI Command sets

The following are a collection of useful VC CLI commands. These CLI commands and many more are documented in detail in Virtual Connect Manager Command Line Interface Version 1.31 (or later) User Guide. The following CLI commands can be copied and pasted into an SSH session with the VCM and will apply immediately upon paste.

VC Domain Configuration #Enclosure Setup #Import Enclosure and Set Domain Name #Ensure password matches the OA password import enclosure username=Administrator password=Administrator set domain name=VC_Domain_1 #Importing additional or multiple Enclosures to an existing VC Domain # Importing an Enclosure into an existing VC Domain (Note: As of this writing (VC firmware 2.30) the following commands must be executed individually and cannot be part of a larger script). #The IP address, login and password information used in this command are from the OA of the enclosure being imported. Import enclosure 10.0.0.60 UserName=Administrator Password=password Import enclosure 10.0.0.30 UserName=Administrator Password=password Import enclosure 10.0.0.40 UserName=Administrator Password=password #Configure MAC and WWN to VC Defined and select pool #1 set domain mactype=vc-defined macpool=1 set domain wwntype=vc-defined wwnpool=1 set serverid type=vc-defined poolid=1 #Change Administrator set user Administrator password=Administrator # Set Advanced Ethernet Settings to "Map VLAN Tags" and set "Force server connections" to disabled set enet-vlan vlantagcontrol=map sharedservervlanid=false # Set Advanced Ethernet Settings to "Tunnel VLAN Tags" set enet-vlan vlantagcontrol=tunnel # Set Advanced Ethernet Settings to a Preferred speed of 500Mb and a Max Speed of 2500Mb set enet-vlan PrefSpeedType=Custom PrefSpeed=500 MaxSpeedType=Custom MaxSpeed=2500 #Add additional User to VCM, creates User jimbo add user jimbo password=password privileges=domain,network,server,storage Creating Shared Uplink Sets #Create Shared Uplink Set "Prod-Net" and configure one uplink VC module 1, port X1 add uplinkport enc0:1:X1 Uplinkset=Prod-Net speed=auto #Create Shared Uplink Set "Prod-Net" and configure multiple uplinks on VC Module 1, Ports X1 – X3 add uplinkset Prod-Net add uplinkport enc0:1:X1 Uplinkset=Prod-Net speed=auto


add uplinkport enc0:1:X2 Uplinkset=Prod-Net speed=auto add uplinkport enc0:1:X3 Uplinkset=Prod-Net speed=auto # Create Networks PROD-A through PROD-D, supporting VLANs 101 through 104 on Shared Uplink Set "Prod-Net" add network VLAN_101 uplinkset=Prod-Net VLanID=101 add network VLAN_102 uplinkset=Prod-Net VLanID=102 add network VLAN_103 uplinkset=Prod-Net VLanID=103 add network VLAN_104 uplinkset=Prod-Net VLanID=104 # (optionally) Set network VLAN_104 as a “Private Network” set network VLAN_104 Private=Enabled Creating vNets #Create vNet "Prod-Net" and configure uplinks add Network Prod-Net add uplinkport enc0:1:X3 Network=Prod-Net speed=auto #Optionally enable the vNet as a Private Network set network Prod-Net Private=Enabled Server Profiles #Create Server Profile App-1, apply this profile to Server Slot 1 and configure NIC 1 to Multiple Networks VLAN_101 and VLAN_102 add profile App-1 -nodefaultenetconn add enet-connection App-1 pxe=Enabled add enet-connection App-1 pxe=Disabled add server-port-map App-1:1 VLAN_101 VLanId=101 add server-port-map App-1:1 VLAN_102 VLanId=102 assign profile App-1 enc0:1 # As an alternative when connection to Multiple Networks, if you want ALL networks # configured on a specific Shared Uplink Set to be presented to a server NIC, ensure that # the “Force VLAN mappings as Shared Uplink Set” check box is enabled. # Shared Uplink Set, use the following commands to do so # This will set the Force same VLAN mappings as Shared Uplink Sets check box to enabled # Result is that only VLANs from this shared uplink will be available to this NIC add server-port-map App-1:1 VLAN_101 Uplinkset=Prod-Net add server-port-map App-1:1 VLAN_102 Uplinkset=Prod-Net # Create Server Profile App-1 – Both NICs are configured on network VLAN_102 add profile App-1 -nodefaultenetconn add enet-connection App-1 pxe=Enabled add enet-connection App-1 pxe=Disabled set enet-connection App-1 1 Network=VLAN_102 set enet-connection App-1 2 Network=VLAN_102 assign profile App-1 enc0:2 # Create Server Profile ESX-1 – Both NICs are configured on both networks VLAN_102 and VLAN_102 add profile ESX-1 -nodefaultenetconn add enet-connection ESX-1 pxe=Enabled add enet-connection ESX-1 pxe=Disabled add server-port-map ESX-1:1 VLAN_101 VLanId=101 add server-port-map ESX-1:1 VLAN_102 VLanId=102


add server-port-map ESX-1:2 VLAN_101 VLanId=101 add server-port-map ESX-1:2 VLAN_102 VLanId=102 assign profile ESX-1 enc0:1 # Create Server Profile Server-1 with 8 Flex-10 NICs configured for specific speeds add profile Server-1 -nodefaultenetconn add enet-connection Server-1 pxe=Enabled Network=Console-101-1 SpeedType=Custom Speed=500 add enet-connection Server-1 pxe=Disabled Network=Console-101-2 SpeedType=Custom Speed=500 add enet-connection Server-1 pxe=Disabled Network=VMotion-102-1 SpeedType=Custom Speed=2500 add enet-connection Server-1 pxe=Disabled Network=VMotion-102-2 SpeedType=Custom Speed=2500 add enet-connection Server-1 pxe=Disabled Network=Prod-103-1 SpeedType=Custom Speed=2000 add enet-connection Server-1 pxe=Disabled Network=Prod-103-2 SpeedType=Custom Speed=2000 add enet-connection Server-1 pxe=Disabled add server-port-map Server-1:7 Prod-104-1 VLanId=104 add server-port-map Server-1:7 Prod-105-1 VLanId=105 add enet-connection Server-1 pxe=Disabled add server-port-map Server-1:8 Prod-104-2 VLanId=104 add server-port-map Server-1:8 Prod-105-2 VLanId=105 Assign profile Server-1 enc0:1 # Add TWO fc connections to Profile ESX-1 with a specific WWN add fc-connection ESX-1 Fabric=SAN_3 AddressType=User-Defined PortWWN=50:06:0B:00:00:C2:ff:00 NodeWWN=50:06:0B:00:00:c2:ff:01 add fc-connection ESX-1 Fabric=SAN_4 AddressType=User-Defined PortWWN=50:06:0B:00:00:C2:ff:02 NodeWWN=50:06:0B:00:00:C2:ff:03 # Add TWO NIC connections to Profile ESX-1 with a specific MAC and iSCSI MAC address add enet-connection ESX-1 AddressType=User-Defined EthernetMac=00-17-00-00-AA-AA IScsiMac=00-17-00-00-BB-BB pxe=Enabled add enet-connection ESX-1 AddressType=User-Defined EthernetMac=00-17-00-00-AA-CC IScsiMac=00-17-00-00-BB-CC pxe=Disabled


FlexFabric additions

#Create the FCoE SAN Fabrics FCoE_A and FCoE_B and configure uplinks as discussed above Add fabric FCoE_A Bay=1 Ports=1,2 LinkDist=Auto Add fabric FCoE_B Bay=2 Ports=1,2 LinkDist=Auto

# Create Server Profile ESX-1 with 6 NICs and 2 FCoE SAN connections add profile ESX-1 set enet-connection ESX-1 1 pxe=Enabled Network=VLAN101-1 SpeedType=Custom Speed-1000 set enet-connection ESX-1 2 pxe=Disabled Network=VLAN101-2 SpeedType=Custom Speed-1000 add enet-connection ESX-1 pxe=Disabled Network=VMotion SpeedType=Custom Speed-1500 add enet-connection ESX-1 pxe=Disabled Network=VMotion SpeedType=Custom Speed-1500 add enet-connection ESX-1 pxe=Disabled add server-port-map ESX-1:5 VLAN_103-1 VLanId=103 add server-port-map ESX-1:5 VLAN_104-1 VLanId=104 add server-port-map ESX-1:5 VLAN_105-1 VLanId=105 add enet-connection ESX-1 pxe=Disabled add server-port-map ESX-1:6 VLAN_103-2 VLanId=103 add server-port-map ESX-1:6 VLAN_104-2 VLanId=104 add server-port-map ESX-1:6 VLAN_105-2 VLanId=105 set fcoe-connection ESX-1:1 Fabric=FCoE_A SpeedType=4Gb set fcoe-connection ESX-1:2 Fabric=FCoE_B SpeedType=4Gb poweroff server 1 assign profile ESX-1 enc0:1

Documents

c02616817