b Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide Release 6.x

Cisco Nexus 2000 Series NX-OS Fabric Extender SoftwareConfiguration GuideFirst Published: May 03, 2012

Last Modified: August 22, 2012

Americas HeadquartersCisco Systems, Inc.170 West Tasman DriveSan Jose, CA 95134-1706USAhttp://www.cisco.comTel: 408 526-4000 800 553-NETS (6387)Fax: 408 527-0883

Text Part Number: OL-25816-02

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITEDWARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITHTHE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain versionof the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDINGANYOTHERWARRANTYHEREIN, ALL DOCUMENT FILES AND SOFTWAREOF THESE SUPPLIERS ARE PROVIDED “AS IS"WITHALL FAULTS.CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OFMERCHANTABILITY, FITNESS FORA PARTICULAR PURPOSEANDNONINFRINGEMENTORARISING FROMACOURSEOFDEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUTLIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERSHAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnershiprelationship between Cisco and any other company. (1110R)

Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shownfor illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.

© 2012 Cisco Systems, Inc. All rights reserved.

http://www.cisco.com/go/trademarks

http://www.cisco.com/go/trademarks

C O N T E N T S

P r e f a c e Preface v

Audience v

Document Conventions v

Related Documentation for Cisco Nexus 7000 Series NX-OS Software vii

Documentation Feedback ix

Obtaining Documentation and Submitting a Service Request ix

C H A P T E R 1 Overview 1

Information About the Cisco Nexus 2000 Series Fabric Extender 2

Fabric Extender Terminology 2

Fabric Interface Features 3

Host Interfaces 3

Layer 3 Host Interfaces 3

Layer 2 Host Interfaces 3

Host Interface Port Channels 4

Layer 3 Host Interface Port Channels 4

Layer 2 Host Interface Port Channels 4

VLANs 5

Protocol Offload 5

Quality of Service 5

Access Control Lists 5

IGMP Snooping 6

Switched Port Analyzer 6

Oversubscription 6

Management Model 7

Forwarding Model 8

Port Channel Fabric Interface Connection 9

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide OL-25816-02 iii

Port Numbering Convention 9

Fabric Extender Image Management 10

Licensing Requirements for the Fabric Extender 10

Guidelines and Limitations 10

Configuration Limits 12

Default Settings 12

C H A P T E R 2 Configuring the Fabric Extender 13

Configuring the Cisco Nexus 2000 Series Fabric Extender 13

Managing the Fabric Extender Feature Set 13

Installing the Fabric Extender Feature Set 14

Enabling the Fabric Extender Feature Set 15

Disallowing the Fabric Extender Feature Set 15

Associating a Fabric Extender to a Fabric Interface 16

Associating a Fabric Extender to a Port Channel 17

Disassociating a Fabric Extender From an Interface 19

Associating a Fabric Extender to an F2 Module 19

Configuring Fabric Extender Global Features 21

Configuration Examples 23

Configuring a FEX with a Layer 3 Host Interface 24

Configuring a Host Interface in a vPC Topology Connected to Two FEXs 24

Dual-homing of a Server to a FEX with FabricPath 25

Verifying the Configuration 26

Verifying the Fabric Extender Configuration 26

Verifying the Chassis Management Information 29

Additional References 33

Related Documents 33

Feature History 33

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guideiv OL-25816-02

Contents

Preface

This preface describes the audience, organization, and conventions of the Cisco Nexus 2000 Series FabricExtender Software Configuration Guide. It also provides information on how to obtain related documentation.

This chapter includes the following sections:

• Audience, page v

• Document Conventions, page v

• Related Documentation for Cisco Nexus 7000 Series NX-OS Software, page vii

• Documentation Feedback , page ix

• Obtaining Documentation and Submitting a Service Request, page ix

AudienceThis publication is for experienced network administrators who configure and maintain Cisco Nexus Seriesdevices.

Document ConventionsCommand descriptions use the following conventions:

DescriptionConvention

Bold text indicates the commands and keywords that you enter literallyas shown.

bold

Italic text indicates arguments for which the user supplies the values.Italic

Square brackets enclose an optional element(keyword or argument).[x]

Square brackets enclosing keywords or arguments separated by a verticalbar indicate an optional choice.

[x | y]

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide OL-25816-02 v


Braces enclosing keywords or arguments separated by a vertical barindicate a required choice.

{x | y}

Nested set of square brackets or braces indicate optional or requiredchoices within optional or required elements. Braces and a vertical barwithin square brackets indicate a required choice within an optionalelement.

[x {y | z}]

Indicates a variable for which you supply values, in context where italicscannot be used.

variable

A nonquoted set of characters. Do not use quotation marks around thestring or the string will include the quotation marks.

string

Examples use the following conventions:


Terminal sessions and information the switch displays are in screen font.screen font

Information you must enter is in boldface screen font.boldface screen font

Arguments for which you supply values are in italic screen font.italic screen font

Nonprinting characters, such as passwords, are in angle brackets.< >

Default responses to system prompts are in square brackets.[ ]

An exclamation point (!) or a pound sign (#) at the beginning of a lineof code indicates a comment line.

!, #

This document uses the following conventions:

Means reader take note. Notes contain helpful suggestions or references to material not covered in themanual.

Note

Means reader be careful. In this situation, you might do something that could result in equipment damageor loss of data.

Caution

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guidevi OL-25816-02

PrefaceDocument Conventions

Related Documentation for Cisco Nexus 7000 Series NX-OSSoftware

The entire Cisco Nexus 7000 Series NX-OS documentation set is available at the following URL:

http://www.cisco.com/en/us/products/ps9402/tsd_products_support_series_home.html

Release Notes

The release notes are available at the following URL:

http://www.cisco.com/en/US/products/ps9402/prod_release_notes_list.html

Configuration Guides

These guides are available at the following URL:

http://www.cisco.com/en/US/products/ps9402/products_installation_and_configuration_guides_list.html

The documents in this category include:

• Cisco Nexus 7000 Series NX-OS Configuration Examples

• Cisco Nexus 7000 Series NX-OS FabricPath Configuration Guide

• Cisco Nexus 7000 Series NX-OS Fundamentals Configuration Guide

• Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide

• Cisco Nexus 7000 Series NX-OS IP SLAs Configuration Guide

• Cisco Nexus 7000 Series NX-OS Layer 2 Switching Configuration Guide

• Cisco Nexus 7000 Series NX-OS LISP Configuration Guide

• Cisco Nexus 7000 Series NX-OS MPLS Configuration Guide

• Cisco Nexus 7000 Series NX-OS Multicast Routing Configuration Guide

• Cisco Nexus 7000 Series NX-OS OTV Configuration Guide

• Cisco Nexus 7000 Series NX-OS Quality of Service Configuration Guide

• Cisco Nexus 7000 Series NX-OS SAN Switching Guide

• Cisco Nexus 7000 Series NX-OS Security Configuration Guide

• Cisco Nexus 7000 Series NX-OS System Management Configuration Guide

• Cisco Nexus 7000 Series NX-OS Unicast Routing Configuration Guide

• Cisco Nexus 7000 Series NX-OS Verified Scalability Guide

• Cisco Nexus 7000 Series NX-OS Virtual Device Context Configuration Guide

• Cisco Nexus 7000 Series NX-OS Virtual Device Context Quick Start

• Cisco Nexus 7000 Series NX-OS OTV Quick Start Guide

• Cisco NX-OS FCoE Configuration Guide for Cisco Nexus 7000 and Cisco MDS 9500

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide OL-25816-02 vii

PrefaceRelated Documentation for Cisco Nexus 7000 Series NX-OS Software


http://www.cisco.com/en/US/products/ps9402/prod_release_notes_list.html

http://www.cisco.com/en/US/products/ps9402/products_installation_and_configuration_guides_list.html

• Cisco Nexus 2000 Series Fabric Extender Software Configuration Guide

Command References

These guides are available at the following URL:

http://www.cisco.com/en/US/products/ps9402/prod_command_reference_list.html

The documents in this category include:

• Cisco Nexus 7000 Series NX-OS Command Reference Master Index

• Cisco Nexus 7000 Series NX-OS FabricPath Command Reference

• Cisco Nexus 7000 Series NX-OS Fundamentals Command Reference

• Cisco Nexus 7000 Series NX-OS High Availability Command Reference

• Cisco Nexus 7000 Series NX-OS Interfaces Command Reference

• Cisco Nexus 7000 Series NX-OS Layer 2 Switching Command Reference

• Cisco Nexus 7000 Series NX-OS LISP Command Reference

• Cisco Nexus 7000 Series NX-OS MPLS Configuration Guide

• Cisco Nexus 7000 Series NX-OS Multicast Routing Command Reference

• Cisco Nexus 7000 Series NX-OS OTV Command Reference

• Cisco Nexus 7000 Series NX-OS Quality of Service Command Reference

• Cisco Nexus 7000 Series NX-OS SAN Switching Command Reference

• Cisco Nexus 7000 Series NX-OS Security Command Reference

• Cisco Nexus 7000 Series NX-OS System Management Command Reference

• Cisco Nexus 7000 Series NX-OS Unicast Routing Command Reference

• Cisco Nexus 7000 Series NX-OS Virtual Device Context Command Reference

• Cisco NX-OS FCoE Command Reference for Cisco Nexus 7000 and Cisco MDS 9500

Other Software Documents

You can locate these documents starting at the following landing page:


• Cisco Nexus 7000 Series NX-OS MIB Quick Reference

• Cisco Nexus 7000 Series NX-OS Software Upgrade and Downgrade Guide

• Cisco Nexus 7000 Series NX-OS Troubleshooting Guide

• Cisco NX-OS Licensing Guide

• Cisco NX-OS System Messages Reference

• Cisco NX-OS XML Interface User Guide

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guideviii OL-25816-02

PrefaceRelated Documentation for Cisco Nexus 7000 Series NX-OS Software



Documentation FeedbackTo provide technical feedback on this document, or to report an error or omission, please send your commentsto [email protected]. We appreciate your feedback.

Obtaining Documentation and Submitting a Service RequestFor information on obtaining documentation, submitting a service request, and gathering additional information,see the monthlyWhat's New in Cisco Product Documentation, which also lists all new and revised Ciscotechnical documentation, at:

http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html

Subscribe to theWhat's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feedand set content to be delivered directly to your desktop using a reader application. The RSS feeds are a freeservice and Cisco currently supports RSS version 2.0.

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide OL-25816-02 ix

PrefaceDocumentation Feedback

http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guidex OL-25816-02

PrefaceObtaining Documentation and Submitting a Service Request

C H A P T E R 1Overview

This chapter provides an architectural overview of the Cisco Nexus 2000 Series Fabric Extender and includesthe following sections:

• Information About the Cisco Nexus 2000 Series Fabric Extender, page 2

• Fabric Extender Terminology, page 2

• Fabric Interface Features , page 3

• Host Interfaces, page 3

• Host Interface Port Channels, page 4

• VLANs, page 5

• Protocol Offload, page 5

• Quality of Service, page 5

• Access Control Lists, page 5

• IGMP Snooping, page 6

• Switched Port Analyzer, page 6

• Oversubscription, page 6

• Management Model, page 7

• Forwarding Model, page 8

• Port Channel Fabric Interface Connection, page 9

• Port Numbering Convention, page 9

• Fabric Extender Image Management, page 10

• Licensing Requirements for the Fabric Extender, page 10

• Guidelines and Limitations, page 10

• Default Settings, page 12

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide OL-25816-02 1

Information About the Cisco Nexus 2000 Series Fabric ExtenderThe Cisco Nexus 2000 Series Fabric Extender, also known as FEX, is a highly scalable and flexible servernetworking solution that works with Cisco Nexus Series devices to provide high-density, low-cost connectivityfor server aggregation. Scaling across 1-Gigabit Ethernet, 10-Gigabit Ethernet, unified fabric, rack, and bladeserver environments, the Fabric Extender is designed to simplify data center architecture and operations.

The Fabric Extender integrates with its parent switch, a Cisco Nexus Series device, to allow automaticprovisioning and configuration taken from the settings on the parent device. This integration allows largenumbers of servers and hosts to be supported using the same feature set as the parent device, including securityand quality-of-service (QoS) configuration parameters, with a single management domain. The Fabric Extenderand its parent switch enable a large multipath, loop-free, active-active data center topology without the useof Spanning Tree Protocol (STP).

The Cisco Nexus 2000 Series Fabric Extender forwards all traffic to its parent Cisco Nexus Series deviceover 10-Gigabit Ethernet fabric uplinks, which allows all traffic to be inspected by policies established on theCisco Nexus Series device.

You must connect the Fabric Extender to its parent Cisco Nexus 7000 Series device equipped with a32-port, 10-Gigabit M1 module (N7K-M132XP-12), a 32-port 10-Gigabit M1-XL module(N7K-M132XP-12L), an M2 module, or an F2 module.

Note

No software is included with the Fabric Extender. Software is automatically downloaded and upgraded fromits parent device.

Fabric Extender TerminologySome terms used in this document are as follows:

• Fabric interface—A10-Gigabit Ethernet uplink port designated for connection from the Fabric Extenderto its parent switch. A fabric interface cannot be used for any other purpose. It must be directly connectedto the parent switch.

A fabric interface includes the corresponding interface on the parent switch. This interfaceis enabled when you enter the switchport mode fex-fabric command.

Note

• Port channel fabric interface—A port channel uplink connection from the Fabric Extender to its parentswitch. This connection consists of fabric interfaces bundled into a single logical channel.

• Host interface—An Ethernet host interface for connection to a server or host system.

Do not connect a bridge or switch to a host interface. These interfaces are designed toprovide end host or server connectivity.

Note

• Port channel host interface—A port channel host interface for connection to a server or host system.

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide2 OL-25816-02

OverviewInformation About the Cisco Nexus 2000 Series Fabric Extender

Fabric Interface FeaturesThe FEX fabric interfaces support static port channels. During the initial discovery and association process,SFP+ validation and digital optical monitoring (DOM) are performed as follows:

• The FEX performs a local check on the uplink SFP+ transceiver. If it fails the security check, the LEDflashes but the link is still allowed to come up.

• The FEX local check is bypassed if it is running its backup image.

• The parent switch performs SFP validation again when the fabric interface is brought up. It keeps thefabric interface down if SFP validation fails.

Once an interface on the parent switch is configured in fex-fabric mode, all other features that were configuredon that port and are not relevant to this mode are deactivated. If the interface is reconfigured to removefex-fabric mode, the previous configurations are reactivated.

For more information about PFC, see the Cisco Nexus 7000 Series NX-OS Quality of Service ConfigurationGuide.

Host Interfaces

Layer 3 Host InterfacesBeginning with Cisco NX-OS Release 5.2, by default, all host interfaces on a Fabric Extender connected toa Cisco Nexus 7000 Series parent switch run in Layer 3 mode.

If you have updated the parent switch to Cisco Nexus Release 5.2, previously configured Fabric Extenderhost interfaces retain their default port mode, Layer 2. You can change these ports to Layer 3 mode withthe no switchport command.

Note

The host interfaces also support subinterfaces. You can create up to 32 subinterfaces on a Fabric Extenderhost interface.

For information about interfaces and subinterfaces, see the Cisco Nexus 7000 Series NX-OS InterfacesConfiguration Guide.

Layer 2 Host InterfacesIn Cisco NX-OS Release 5.1 and earlier releases, the default port mode is Layer 2.

To run a host interface in Layer 2 mode, use the switchport command. For Cisco NX-OS Release 5.2 andlater releases, to change the port mode to Layer 3, use the no switchport command.

The Fabric Extender provides connectivity for computer hosts and other edge devices in the network fabric.The following guidelines should be followed when connecting devices to Fabric Extender host interfaces:

• All Fabric Extender host interfaces run as spanning tree edge ports with BPDUGuard enabled and cannotbe configured as spanning tree network ports.


OverviewFabric Interface Features

• Servers utilizing active/standby teaming, 802.3ad port channels, or other host-based link redundancymechanisms can be connected to Fabric Extender host interfaces.

• Any device running spanning tree connected to a Fabric Extender host interface results in that hostinterface being placed in an error-disabled state when a BPDU is received.

• Any edge switch that leverages a link redundancy mechanism not dependent on spanning tree such asCisco FlexLink or vPC (with the BPDU Filter enabled) can be connected to a Fabric Extender hostinterface. Because spanning tree is not used to eliminate loops, you should ensure a loop-free topologybelow the Fabric Extender host interfaces.

Ingress and egress packet counters are provided on each host interface.

For more information about BPDU Guard, see the Cisco Nexus 7000 Series NX-OS Layer 2 SwitchingConfiguration Guide.

Host Interface Port Channels

Layer 3 Host Interface Port ChannelsThe Fabric Extender supports host interface port channel configurations. You can combine up to 8 interfacesin a standard mode port channel and 16 interfaces when configured with the Link Aggregation Control Protocol(LACP).

Port channel resources are allocated when the port channel has one or more members.Note

All members of the port channel must be Fabric Extender host interfaces and all host interfaces must be fromthe same Fabric Extender. You cannot mix interfaces from the Fabric Extender and the parent switch.

Layer 3 mode is supported on host interface port channels.

A host interface port channel also supports subinterfaces. You can create up to 1000 subinterfaces on a FabricExtender host interface port channel.

For more information about port channels, see the Cisco Nexus 7000 Series NX-OS Interfaces ConfigurationGuide.

Layer 2 Host Interface Port ChannelsThe Fabric Extender supports host interface port channel configurations. You can combine up to 8 interfacesin a standard mode port channel and 16 interfaces when configured with the Link Aggregation Control Protocol(LACP).

Port channel resources are allocated when the port channel has one or more members.Note

All members of the port channel must be Fabric Extender host interfaces and all host interfaces must be fromthe same Fabric Extender. You cannot mix interfaces from the Fabric Extender and the parent switch.

Layer 2 mode is supported on host interface port channels.


OverviewHost Interface Port Channels

You can configure Layer 2 port channels as access or trunk ports.

Beginning with Cisco NX-OS Release 5.2(1), Fabric Extenders support the host vPC feature where a servercan be dual-attached to two different FEXs through a port channel. You must configure parent switches thatconnect each Fabric Extender (one parent switch per FEX) in a vPC domain.

VLANsThe Fabric Extender supports Layer 2 VLAN trunks and IEEE 802.1Q VLAN encapsulation.

For more information about VLANs, see theCisco Nexus 7000 Series NX-OS Layer 2 Switching ConfigurationGuide.

Note: The Fabric Extender does not support Private VLANs (PVLANs).Note

Protocol OffloadTo reduce the load on the control plane of the Cisco Nexus Series device, Cisco NX-OS allows you to offloadlink-level protocol processing to the Fabric Extender CPU. The following protocols are supported:

• Link Layer Discovery Protocol (LLDP) and Data Center Bridging Exchange (DCBX)

• Cisco Discovery Protocol (CDP)

• Link Aggregation Control Protocol (LACP)

Quality of ServiceThe Fabric Extender uses IEEE 802.1p class of service (CoS) values to associate traffic with the appropriateclass. Per-port QoS configuration is also supported.

Host interfaces support pause frames, which are implemented using IEEE 802.3x link-level flow control(LLC). By default, flow control send is on and flow control receive is off on all host interfaces. Autonegotiationis enabled on the host interfaces. Per-class flow control is set according to the QoS classes.

For more information about LLC and quality-of-service, see the Cisco Nexus 7000 Series NX-OS Quality ofService Configuration Guide.

Access Control ListsThe Fabric Extender supports the full range of ingress access control lists (ACLs) that are available on itsparent Cisco Nexus Series device.

For more information about ACLs, see the Cisco Nexus 7000 Series NX-OS Security Configuration Guide.


OverviewVLANs

IGMP SnoopingIGMP snooping is supported on all host interfaces of the Fabric Extender.

The Fabric Extender and its parent switch support IGMPv2 and IGMPv3 snooping based only on the destinationmulticast MAC address. It does not support snooping based on the source MAC address or on proxy reports.

For more information about IGMP snooping, see http://tools.ietf.org/wg/magma/draft-ietf-magma-snoop/rfc4541.txt. Also, see the Cisco Nexus 7000 Series NX-OS Multicast Routing Configuration Guide.

Note

Switched Port AnalyzerYou can configure the host interfaces on the Fabric Extender as Switched Port Analyzer (SPAN) source ports.You cannot configure Fabric Extender ports as a SPAN destination. Only one SPAN session is supported forall the host interfaces on the same Fabric Extender. Ingress source (Rx), egress source (Tx), or both ingressand egress monitoring are supported.

All IP multicast traffic on the VLANs that a Fabric Extender host interface belongs to is captured in theSPAN session. You cannot separate the traffic by IP multicast group membership.

If ingress monitoring and egress monitoring is configured for host interfaces on the same Fabric Extender,you might see a packet twice: once as the packet ingresses on an interface with Rx configured, and againas the packet egresses on an interface with Tx configured.

Note

For more information about SPAN, see theCisco Nexus 7000 Series NX-OS SystemManagement ConfigurationGuide.

OversubscriptionIn a switching environment, oversubscription is the practice of connecting multiple devices to the sameinterface to optimize port usage. An interface can support a connection that runs at its maximum speed. Becausemost interfaces do not run at their maximum speeds, you can take advantage of unused bandwidth by sharingports. Oversubscription, which is a function of the available fabric interfaces to active host interfaces, providescost-effective scalability and flexibility for Ethernet environments.

The Cisco Nexus 2248TP Fabric Extender has 4 10-Gigabit Ethernet fabric interfaces and 48 100/1000BASE-T(100-Mb/1-Gigabit) Ethernet host interfaces. When its host interfaces are running in Gigabit Ethernet mode,it offers the following configurations:

• No oversubscription (40 host interfaces for four fabric interfaces)

• 1.2 to 1 oversubscription (48 host interfaces for four fabric interfaces)

• 4.8 to 1 oversubscription (48 host interfaces for one fabric interface)


OverviewIGMP Snooping

http://tools.ietf.org/wg/magma/draft-ietf-magma-snoop/rfc4541.txt

http://tools.ietf.org/wg/magma/draft-ietf-magma-snoop/rfc4541.txt

The Cisco Nexus 2248TP can be run with no oversubscription when its host interfaces are running in 100-Mbmode.

The Cisco Nexus 2248TP-E Fabric Extender has 4 10-Gigabit Ethernet fabric interfaces and 48100/1000BASE-T (100-Mb/1-Gigabit) Ethernet host interfaces.When its host interfaces are running in GigabitEthernet mode, it offers 1.2 to 1 oversubscription (48 host interfaces for four fabric interfaces).

The Cisco Nexus 2232PP Fabric Extender has 8 10-Gigabit Ethernet fabric interfaces and 32 10-GigabitEthernet host interfaces. All host interfaces use all of the available fabric interfaces. (Static pinning is notsupported. Port-channel mode is supported only on fabric interfaces.) When all host interfaces are sendingtraffic to all fabric interfaces, the maximum oversubscription ratio for the Cisco Nexus 2232PP is 4:1.

The Cisco Nexus 2232TM Fabric Extender has 8 10-Gigabit Ethernet fabric interfaces and 32 Gigabit and10-Gigabit Ethernet host interfaces. All host interfaces use all of the available fabric interfaces. (Static pinningis not supported. Port-channel mode is supported only on fabric interfaces.) When all host interfaces aresending traffic to all fabric interfaces, the maximum oversubscription ratio for the Cisco Nexus 2232TM is4:1.

The Cisco Nexus 2224TP Fabric Extender has 2 10-Gigabit Ethernet fabric interfaces and 24 100/1000BASE-T(100-Mb/1-Gigabit) Ethernet host interfaces. With this system, you can configure a 1.2 to 1 oversubscription(24 host interfaces for 2 fabric interfaces) or higher.

Management ModelThe Cisco Nexus 2000 Series Fabric Extender is managed by its parent switch over the fabric interfacesthrough a zero-touch configuration model. The switch discovers the Fabric Extender by detecting the fabricinterfaces of the Fabric Extender.

After discovery, if the Fabric Extender has been correctly associated with the parent switch, the followingoperations are performed:

1 The switch checks the software image compatibility and upgrades the Fabric Extender if necessary.

2 The switch and Fabric Extender establish in-band IP connectivity with each other. The switch assigns anIP address in the range of loopback addresses (127.15.1.0/24) to the Fabric Extender to avoid conflictswith IP addresses that might be in use on the network.

3 The switch pushes the configuration data to the Fabric Extender. The Fabric Extender does not store anyconfiguration locally.

4 The Fabric Extender updates the switch with its operational status. All Fabric Extender information isdisplayed using the switch commands for monitoring and troubleshooting.


OverviewManagement Model

Forwarding ModelThe Cisco Nexus 2000 Series Fabric Extender does not perform any local switching. All traffic is sent to theparent switch that provides central forwarding and policy enforcement, including host-to-host communicationsbetween two systems that are connected to the same Fabric Extender as shown in the following figure.

Figure 1: Forwarding Model

The forwarding model facilitates feature consistency between the Fabric Extender and its parent Cisco NexusSeries device.

The Fabric Extender provides end-host connectivity into the network fabric. As a result, BPDU Guard isenabled on all its host interfaces. If you connect a bridge or switch to a host interface, that interface isplaced in an error-disabled state when a BPDU is received.

You cannot disable BPDU Guard on the host interfaces of the Fabric Extender.

Note

The Fabric Extender supports egress multicast replication from the network to the host. Packets sent from theparent switch for multicast addresses attached to the Fabric Extender are replicated by the Fabric ExtenderASICs and are then sent to corresponding hosts.


OverviewForwarding Model

Port Channel Fabric Interface ConnectionTo provide load balancing between the host interfaces and the parent switch, you can configure the FabricExtender to use a port channel fabric interface connection. This connection bundles 10-Gigabit Ethernet fabricinterfaces into a single logical channel as shown in the following figure.

Figure 2: Port Channel Fabric Interface Connection

When you configure the Fabric Extender to use a port channel fabric interface connection to its parent switch,the switch load balances the traffic from the hosts that are connected to the host interface ports by using thefollowing load-balancing criteria to select the link:

• For a Layer 2 frame, the switch uses the source and destination MAC addresses.

• For a Layer 3 frame, the switch uses the source and destination MAC addresses and the source anddestination IP addresses.

A fabric interface that fails in the port channel does not trigger a change to the host interfaces. Traffic isautomatically redistributed across the remaining links in the port channel fabric interface. If all links inthe fabric port channel go down, all host interfaces on the FEX are set to the down state.

Note

Port Numbering ConventionThe following port numbering convention is used for the Fabric Extender:

interface ethernet chassis/slot/port

where

• chassis is configured by the administrator. A Fabric Extender must be directly connected to its parentCisco Nexus Series device via a port channel fabric interface. You configure a chassis ID on a portchannel on the switch to identify the Fabric Extender that is discovered through those interfaces.


OverviewPort Channel Fabric Interface Connection

The chassis ID ranges from 101 to 199.

The chassis ID is required only to access a host interface on the Fabric Extender. Avalue of less than 101 indicates a slot on the parent switch. The following port numberingconvention is used for the interfaces on the switch:

interface ethernet slot/port

Note

• slot identifies the slot number on the Fabric Extender.

• port identifies the port number on a specific slot and chassis ID.

Fabric Extender Image ManagementNo software ships with the Cisco Nexus 2000 Series Fabric Extender. The Fabric Extender image is bundledinto the system image of the parent switch. The image is automatically verified and updated (if required)during the association process between the parent switch and the Fabric Extender.

When you enter the install all command, it upgrades the software on the parent Cisco Nexus Series switchand also upgrades the software on any attached Fabric Extender. To minimize downtime as much as possible,the Fabric Extender remains online while the installation process loads its new software image. Once thesoftware image has successfully loaded, the parent switch and the Fabric Extender both automatically reboot.

This process is required to maintain version compatibility between the parent switch and the Fabric Extender.

Licensing Requirements for the Fabric ExtenderThe following table shows the licensing requirements for the Cisco Nexus 2000 Series Fabric Extender:

License RequirementProduct

The Cisco Nexus 2000 Series Fabric Extenderrequires no license. Any feature not included in alicense package is bundled with the Cisco NX-OSsystem images and is provided at no extra charge toyou. For an explanation of the licensing scheme, seethe Cisco NX-OS Licensing Configuration Guide.

Cisco NX-OS

Guidelines and LimitationsThe Cisco Nexus 2000 Series Fabric Extender has the following configuration guidelines and limitations:

• Beginning with Cisco NX-OS Release 5.2(1), the default port mode is Layer 3. Before Cisco NX-OSRelease 5.2(1), the default port mode was Layer 2.


OverviewFabric Extender Image Management

• You must enable the Fabric Extender feature set in the default virtual device context (VDC). After youenable the feature set in the default VDC, the FEX can belong to any VDC and can be configured fromthose VDCs.

• All the uplinks and host ports of a Fabric Extender belong to a single VDC. The ports cannot be allocatedor split among multiple VDCs.

• You must connect the Fabric Extender to its parent Cisco Nexus 7000 Series device equipped with a32-port 10-Gigabit M1 module (N7K-M132XP-12), a 32-port, 10-Gigabit M1-XL module(N7K-M132XP-12L), an M2 module, or an F2 module.

• The Fabric Extender feature set operation might cause the standby supervisor to reload if it is in anunstable state, such as following a service failure or powering up. You can check whether the standbysupervisor is stable with the show modules command. When the standby supervisor is stable, it isindicated as ha-standby.

• You can configure the Fabric Extender host interfaces as edge ports only. The interface is placed in anerror disabled state if a downstream switch is detected.

• When you connect a FEX is to a Cisco Nexus 7000 series device, the queuing capability on the FEXhost interface is limited. A router connected to an Layer 2 (using SVI interfaces) or Layer 3 FEX interfacecannot participate in routing protocol adjacency. The FEX cannot be used as a peer because whencongestion occurs on the FEX host interface, the control plane traffic is not prioritized. This limitationalso applies to the FEX when it is connected to other Layer 3 devices, such as an ASA firewall, an ACEload balancer, or other Layer 3 networking devices running a dynamic routing protocol. Static routes torouters, ASA firewalls, ACE load balancers, and other Layer 3 network devices are supported.

• The Fabric Extender does not support PVLANs.

Associating with F2 Series modules

• F2 modules are only supported by the following FEX devices:

◦ 2248TP

◦ 2248TP-E

◦ 2232TP

◦ 2232PP

◦ 2232TM

◦ 2224TP

• Each port in the ASIC has an index. Allow only ports with similar indices across ASICs to be added toa port channel.

For example, if port 1 has an index of 1 and port 2 has an index of 2, then

◦ Supported: Port 1 of ASIC 1 and port 1 of ASIC 2 are added to a port channel.

◦ Not supported: Port 1 of ASIC 1 and port 2 of ASIC 2 to form a port channel.

In general, a set of ports from an ASIC that has an index sub-set S, such as {1,2,4}, is allowed to beadded to a port-channel only if the port-channel has an equivalent or an empty set.


OverviewGuidelines and Limitations

Configuration LimitsThe configuration limits are documented in the Cisco Nexus 7000 Series NX-OS Verified Scalability Guide.

Default SettingsThis table lists the default settings for the Fabric Extender parameters.

Table 1: Default Cisco Nexus 2000 Series Fabric Extender Parameter Settings

DefaultParameters

Disabledfeature-set fex command

Layer 3 (CiscoNX-OSRelease 5.2 and later releases).

Layer 2 (Cisco NX-OS Release 5.1 and earlierreleases).

Port mode


OverviewConfiguration Limits

C H A P T E R 2Configuring the Fabric Extender

This chapter describes how to configure a Cisco Nexus 2000 Series Fabric Extender using a parent CiscoNexus 7000 Series device and includes the following sections:

• Configuring the Cisco Nexus 2000 Series Fabric Extender, page 13

• Verifying the Configuration, page 26

• Additional References, page 33

Configuring the Cisco Nexus 2000 Series Fabric ExtenderThis section describes how to configure the Fabric Extender.

Managing the Fabric Extender Feature SetYou can install and manage the Fabric Extender feature set.

SUMMARY STEPS

1. Installing the Fabric Extender Feature Set, on page 142. Enabling the Fabric Extender Feature Set, on page 153. (Optional) Disallowing the Fabric Extender Feature Set, on page 15

DETAILED STEPS

Step 1 Installing the Fabric Extender Feature Set, on page 14Step 2 Enabling the Fabric Extender Feature Set, on page 15Step 3 (Optional) Disallowing the Fabric Extender Feature Set, on page 15


Installing the Fabric Extender Feature Set

You must enable the Fabric Extender feature set in the default VDC. Once enabled in the default VDC,the FEX can belong to any VDC and can be configured from those VDCs. A single Fabric Extenderbelongs exclusively to a single VDC.

Note

Before You Begin

Ensure that you are in the default VDC.

Ensure that you have disabled the IDS reserved addresses check (it is disabled by default). Use the showhardware ip verify command and look for the string "address reserved" in the output. If the IDS reservedaddresses check is enabled, disable it with the no hardware ip verify address reserved command.

SUMMARY STEPS

1. configure terminal2. install feature-set fex3. exit

DETAILED STEPS

PurposeCommand or Action

Enters configuration mode.configure terminal

Example:switch# configure terminalswitch(config)#

Step 1

Installs the Fabric Extender feature set in the default VDC.install feature-set fexStep 2

Example:switch(config)# install feature-set fex

To uninstall the Fabric Extender feature set, use the no installfeature-set fex command. Before you can uninstall the feature set,you must ensure the following:

• The feature set must be installed in the default VDC.

• The feature set must not be enabled in any VDC.

Exits configuration mode.exit

Example:switch(config)# exitswitch#

Step 3


Configuring the Fabric ExtenderManaging the Fabric Extender Feature Set

Enabling the Fabric Extender Feature SetYou can enable the installed Fabric Extender feature set in any VDC on the device.

Before You Begin

Ensure that you have installed the Fabric Extender feature set in the default VDC.

Ensure that you are in the correct VDC or use the switchto vdc command.

SUMMARY STEPS

1. configure terminal2. feature-set fex3. exit

DETAILED STEPS




Step 1

Enables the Fabric Extender feature set. The feature set must be installedbefore it shows as an option to this command.

feature-set fex

Example:switch(config)# feature-set fex

Step 2

To disable the Fabric Extender feature set, use the no feature-set fexcommand. Before you can disable a feature set, you must install the featureset in the default VDC.

The no feature-set fex commandmight take some time to completeif the size of the configuration is very large. The command cleansup all of the configurations associated with the Fabric Extenderfeature set.

Note

Exits configuration mode.exit

Example:switch(config)# exitswitch#

Step 3

Disallowing the Fabric Extender Feature SetBy default, when you install the Fabric Extender feature set, it is allowed in all VDCs. You can disallow theinstalled Fabric Extender feature set in a specific VDC on the device.

Before You Begin

Ensure that you have installed the feature set in the default VDC.


Configuring the Fabric ExtenderManaging the Fabric Extender Feature Set

Ensure that you have not enabled the feature set in the specified VDC.

SUMMARY STEPS

1. configure terminal2. vdc vdc_ID3. no allow feature-set fex4. end

DETAILED STEPS




Step 1

Specifies a VDC and enters VDC configuration mode.vdc vdc_ID

Example:switch(config)# vdc 1

Step 2

Disallows the feature set in the VDC. You cannot disallow a featureset that is enabled in the specified VDC.

no allow feature-set fex

Example:switch(config-vdc)# no allow feature-setfex

Step 3

By default, the installed Fabric Extender feature set is allowed in allVDCs on the device. You can disallow a feature set in a specificVDC. Subsequently, you can change the status back to allowed withthe allow feature-set fex command.

Exits VDC configuration mode and returns to EXEC mode.end

Example:switch(config-vdc)# endswitch#

Step 4

Associating a Fabric Extender to a Fabric InterfaceA FEX is connected to its parent device through a port channel. By default, the parent device does not allowthe attached Fabric Extender to connect until it has been assigned a FEX-number and is associated with theconnected interface.

You must have installed and enabled the Fabric Extender features with the install feature-set fex andfeature-set fex commands before you can configure and use a Fabric Extender that is connected to theparent switch.

Note


Configuring the Fabric ExtenderAssociating a Fabric Extender to a Fabric Interface

Associating a Fabric Extender to a Port Channel

Before You Begin

Ensure that you have installed and enabled the Fabric Extender feature set.

Ensure that you are in the correct VDC (or use the switchto vdc command).

SUMMARY STEPS

1. configure terminal2. interface port-channel channel3. switchport4. switchport mode fex-fabric5. fex associate FEX-number6. (Optional) show interface port-channel channel fex-intf

DETAILED STEPS




Step 1

Specifies a port channel to configure.interface port-channel channelStep 2

Example:switch(config)# interface port-channel 4switch(config-if)#

All interfaces in the port channel must be in thesame VDC.

Note

Sets the interface as a Layer 2 switching port.switchport

Example:switch(config-if)# switchport

Step 3

Sets the port channel to support an external FabricExtender.

switchport mode fex-fabric

Example:switch(config-if)# switchport mode fex-fabric

Step 4

Associates a FEX number to the Fabric Extender unitattached to the interface. The range is from 101 to 199.

fex associate FEX-number

Example:switch(config-if)# fex associate 101

Step 5

(Optional)Displays the association of a Fabric Extender to a portchannel interface.

show interface port-channel channel fex-intf

Example:switch# show interface port-channel 4 fex-intf

Step 6



Examples

This example shows how to associate the Fabric Extender to a port channel interface on the parent device:switch# configure terminalswitch(config)# interface ethernet 1/28switch(config-if)# channel-group 4switch(config-if)# no shutdownswitch(config-if)# exitswitch(config)# interface ethernet 1/29switch(config-if)# channel-group 4switch(config-if)# no shutdownswitch(config-if)# exitswitch(config)# interface ethernet 1/30switch(config-if)# channel-group 4switch(config-if)# no shutdownswitch(config-if)# exitswitch(config)# interface ethernet 1/31switch(config-if)# channel-group 4switch(config-if)# no shutdownswitch(config-if)# exitswitch(config)# interface port-channel 4switch(config-if)# switchportswitch(config-if)# switchport mode fex-fabricswitch(config-if)# fex associate 101

As a best practice, only enter the fex associate command from the port channel interface, not from thephysical interface.

If you try to associate a physical port to a FEX before that physical port is joined to a port channel, thephysical port moves to the err-disable state and the Cisco Nexus 7000 series device does not communicatewith the FEX on that link. You must enter the shutdown command and the no shutdown command onthe Ethernet interface (not the port channel interface) to clear the err-disable state and bring the link up.(Note that this does not apply if the configuration is performed prior to cabling.)

Tip

When adding physical interfaces to port channels, all configurations on the port channel and physicalinterface must match.

Note

This example shows how to display the association of the Fabric Extender and the parent device:switch# show interface port-channel 4 fex-intfFabric FEXInterface Interfaces---------------------------------------------------Po4 Eth101/1/48 Eth101/1/47 Eth101/1/46 Eth101/1/45

Eth101/1/44 Eth101/1/43 Eth101/1/42 Eth101/1/41Eth101/1/40 Eth101/1/39 Eth101/1/38 Eth101/1/37Eth101/1/36 Eth101/1/35 Eth101/1/34 Eth101/1/33Eth101/1/32 Eth101/1/31 Eth101/1/30 Eth101/1/29Eth101/1/28 Eth101/1/27 Eth101/1/26 Eth101/1/25Eth101/1/24 Eth101/1/23 Eth101/1/22 Eth101/1/21Eth101/1/20 Eth101/1/19 Eth101/1/18 Eth101/1/17Eth101/1/16 Eth101/1/15 Eth101/1/14 Eth101/1/13Eth101/1/12 Eth101/1/11 Eth101/1/10 Eth101/1/9Eth101/1/8 Eth101/1/7 Eth101/1/6 Eth101/1/5Eth101/1/4 Eth101/1/3 Eth101/1/2 Eth101/1/1



Disassociating a Fabric Extender From an Interface

Before You Begin



SUMMARY STEPS

1. configure terminal2. interface {ethernet slot/port | port-channel channel}3. no fex associate

DETAILED STEPS




Step 1

Specifies the interface to configure. The interface canbe an Ethernet interface or a port channel.

interface {ethernet slot/port | port-channel channel}

Example:switch(config)# interface port-channel 4switch(config-if)#

Step 2

Disassociates the Fabric Extender unit attached to theinterface.

no fex associate

Example:switch(config-if)# no fex associate

Step 3

Associating a Fabric Extender to an F2 ModuleYou can associate the Fabric Extender to an F2 module.

Before You Begin


• switch# configure terminal

• switch(config)# install feature-set fex

• switch(config)# feature-set fex



F2 modules are only supported by the following FEX devices:Note

• 2248TP

• 2248TP-E

• 2232TP

• 2232PP

• 2232TM

• 2224TP

Each port in the ASIC has an index. Allow only ports with similar indices across ASICs to be added to aport channel.

For example, if port 1 has an index of 1 and port 2 has an index of 2, then

Note

• Supported: Port 1 of ASIC 1 and port 1 of ASIC 2 are added to a port channel.

• Not supported: Port 1 of ASIC 1 and port 2 of ASIC 2 to form a port channel.

In general, a set of ports from an ASIC that has an index sub-set S, such as {1,2,4}, is allowed to be addedto a port-channel only if the port-channel has an equivalent or an empty set.

SUMMARY STEPS

1. vdc switch2. limit-resource module-type f23. interface ethernet <mod-number><port-range>4. allocate interface ethernet <slot-port>5. switchport mode fex -fabric6. fex associate <FEX chassis number>7. channel-group <port-channel number>8. no shutdown9. int po <port-channel number>10. no shutdown

DETAILED STEPS


Specifies the VDC.vdc switch

Example:switch(config)# vdc switch

Step 1




Specifies the module type.limit-resource module-type f2

Example:switch(config)# limit-resource module-type f2

Step 2

Specifies the interface.interface ethernet <mod-number><port-range>

Example:switch(config)# interface ethernet 1/1

Step 3

Allocates the F2 interfaces to VDC.allocate interface ethernet <slot-port>

Example:switch(config)# allocate interface ethernet 1

Step 4

Specifies the FEXswitchport mode fex -fabric

Example:switch(config-if)# switchport mode fex -fabric

Step 5

Specifies the chassis.fex associate <FEX chassis number>

Example:switch(config-if)# fex associate 101

Step 6

Specifies the port channel number.channel-group <port-channel number>

Example:switch(config-if)# channel-group 1

Step 7

Brings up port.no shutdown

Example:switch(config-if)# no shutdown

Step 8

Specifies the port channel.int po <port-channel number>

Example:switch(config-if)# int po 1

Step 9

Brings up the port channel.no shutdown

Example:switch(config-if)# no shutdown

Step 10

Configuring Fabric Extender Global FeaturesYou can configure global features on the Fabric Extender.


Configuring the Fabric ExtenderConfiguring Fabric Extender Global Features

Before You Begin

Ensure that you have enabled the Fabric Extender feature set.


SUMMARY STEPS

1. configure terminal2. fex FEX-number3. (Optional) description desc4. (Optional) no description5. (Optional) type FEX-type6. (Optional) no type7. (Optional) serial serial8. (Optional) no serial

DETAILED STEPS




Step 1

Enters configuration mode for the specified Fabric Extender. The range of theFEX-number is from 101 to 199.

fex FEX-number

Example:switch(config)# fex 101switch(config-fex)#

Step 2

(Optional)Specifies the description. The default is the string FEXxxxx where xxxx is theFEX-number. If the FEX-number is 123, the description is FEX0123.

description desc

Example:switch(config-fex)# descriptionRack7A-N2K

Step 3

(Optional)Deletes the description.

no description

Example:switch(config-fex)# no description

Step 4

(Optional)Specifies the type of Fabric Extender. FEX-type is one of the following:

type FEX-type

Example:switch(config-fex)# type N2248T

Step 5

• N2224TP—24 100Base-T/1000Base-T Ethernet host interfaces and 210-Gigabit SFP+ Ethernet fabric interfaces module

• N2232P and N2232TM—32 10-Gigabit SFP+ Ethernet host interfacesand 8 10-Gigabit SFP+ Ethernet fabric interfaces module


Configuring the Fabric ExtenderConfiguring Fabric Extender Global Features


• N2248T and N2248TP-E—48 100Base-T/1000Base-T Ethernet hostinterfaces and 4 10-Gigabit SFP+ Ethernet fabric interfaces module.

The parent Cisco Nexus Series device remembers the type of the FabricExtender in its binary configuration.When this feature is configured, the FabricExtender is only allowed to come online if its type matches the configuredFEX-type.

(Optional)Deletes the FEX type. When a Fabric Extender is connected to the fabricinterfaces and does not match the configured type saved in the binary

no type

Example:switch(config-fex)# no type

Step 6

configuration on the parent switch, all configurations for all interfaces on theFabric Extender are deleted.

(Optional)Defines a serial number string. If this command is configured, a switch onlyallows the corresponding chassis ID to associate (using the fex associatecommand) if the Fabric Extender reports a matching serial number string.

serial serial

Example:switch(config-fex)# serialJAF1339BDSK

Step 7

Configuring a serial number that does not match the specifiedFabric Extender forces the Fabric Extender offline.

Caution

(Optional)Deletes the serial number string.

no serial

Example:switch(config-fex)# no serial

Step 8

Configuration ExamplesThis section contains examples of FEX configurations.


Configuring the Fabric ExtenderConfiguration Examples

Configuring a FEX with a Layer 3 Host InterfaceThis example shows how to configure a Fabric Extender with a Layer 3 host interface (at the interface level,subinterface level, port channel level, and port channel subinterface level):

Layer 3 Sub-interface ConfigurationLayer 3 Configuration

config tinterface ethernet 101/1/1.12ip address 192.0.2.1/24encapsulation dot1Q 12mtu 850no shutdown

config tinterface ethernet 101/1/1no switchportip address 192.0.1.1/24Mtu 9000no shutdown

Layer 3 Host Interface Port Channel Sub-interfaceConfiguration

Layer 3 Host Interface Port Channel Configuration

config tinterface ethernet 101/1/1-2no switchportchannel-group 12no shutdown

interface port-channel 12.14ip address 192.0.4.1/24encapsulation dot1Q 14mtu 1700no shutdown

config tinterface ethernet 101/1/1-2no switchportchannel-group 12no shutdown

interface port-channel 12ip address 192.0.3.1/24mtu 2000no shutdown

The VLAN used in the Layer 3 host interface (HIF) or host interface port channel (HIFPC) subinterfacehas only local significance to its parent interface. The same VLAN ID can be reused between Layer 3subinterfaces in the same switch or VDC.

Note

Configuring a Host Interface in a vPC Topology Connected to Two FEXsThis example shows how to configure a host vPC with a FEX (host vPC attached to 2 different FEXs):

Switch 2 ConfigurationSwitch 1 Configuration

config tfeature lacpint e101/1/1-2channel-group 12 mode activeno shutdown

Int port-channel10switchportswitchport mode trunkswitchport trunk allowed vlan 1-20

vpc 10

config tfeature lacpint e101/1/1-2channel-group 12 mode activeno shutdown

Int port-channel10switchportswitchport mode trunkswitchport trunk allowed vlan 1-20

vpc 10



Dual-homing of a Server to a FEX with FabricPathTo configure FabricPath interfaces for two Cisco Nexus 7000 switches that are connected with FabricPath,perform the following tasks on each switch:

• Enable FabricPath on each switch.

• Configure the interfaces that you want to designate as FabricPath interfaces.

• Set the STP priority device to 8192 on all FabricPath Layer 2 gateway devices.

• (Optional) Set the STP domain ID for each of the separate STP domains that are connected to theFabricPath network.

• (Optional) Configure a FEX switch ID.

To configure FabricPath interfaces, follow these steps:

1 (Optional) Enable FabricPath on each switch.switch# config terminalswitch(config)# feature fabricpathswitch(config-lldp)# exitswitch(config)#

2 After you enable FabricPath on the switch, configure the specified interface as FabricPath interfaces.switch(config)# interface ethernet 1/2switch(config-if)# switchport mode fabricpathswitch(config-if)# exitswitch(config)#

3 Configure the STP priority for all Rapid PVST+ VLANs as 8192.switch# config terminalswitch(config)# spanning-tree vlan 11-20 priority 8192switch(config)#

4 Configure the STP priority for all MST instances as 8192.switch# config terminalswitch(config)# spanning-tree mst 1-5 priority 8192switch(config)#

5 (Optional) Configure the STP domain ID on each FabricPath Layer 2 gateway switch attached to theFabricPath network.switch# config terminalswitch(config)# spanning-tree domain 5switch(config)

6 (Optional) Configure the FEX switch ID.

See the Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide for information on configuringFEX.

Note

FEX VPC+ configurations are only supported on F2 modules.Note

If you are setting up an initial FEX VPC+ configuration on an F Series module, follow these steps:

a In the VPC domain configuration mode, enable partial DF mode with the fabricpath multicastload-balance command.



b If disabled, enable TRILL style mac-address learning with themac address-table core-port-learningcommand.

c In the VPC domain configurationmode, configure the emulated switch IDwith the fabricpath switchid<switchid#> command.

d On each of the VPC/VPC+ peer link interfaces in interface configuration mode, enter the switchportmode fabricpath command.

e On each VPC/VPC+ peer link port channel, enter the VPC peer-link command.

f Configure the VPC ID with the vpc vpcid command.

If you are changing an existing FEX VPC configuration to a FEX VPC+ configuration on an F Seriesmodule, follow these steps:

a In the VPC domain configuration mode, enable partial DF mode with the fabricpath multicastload-balance command.

b If disabled, enable trill style mac-address learning with themac address-table core-port-learningcommand.

c In the VPC domain configurationmode, configure the emulated switch IDwith the fabricpath switchid<switchid#> command.

7 Copy the configuration.switch(config)# copy running-config startup-configswitch(config)#

Verifying the ConfigurationThis section describes how to display the configuration of the Fabric Extender and verify the chassis hardwarestatus.

Verifying the Fabric Extender ConfigurationTo display configuration information about the defined interfaces on a Fabric Extender, perform one of thefollowing tasks:


Displays information about a specific Fabric Extenderor all attached units.

show fex [FEX-number] [detail]

Displays the Fabric Extender ports that are pinned toa specific switch interface.

show interface type number fex-intf

Displays the switch interfaces that have detected aFabric Extender uplink.

show interface fex-fabric


Configuring the Fabric ExtenderVerifying the Configuration


Displays the SFP+ transceiver and diagnostic opticalmonitoring (DOM) information for the FabricExtender uplinks.

show interface ethernet number transceiver[fex-fabric]

Displays the status of the feature sets on the device.show feature-set

Configuration Examples for the Fabric Extender

This example shows how to display all the attached Fabric Extender units:switch# show fexFEX FEX FEX FEX

Number Description State Model Serial------------------------------------------------------------------------101 FEX0101 Online N2K-C2248TP-1GE JAF1418AARL

This example shows how to display the detailed status of a specific Fabric Extender:switch# show fex 101 detailFEX: 101 Description: FEX0101 state: OnlineFEX version: 5.1(1) [Switch version: 5.1(1)]FEX Interim version: 5.1(0.159.6)Switch Interim version: 5.1(1)Extender Model: N2K-C2248TP-1GE, Extender Serial: JAF1418AARLPart No: 73-12748-05Card Id: 99, Mac Addr: 54:75:d0:a9:49:42, Num Macs: 64Module Sw Gen: 21 [Switch Sw Gen: 21]pinning-mode: static Max-links: 1Fabric port for control traffic: Po101Fabric interface state:Po101 - Interface Up. State: ActiveEth2/1 - Interface Up. State: ActiveEth2/2 - Interface Up. State: ActiveEth4/1 - Interface Up. State: ActiveEth4/2 - Interface Up. State: Active

Fex Port State Fabric Port Primary FabricEth101/1/1 Up Po101 Po101Eth101/1/2 Up Po101 Po101Eth101/1/3 Down Po101 Po101Eth101/1/4 Down Po101 Po101Eth101/1/5 Down Po101 Po101Eth101/1/6 Down Po101 Po101Eth101/1/7 Down Po101 Po101Eth101/1/8 Down Po101 Po101Eth101/1/9 Down Po101 Po101Eth101/1/10 Down Po101 Po101Eth101/1/11 Down Po101 Po101Eth101/1/12 Down Po101 Po101Eth101/1/13 Down Po101 Po101Eth101/1/14 Down Po101 Po101Eth101/1/15 Down Po101 Po101Eth101/1/16 Down Po101 Po101Eth101/1/17 Down Po101 Po101Eth101/1/18 Down Po101 Po101Eth101/1/19 Down Po101 Po101Eth101/1/20 Down Po101 Po101Eth101/1/21 Down Po101 Po101Eth101/1/22 Down Po101 Po101Eth101/1/23 Down Po101 Po101Eth101/1/24 Down Po101 Po101Eth101/1/25 Down Po101 Po101Eth101/1/26 Down Po101 Po101Eth101/1/27 Down Po101 Po101Eth101/1/28 Down Po101 Po101Eth101/1/29 Down Po101 Po101


Configuring the Fabric ExtenderVerifying the Fabric Extender Configuration

Eth101/1/30 Down Po101 Po101Eth101/1/31 Down Po101 Po101Eth101/1/32 Down Po101 Po101Eth101/1/33 Down Po101 Po101Eth101/1/34 Down Po101 Po101Eth101/1/35 Down Po101 Po101Eth101/1/36 Down Po101 Po101Eth101/1/37 Down Po101 Po101Eth101/1/38 Down Po101 Po101Eth101/1/39 Down Po101 Po101Eth101/1/40 Down Po101 Po101Eth101/1/41 Down Po101 Po101Eth101/1/42 Down Po101 Po101Eth101/1/43 Down Po101 Po101Eth101/1/44 Down Po101 Po101Eth101/1/45 Down Po101 Po101Eth101/1/46 Down Po101 Po101Eth101/1/47 Down Po101 Po101Eth101/1/48 Down Po101 Po101

Logs:09/21/2010 21:14:26.843850: Module register received09/21/2010 21:14:26.845778: Registration response sent09/21/2010 21:14:27.663073: Module Online Sequence09/21/2010 21:14:30.191121: Module Online

This example shows how to display the Fabric Extender interfaces pinned to a specific switch interface:switch# show interface port-channel 101 fex-intfFabric FEXInterface Interfaces---------------------------------------------------Po101 Eth101/1/2 Eth101/1/1

This example shows how to display the switch interfaces that are connected to a Fabric Extender uplink:switch# show interface fex-fabric

Fabric Fabric Fex FEXFex Port Port State Uplink Model Serial---------------------------------------------------------------101 Eth2/1 Active 1 N2K-C2248TP-1GE JAF1418AARL101 Eth2/2 Active 2 N2K-C2248TP-1GE JAF1418AARL101 Eth4/1 Active 3 N2K-C2248TP-1GE JAF1418AARL101 Eth4/2 Active 4 N2K-C2248TP-1GE JAF1418AARL

This example shows how to display the SFP+ transceiver and diagnostic optical monitoring (DOM) informationfor Fabric Extender uplinks for an SFP+ transceiver that is plugged into the parent switch interface:switch# show interface ethernet 1/40 transceiverEthernet1/40

sfp is presentname is CISCO-MOLEX INCpart number is 74752-9026revision is A0serial number is MOC13321057nominal bitrate is 12000 MBits/secLink length supported for copper is 3 m(s)cisco id is --cisco extended id number is 4

This example shows how to display the SFP+ transceiver and DOM information for Fabric Extender uplinksfor an SFP+ transceiver that is plugged into the uplink port on the Fabric Extender:switch# show interface ethernet 1/40 transceiver fex-fabricEthernet1/40

sfp is presentname is CISCO-MOLEX INCpart number is 74752-9026revision is A0serial number is MOC13321057nominal bitrate is 12000 MBits/secLink length supported for 50/125mm fiber is 0 m(s)Link length supported for 62.5/125mm fiber is 0 m(s)


Configuring the Fabric ExtenderVerifying the Fabric Extender Configuration

cisco id is --cisco extended id number is 4

The following example shows how to display the status of the feature sets on the device:switch# show feature-setFeature Set Name ID State-------------------- -------- --------fcoe 1 installedfabricpath 2 enabledfex 3 enabledswitch#

The following example shows how to display the services used by the Fabric Extender feature set:switch# show feature-set services fexvntag_mgrfex2 services in feature set fexswitch#

Verifying the Chassis Management InformationTo display configuration information used on the switch supervisor to manage the Fabric Extender, performone of the following commands:


Displays the environmental sensor status.show environment fex {all | FEX-number}[temperature | power | fan]

Displays inventory information for a Fabric Extender.show inventory fex FEX-number

Displaysmodule information about a Fabric Extender.show module fex [ FEX-number ]

Displays the contents of the serial PROM (SPROM)on the Fabric Extender.

show sprom fex FEX-number {all | backplane |powersupply ps-num} | all

Configuration Examples for Chassis Management

This example shows how to display the module information about all connected Fabric Extender units:switch# show module fex

FEX Mod Ports Card Type Model Status.--- --- ----- ---------------------------------- ------------------ -----------101 1 48 Fabric Extender 48x1GE + 4x10G M N2K-C2248TP-1GE ok

FEX Mod Sw Hw World-Wide-Name(s) (WWN)--- --- -------------- ------ -----------------------------------------------101 1 5.1(1) 3.5 --FEX Mod MAC-Address(es) Serial-Num--- --- -------------------------------------- ----------101 1 5475.d0a9.4940 to 5475.d0a9.496f JAF1418AARL

This example shows how to display the inventory information about a specific Fabric Extender:switch# show inventory fex 101NAME: "FEX 101 CHASSIS", DESCR: "N2K-C2248TP-1GE CHASSIS"PID: N2K-C2248TP-1GE , VID: V00 , SN: SSI13380FSM


Configuring the Fabric ExtenderVerifying the Chassis Management Information

NAME: "FEX 101 Module 1", DESCR: "Fabric Extender Module: 48x1GE, 4x10GE Supervisor"PID: N2K-C2248TP-1GE , VID: V00 , SN: JAF1339BDSK

NAME: "FEX 101 Fan 1", DESCR: "Fabric Extender Fan module"PID: N2K-C2248-FAN , VID: N/A , SN: N/A

NAME: "FEX 101 Power Supply 2", DESCR: "Fabric Extender AC power supply"PID: NXK-PAC-400W , VID: 000, SN: LIT13370QD6

This example shows how to display the environment status for a specific Fabric Extender:switch# show environment fex 101

Temperature Fex 101:-----------------------------------------------------------------Module Sensor MajorThresh MinorThres CurTemp Status

(Celsius) (Celsius) (Celsius)-----------------------------------------------------------------1 Outlet-1 60 50 33 ok1 Outlet-2 60 50 38 ok1 Inlet-1 50 40 35 ok1 Die-1 100 90 44 ok

Fan Fex: 101:------------------------------------------------------Fan Model Hw Status------------------------------------------------------Chassis N2K-C2148-FAN -- okPS-1 -- -- absentPS-2 NXK-PAC-400W -- ok

Power Supply Fex 101:---------------------------------------------------------------------------Voltage: 12 Volts-----------------------------------------------------PS Model Power Power Status

(Watts) (Amp)-----------------------------------------------------1 -- -- -- --2 NXK-PAC-400W 4.32 0.36 ok

Mod Model Power Power Power Power StatusRequested Requested Allocated Allocated(Watts) (Amp) (Watts) (Amp)

--- ------------------- ------- ---------- --------- ---------- ----------1 N2K-C2248TP-1GE 0.00 0.00 0.00 0.00 powered-up

Power Usage Summary:--------------------Power Supply redundancy mode: redundant

Total Power Capacity 4.32 W

Power reserved for Supervisor(s) 0.00 WPower currently used by Modules 0.00 W

-------------Total Power Available 4.32 W

-------------

This example shows how to display the SPROM for a specific Fabric Extender:switch# show sprom fex 101 allDISPLAY FEX 101 SUP sprom contentsCommon block:Block Signature : 0xababBlock Version : 3Block Length : 160



Block Checksum : 0x1a1eEEPROM Size : 65535Block Count : 3FRU Major Type : 0x6002FRU Minor Type : 0x0OEM String : Cisco Systems, Inc.Product Number : N2K-C2248TP-1GESerial Number : JAF1339BDSKPart Number : 73-12748-01Part Revision : 11Mfg Deviation : 0H/W Version : 0.103Mfg Bits : 0Engineer Use : 0snmpOID : 9.12.3.1.9.78.3.0Power Consump : 1666RMA Code : 0-0-0-0CLEI Code : XXXXXXXXXTBDV00VID : V00Supervisor Module specific block:Block Signature : 0x6002Block Version : 2Block Length : 103Block Checksum : 0x2686Feature Bits : 0x0HW Changes Bits : 0x0Card Index : 11016MAC Addresses : 00-00-00-00-00-00Number of MACs : 0Number of EPLD : 0Port Type-Num : 1-48;2-4Sensor #1 : 60,50Sensor #2 : 60,50Sensor #3 : -128,-128Sensor #4 : -128,-128Sensor #5 : 50,40Sensor #6 : -128,-128Sensor #7 : -128,-128Sensor #8 : -128,-128Max Connector Power: 4000Cooling Requirement: 65Ambient Temperature: 40

DISPLAY FEX 101 backplane sprom contents:Common block:Block Signature : 0xababBlock Version : 3Block Length : 160Block Checksum : 0x1947EEPROM Size : 65535Block Count : 5FRU Major Type : 0x6001FRU Minor Type : 0x0OEM String : Cisco Systems, Inc.Product Number : N2K-C2248TP-1GESerial Number : SSI13380FSMPart Number : 68-3601-01Part Revision : 03Mfg Deviation : 0H/W Version : 1.0Mfg Bits : 0Engineer Use : 0snmpOID : 9.12.3.1.3.914.0.0Power Consump : 0RMA Code : 0-0-0-0CLEI Code : XXXXXXXXXTDBV00VID : V00Chassis specific block:Block Signature : 0x6001Block Version : 3Block Length : 39Block Checksum : 0x2cfFeature Bits : 0x0



HW Changes Bits : 0x0Stackmib OID : 0MAC Addresses : 00-0d-ec-e3-28-00Number of MACs : 64OEM Enterprise : 0OEM MIB Offset : 0MAX Connector Power: 0WWN software-module specific block:Block Signature : 0x6005Block Version : 1Block Length : 0Block Checksum : 0x66wwn usage bits:00 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00 00 00 00 00 00 0000 00License software-module specific block:Block Signature : 0x6006Block Version : 1Block Length : 16Block Checksum : 0x86flic usage bits:ff ff ff ff ff ff ff ff

DISPLAY FEX 101 power-supply 2 sprom contents:Common block:Block Signature : 0xababBlock Version : 3Block Length : 160Block Checksum : 0x1673EEPROM Size : 65535Block Count : 2FRU Major Type : 0xab01FRU Minor Type : 0x0OEM String : Cisco Systems Inc NXK-PAC-400WProduct Number : NXK-PAC-400WSerial Number : LIT13370QD6Part Number : 341Part Revision : -037CLEI Code : 5-01 01 000VID : 000snmpOID : 12336.12336.12336.12336.12336.12336.12374.12336H/W Version : 43777.2Current : 36



RMA Code : 200-32-32-32Power supply specific block:Block Signature : 0x0Block Version : 0Block Length : 0Block Checksum : 0x0Feature Bits : 0x0Current 110v : 36Current 220v : 36Stackmib OID : 0

Additional ReferencesThis section includes additional information that is related to configuring the Cisco Nexus 2000 Series FabricExtender.

Related DocumentsDocument TitleRelated Topic

Cisco NX-OS Licensing GuideCisco NX-OS Licensing

Cisco Nexus 7000 Series NX-OS Virtual Device Context ConfigurationGuide

Virtual device contexts (VDC)

Cisco Nexus 7000 Series NX-OS Interfaces Configuration GuideInterface configuration

Cisco Nexus 7000 Series Command References available at the followingURL: http://www.cisco.com/en/US/products/ps9402/prod_command_reference_list.html

Command reference

Feature HistoryThis table lists the release history for this feature.

Table 2: Feature History for the Cisco Nexus 2000 Series Fabric Extender

Feature InformationReleasesFeature Name

Support was added when the FEX is connected to the48-port, 100/1000BASE-T (100-Mb/1-Gigabit)N2248TP-E and 32-port 10-Gigabit SFP+N2248TP-E.

6.1(1)Support for the Cisco Nexus 7000 Series

Support for M2 series modules.6.1(1)Support for M2 series modules

Support was added when the FEX is connected to the48-port, 1/10-Gigabit F2 module (N7K-F248XP-25).



Configuring the Fabric ExtenderAdditional References



Feature InformationReleasesFeature Name

Support was added when the FEX is connected to the32-port, 10-Gigabit M1 module (N7K-M132XP-12)or the 32-port, 10-Gigabit M1 XL module(N7K-M132XP-12L).


The Fabric Extender is enabled on the parent CiscoNexus 7000 Series device with the install feature-setand feature-set commands.

5.1(1)Feature set commands

Support was added for port channels and theintegration of vPC on the host interfaces.

5.2(1)Port channel and vPC support

Layer 3 capability was added to the Fabric Extenderhost interface ports including IPv4 and IPv6 andIGMP snooping.

5.2(1)Layer 3 support


Configuring the Fabric ExtenderFeature History

I N D E X

A

ACL support 5associating fabric extender 16

B

BPDU Guard 3, 8

C

chassis configuration mode 21chassis ID 9class of service, See CoSconfiguration data 7CoS 5

D

Data Center Bridging Exchange, See DCBXDCBX 5description 21digital optical monitoring, See DOMDOM 3

E

edge port (PortFast) 3Ethernet fabric interface 2

F

fabric extender 10guidelines 10licensing requirements 10limitations 10

fabric interface 2fabric interface Port Channel 9fail-over load balancing 9FEX-number 9

G

guidelines 10

H

host interface 2host interface autonegotiation 5

I

IEEE 802.1p 5IEEE 802.3x 5IGMP snooping 6image management 10

L

LACP 5layer 3 ports 3licensing 10

fabric extender 10limitations 10Link Aggregation Control Protocol, See LACPLink Layer Discovery Protocol, See LLDPLLDP 5local switching 8loopback address assignment 7loopback address range 7

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide OL-25816-02 IN-1

M

multicast replication 8

O

oversubscription 6oversubscription ratio 6

P

packet counter 3PFC 3port channel 9port channel fabric interface 2, 3port channel host interface 2port numbering 9priority flow control, See PFC

Q

QoS 5quality-of-service, See QoS

R

related documents 33

S

serial number 21SFP+ validation 3show diagnostics 29show environment 29show fex 26show inventory 29show modules 29show SPROM 29show transceiver status 26SPAN restrictions 6SPAN source ports 6subinterfaces 3switchport fex-fabric mode 3switchport saved configuration 3

T

type 21

V

version compatibility 10VLANs 5

Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration GuideIN-2 OL-25816-02

Index