Nexus7000VirtualPortChannelOverview 09starmer Trans

8/22/2019 Nexus7000VirtualPortChannelOverview 09starmer Trans

http://slidepdf.com/reader/full/nexus7000virtualportchanneloverview-09starmer-trans 1/42

© 2008 Cisco Systems, Inc. All rights reserved. 1Cisco Restricted and Confidential

Nexus 7000 virtual

Port ChannelOverview

Nexus 7000 TME

Data Center Business Unit

Version 1.6 September 29, 2008




Current Network Scaling ChallengesData Center

Traditional Data Center designs are requiring ever increasing Layer 2 adjacencies between

servers due to applications, Virtualization technology and server growth. The size of Layer 2

networks is stretched, placing more burden on loop-avoidance protocols (Spanning Tree)

L3 Core

L2/L3

Aggregation

L2 Access

Dual-Homed Servers,

Single active uplink per

VLAN (PVST), L2

reconvergence

FHRP, Single active

uplink per VLAN, L2

reconvergence,

excessive BPDUs

BGP, IGP, ECMP

Policy Management

DC Pod: L2 Domain




Nexus 7000 L2 POD Innovation

L2

L3

L3

L2

IP Cloud

vPC

Core

Aggregation

Access

Servers

vPC

vPC

STP+ vPC OTV DCE

Now Q4CY08 2HCY09 Q1CY10

STPEnhancements

BridgeAssurance

NIC Teaming

Simplifiedloop-free trees

2x Multi-pathing

Inter-PODConnectivity

across L3

FailureBoundary

Preservation

16x ECMP

Low Latency

Lossless

OperationalFlexibility

DCE

Failure

Boundary




virtual Port-ChannelFeature Overview

Allow a single device to use aport channel across twoupstream switches

Eliminate STP blocked ports

Uses all available uplinkbandwidth

Dual-homed server operate inactive-active mode

Provide fast convergenceupon link/device failure

Reduce CAPEX and OPEX

Available in NX-OS 4.1 withcurrent and future hardware

Logical Topology without vPC

Logical Topology with vPC

Available 2HCY08




vPC Terminology

vPC peer – a vPC switch, one of apair

vPC member port – one of a set of ports (port channels) that form a vPC

vPC – the combined port channelbetween the vPC peers and thedownstream device

vPC peer-link – Link used tosynchronize state between vPC peer devices, must be 10GbE

vPC ft-link – the fault tolerant linkbetween vPC peer devices, i.e.,backup to the vPC peer-link

CFS – Cisco Fabric Servicesprotocol, used for statesynchronization and configurationvalidation between vPC peer devices

vPC

vPC peer-link

vPC peer

non-vPC

device

vPC ft-link

vPC

member

port

vPC

vPC

member

port

CFS protocol




vPC system requirements

Support for vPC is on Nexus-OS 4.1.2 or later on the Nexus 7000 platform.

vPC being investigated for use on other platforms

vPC peer link must be on a N7K-M132XP-12,dedicated mode port is recommended

vPC ft-link is supported on any L3 reachableinterface (e.g., the system managementinterface)

Redundant ft-link requires dual sups asrecommended

Support for a maximum of 752 vPC portchannels


http://slidepdf.com/reader/full/nexus7000virtualportchanneloverview-09starmer-trans 7/42© 2007 Cisco Systems, Inc. All rights reserved. 7Cisco Restricted and Confidential

Port Channel Properties

Downstream end:

–Standard link load balancing protocolsavailable (depends on downstreamdevice; src/dst-mac, round-robin, etc.)

–Works with LACP and manuallyconfigured links

vPC end:

–Load-balancing scheme modified to keeptraffic forwarding local (i.e., packetheaded into the link aggregation groupwill use one of the local links rather thanacross the vPC peer-link)

Standard PortChannel on

Downstream

Switches

Standard PortChannel on

Downstream

Switches

Standard Port

Channel on

Downstream

Switches

vPC on vPC

peers with

local

forwarding



vPC peer interaction vPC Primary

–Primary is manually defined, withmanual failback in case of systemfailure

–STP root highest priority

–HSRP active highest priority

–PIM DR highest priority

vPC Secondary –STP root lower priority

–HSRP standby

–PIM DR standby

STP is used for backup in caseof vPC failure

STP, HSRP, PIMprimary/secondary configurationshould follow vPCprimary/secondary to simplifydebug

STP/HSPR/PIM failover tosecondary/standby is not forcedby vPC, follows standard

failover operation

vPC

Secondary

vPC Primary

STP root

HSRP Active

PIM DR

STP backup

HSRP Standby



Forwarding Basics

vPC acts as one link with distributedcontrol

Unknown unicast, multicast frames are

flooded, and STP BPDU packets areforwarded across the vPC peer-link

STP port blocking operations disabled onvPC member ports

STP BDPUs sent down vPC peer-link andvPC path on Primary vPC peer



vPC L3 Interaction on N7K

Some L3 functions may use VPC peer-linkfor direct communication

–FHRP

–PIM/Multicast

HSRP and PIM enhanced to support moreefficient forwarding and reduced VPC peer-link use for data-plane traffic



vPC Control Plane

vPC peer switches are separate controlplanes, with separate configurations

Configuration should be identical for:

–STP, HSRP, PIM

–Port channel configuration

•Port ACLs

•Buffer and Queue configuration

Non critical forwarding configurations donot keep vPC from creating a port channel,but may cause odd forwarding behavior

Use DCNM to manage vPC on N7K



Configuration Element Types

Type 1 Configuration elements –Configuration elements that keep vPC from

running (no port channels can be created)

– vPC, STP, and HSRP configuration

–vPC member port channel configuration:secondary switch port(s) will not join the channel

Type 2 Configuration elements

–Configuration elements that will not keep a portchannel from being created, but may cause oddforwarding behavior

– ACLs

–Syslog message will be generated on configurationmismatch



vPC configuration commands

configure vPC, and start the ft-link on both peers:

(config)# feature vpc

(config)# vpc domain 1

(config-vpc-domain)# peer-keepalive destination x.x.x.x sourcex.x.x.y

(conifg)# int port-channel 10

(config-int)# vpc peer-link

Move any port-channels into appropriate vPC groups

(config)# int port-channel 20

(config-int)# vpc 20



Forwarding in detail



vPC on Nexus 7000Unicast Forwarding Mac_A to Mac_B

MAC_A MAC_BSW3

SW2

SW4

SW1

vPC1 vPC2

vPC_PL

L2

L3

ECMP

Packet Send

ECMP

Port channel

path selection

Packet

Flooding

Packet

Flooding

Packet(s)blocked on

vPC member

ports, vPC

peer-link

traversed

vPC FT-Link



vPC on Nexus 7000Unicast Forwarding Reply Mac_B to Mac_A

MAC_A MAC_BSW3

SW2

SW4

SW1

vPC1 vPC2

vPC_PL

L2

L3

ECMP

Packet Send

ECMP

Port channelpath selection

Local

forwarding,

previously

learned

destination

vPC FT-Link



vPC on Nexus 7000BPDU forwarding

MAC_A MAC_BSW3

SW2

SW4

SW1

vPC1 vPC2

vPC_PL

L2

L3

ECMP

Packet Send

ECMP

STPRoot

STP RootBackup

Packet

Flooding

Packet

Flooding

STP process

updated,

BDPUs notforwarded

on vPC

member

portsBPDUs

forwarded

vPC FT-Link



vPC on Nexus 7000L2 IGMP Interaction

MAC_A MAC_BSW3

SW2

SW4

SW1

vPC1 vPC2

vPC_PL

L2

L3

ECMP

Packet Send

ECMP

vPCPrimary

Join/Leave

received by

IGMP process,

IGMP forwarded join/leave

messages to

peer

vPC FT-Link

vPCSecondary

Join/Leave

received by

IGMP process,

IGMP forwarded join/leave

messages to

peer



L3 enhancements to

forwarding



HSRP on a vPC based SVI

No changes to HSRP control or wire protocol

HSRP process on vPC peers use the sameshared MAC (derived from the initial activeHSRP member)

HSRP active responds to ARP requests

vPC peers are aware of HSRP active/standby:data sent to the remote “active” HSRP peer are in fact forwarded to the local HSRP

standby peer

Reduces vPC peer-link use

Improves L3 upstream bandwidth(active/active)



vPC on Nexus 7000HSRP Interaction

MAC_A MAC_BSW3

SW2

SW4

SW1

vPC1 vPC2

vPC_PL

L2

L3

ECMP

Packet Send

ECMP

HSRPActive

HSRPStandby

HSRP active MAC

is populated into

the L3 hardware

forwarding tables,

creating a localforwarding

capability on the

HSRP standby

device

vPC FT-Link

HSRP active

process

communicates the

active MAC to its

neighbor. Only the

HSRP active

process responds

to ARP requests



Multicast on SVI with VPC

No changes to PIM configuration or wire protocol

Both VPC peers function as DR for VPC VLANs

–Only “real” DR functions as DR for non-VPC VLANs

Only “real” DR builds L3 multicast tree to PIM RP

IGMP snooping state (L2 table) synchronized betweenVPC peers (via CFS)



Failure response



vPC Failure State DiagramStart

vPC PLfailed?

(UDLD/Linkstate)

CFSmessagedeliveryfailure?

vPC ft-linkheartbeatdetect?

vPCsecondary

peer?

Suspend vPCmember ports

Other processes takeover based on priority(STP root, HSRPactive, PIM DR)

No

No

No

Yes

Yes

Yes

Yes

vPC peer recovered?

YesNo

Recover vPCmember ports




vPC on Nexus 7000Failure Reaction vPC peer-link failure (link loss)

SW3 SW4

vPC1 vPC2

vPC_PLvPCPrimary vPCSecondary

vPC FT-Link

In case vPC peer-link fails

Check active status of remote vPC peer via vPC ft-link (heartbeat)

If both peers are active, thenSecondary will disable allvPC ports to prevent loops

Data will automaticallyforward down remainingactive port channel ports

Failover gated on CFSmessage failure, or UDLD/Link state detection

Suspend AllvPC Member

Ports




vPC on Nexus 7000Failure Reaction vPC ft-link

SW3 SW4

vPC1 vPC2

vPC_PL

vPC FT-Link

In case vPC ft-link fails

Don’t care, vPC peer -linkstill active

vPCPrimary vPCSecondary




vPC on Nexus 7000Failure Reaction vPC Peer

SW3 SW4

vPC1 vPC2

vPC_PL

vPC FT-Link

In case vPC peer fails

Check active status of remote vPC peer via vPC ft-link

Remaining active peer becomes root for STP,HSRP active, PIM DR

Data will automaticallyforward down remainingactive port channel ports






vPC on Nexus 7000Failure Reaction vPC peer-link active, control planefailure (no CFS messages)

SW3 SW4

vPC1 vPC2

vPC_PL


Ports

vPC FT-Link

CFS messages not received(Unidirectional link?), butlink is still active

Check to see if vPC peer isactive via vPC ft-link(heartbeat)

If Primary is active,Secondary disables vPCmember ports

If no Primary, Secondarytakes over root functions






vPC on Nexus 7000Failure Reaction vPC Primary control plane fails (or supervisor switchover)

SW3 SW4

vPC1 vPC2

vPC_PL

vPC FT-Link

In case Primary control planefails, sysmgr process willrestart the process (up to 3x)and then failover to the localstandby sup and as a lastattempt, reboot the entireswitch

Check active status of remotevPC peer via vPC ft-link

HSRP/PIM/STP will fail toactive on the Secondary vPCpeer switch

Data will automatically forward,

but learning may be broken for new flows that are directed tothe dead Primary

Failover gated on CFSmessage failure, or UDLD/Linkstate detection





vPC on Nexus 7000Failure Reaction vPC peer-link and ft-link fail

SW3 SW4

vPC1 vPC2

vPC_PL

vPC FT-Link

In case vPC peer-link & ft-link fails

Check active status of remote vPC peer via vPC ft-link (which fails)

HSRP/PIM/STP will fail toactive on the SecondaryvPC peer switch as well asthe primary

Data will continue to

forward, but learning will bebroken for new flows



DOUBLEFAILURE




vPC on Nexus 7000Failure Reaction vPC Peer with upstream STP root

SW3 SW4

vPC1 vPC2

vPC_PL

vPC FT-Link

NOTE: This is not arecommended configuration

In case vPC peer-link and ft-link fail

Upstream STP process willunblock port to Secondary

Any packet sent via the STProot will be re-broadcastdown the vPC link to theedge device

No Loop (upstream is part of an etherchannel still)


May occur with servicesplatforms, system in transition




vPC on Nexus 7000Failure Reaction vPC peer-link failure, single attacheddevice

SW3 SW4

vPC1 vPC2

vPC_PL


Ports

vPC FT-Link

NOTE: This is not arecommended configuration

vPC peer-link failure

Check to see if vPC peer isactive via vPC ft-link(heartbeat)

If Primary is active,Secondary disables vPCmember ports

Single attached device (onSecondary) is on an island


Potentially due toa double failure




Deployment

Scenarios




Deployment Best Practices

All non vPC peer devices should be attached via vPC to the switchsystem

STP root should be on a vPC peer, or on a vPC attached device

When possible, use SVIs on vPC peer devices and HSRP for L3interfaces

Use N7K-M132XP-12 as vPC peer-link 10GbE port in dedicatedmode (shared mode is supported)

IGMP snooping should either be enabled on both vPC peers or disabled on both vPC peers

When possible:

–Use 2 different modules for redundant vPC peer-links

–Use UDLD on the vPC peer-links

–Use port channel of multiple vPC peer-link connections.

–Use the management interface(s) for the vPC ft-link




Multi-level vPC

SW4SW3

vPC_PL

vPC FT-Link

SW2SW1

vPC_PL

vPC FT-Link

SW4SW3

vPC_PL

vPC FT-Link

SW2SW1

vPC_PL

vPC FT-Link

Physical

View

Logical

View

Up to 8 links between both sets of switches (maximum port channel configfor current line cards), 4 ports from s1-s3, s1-s4, s2-s3, s2-s4

Provides maximum non-blocking bandwidth between sets of switch peers

Is not limited to one layer, can be extended indefinitely




vPC at the Agg and Access Layer

Servers and/or bladeswitches need to support802.3ad LACP linkaggregation

Can now haveactive/active edgeaccess without oddforwarding paradigms(multi-IP, active/passivein, active/active out)

No loop topology from L3to the server with fullbandwidth available!

L2

L3




Multi-level vPC

SW4SW3

vPC_PL

vPC FT-Link

SW2SW1

vPC_PL

vPC FT-Link

SW4SW3

vPC_PL

vPC FT-Link

SW2SW1

vPC_PL

vPC FT-Link

Physical

View

Logical

View

Up to 8 links between both sets of switches (maximum port channel configfor current line cards), 4 ports from s1-s3, s1-s4, s2-s3, s2-s4

Provides maximum non-blocking bandwidth between sets of switch peers

Is not limited to one layer, can be extended indefinitely




Invalid vPC Topologies

SW5

SW3

vPC_PL

vPC FT-LinkSW2SW1

vPC_PL

vPC FT-Link

SW4

No more than 2 switches in a vPC peer group

More than 2 way multi-path topologies are not valid, L2MP on DCE is therecommended path to support this




Modular virtual Port-Channel

Nexus 7000 Separate control plane

Separate management planereliable vPC state

synchronization (CFS) Redundant supervisor per

chassis

Manual port sync config

Local SVI HSRP forwardingenhancement to act asactive-active pair

Catalyst 6500 VSS Single control plane

Single management plane

Single supervisor per chassis

Port config sync (singlecontrol plane)

Single L3 domain (singleSVI) no need for FHRP,

reduced route peers in L3domain




Invalid vPC Topologies

SW5

SW3

vPC_PL

vPC FT-LinkSW2SW1

vPC_PL

vPC FT-Link

SW4

No more than 2 switches in a vPC peer group

More than 2 way multi-path topologies are not valid, L2MP on DCE is therecommended path to support this




Modular virtual Port-Channel

Nexus 7000 Separate control plane

Separate management planereliable vPC state

synchronization (CFS) Redundant supervisor per

chassis

Manual port sync config

Local SVI HSRP forwardingenhancement to act asactive-active pair

Catalyst 6500 VSS Single control plane

Single management plane

Single supervisor per chassis

Port config sync (singlecontrol plane)

Single L3 domain (singleSVI) no need for FHRP,

reduced route peers in L3domain



Documents

Nexus7000VirtualPortChannelOverview 09starmer Trans