Learn. Practice. Achieve. @ CCIEin8Weeks · 2018-08-23 · Learn. Practice. Achieve. @ CCIEin8Weeks.com 4 Contents at a Glance Part 1 Cisco Data Center L2/L3 Technologies Chapter

Learn. Practice. Achieve. @ CCIEin8Weeks.com

2

All-in-One CCIE Data Center V2.1

400-151 Written Exam Cert Guide

3rd Edition


3


4

Contents at a Glance

Part 1 Cisco Data Center L2/L3 Technologies

Chapter 1: Design, implement, and troubleshoot Layer 2 technologies

Chapter 2: Design, implement, and troubleshoot overlays

Chapter 3: Design, implement, and troubleshoot routing protocols and features

Chapter 4: Design, implement, and troubleshoot multicast protocols

Chapter 5: Describe inter-fabric connectivity

Chapter 6: Design, implement, and troubleshoot external fabric connectivity

Chapter 7: Design, implement, and troubleshoot traffic management

Part 2 Cisco Data Center Network Services

Chapter 8: Design, implement, and troubleshoot network services insertion and redirection

Chapter 9: Design, implement, and troubleshoot services

Chapter 10: Design, implement, and troubleshoot RBAC

Chapter 11: Design, implement, and troubleshoot maintenance tasks

Chapter 12: Design, implement, and troubleshoot security features

Part 3 Data Center Storage Networking and Compute

Chapter 13: Describe, configure, and troubleshoot infrastructure to support block storage

protocols

Chapter 14: Design, implement, and troubleshoot data center storage networking features

Chapter 15: Design, implement, and troubleshoot compute policies and profiles

Chapter 16: Design, implement, and troubleshoot data center connectivity

Part 4 Data Center Automation and Orchestration

Chapter 17: Implement and troubleshoot data center tasks using provided Python scripts

Chapter 18: Describe and design data center orchestration using tools

Part 5 Data Center Fabric Infrastructure

Chapter 19: Configure and troubleshoot physical fabric components

Chapter 20: Design, implement, and troubleshoot fabric policies

Chapter 21: Design, implement, and troubleshoot tenant policies


5

Chapter 22: Analyze and troubleshoot logical fabric elements

Chapter 23: Design, implement, and troubleshoot virtual networking

Part 6 Evolving Technologies V1.1

Chapter 24: Cloud

Chapter 25: Network Programmability

Chapter 26: Internet of Things (IoT)


6

Table of Contents

PART 1 CISCO DATA CENTER L2/L3 TECHNOLOGIES..................................................... 16

CHAPTER 1: DESIGN, IMPLEMENT, AND TROUBLESHOOT LAYER 2 TECHNOLOGIES .............................................................................................................................17

Link Aggregation ................................................................................................................................................................................... 18 NX-OS Version Requirement for vPC ...................................................................................................................................... 20 NX-OS License Requirement for vPC ...................................................................................................................................... 21 Components of vPC ............................................................................................................................................................................ 21 vPC Peer Link......................................................................................................................................................................................... 26 vPC Peer-Keepalive............................................................................................................................................................................ 27 vPC Ports, and Orphaned Ports .................................................................................................................................................. 28 Traffic Flows ............................................................................................................................................................................................ 28 Dual-Control Plane with Single Layer 2 Node Behavior ................................................................................................ 29 Link Aggregation Group Identifier ............................................................................................................................................... 29 Dual-Control Plane with Single Layer 2 Node Behavior ................................................................................................ 30 Link Aggregation Group Identifier ............................................................................................................................................... 30 System ID in a vPC System ........................................................................................................................................................... 32 Primary and Secondary vPC Roles ........................................................................................................................................... 32 Spanning Tree ........................................................................................................................................................................................ 33 Cisco Fabric Services over Ethernet Synchronization Protocol ................................................................................ 34 vPC Configuration Changes When the Peer Link Fails ................................................................................................. 35 vPC Reload Restore ........................................................................................................................................................................... 40 vPC Configuration Consistency ................................................................................................................................................... 41 vPC Consistency Check ................................................................................................................................................................... 41 vPC Configuration Synchronization ........................................................................................................................................... 42 Duplicate Frames Prevention in vPC ........................................................................................................................................ 44 vPC and Object Tracking ................................................................................................................................................................. 45 In-Service Software Upgrade and vPC .................................................................................................................................... 46 Interactions Between vPC and Routing ................................................................................................................................... 47 HSRP Gateway Considerations ................................................................................................................................................... 48 HSRP Configuration and Best Practices for vPC .............................................................................................................. 48 ARP Synchronization ......................................................................................................................................................................... 49 Peer Gateway ......................................................................................................................................................................................... 49 Layer 3 Link Between vPC Peers ............................................................................................................................................... 51 IGMP Snooping and vPC ................................................................................................................................................................. 53 Multicast Traffic Forwarding with vPC ...................................................................................................................................... 54 LACP............................................................................................................................................................................................................ 55 Spanning Tree Protocol (STP)...................................................................................................................................................... 57 Rules of Operation ............................................................................................................................................................................... 58 Spanning Tree Protocol Failure ................................................................................................................................................... 59 Exam Essentials.................................................................................................................................................................................... 60

CHAPTER 2: DESIGN, IMPLEMENT, AND TROUBLESHOOT OVERLAYS .............61 VXLAN ........................................................................................................................................................................................................ 62 VXLAN Encapsulation and Packet Format ............................................................................................................................ 62 VXLAN Tunnel Endpoint .................................................................................................................................................................. 63 VXLAN Packet Forwarding Flow ................................................................................................................................................. 64 Configuring a Layer 2 VXLAN Gateway on a Cisco Nexus 5600 ............................................................................. 66 Cisco Nexus 5600 Deployment Scenarios ............................................................................................................................ 67 Cisco Nexus 5600 as a Layer 3 VXLAN Gateway ............................................................................................................ 69 Hypervisor-Originated VXLAN ...................................................................................................................................................... 69 Underlay Requirements for VXLAN ........................................................................................................................................... 70


7

VXLAN Designs ..................................................................................................................................................................................... 70 Centralized Gateway with Inter-VXLAN Routing in the Core/Aggregation.......................................................... 70 Single and Dual Attached Physical Machines to Hardware VTEPs ........................................................................ 71 Single and Dual Attached Virtual Machines to Hardware VTEPs ............................................................................ 72 Single and Dual Attached Physical Machines to FEX ..................................................................................................... 72 Single and Dual Attached Virtual Machines to FEX ......................................................................................................... 73 Single and Dual Attached (Distributed Gateway) Hypervisor-Originated VXLAN to the Top of Rack (ToR) ............................................................................................................................................................................................................ 73 Single Attached Hypervisor-Originated VXLAN to FEX ................................................................................................. 74 Hypervisor-Originated VXLAN with no VTEPs on Physical Switches .................................................................... 75 Bud-Node VTEP.................................................................................................................................................................................... 76 VXLAN and Virtual PortChannel (vPC).................................................................................................................................... 77 EVPN ........................................................................................................................................................................................................... 80 MP-BGP EVPN Control Plane Overview ................................................................................................................................ 80 Software and Hardware Support for the MP-BGP EVPN Control Plane .............................................................. 81 IP Transport Devices Running MP-BGP EVPN .................................................................................................................. 81 VTEPs Running MP-BGP EVPN ................................................................................................................................................. 82 Inter-VXLAN Routing .......................................................................................................................................................................... 82 Multitenancy in MP-BGP EVPN ................................................................................................................................................... 82 MP-BGP EVPN NLRI and L2VPN EVPN Address Family ........................................................................................... 83 Integrated Routing and Bridging with the MP-BGP EVPN Control Plane............................................................ 85 Local-Host Learning ............................................................................................................................................................................ 86 EVPN Route Advertisement and Remote-Host Learning ............................................................................................. 87 MP-BGP EVPN VTEP Configuration ........................................................................................................................................ 88 OTV .............................................................................................................................................................................................................. 93 Overlay Interfaces ................................................................................................................................................................................ 94 MAC Address Learning ..................................................................................................................................................................... 94 Multicast Group Addresses and IGMP Snooping .............................................................................................................. 95 High Availability and ISSU .............................................................................................................................................................. 95 Virtualization Support ......................................................................................................................................................................... 96 Guidelines and Limitations for OTV ........................................................................................................................................... 96 Exam Essentials.................................................................................................................................................................................... 98

CHAPTER 3: DESIGN, IMPLEMENT, AND TROUBLESHOOT ROUTING PROTOCOLS AND FEATURES .................................................................................................99

OSPF ........................................................................................................................................................................................................ 100 LSA types (1, 2, 3, 4, 5, 7, 9) ...................................................................................................................................................... 100 Table below summarizes various LSA types and their description ...................................................................... 100 Table below, shows various OSPF network types and traffic that are allowed ............................................. 101 Route types (N1, N2, E1, E2) ..................................................................................................................................................... 101 Implement and troubleshoot neighbor relationship ........................................................................................................ 102 Enabling OSPFv2 .............................................................................................................................................................................. 107 Configuration Steps .......................................................................................................................................................................... 107 Describe basic ISIS network ....................................................................................................................................................... 108 Enabling IS-IS...................................................................................................................................................................................... 110 Configuration Steps .......................................................................................................................................................................... 110 Creating an IS-IS Instance ........................................................................................................................................................... 110 Configuration Steps .......................................................................................................................................................................... 110 BGP Autonomous Systems ......................................................................................................................................................... 111 4-Byte AS Number Support ......................................................................................................................................................... 111 Administrative Distance ................................................................................................................................................................. 111 BGP Peers............................................................................................................................................................................................. 112 BGP Sessions ..................................................................................................................................................................................... 112 BGP Router Identifier ...................................................................................................................................................................... 113 BGP Path Selection ......................................................................................................................................................................... 113


8

BGP and the Unicast RIB ............................................................................................................................................................. 114 BGP Prefix Independent Convergence Core .................................................................................................................... 114 BGP Virtualization ............................................................................................................................................................................. 114 Multiprotocol BGP ............................................................................................................................................................................. 115 Enabling BGP ...................................................................................................................................................................................... 115 Configuration Steps .......................................................................................................................................................................... 115 Creating a BGP Instance .............................................................................................................................................................. 116 Configuration Steps .......................................................................................................................................................................... 116 Policy-Based Routing (PBR) ....................................................................................................................................................... 116 Policy Route Maps ............................................................................................................................................................................ 117 BFD............................................................................................................................................................................................................ 117 Asynchronous Mode ........................................................................................................................................................................ 118 BFD Detection of Failures ............................................................................................................................................................ 118 BFD Echo Function .......................................................................................................................................................................... 119 Enabling the BFD Feature ............................................................................................................................................................ 119 Configuring Global BFD Parameters...................................................................................................................................... 119 Further Reading ................................................................................................................................................................................. 120 FHRP ........................................................................................................................................................................................................ 120 VRRP Operation ................................................................................................................................................................................ 121 VRRP Router Priority and Preemption .................................................................................................................................. 122 vPC and VRRP ................................................................................................................................................................................... 123 vPC and HSRP ................................................................................................................................................................................... 124 GLBP ........................................................................................................................................................................................................ 125 Virtualization Support ...................................................................................................................................................................... 126 Exam Essentials................................................................................................................................................................................. 126

CHAPTER 4: DESIGN, IMPLEMENT, AND TROUBLESHOOT MULTICAST PROTOCOLS .................................................................................................................................. 127

PIM ............................................................................................................................................................................................................. 128 PIM Dense Mode (PIM-DM) ........................................................................................................................................................ 128 PIM Sparse Mode (PIM-SM) ....................................................................................................................................................... 129 Bidirectional PIM (Bidir-PIM) ....................................................................................................................................................... 129 IGMP ......................................................................................................................................................................................................... 130 Table below shows IGMPv2 intervals and their default values............................................................................... 132 Further Reading ................................................................................................................................................................................. 133 IGMP Snooping .................................................................................................................................................................................. 134 IGMP Querier ....................................................................................................................................................................................... 134 Further Reading ................................................................................................................................................................................. 135 IGMP Filter ............................................................................................................................................................................................ 135 Further Reading ................................................................................................................................................................................. 135 IGMP proxy ........................................................................................................................................................................................... 136 Further Reading ................................................................................................................................................................................. 136 Rendezvous Point (RP) ................................................................................................................................................................. 136 BSR ........................................................................................................................................................................................................... 137 Protocol Independent Multicast (PIM) and vPC ............................................................................................................... 138 Exam Essentials................................................................................................................................................................................. 139

CHAPTER 5: DESCRIBE INTER-FABRIC CONNECTIVITY.......................................... 140 Multipod ................................................................................................................................................................................................... 141 Multisite ................................................................................................................................................................................................... 141 Cisco ACI Multi-Site policy manager ...................................................................................................................................... 142 Exam Essentials................................................................................................................................................................................. 143

CHAPTER 6: DESIGN, IMPLEMENT, AND TROUBLESHOOT EXTERNAL FABRIC CONNECTIVITY ............................................................................................................................. 144


9

Configuring a BGP Layer 3 Outside Network Connection ......................................................................................... 145 Configuring BGP External Routed Network Using the NX-OS Style CLI .......................................................... 148 Configuring a Tenant Layer 3 Outside Network Connection .................................................................................... 148 Shared Services Contracts Usage .......................................................................................................................................... 153 Shared Layer 3 Out .......................................................................................................................................................................... 154 ACI Transit Routing .......................................................................................................................................................................... 155 Transit Routing Overview ............................................................................................................................................................. 155 Route Distribution Within the ACI Fabric ............................................................................................................................. 157 External Layer 3 Outside Connection Types ..................................................................................................................... 157 Further Reading ................................................................................................................................................................................. 158 ACI Virtual Port Channel Workflow ......................................................................................................................................... 158 Configure the Virtual Port Channel ......................................................................................................................................... 159 Configure the Application Profile .............................................................................................................................................. 159 Networking Domains ....................................................................................................................................................................... 159 Attachable Entity Profile ................................................................................................................................................................ 160 Exam Essentials................................................................................................................................................................................. 161

CHAPTER 7: DESIGN, IMPLEMENT, AND TROUBLESHOOT TRAFFIC MANAGEMENT .............................................................................................................................. 163

Configuring Queuing and Scheduling .................................................................................................................................... 165 Configuring Color-Aware Policing ............................................................................................................................................ 166 RDMA over Converged Ethernet (RoCE)............................................................................................................................ 166 Exam Essentials................................................................................................................................................................................. 167

PART 2 CISCO DATA CENTER NETWORK SERVICES ................................................... 168

CHAPTER 8: DESIGN, IMPLEMENT, AND TROUBLESHOOT NETWORK SERVICES INSERTION AND REDIRECTION .................................................................... 169

Policy-based Routing (PBR) ....................................................................................................................................................... 170 Policy Route Maps ............................................................................................................................................................................ 170 Enabling the Policy-Based Routing Feature ...................................................................................................................... 171 Configuring a Route Policy .......................................................................................................................................................... 171 Policy-based Redirection (PBR) ............................................................................................................................................... 172 VRF Stitching ....................................................................................................................................................................................... 173 Exam Essentials................................................................................................................................................................................. 175

CHAPTER 9: DESIGN, IMPLEMENT, AND TROUBLESHOOT SERVICES ............ 176 PTP Overview...................................................................................................................................................................................... 177 PTP Device Types ............................................................................................................................................................................ 177 Configuration Steps .......................................................................................................................................................................... 178 Name Servers...................................................................................................................................................................................... 180 DHCP Relay Agent ........................................................................................................................................................................... 182 Exam Essentials................................................................................................................................................................................. 182

CHAPTER 10: DESIGN, IMPLEMENT, AND TROUBLESHOOT RBAC ................... 183 RADIUS ................................................................................................................................................................................................... 184 TACACS+ .............................................................................................................................................................................................. 185 LDAP......................................................................................................................................................................................................... 185 LDAP Authentication and Authorization ............................................................................................................................... 186 Exam Essentials................................................................................................................................................................................. 186

CHAPTER 11: DESIGN, IMPLEMENT, AND TROUBLESHOOT MAINTENANCE TASKS............................................................................................................................................... 187

Backup and restore .......................................................................................................................................................................... 188 Firmware upgrades and downgrades .................................................................................................................................... 188


10

CHAPTER 12: DESIGN, IMPLEMENT, AND TROUBLESHOOT MONITORING SERVICES ....................................................................................................................................... 190

Netflow v5, v9 ...................................................................................................................................................................................... 191 Table below describes various NetFlow versions and their description ............................................................ 191 Further Reading ................................................................................................................................................................................. 192 Nx-OS Features ................................................................................................................................................................................. 192 Configuration Guidelines ............................................................................................................................................................... 193 Configuration Examples ................................................................................................................................................................ 193 SPAN ........................................................................................................................................................................................................ 195 SPAN Destinations ........................................................................................................................................................................... 196 SNMP ....................................................................................................................................................................................................... 196 Syslog....................................................................................................................................................................................................... 197 Exam Essentials................................................................................................................................................................................. 198

CHAPTER 13: DESIGN, IMPLEMENT, AND TROUBLESHOOT SECURITY FEATURES ...................................................................................................................................... 199

CoPP......................................................................................................................................................................................................... 200 ACLs ......................................................................................................................................................................................................... 201 ACL Types and Applications ....................................................................................................................................................... 201 Dynamic ARP Inspection (DAI) ................................................................................................................................................. 201 Storm control ........................................................................................................................................................................................ 202 First-hop security ............................................................................................................................................................................... 204 VRRP Authentication....................................................................................................................................................................... 204 HSRP Authentication....................................................................................................................................................................... 204 GLBP Authentication ....................................................................................................................................................................... 205 Contracts ................................................................................................................................................................................................ 205 Port Security and Port Types ...................................................................................................................................................... 206 Virtualization Support ...................................................................................................................................................................... 206 Configuration Steps .......................................................................................................................................................................... 207 Cisco TrustSec and Authentication ......................................................................................................................................... 207 SGACLs and SGTs .......................................................................................................................................................................... 208 Exam Essentials................................................................................................................................................................................. 212

PART 3 DATA CENTER STORAGE NETWORKING AND COMPUTE ......................... 213

CHAPTER 14: DESCRIBE, CONFIGURE, AND TROUBLESHOOT INFRASTRUCTURE TO SUPPORT BLOCK STORAGE PROTOCOLS ................... 214

Describe, Configure and Troubleshoot infrastructure to support Block Storage Protocols for example FC, FCoE, iSCSI, DCB .................................................................................................................................................................. 215 Describe SCSI standards and protocols .............................................................................................................................. 217 Describe iSCSI standards and protocols ............................................................................................................................. 218 SCSI initiator ........................................................................................................................................................................................ 218 SCSI target ............................................................................................................................................................................................ 219 SCSI Flow Statistics......................................................................................................................................................................... 220 Describe iSCSI and its features ................................................................................................................................................ 221 Initiator Presentation Modes ....................................................................................................................................................... 221 Transparent Initiator Mode ........................................................................................................................................................... 222 Proxy Initiator Mode ......................................................................................................................................................................... 222 iSCSI Routing Modes ...................................................................................................................................................................... 222 iSCSI Immediate Data and Unsolicited Data Features ............................................................................................... 224 iSCSI Interface Advanced Features ....................................................................................................................................... 224 Parallel SCSI ........................................................................................................................................................................................ 224 SCSI-1 ..................................................................................................................................................................................................... 225 SCSI-2 ..................................................................................................................................................................................................... 225 Fibre Channel over Ethernet Overview................................................................................................................................. 225


11

Design and implement data center bridging protocol and lossless Ethernet .................................................. 226 Lossless Ethernet .............................................................................................................................................................................. 226 Logical Link Up/Down ..................................................................................................................................................................... 226 Converged Network Adapters (CNAs) .................................................................................................................................. 227 FCoE Topologies ............................................................................................................................................................................... 228 Directly Connected CNA Topology ......................................................................................................................................... 228 Remotely Connected CNA Topology ..................................................................................................................................... 228 Configuring FCoE .............................................................................................................................................................................. 229 Enabling FCoE .................................................................................................................................................................................... 229 Configuration Steps .......................................................................................................................................................................... 229 Disabling LAN Traffic on an FCoE Link ................................................................................................................................ 229 Configuration Steps .......................................................................................................................................................................... 230 Configuring the FC-Map ................................................................................................................................................................ 230 Configuration Steps .......................................................................................................................................................................... 230 Configure and Troubleshoot infrastructure to support File Storage Protocols for example NFS, CIFS...................................................................................................................................................................................................................... 230 Further Reading ................................................................................................................................................................................. 233 Exam Essentials................................................................................................................................................................................. 234

CHAPTER 15: DESIGN, IMPLEMENT, AND TROUBLESHOOT DATA CENTER STORAGE NETWORKING FEATURES................................................................................ 235

Design basic and enhanced zoning ........................................................................................................................................ 236 Describe multihop FCoE ............................................................................................................................................................... 237 Design Consideration for Multihop FCoE with Edge-Core-Edge Design .......................................................... 238 Access Layer and SAN Edge Fabric ...................................................................................................................................... 238 Aggregation Layer and SAN Core Fabric ............................................................................................................................ 239 Storage Edge Fabric ........................................................................................................................................................................ 239 Further Reading ................................................................................................................................................................................. 239 Exam Essentials................................................................................................................................................................................. 239 Exam Essentials................................................................................................................................................................................. 240

CHAPTER 16: DESIGN, IMPLEMENT, AND TROUBLESHOOT COMPUTE POLICIES AND PROFILES ....................................................................................................... 241

Cisco UCS Manager ........................................................................................................................................................................ 242 Faults ........................................................................................................................................................................................................ 242 Events ...................................................................................................................................................................................................... 243 Audit Log ................................................................................................................................................................................................ 243 System Event Log ............................................................................................................................................................................. 243 Syslog....................................................................................................................................................................................................... 244 Technical Support Files ................................................................................................................................................................. 244 Cisco Intersight ................................................................................................................................................................................... 245 Management as a service ............................................................................................................................................................ 246 Telemetry data collection .............................................................................................................................................................. 246 Customizable dashboard .............................................................................................................................................................. 247 Simplified support .............................................................................................................................................................................. 248 Platform compliance ........................................................................................................................................................................ 248 Exam Essentials................................................................................................................................................................................. 248

CHAPTER 17: DESIGN, IMPLEMENT, AND TROUBLESHOOT DATA CENTER CONNECTIVITY ............................................................................................................................. 249

SAN/LAN uplinks ............................................................................................................................................................................... 250 Direct Connect Mode....................................................................................................................................................................... 251 Management Connection Policy and Connection Mode ............................................................................................. 252 Why Appliance Port VLANs Should be Allowed on Uplinks ..................................................................................... 253 Definition of a Unified Storage Port......................................................................................................................................... 253


12

Appliance Port Port-Channel ...................................................................................................................................................... 253 When to Use Trunk or Access Mode ..................................................................................................................................... 254 Situations to Avoid ............................................................................................................................................................................ 254 Appliance Port Failover .................................................................................................................................................................. 254 Network Uplink Failure ................................................................................................................................................................... 255 Appliance Port Troubleshooting ................................................................................................................................................ 255 Further Reading ................................................................................................................................................................................. 257 Exam Essentials................................................................................................................................................................................. 257

PART 4 DATA CENTER AUTOMATION AND ORCHESTRATION ................................. 258

CHAPTER 18: IMPLEMENT AND TROUBLESHOOT DATA CENTER TASKS USING PROVIDED PYTHON SCRIPTS ................................................................................ 259

Infrastructure Provisioning and Operations ........................................................................................................................ 261 Development and Operations Models and Continuous Integration...................................................................... 262 Monitoring and Advanced Analytics ....................................................................................................................................... 263 Standard Network Manageability Features ........................................................................................................................ 264 Advanced Automation Features................................................................................................................................................ 264 Extensible Messaging and Presence Protocol Support .............................................................................................. 264 Puppet and Chef Integration ....................................................................................................................................................... 265 OpenStack Integration .................................................................................................................................................................... 266 OpenDayLight Integration and OpenFlow Support ........................................................................................................ 266 Comprehensive Programmability Support .......................................................................................................................... 267 Cisco NX-API Support .................................................................................................................................................................... 267 Python Scripting ................................................................................................................................................................................. 268 Cisco onePK ......................................................................................................................................................................................... 270 Further Reading ................................................................................................................................................................................. 270 Exam Essentials................................................................................................................................................................................. 271

CHAPTER 19: DESCRIBE AND DESIGN DATA CENTER ORCHESTRATION USING TOOLS ............................................................................................................................... 272

Cisco UCS Director .......................................................................................................................................................................... 273 Cisco UCS Director Orchestrator ............................................................................................................................................. 273 Tasks ........................................................................................................................................................................................................ 274 Workflows............................................................................................................................................................................................... 274 Service Requests .............................................................................................................................................................................. 275 Approvals ............................................................................................................................................................................................... 275 Rollback .................................................................................................................................................................................................. 275 Further Reading ................................................................................................................................................................................. 276 Exam Essentials................................................................................................................................................................................. 277

PART 5 DATA CENTER FABRIC INFRASTRUCTURE ...................................................... 278

CHAPTER 20: CONFIGURE AND TROUBLESHOOT PHYSICAL FABRIC COMPONENTS .............................................................................................................................. 279

Fabric discovery ................................................................................................................................................................................. 280 Controllers/network managers ................................................................................................................................................... 281 Switches.................................................................................................................................................................................................. 283 Expansion Modules .......................................................................................................................................................................... 283 Leaf Switches ...................................................................................................................................................................................... 284 Spine Switches ................................................................................................................................................................................... 285 Exam Essentials................................................................................................................................................................................. 285

CHAPTER 21: DESIGN, IMPLEMENT, AND TROUBLESHOOT FABRIC POLICIES ............................................................................................................................................................. 286

Setting up the APIC .......................................................................................................................................................................... 287


13

Accessing the CLI ............................................................................................................................................................................. 290 Switch Discovery with the APIC ................................................................................................................................................ 290 Switch Discovery Validation and Switch Management from the APIC ............................................................... 291 Validating the Fabric Topology .................................................................................................................................................. 291 Unmanaged Switch Connectivity in VM Management ................................................................................................. 291 Registering the Unregistered Switches Using the CLI ................................................................................................. 292 Creating User Accounts ................................................................................................................................................................. 292 Configuring a Local User............................................................................................................................................................... 292 Configuring a Remote User ......................................................................................................................................................... 293 Adding Management Access ...................................................................................................................................................... 293 Management Connectivity Modes ........................................................................................................................................... 294 Attach Entity Profiles ....................................................................................................................................................................... 294 Deploying an Application Policy ................................................................................................................................................ 295 Three-Tier Application Deployment ........................................................................................................................................ 295 Further Reading ................................................................................................................................................................................. 296 Pre-Provision Interface Policy Groups .................................................................................................................................. 297 The Cisco ACI Policy Management Information Model............................................................................................... 299 Tenants ................................................................................................................................................................................................... 302 Contexts .................................................................................................................................................................................................. 302 Application Profiles ........................................................................................................................................................................... 303 Endpoint Groups ................................................................................................................................................................................ 303 Microsegmentation ........................................................................................................................................................................... 304 Intra-EPG Endpoint Isolation ...................................................................................................................................................... 304 Contracts ................................................................................................................................................................................................ 305 Labels, Filters, and Subjects Govern EPG Communications .................................................................................. 305 What vzAny Is...................................................................................................................................................................................... 305 Default Policies ................................................................................................................................................................................... 306 Further Reading ................................................................................................................................................................................. 306 Exam Essentials................................................................................................................................................................................. 306

CHAPTER 22: DESIGN, IMPLEMENT, AND TROUBLESHOOT TENANT POLICIES ............................................................................................................................................................. 308

Policy Identification and Enforcement ................................................................................................................................... 309 Encapsulation Normalization ...................................................................................................................................................... 310 Extension to Virtualized Servers and VLAN and VxLAN Normalization ............................................................ 311 Routed Design with VxLAN Overlays .................................................................................................................................... 312 Load Balancing ................................................................................................................................................................................... 312 Configuring Static In-Band Management Access Using the NX-OS Style CLI .............................................. 314 Further Reading ................................................................................................................................................................................. 315 Exam Essentials................................................................................................................................................................................. 315

CHAPTER 23: ANALYZE AND TROUBLESHOOT LOGICAL FABRIC ELEMENTS ............................................................................................................................................................. 316

Viewing a Health Score Using the NX-OS-Style CLI .................................................................................................... 319 Viewing a Health Score Using the iShell ............................................................................................................................. 319 Understanding Faults ...................................................................................................................................................................... 320 How Health Scores Are Calculated......................................................................................................................................... 320 Using Health Scores for Proactive Monitoring .................................................................................................................. 321 Using Health Scores for Reactive Monitoring ................................................................................................................... 321 Further Reading ................................................................................................................................................................................. 322 Exam Essentials................................................................................................................................................................................. 322

CHAPTER 24: DESIGN, IMPLEMENT, AND TROUBLESHOOT VIRTUAL NETWORKING ............................................................................................................................... 323

Cisco AVE .............................................................................................................................................................................................. 324


14

vSphere Distributed Switch.......................................................................................................................................................... 324 Hyper-V switch .................................................................................................................................................................................... 325 Uses for Hyper-V Virtual Switch................................................................................................................................................ 326 Exam Essentials................................................................................................................................................................................. 327

PART 6 EVOLVING TECHNOLOGIES ...................................................................................... 328

CHAPTER 25: CLOUD ................................................................................................................ 329 Compare and Contrast Public, Private, Hybrid, and Multi-cloud Design Considerations ........................ 330 Public Cloud ......................................................................................................................................................................................... 331 Private Cloud........................................................................................................................................................................................ 332 Virtual Private Cloud (VPC) ......................................................................................................................................................... 333 Hybrid Cloud ......................................................................................................................................................................................... 333 Multi-cloud ............................................................................................................................................................................................. 334 Infrastructure as a service (IaaS) ............................................................................................................................................. 336 Platform as a service (PaaS) ...................................................................................................................................................... 337 Software as a Service (SaaS) .................................................................................................................................................... 337 Consolidation ....................................................................................................................................................................................... 339 Virtualization ......................................................................................................................................................................................... 339 Automation ............................................................................................................................................................................................ 339 Performance, Scalability, and High Availability ................................................................................................................ 340 Performance ......................................................................................................................................................................................... 340 Scalability and High Availability ................................................................................................................................................. 342 Security Implications, Compliance, and Policy ................................................................................................................. 343 Workload Migration .......................................................................................................................................................................... 345 Describe Cloud Infrastructure and Operations ................................................................................................................. 347 Compute Virtualization (Containers and Virtual Machines) ...................................................................................... 347 Installing Docker................................................................................................................................................................................. 350 Using Docker Commands............................................................................................................................................................. 352 Connectivity (Virtual Switches, SD-WAN and SD-Access) ....................................................................................... 352 Virtual Switches .................................................................................................................................................................................. 354 Virtual Machine Device Queues (VMDq) ............................................................................................................................. 356 Single Root IO Virtualization (SR-IOV) ................................................................................................................................. 356 SD-WAN and SD-Access ............................................................................................................................................................. 357 Cisco SD-WAN Solution (formerly Viptela) ........................................................................................................................ 357 Software-Defined Access (or SD-Access) .......................................................................................................................... 362 Virtualization Functions (NFVI, VNF, and L4/L1) ............................................................................................................ 365 NFVI and VNFs................................................................................................................................................................................... 365 Virtual Topology Forwarder (VTF) ........................................................................................................................................... 367 Automation and Orchestration Tools (CloudCenter, DNA-center, and Kubernetes) .................................. 369 Cisco CloudCenter Manager and Orchestrator................................................................................................................ 369 Cisco DNA Center ............................................................................................................................................................................. 371 Kubernetes ............................................................................................................................................................................................ 373 Exam Essentials................................................................................................................................................................................. 386 Further Reading ................................................................................................................................................................................. 388

CHAPTER 26: NETWORK PROGRAMMABILITY ............................................................ 389 Describe Architectural and Operational Considerations for a Programmable Network ............................ 390 Data Models and Structures (YANG, JSON and XML) ............................................................................................... 390 YANG ....................................................................................................................................................................................................... 391 YAML ........................................................................................................................................................................................................ 395 JSON ........................................................................................................................................................................................................ 396 JSON Example ................................................................................................................................................................................... 396 XML ........................................................................................................................................................................................................... 398 XML Example ...................................................................................................................................................................................... 398 Device Programmability (gRPC, NETCONF and RESTCONF) ............................................................................. 399


15

gRPC ........................................................................................................................................................................................................ 399 NETCONF ............................................................................................................................................................................................. 402 NETCONF Example......................................................................................................................................................................... 403 RESTCONF .......................................................................................................................................................................................... 404 Using RESTCONF to Retrieve Full Running Configuration ...................................................................................... 404 Using RESTCONF to Retrieve Interface Specific Attributes .................................................................................... 405 Controller Based Network Design (Policy Driven Configuration and Northbound/Southbound APIs)...................................................................................................................................................................................................................... 405 Policy-driven Configuration .......................................................................................................................................................... 405 Northbound and Southbound APIs ......................................................................................................................................... 406 Configuration Management Tools (Agent and Agentless) and Version Control Systems (Git and SVN).......................................................................................................................................................................................................... 408 Configuration Management Tools (Agent and Agentless) ......................................................................................... 408 Version Control Systems (Git and SVN) .............................................................................................................................. 411 Creating Configuration Change ................................................................................................................................................. 414 Building New Configuration ......................................................................................................................................................... 414 Testing New Configuration ........................................................................................................................................................... 415 Deploying New Configuration ..................................................................................................................................................... 415 Exam Essentials................................................................................................................................................................................. 422 Further Reading ................................................................................................................................................................................. 423

CHAPTER 27: INTERNET OF THINGS ................................................................................. 425 Describe Architectural Framework and Deployment Considerations for Internet of Things (IoT) ...... 426 IoT Technology Stack (IoT Network Hierarchy, Data Acquisition and Flow) .................................................. 430 Embedded Systems Layer ........................................................................................................................................................... 431 Multi-Service Edge (or Access) Layer ................................................................................................................................... 431 Core Network Layer ......................................................................................................................................................................... 432 Data Center Cloud Layer .............................................................................................................................................................. 432 Data Acquisition and Flow ............................................................................................................................................................ 433 IoT Standards and Protocols (characteristics within IT and OT environment)............................................... 434 IoT Security (network segmentation, device profiling, and secure remote access) .................................... 437 IoT Edge and Fog Computing (data aggregation and edge intelligence) ......................................................... 438 Data Aggregation ............................................................................................................................................................................... 439 Edge Intelligence ............................................................................................................................................................................... 440 Exam Essentials................................................................................................................................................................................. 441 Further Reading ................................................................................................................................................................................. 443


16

PART 1 CISCO DATA CENTER L2/L3 TECHNOLOGIES

Chapter 1: Design, implement, and troubleshoot Layer 2 technologies

Chapter 2: Design, implement, and troubleshoot overlays

Chapter 3: Design, implement, and troubleshoot routing protocols and features

Chapter 4: Design, implement, and troubleshoot multicast protocols

Chapter 5: Describe inter-fabric connectivity

Chapter 6: Design, implement, and troubleshoot external fabric connectivity

Chapter 7: Design, implement, and troubleshoot traffic management


17

CHAPTER 1: DESIGN, IMPLEMENT, AND TROUBLESHOOT LAYER 2

TECHNOLOGIES

This chapter covers the following exam topics from Cisco’s official 400-151 V2.1 written exam

blueprint.

● Link aggregation

● Tagging and Trunking

● Spanning Tree Protocol (STP)

https://www.cciein8weeks.com/cisco-ccie-data-center-400-151-written-and-lab-exam-v2-1/


18

Link Aggregation

Link aggregation allows multiple ports to merge logically in the form of a single link. Link

aggregation offers higher aggregate bandwidth to traffic-heavy servers and reroute

capability in case of a single port or cable failure. Link aggregation provides increased

bandwidth and redundancy. It is sometimes also known as port trunking or link bonding.

Link Aggregation Control Protocol (LACP) is part of an IEEE specification (802.3ad) that

allows you to bundle several physical ports together to form a single logical channel.

LACP allows a switch to negotiate an automatic bundle by sending LACP packets to the

peer. It performs a similar function as Port Aggregation Protocol (PAgP) with Cisco

EtherChannel. LACP allows a network device to negotiate an automatic bundling of

links by sending LACP packets to the peer (directly connected device that also

implements LACP).

vPC is a virtualization technology that presents both Cisco Nexus 7000 Series paired

devices as a unique Layer 2 logical node to access layer devices or endpoints. vPC

belongs to Multichassis EtherChannel [MCEC] family of technology.

A virtual port channel (vPC) allows links that are physically connected to two different

Cisco Nexus 7000 Series devices to appear as a single port channel to a third device.

The third device can be a switch, server, or any other networking device that supports

link aggregation technology.

• vPC provides the following technical benefits:

• Eliminates Spanning Tree Protocol (STP) blocked ports

• Uses all available uplink bandwidth

• Allows dual-homed servers to operate in active-active mode


19

• Provides fast convergence upon link or device failure

• Offers dual active/active default gateways for servers

vPC also leverages native split horizon/loop management provided by port-channeling

technology: a packet entering a port-channel cannot immediately exit that same port-

channel.

By using vPC, users get the immediate operational and architectural advantages:

• Simplifies network design

• Build highly resilient and robust Layer 2 network

• Enables seamless virtual machine mobility and server high-availability clusters

• Scales available Layer 2 bandwidth, increasing bisectional bandwidth

• Grows the size of the Layer 2 network

(Source: Cisco.com)


20

• vPC leverages both hardware and software redundancy aspects:

• vPC uses all port-channel member links available so that in case an individual

link fails, hashing algorithm will redirect all flows to the remaining links.

• vPC domain is composed of two peer devices. Each peer device processes half

of the traffic coming from the access layer. In case a peer device fails, the other

peer device will absorb all the traffic with minimal convergence time impact.

• Each peer device in the vPC domain runs its own control plane, and both devices

work independently. Any potential control plane issues stay local to the peer

device and does not propagate or impact the other peer device.

From a Spanning-Tree standpoint, vPC eliminates STP blocked ports and uses all

available uplink bandwidth. Spanning-Tree is used as a fail-safe mechanism and does

not dictate L2 path for vPC-attached devices.

Within a vPC domain, user can connect access devices in multiple ways: vPC-attached

connections leveraging active/active behavior with port-channel, active/standby

connectivity using spanning-tree, single attachment without spanning-tree running on

the access device.

NX-OS Version Requirement for vPC

vPC technology is supported since NX-OS 4.1.3. (i.e. since the inception of Nexus 7000

platform).

NX-OS appropriate version depends on line cards configuration (M1, F1 or F2), chassis

type (7010, 7018 or 7009) and Fabric Module generation (FM generation 1 [46 Gbps

per module] or generation 2 [110 Gbps per module]).


21

NX-OS License Requirement for vPC

vPC feature is included in the base NX-OS software license.

● Hot Standby Router Protocol (HSRP), Virtual Router Redundancy Protocol

(VRRP), Link Aggregation Control Protocol (LACP) are also included in this base

license.

● Layer 3 features like Open Shortest Path First (OSPF) protocol or Intermediate-

System-to-Intermediate System (IS-IS) protocol require

LAN_ENTERPRISE_SERVICES_PKG license.

● Virtual device contexts (VDCs) requires LAN_ADVANCED_SERVICES_PKG

license.

Components of vPC

Table below lists important terms you need to know to understand vPC technology.

These terms are used throughout this guide.

Term Description

vPC

The combined port-channel between the vPC peers and the

downstream device.

A vPC is a L2 port type: switchport mode trunk or switchport

mode access

vPC peer device A vPC switch (one of a Cisco Nexus 7000 Series pair).

vPC

domain Domain containing the 2 peer devices.


22

Only 2 peer devices max can be part of same vPC domain.

vPC member port

One of a set of ports (that is, port-channels) that form a vPC

(or port-channel member of a vPC).

vPC peer-link

Link used to synchronize the state between vPC peer

devices. It must be a 10-Gigabit Ethernet link.

vPC peer-link is a L2 trunk carrying vPC VLAN.

vPC peer-keepalive

link

The keepalive link between vPC peer devices; this link is

used to monitor the liveness of the peer device.

vPC

VLAN

VLAN carried over the vPC peer-link and used to

communicate via vPC with a third device.

As soon as a VLAN is defined on vPC peer-link, it becomes

a vPC VLAN

non-vPC VLAN

A VLAN that is not part of any vPC and not present on vPC

peer-link.

Orphan

port A port that belong to a single attached device.

vPC VLAN is typically used on this port.


23

Cisco Fabric

Services (CFS)

Underlying protocol running on top of vPC peer-link providing

reliable synchronization and consistency

protoco

l check mechanisms between the 2 peer devices.

Virtual PortChannels (vPCs) allow links that are physically connected to two different

Cisco® switches to appear to a third downstream device to be coming from a single

device and as part of a single PortChannel. The third device can be a switch, a server,

or any other networking device that supports IEEE 802.3ad PortChannels. vPC allows

the creation of Layer 2 PortChannels that span two switches. At the time of this writing,

vPC is implemented on the Cisco Nexus 7000 and 5000 Series platforms (with or

without Cisco Nexus 2000 Series Fabric Extenders).

vPCs consist of two vPC peer switches connected by a peer link. Of the vPC peers, one

is primary and one is secondary. The system formed by the switches is referred to as a

vPC domain.


24

(Source: Cisco.com)


25

(Source: Cisco.com)

Following is a list of some possible Cisco Nexus vPC topologies, refer to diagrams

above:

• vPC on the Cisco Nexus 7000 Series (topology A): This topology consists of

access layer switches dual-homed to the Cisco Nexus 7000 Series with a switch

PortChannel with Gigabit Ethernet or 10 Gigabit Ethernet links. This topology can

also consist of hosts connected with virtual PortChannels to each Cisco Nexus

7000 Series Switch.

• vPC on Cisco Nexus 5000 Series (topology B): This topology consists of

switches dual-connected to the Cisco Nexus 5000 Series with a switch

PortChannel with 10 Gigabit Ethernet links, with one or more links to each Cisco

Nexus 5000 Series Switch. Like topology A, topology B can consist of servers

connected to each Cisco Nexus 5000 Series Switch via virtual PortChannels.

• vPC on the Cisco Nexus 5000 Series with a Cisco Nexus 2000 Series Fabric

Extender single-homed (also called straight-through mode) (topology C): This

topology consists of a Cisco Nexus 2000 Series Fabric Extender single-homed

with one to eight 10 Gigabit Ethernet links (depending on the fabric extender

model) to a single Cisco Nexus 5000 Series Switch, and of Gigabit Ethernet or

10 Gigabit Ethernet-connected servers that form virtual PortChannels to the

fabric extender devices. Note that each fabric extender connects to a single

Cisco Nexus 5000 Series Switch and not to both, and that the virtual

PortChannel can be formed only by connecting the server network interface

cards (NICs) to two fabric extenders, where fabric extender 1 depends on Cisco

Nexus 5000 Series Switch 1 and fabric extender 2 depends on Cisco Nexus

5000 Series Switch 2. If both fabric extender 1 and fabric extender 2 depend on

switch 1 or both of them depend on switch2, the PortChannel cannot be formed.

• Dual-homing of the Cisco Nexus 2000 Series Fabric Extender (topology D): This

topology is also called Cisco Nexus 2000 Series Fabric Extender (FEX for brief)

Active/Active. In this topology each FEX is connected to each Cisco Nexus 5000


26

Series device with a virtual PortChannel. With this topology, the server cannot

create a PortChannel split between two fabric extenders. The servers can still be

dual-homed with active-standby or active-active transmit-load-balancing (TLB)

teaming.

The vPC peer switches are connected through a link called a peer link, also known as a

multi-chassis EtherChannel trunk (MCT).

vPC Peer Link

The vPC peer link is the most important connectivity element in the vPC system. This

link is used to create the illusion of a single control plane by forwarding Bridge Protocol

data units (BPDUs) or Link Aggregation Control Protocol (LACP) packets to the primary

vPC switch from the secondary vPC switch. The peer link is used to synchronize MAC

addresses between aggregation groups 1 and 2, to synchronize IGMP entries for the

purpose of IGMP snooping, it provides the necessary transport for multicast traffic and

for the communication of orphaned ports. The term “orphaned ports” refers to switch

ports connected to single-attached hosts, or vPC ports whose members are all

connected to a single vPC peer.

In the case of a vPC device that is also a Layer 3 switch, the peer link also carries Hot

Standby Router Protocol (HSRP) frames. For a vPC to forward a VLAN, that VLAN

must exist on the peer link and on both vPC peers, and it must appear in the allowed list

of the switch port trunk for the vPC itself. If either of these conditions is not met, the

VLAN is not displayed when you enter the command show vpc brief, nor is it a vPC

VLAN. When a PortChannel is defined as a vPC peer link, Bridge Assurance is

automatically configured on the peer link.


27

(Source: Cisco.com)

vPC Peer-Keepalive

A routed “link” (it is more accurate to say “path”) is used to resolve dual-active scenarios

in which the peer link connectivity is lost. This link is referred to as a vPC peer-keepalive

or fault-tolerant link. The peer-keepalive traffic is often transported over the

management network through the management 0 port of the Cisco Nexus 5000 Series

Switch or the management 0 ports on each Cisco Nexus 7000 Series supervisor. The

peer-keepalive traffic is typically routed over a dedicated Virtual Routing and Forwarding

(VRF) instance (which could be the management VRF, for example). The keepalive can

be carried over a routed infrastructure; it does not need to be a direct point-to-point link,

and, in fact, it is desirable to carry the peer-keepalive traffic on a different network

instead of on a straight point-to-point link.


28

vPC Ports, and Orphaned Ports

A vPC port is a port that is assigned to a vPC channel group. The ports that form the

virtual PortChannel are split between the vPC peers and are referred to as vPC member

ports.

A non-vPC port, also known as an orphaned port, is a port that is not part of a vPC.

To summarize, a vPC system consists of the following components:

● Two peer devices: the vPC peers, of which one is primary, and one is secondary; both

are part of a vPC domain

● A Layer 3 Gigabit Ethernet link called a peer-keepalive link to resolve dual-active

scenarios

● A redundant 10 Gigabit Ethernet PortChannel called a peer link which is used to carry

traffic from one system to the other when needed and to synchronize forwarding tables

● vPC member ports forming the virtual PortChannel

Traffic Flows

vPC configurations are optimized to help ensure that traffic through a vPC-capable

system is symmetric. In Figure 6, for example, the flow on the left (in blue) reaching a

Cisco Nexus switch (Agg1 in the figure) from the core is forwarded toward the access

layer switch (Acc1 in the figure without traversing the peer Cisco Nexus switch device

(Agg2). Similarly, traffic from the server directed to the core reaches a Cisco Nexus

Switch (Agg1), and the receiving Cisco Nexus Switch routes this traffic directly to the

core without unnecessarily passing it to the peer Cisco Nexus device. This process

occurs regardless of which Cisco Nexus device is the primary HSRP device for a given

VLAN.


29

(Source: Cisco.com)

Dual-Control Plane with Single Layer 2 Node Behavior

While still operating with two separate control planes, vPC helps ensure that the

neighboring devices connected in vPC mode see the vPC peers as a single spanning-

tree and LACP entity. For this to happen, the system has to perform IEEE 802.3ad

control-plane operations in a slightly modified way (which is not noticeable to the

neighbor switch).

Link Aggregation Group Identifier

IEEE 802.3ad specifies the standard implementation of PortChannels. PortChannel

specifications provide LACP as a standard protocol, which enables negotiation of port

bundling.


30

LACP makes misconfiguration less likely, because if ports are mismatched, they will not

form a PortChannel.

LACP discerns that the only ports that can be bundled are port 1 going to port 4.

According to the IEEE specifications, to allow LACP to determine whether a set of links

connect to the same system and to determine whether those links are compatible with

aggregation, you need to be able to establish two types of identification:

• You need a globally unique identifier for each system that participates in link

aggregation (that is, the switch itself needs to be unique. This number is referred

to as the system ID and is composed of a priority and a MAC address that

uniquely identifies the switch. Figure 7 illustrates the system ID.

• You need a means of identifying a link aggregation group.

Dual-Control Plane with Single Layer 2 Node Behavior

While still operating with two separate control planes, vPC helps ensure that the

neighboring devices connected in vPC mode see the vPC peers as a single spanning-

tree and LACP entity. For this to happen, the system has to perform IEEE 802.3ad

control-plane operations in a slightly modified way (which is not noticeable to the

neighbor switch).

Link Aggregation Group Identifier

IEEE 802.3ad specifies the standard implementation of PortChannels. PortChannel

specifications provide LACP as a standard protocol, which enables negotiation of port

bundling.

LACP makes misconfiguration less likely, because if ports are mismatched, they will not

form a PortChannel.

ACP makes misconfiguration less likely, because if ports are mismatched, they will not

form a PortChannel. Consider example A in the figure below, in which switch 1 connects


31

to switch 2. Port 1 on switch 1 connects to port 4 on switch 2, and port 2 on switch 1

connects to port 6 on switch 2.

Now imagine that the administrator configured a PortChannel on switch 1 between ports

1 and 2, while on switch 2 the PortChannel is configured between ports 5 and 3.

Without LACP, the ports could be configured in channel-group mode, and you would not

discover that this is a misconfiguration until you notice that traffic has dropped. LACP

discerns that the only ports that can be bundled are port 1 going to port 4. According to

the IEEE specifications, to allow LACP to determine whether a set of links connect to

the same system and to determine whether those links are compatible with aggregation,

you need to be able to establish two types of identification:

• You need a globally unique identifier for each system that participates in link

aggregation (that is, the switch itself needs to be unique. This number is referred

to as the system ID and is composed of a priority and a MAC address that

uniquely identifies the switch.

• You need a means of identifying a link aggregation group.

(Source: Cisco.com)


32

System ID in a vPC System

Spanning tree and LACP use the switch MAC address for, respectively, the bridge ID

field in the spanning-tree BPDU and as part of LACP LAGID. In a single chassis, they

use the systemwide MAC address for this purpose. For systems that use vPCs, use of

the systemwide MAC address would not work because the vPC peers needs to appear

as a single entity as shown in example C in Figure 8. To meet this requirement, vPC

offers both an automatic configuration and a manual configuration of the system ID for

the vPC peers.

The automatic solution implemented by vPC consists of generation of a system ID

composed of a priority and a MAC address, with the MAC derived from a reserved pool

of MAC addresses combined with the domain ID specified in the vPC configuration. The

domain ID is encoded in the last octet and the trailing 2 bits of the previous octet of the

MAC address.

By configuring domain IDs to be different on adjacent vPCs complexes (and to be

identical on each vPC peer complex), you will help ensure the uniqueness of the system

ID for LACP negotiation purposes. You also help ensure that the spanning-tree BPDUs

use a MAC address that is representative of the vPC complex.

You can override the automatic generation of the system ID by using the command-line

interface (CLI) and configuring the system ID on both vPC peers manually, as follows:

(config-vpc-domain)#system-mac <mac>

Primary and Secondary vPC Roles

In a vPC system, one vPC switch is defined as primary and one is defined as

secondary, based on defined priorities. The lower number has higher priority, so it wins.

Also, these roles are non-preemptive, so a device may be operationally primary, but

secondary from a configuration perspective. To understand the operational role of a


33

vPC member, you need to consider the status of the peer-keepalive link and the peer

link.

When the two vPC systems are joined to form a vPC domain, the priority decides which

device is the vPC primary and which is the vPC secondary. If the primary device were to

reload, when the system comes back online and connectivity to the vPC secondary

device (now the operational primary) is restored, the operational role of the secondary

device (operational primary) will not change, to avoid unnecessary disruptions. This

behavior is achieved with a sticky-bit method, whereby the sticky information is not

saved in the startup configuration, thus making the device that is up and running win

over the reloaded device. Hence, the vPC primary becomes the vPC operational

secondary.

If the peer link is disconnected but the vPC peers are still connected through the vPC

peer keepalive link, the vPC operational roles stay unchanged.

If both the peer link and peer-keepalive link are disconnected, both vPC peers become

operational primary, but upon reconnection of the peer-keepalive link and the peer link,

the vPC secondary device (operational primary) keeps the primary role, and the vPC

primary becomes the operational secondary device.

Spanning Tree

vPC modifies the way in which spanning tree works on the switch to help ensure that

the vPC peers in a vPC domain appear as a single spanning-tree entity on vPC ports.

Also, vPC helps ensure that devices can connect to a vPC domain in a non-vPC fashion

with classic spanning-tree topology. vPC is designed to support hybrid topologies.

Depending on the Cisco NX-OS Software release, this can be achieved in slightly

different ways.

In all Cisco NX-OS releases, the peer link is always forwarding because of the need to

maintain the MAC address tables and Internet Group Management Protocol (IGMP)

entries synchronized.


34

vPC by default ensures that only the primary switch forwards BPDUs on vPCs. This

modification is strictly limited to vPC member ports. As a result, the BPDUs that may be

received by the secondary vPC peer on a vPC port are forwarded to the primary vPC

peer through the peer link for processing. Non-vPC ports operate like regular spanning-

tree ports.

The special behavior of the primary vPC member applies uniquely to ports that are part

of a vPC.

Starting from Cisco NX-OS Releases 4.2(6) and 5.0(2), vPC allows the user to choose

the peer-switch option. This option optimizes the behavior of spanning tree with vPC as

follows:

● The vPC primary and secondary are both root devices and both originate BPDUs

● The BPDUs originated by both the vPC primary and the vPC secondary have the

same designated bridge ID on vPC ports

● The BPDUs originated by the vPC primary and secondary on non-vPC ports maintain

the local bridge ID instead of the vPC bridge ID and advertise the Bridge ID of the vPC

system as the root

The peer-switch option has the following advantages:

● It reduces the traffic loss upon restoration of the peer link after a failure.

● It reduces the disruption associated with a dual-active failure (whereby both vPC

members become primary). Both devices keep sending BPDUs with the same bridge ID

information on vPC member ports, which prevents errdisable from potentially disabling

the PortChannel for an attached device.

● It reduces the potential loss of BPDUs if the primary and secondary roles change.

Cisco Fabric Services over Ethernet Synchronization Protocol

The vPC peers use the Cisco Fabric Services protocol to synchronize forwarding-plane

information and implement necessary configuration checks.

vPC peers must synchronize the Layer 2 forwarding table - that is, the MAC address

information between the vPC peers. This way, if one vPC peer learns a new MAC


35

address, that MAC address is also programmed on the Layer 2 forwarding table of the

other peer device.

The Cisco Fabric Services protocol travels on the peer link and does not require any

configuration by the user.

To help ensure that the peer link communication for the Cisco Fabric Services over

Ethernet protocol is always available, spanning tree has been modified to keep the

peer-link ports always forwarding. The Cisco Fabric Services over Ethernet protocol is

also used to perform compatibility checks to validate the compatibility of vPC member

ports to form the channel, to synchronize the IGMP snooping status, to monitor the

status of the vPC member ports, to synchronize the Address Resolution Protocol (ARP)

table (starting from Cisco NX-OS 4.2(6) and future Release 5.0 releases). If the peer

link is disconnected between two vPC peers, the synchronization between vPC peers is

interrupted, which may lead to traffic drop for multicast traffic and to flooding for unicast

traffic.

vPC Configuration Changes When the Peer Link Fails

The correct sequence for setting up vPCs requires that the two participating vPC

switches see each other over the peer link and that they can communicate over the vPC

peer keepalive link.

Figure below illustrates this fundamental concept: the configuration depicted in Figure 9

(2) requires starting from Figure (1). If you try to configure a vPC like in Figure (2)

without having established vPC peer-link and vPC peer keepalive connectivity, vPC

ports won’t go into forwarding state.

Once you are in the state depicted in Figure (1) (which is a fully formed vpc domain) the

peer keepalive connectivity is not strictly required in order to create or modify vPCs. In

other words, you can configure the vPC in Figure (2) even if there’s a loss of vPC peer

keepalive connectivity after the configuration depicted in Figure (1).


36

It is necessary and recommended to have functional vPC peer keepalive connectivity

for the correct behavior of vPC in presence of failures, but from a traffic forwarding and

configuration purpose, the temporary failure of the peer keepalive link doesn’t have any

impact.

The vPC peer-link connectivity is more important for the correct traffic forwarding

operations as well as for the ability to create or modify vPCs.

(Source: Cisco.com)

The initial implementation of vPC did not allow configuration changes when the peer link

was disconnected to avoid, upon reconnection of the peer switch, inconsistencies that

could bring these links down. Thus, if the peer-link was lost, the interfaces on the vPC

primary could not be flapping (i.e. if they went down, they stayed down up until the peer-

link was restored), and while the peer link was down, a new vPC member port could not

be activated.

Example A in Figure shows the case of a single vPC peer. The user cannot activate any

vPC member port until a vPC peer switch is present (example B in Figure).


37

(Source: Cisco.com)

Because of this behavior, if the peer-link connection is lost, by default the user cannot

add any vPC ports and activate them, nor can an interface flap. If a vPC interface flaps,

the port will stay down after flapping. For example, imagine a vPC setup with

PortChannel 8 configured as vPC 8:

vPC status

----------------------------------------------------------------------------

id Port Status Consistency Reason Active vlans

------ ----------- ------ ----------- -------------------------- -----------

8 Po8 up success success 23,50

After the peer-link failure only the primary keeps the vPC interfaces up. If the interface

associated with PortChannel 8 flaps, it never goes up again.

vPC status

----------------------------------------------------------------------------


------ ----------- ------ ----------- -------------------------- -----------

8 Po8 down failed Peer-link is down

To restore connectivity, you need to first restore the peer link, and then enter a shut/no

shut command under the interface on the primary device. Similarly, if you want to create

a new vPC interface and to activate it, a vPC peer needs to be present and connected


38

through the peer link. While a peer link is not connected, vPC prevents activation of any

new vPC member port.

This vPC behavior may manifest itself in several scenarios:

● A vPC pair in which the peer link is lost but the peer-keepalive is still connected

(shown as case 1 in Figure)

● A vPC pair in which peer link and peer-keepalive links are lost (split brain)

● A switch that was part of a vPC but has been reloaded; upon coming online, the vPC

peer is unavailable (shown as case 2 in Figure)

● A vPC switch that has never been part of a vPC domain because it has just been

powered up and configured for vPC, but no peer-link and peer-keepalive connectivity

has been established yet

(Source: Cisco.com)

Case 1 has been addressed by the Cisco Nexus 7000 Series with Cisco NX-OS 4.2(3).

This solution allows the vPC device to shut/no shut existing vPC member ports as long

as the vPC secondary device can still be reached through the peer-keepalive link. This

same behavior on the Cisco Nexus 5000 Series requires configuration of the keyword

peer-check-config-bypass under the vPC domain configuration, but this process is

superseded with NXOS 5.0(2)N1(1).

For case 1, to add a new vPCs you need the reload restore command of the Cisco

Nexus 7000 Series and on the Cisco Nexus 5000 Series running NXOS 5.0(2)N1(1) or


39

higher. You can achieve the same results with peer-check-config-bypass for the Cisco

Nexus 5000 Series if running an earlier version of code.

The case of complete disconnection between the vPC primary and secondary devices

(split brain) is addressed in the Cisco Nexus 7000 Series or Cisco Nexus 5000 Series

running NXOS 5.0(2)N1(1) or higher (for existing vPC member ports) and reload restore

(for new vPC member ports). The equivalent command on the Cisco Nexus 5000 Series

for earlier releases is peer-check-config-bypass.

The third case, reload of a vPC device, is addressed with the vPC reload restore

command.

To override the default vPC behavior, which prevents activation of new vPC member

ports when the peer link is down, you can use the command peer-config-check-bypass

under the vPC domain configuration (on the Cisco Nexus 5000 Series only).

As an example:

vpc domain 2

role priority 100

peer-keepalive destination 10.51.35.18

peer-config-check-bypass

The peer-config-check-bypass command was introduced in Cisco NX-OS 4.1(3)N2(1),

and it will be superseded by the vPC reload restore command when that command is

available.

With this command in place, even if the Cisco Nexus 5000 Series peer link is down, a

vPC member port can flap, and you can also create new vPC member ports and

activate them.


40

vPC Reload Restore

The vPC reload restore feature was introduced with Cisco NX-OS 5.0(2). With vPC

reload restore, when the peer link is down, or when the both peer link and peer-

keepalive links are lost, the vPC primary can activate new vPCs ports.

If the vPC peer switch is brought online or connected to the existing vPC peer switch,

the configuration checks are performed; if inconsistencies are found, the vPC ports are

shut down.

In addition, with reload restore a vPC peer can reload, and after a reload, if no vPC peer

exists (a condition that is detected by a timer), the user can create vPCs on the

standalone vPC peer.

Upon reload, Cisco NX-OS starts a user-configurable timer (with a default of 240

seconds). If the peer-link port comes up physically or if the peer keepalive is functional,

the timer is stopped and the device waits for the peer adjacency to form. If at timer

expiration no peer-keepalive- or peer-link-up packets have been received, Cisco NX-OS

assumes the primary role. The software reinitializes the vPCs, activating its local ports.

Because there are no peers, the consistency check is bypassed for the local vPC ports.

The timer is user configurable and defines how long the standalone vPC device waits to

detect a vPC peer. If at the timer expiration no peer-keepalive- or peer-link-up packets

have been received, the software reinitializes the vPCs, activating its local ports.

Because there are no peers, the consistency check is bypassed for the local vPC ports.

The following output shows the status of a virtual PortChannel configured on a

standalone vPC system with restore reload:

----------------------------------------------------------------------


-- ---- ------ ----------- -------------------------- ------------

51 Po51 up success Type checks were bypassed 10-14, 21-24

for the vPC ,50,60


41

Documents

Learn. Practice. Achieve. @ CCIEin8Weeks · 2018-08-23 · Learn. Practice. Achieve. @ CCIEin8Weeks.com 4 Contents at a Glance Part 1 Cisco Data Center L2/L3 Technologies Chapter