View
265
Download
5
Category
Preview:
Citation preview
CISCO WHITE PAPER
WAN MACsec Deployment White Paper
August 2016
Cisco White Paper
Table of ContentsIntroduction ....................................................................................................................................................................1
Technology Overview .....................................................................................................................................................2
MACsec Frame Format ....................................................................................................................................................3
Ethernet WAN Transition for Carrier Services ...................................................................................................................3
Layer 2 Adjacency with MACsec ......................................................................................................................................5
WAN MACsec and IPsec Comparison .............................................................................................................................6
Target Use Cases for MACsec ........................................................................................................................................8
WAN MACsec Deployment Models ...............................................................................................................................10
WAN MACsec Enhancements .......................................................................................................................................12
Clear Tag Option to Enable Point-to-Multipoint Deployments ....................................................................................... 12
Access Control Option for Smoother Migration ............................................................................................................. 12
Configurable EAPoL Destination Address ...................................................................................................................... 13
Configurable Replay-Protection-Window Size .............................................................................................................. 14
Configurable Key Lifetime and Hitless Key Rollover ....................................................................................................... 14
Configurable 128/256-bit Encryption and Hitless Rollover ............................................................................................ 15
CLI Syntax for Configuring Encryption Algorithm for Data Packets ................................................................................ 15
CLI Syntax to Configure Encryption Algorithm for MKA Control Packets ....................................................................... 15
Configuration Guidelines ...............................................................................................................................................16
Use Case 1: Point to Point E-Line Service .................................................................................................................... 17
Use Case 2: E-LAN Service (VPLS Service) ................................................................................................................. 22
Use Case 3: MACsec + SGT Transport ......................................................................................................................... 26
Configurable MKA, Key Chain & MACsec Parameters ...................................................................................................28
Performing Maintenance Tasks .....................................................................................................................................30
Performing Maintenance Tasks (without Impacting Traffic) ........................................................................................... 30
Performing Maintenance Tasks (Traffic Impacting) ........................................................................................................ 32
Scalability .....................................................................................................................................................................34
Monitoring and Troubleshooting ...................................................................................................................................35
Monitoring MACsec Sessions: Sample Output .............................................................................................................. 35
Monitoring MKA Sessions: Sample Output .................................................................................................................... 36
Network Design Considerations When Leveraging MACsec over Ethernet Service Offerings ........................................38
WAN MACsec Best Practices .......................................................................................................................................42
Caveats and Restrictions ..............................................................................................................................................43
Feature Limitations ........................................................................................................................................................ 43
Glossary .......................................................................................................................................................................44
page 1Cisco White Paper
Introduction
IntroductionThis white paper provides a reference point for Cisco sales staff, support teams, and organizations to use when deploying media access control security (MACsec) in common use cases for WAN.
You can use WAN MACsec innovation from Cisco in order to optimize and simplify the overall deployment, func-tionality, and operation of networks that require high-speed encryption beyond what IPsec can deliver, without sacrificing performance and packet size agility. WAN MACsec provides a line-rate network encryption solution over Layer 2 Ethernet transport services. MACsec is no longer just a LAN technology and can be leveraged outside campus networks, whether it be over Metro Ethernet transport or Data Center Interconnect (DCI) links. MACsec also secures WAN connections that are leveraging Ethernet as the link-layer media.
This paper provides an overview of MACsec beyond the campus LAN. MACsec can be a formidable encryption solution for WAN and Metro Ethernet links. This document provides an overview of MACsec, compares MACsec with current IP-based encryption solutions, and highlights key WAN/Metro use cases.
The goal of this white paper is to provide the details necessary to configure WAN MACsec, a line-rate network encryption solution over Layer 2 Ethernet Transport Services in the Cisco ASR 1001-X platform. This document also describes the best practices and recommendations for Cisco WAN MACsec deployments on ASR 1001-X platforms. The solution has been thoroughly validated during the system test cycle, and some of the deployment recommendations are included.
page 2Cisco White Paper
Technology Overview
Technology OverviewFigure 1 Data encryption using MACsec for a WAN connection between routers
Encrypted Data
MACsec
Encrypt
Decrypt
Encrypted Data
EVCs
L2 Service ProviderNetwork
60
20
F
The WAN MACsec solution:
• Is an IEEE 802.1AE standards-based Layer 2 hop-by-hop encryption that provides data confidentiality and integrity for media access independent protocols.
• Encrypts all data except for the source and destination MAC addresses of an Ethernet packet.
• Mitigates packet eavesdropping, tampering, and injection.
• Supports:
◦ IEEE 802.1AE-2006 compliant GCM-AES-128 and IEEE 802.1AEbn-2011 compliant GCM-AES-256 MACsec, NIST-approved, 1/10 Gb line-rate encryption.
◦ Point-to-point and point-to-multipoint configurations.
◦ Extending from hop-to-hop service to peer-to-peer service.
◦ The option to use VLAN tag in clear.
◦ The option to modify Extensible Authentication Protocol over LAN (EAPoL) Destination MAC and EtherType to avoid MACsec key agreement (MKA) protocol data unit being consumed by intermediate devices.
◦ The option to change access-control for smoother migration.
• Is typically done through hardware (ASIC/PHY) and provides line-rate throughput.
page 3Cisco White Paper
Technology Overview
MACSEC FRAME FORMATThe MACsec frame format is similar to that of standard Ethernet but includes an additional 32-byte MACsec header, which includes a well-known EtherType field.
Figure 2 MACsec frame format
Encrypted
Authenticated
DMAC
MACsec EtherType TCI/AN SL Packet Number SCI (optional)
SMAC 802.1AE Header 802.1Q CMD ETYPE PAYLOAD ICV CRC
TrustSec Frame Format
MACsec Tag Format
0x88E5
60
21
F
As shown in the above figure, the frame format using MACsec leverages a tag format with a MACsec EtherType (0x88E5), while allowing the Ethernet source/destination MAC addresses to be left in the clear for Ethernet frame forwarding. Also note that MACsec encrypts all fields behind the source/destination MAC addresses, so unless the ability to offset the encrypted field exists, fields such as MPLS labels and 802.1Q tags are encrypted and not able to be used when the Ethernet frame traverses the underlying transport between encrypted stations. The Cisco implementation of WAN MACsec described in this white paper allows the 802.1Q header to be left in the clear, offering a vast amount of network design flexibility as it relates to MACsec frames traversing a public Ether-net transport.
ETHERNET WAN TRANSITION FOR CARRIER SERVICESWAN and metro service provider offerings are replacing existing T1, ATM/frame relay, and SONET options for their customers in favor of low-cost Ethernet transport, which offers:
• Highly flexible, granular and scalable bandwidth.
• Simple troubleshooting.
• Enterprise-maintained networking and routing decisions.
• Easily added new locations (to L2VPN).
• Ubiquitous use for router ports with Ethernet support.
MACsec, a hop-by-hop encryption technology (per link), offers several advantages over existing IPsec encryption solutions in certain designs.
page 4Cisco White Paper
Technology Overview
Figure 3 MACsec encryption process, applied per link
Enterprise WAN
802.1AE MACsec Secured Ethernet WAN Link
A
B
EC
D F
Per “Hop” LinkEncryption over
the WAN
EnterpriseLAN
EnterpriseLAN
EnterpriseLAN
EnterpriseLAN
60
23
F
As shown in the above figure, the MACsec encryption process is applied per link, so as the Ethernet frame enters the PHY layer encapsulation process, the MACsec encryption is applied as well.
The PHY layer processing for MACsec is performed as the last process prior to the frame being sent to the egress port to be put on the physical connection to the neighbor device. Leveraging the encryption process late in the frame-forwarding procedure offers several advantages, most notably:
• It has no impact on Ethernet frame markings (802.1p for QoS, 802.1Q tag, Q-in-Q tags).
• Certain implementations have the option to offset where MACsec encryption is applied, leaving chosen tags in the clear, which can be advantageous to the Ethernet transport provider and the services offered based on 802.1Q and/or 802.1p bits (example: class of service).
In the case where an IP packet traverses multiple router hops (example: multiple P routers in an MPLS backbone) to get from router A to router E, because the links are MACsec encrypted over the Ethernet link, each router hop must encrypt/decrypt the frame. Some designers may believe this is a disadvantage in using MACsec, but it is no different than networks where the source/destination of the packet traverses multiple router hops, because the encryption process is done in the hardware PHY of the MAC ASIC. This is case with the ASR 1001-X — there is no encryption processing penalty when you enable MACsec.
page 5Cisco White Paper
Technology Overview
LAYER 2 ADJACENCY WITH MACSECCharacteristics of CE-to-CE deployment packet format:
• There is Layer 2 adjacency between CEs via EoMPLS or VPLS cloud.
• MACsec is required on CEs only. The cloud doesn’t need to be MACsec-capable.
• The purpose is to secure the untrusted WAN link.
Figure 4 Layer 2 adjacency with MACsec
CEASR1K
Egress
PE
PE
PE
CEASR1K
CEASR1K
Remote Site 3with ASR1K at edge
Remote Site 2with ASR1K at edgeMACsec encryption between L2
adjacent CEs at remote site edges
MPLSCloud
EoMPLS/VPLS
Remote Site 1with ASR1K at edge
60
24
F
page 6Cisco White Paper
WAN MACsec and IPsec Comparison
WAN MACsec and IPsec ComparisonWhen considering encryption options over a WAN/MAN, IPsec has clearly been the preferred choice given it leverages the ubiquitous nature and broad footprint of IP. This allows a very broad set of transport options, from private Layer 3 VPN services to the public Internet.
Although IPsec solves most encryption requirements, one of its limitations has been performance, specifically as it relates high-speed links that require line-rate performance, regardless of the packet size and speed.
When comparing MACsec and IPsec, the more generic comparison is Layer 2 link encryption to Layer 3 IP en-cryption. Layer 2 encryption dictates that the encryption is being done at Layer 2 of the OSI model, so examples of Layer 2 encryption solutions available include ATM, serial, SONET, and Ethernet. With link encryption, the en-cryption is done at Layer 2 for any of the chosen technologies, and applications and protocols at the layers above Layer 2 (Layers 3-7) are transparent, and part of the payload being encrypted. Because Layer 2 encryption is link-layer aware, it is dependent on the Layer 2 transport chosen for the network design (example: SONET net-work for SONET, Ethernet transport for Ethernet encryption, etc.), thus leveraging the forwarding of the encrypted frame to be performed in the transport of choice.
Layer 3 encryption (example: IPsec), on the other hand, operates at the IP layer of the OSI model, thus encrypt-ing data packets within the IP protocol. IPsec, as described in RFC 4301, offers packet-based encryption, which drastically changes the encryption model to that of Layer 2 link encryption. With Layer 3, encryption between end-stations is not limited to a Layer 2 technology but instead uses IP, allowing the underlying IP transport to be leveraged for connectivity between those end-stations. Because IP networks span globally (example: the Inter-net), Layer 3 encryption is much more flexible because it can leverage any packet-based IP transport. Layer 2 encryption, in contrast, is limited to the specific Layer 2 transport and does not normally span as broad a network footprint. Layer 3 encryption is unaware of the Layer 2 transport networks/links the packet traverses, and when one thinks of scale and reachability, nothing can rival the ubiquitous nature of IP, which IPsec leverages.
Given this comparison, one may ask “why even consider Layer 2 encryption over Layer 3?” Although IP has dominated network designs, both solutions have advantages and disadvantages and can simplify the integration of encryption in certain network design use cases, and Layer 2 encryption does have some clear differentiators that offer the network designer options.
One of the key differentiators MACsec offers over IPsec (beyond a simpler implementation) is its abilities to offer an AES-128/AES-256 encrypted solution at 100Mbps and to remain transparent to the IPv4/v6 packet layer, as well as not impacting the use of MPLS label(s) appended to the Ethernet frame.
When evaluating the various encryption solutions that exist in network deployments, it is important to not to evalu-ate whether one technology is superior to the other. Instead, your focus should be which technology:
• Best meets the business objectives of the end-user and application experience.
• Simplifies operations.
• Offers the most cost-effective solution.
These technologies are never one-size-fits-all, so it is up to the network designer to understand the holistic view of the network so the proposed solution aligns with the business objectives and services the network aims to offer. This is a key component when evaluating encryption technologies or any network solution and transport overall.
page 7Cisco White Paper
WAN MACsec and IPsec Comparison
Table 1 Comparison of WAN MACsec and IPsec encryption solutions
Category WAN MACsec IPsec
Market positioning Aggregate deployments such as regional hubs
Large branches that require high throughput
Data center interconnects
Small branches
High scale deployments
Low throughput branches
Beyond MetroE (interna-tional) reach
Link requirement Requires dedicated MetroE EVC circuits for L2 connectivity between sites
Easily routable over many commonly available public networks
Encryption performance Per PHY link speed (1 Gb, 10 Gb)
Constrained by IPsec crypto engine perfor-mance
Services enablement No impact to encryption throughput
Impacts encryption throughput
Peers scale Limited by hardware resources Highly scalable
Throughput Up to line rate on each port (limited only by the forwarding capability)
Aggregate throughput (limited by the encryption throughput)
Configurability Simple configuration More complex configura-tion and policy choices
Layer 3 visibility for monitoring
No. Except Layer 2 headers (and optionally VLAN/MPLS labels). Everything else is en-crypted.
Visible. Layer 3 info can be used for monitoring and policy enforcement purposes
NAT environment L3 header is not accessible. Works with NAT environ-ment.
page 8Cisco White Paper
Target Use Cases for MACsec
Target Use Cases for MACsecAs discussed, both Layer 2 and Layer 3 encryption technologies have advantages and disadvantages. However, MACsec does offer several unique advantages, given the required transport to carry an Ethernet frame exists. Link-layer encryption, within itself, offers an advantage over IPsec in that it does not require an overlay transport technology (example: IP in IP). Secondly, for MPLS deployments, technologies that require the use of MPLS over IP for use with IPsec (UDP or GRE) are not required, because the MPLS label is imposed on the Ethernet frame prior to MACsec encryption. This offers several advantages, including:
• Eliminating the need to implement MPLS over IP or any over-the-top solutions when encryption is required, thus simplifying the implementation.
• Reducing the packet overhead (MACsec = 32 bytes) versus IPsec (GRE+Tunnel mode = ~74 bytes).
• Encrypting (that is, hiding) the label and/or 802.1Q tag, leaving only the source/destination MAC address in the clear between stations.
• Allowing customers the ability to leverage cost-effective Ethernet transport services.
• Encrypting at Ethernet speeds.
• Targeting peer-model network designs (versus flat multipoint Ethernet network transport).
Considering these advantages, there are several key situations when MACsec could be considered over IPsec in the design, including when:
• High encryption rate is required.
• The WAN/MAN transport service used is Ethernet.
• The backbone routers are interconnected with either fiber/wavelength or Ethernet, in a point-to-point and/or partial mesh topology design
• Per-link encryption is suitable for the given design for core/edge router connectivity within the backbone topology (that is, the core/edge routers leverage point-to-point Ethernet links for interconnection).
As the MACsec technology continues to evolve, consider the use cases that extend far beyond campus and data center designs. MACsec opens up a whole new paradigm as it relates to secure WAN and Metro Ethernet designs. The following WAN and Metro implementations require data encryption over untrusted networks and can benefit from Layer 2 MACsec encryption:
• Securing router core links (IP/MPLS, PE-P, P-P)—Secure high-speed backbone transport links (point-to-point today), such as the following:
◦ Optical transport hand-off is client optics from a DWDM system, which are easy to tap into and intercept
◦ Service provider hand-off is standard Ethernet, which is also easy to intercept
• Securing Metro Ethernet Services—Offer (as a service provider) or leverage secure 10/40/100GE Metro Ethernet services, each link leveraging 802.1AE for encryption
◦ Secure Ethernet Service (point to point)
◦ Secure Ethernet Multipoint Service (example: VPLS) (future)
page 9Cisco White Paper
Target Use Cases for MACsec
• Securing PE-CE link transport integration for MPLS IP VPN Services—Secure back-haul to an MPLS BGP VPN service (L3 service)
• Securing N-PE to U-PE/CPE—Backbone PE (L2 service offering)
• Securing links over any Ethernet service—Applications include DCI, access to Cloud utilities, storage replica-tion
• Securing Over-the-Top Ethernet Links—Enterprise/Public Sector encrypts their Ethernet links on their own CPE routers over-the-top of the SP Ethernet service (supported today on point-to-point Ethernet services only)
As discussed in “WAN MACsec and IPsec Comparison,” one technology is not necessarily better than the other; the network designer has options and can evaluate which solution best fits the network requirement given the transport and design criteria that exist. MACsec clearly offers significant design options, given the popularity of Ethernet and the ever-growing Ethernet services that service providers offer customers.
page 10Cisco White Paper
WAN MACsec Deployment Models
WAN MACsec Deployment ModelsStandard WAN MACsec definitions call out the following deployment models:
• Data center interconnect
• Branch connectivity
• Aggregation deployment
• Metro-E Ring/EoMPLS
• L2TPv3 (Layer 2 Tunneling Protocol)
• Overlay transport virtualization
WAN MACsec deployment modes:
• Point-to-point, CE-to-CE connectivity using Ethernet private line (EPL) service (port-mode) Typical use cases: head-office-to-large-branch-office connectivity, data center interconnect
• Point-to-point, hub and spoke connectivity using Ethernet virtual private line (EVPL) service (VLAN-mode) Typical use cases: head-office-to-multiple-branch-offices connectivity, interconnecting multiple data cen-ters, connectivity between aggregation sites such as regional hubs, and allowing service multiplexing such that multiple P2P can co-exist on the same physical interface
• Point-to-point, hub-and-spoke connectivity with MACsec and non-MACsec spokes Typical use cases: migration from non-MACsec branches to MACsec branches, service multiplexing such as Metro-E circuit and Internet or IPsec connection on the same physical interface.
• Multipoint-to-multipoint, hub-and-spoke connectivity using Ethernet private LAN (EP-LAN) service (port-mode) Typical use cases: head-office-to-multiple-branch-offices connectivity, interconnecting multiple data cen-ters, connectivity between aggregations sites such as regional hubs in a LAN fashion. Easy to provision and manage.
• Multipoint-to-multipoint, hub-and-spoke connectivity using Ethernet virtual private LAN (EVP-LAN) service (VLAN-mode) Typical use cases: head-office-to-multiple-branch-offices connectivity, interconnecting multiple data cen-ters, connectivity between aggregation sites such as regional hubs in a LAN fashion, and allowing service multiplexing such that multiple LAN can co-exist on the same physical interface. Easy to provision and man-age.
• MACsec with security group tag (SGT) transport Typical use case: carrying SGT over WAN MACsec
page 11Cisco White Paper
WAN MACsec Deployment Models
MACsec is commonly deployed in a Metro Ethernet WAN. The following four Metro Ethernet Service types are supported as part of this solution as defined in the Metro Ethernet Forum (MEF).
Table 2 MEF Ethernet Service Types
Service typePort-based services VLAN-based services
E-Line EPL EVPL
Point-to-point, UNI-to-UNI
EPL EVPL
E-LAN EP-LAN EVP-LAN
Multipoint-to-multipoint, UNIs-to-UNIs
EP-LAN EVP-LAN
page 12Cisco White Paper
WAN MACsec Enhancements
WAN MACsec EnhancementsThis section details the specific enhancements made to accommodate MACsec in the WAN environment.
CLEAR TAG OPTION TO ENABLE POINT-TO-MULTIPOINT DEPLOYMENTSClear Tag is a Cisco MACsec innovation implemented to enable service multiplexing at the CE by providing tun-neling information such as VLAN tag in the clear so that multiple point-to-point or multipoint services can co-exist on a single physical interface. Enabling this option allows customers to configure a VLAN (802.1Q) tag to bypass the MACsec encryption, as shown in the following figure.
Figure 5 VLAN tag configured to bypass MACsec encryption
Encrypted
AuthenticatedAuthenticated
Eth 802.1Q 802.1AE ETYPE PAYLOAD ICV CRC
60
26
F
The macsec dot1q-in-clear 1 command controls the behavior of allowing 802.1Q tag in the clear. The default is no macsec dot1q-in-clear (that is, no dot1q tag in the clear).
Tech Tip
With Cisco ASR1001-X, macsec dot1q-in-clear 1 can only be configured on physical interface, and the setting is automatically inherited by the sub-interfaces.
ACCESS CONTROL OPTION FOR SMOOTHER MIGRATIONBy default, when MACsec is enabled on an interface, it is expected that the customer’s intention is to secure the entire interface traffic; as such, MACsec does not allow any unencrypted packets to be transmitted or received from the same physical interface. But it is not uncommon to see a mix of MACsec-enabled and non-MACsec-enabled sub-interfaces co-existing, such as during migration (example: customers may wish to enable MACsec for one branch at a time). To accommodate this, an additional Cisco proprietary extension has been implemented to allow unencrypted packets to be transmitted or received from the same physical interface.
The macsec access-control [must-secure | should-secure] command controls the behavior of unencrypted packets processing as follows:
• should-secure allows unencrypted packets to be transmitted and received from the physical interface or sub interfaces.
• must-secure does not allow transmit/receive of unencrypted packets from physical interface or sub inter-faces and drops the packet except for MKA control protocol packets.
• If a mix of MACsec and non-MACsec sub interfaces (example: sub interfaces with IPsec configured) co-exists, then should-secure configuration is required.
page 13Cisco White Paper
WAN MACsec Enhancements
The default is macsec access-control must-secure, which is effective as soon as MACsec CLI is configured (that is, unencrypted packets are not allowed to be transmitted or received except for MKA control protocol pack-ets).
Tech Tip
Because should-secure operates on a physical interface level, it cannot be configured on a sub-interface. More importantly, configuring should-secure allows unencrypted traffic to be received even on a secured MACsec session.
CONFIGURABLE EAPOL DESTINATION ADDRESSWAN MACsec provides ability to change the MKA packet’s MAC address in order to adjust to Ethernet networks that do not support MKA transport.
Before establishing a MACsec secure session, MKA as a control protocol is used to select the cipher suite for encryption and exchanges the required keys and parameters between the peers.
MKA uses EAPoL as a transport protocol to transmit MKA messages. By default, EAPoL uses a destination mul-ticast MAC address of 01:80:C2:00:00:03 to multicast the packets to multiple destinations. Since EAPoL is a standards-based protocol and other authentication mechanisms such as IEEE 802.1X also use the same proto-col, there may be situations where the switches in the service provider cloud might consume this packet (based on the destination multicast MAC address)—assuming the packet is destined for them, trying to process it, and eventually dropping it, essentially causing MKA session establishment to fail. To avoid this situation, another Cisco proprietary extension has been implemented; it allows an administrator to change the destination MAC address of an EAPoL packet that is transmitted on an interface towards the service provider so that the service provider tun-nels the packet like any other data packet instead of consuming it.
The eapol destination-address command is used to configure the EAPoL destination MAC address. The follow-ing options are supported:
• H.H.H (any MAC address)
• Bridge-group address (0180.C200.0000)
• LLDP-multicast address (0180.C200.000E)
• Broadcast
• Default: (0180.C200.0003)
Tech Tip
You can configure the EAPoL destination address independently on either physical or sub interface level. If it is configured on the physical interface, it is automatically inherited by the sub interfaces. Explicit configuration on a sub-interface overrides the inherited value or policy for that sub-interface.
page 14Cisco White Paper
WAN MACsec Enhancements
CONFIGURABLE REPLAY-PROTECTION-WINDOW SIZEReplay protection is a MACsec feature to counter against replay attacks by assigning a unique sequence number to each encrypted packet and verifying the sequence received at the remote end. Although it is not common to see packets reordered in a campus environment and hence strict replay protection may be used, frames trans-mitted through a Metro Ethernet SP network are highly susceptible to reordering due to various prioritization and load-balancing mechanisms used within the SP network that increase the chances of frames arriving out of order and hence may require a high tolerance to packet reordering.
Again, a replay window is necessary to support use of MACsec over provider networks that reorder frames for various reasons. Frames within the window can be received out of order but are not replay protected. The default window size is set to 64, the same as IPsec. This can be changed to a lower or higher value through the following command:
macsec replay-protection window-size [0-4294967295]
The replay protection window may be set to zero to enforce strict reception ordering and replay protection.
Tech Tip
You can configure a replay protection window independently on either physical or sub interface. If you configure it on the physical interface, it is automatically inherited by the sub interfaces. Explicit configuration on sub-interface overrides the inherited value or policy for that sub-interface.
CONFIGURABLE KEY LIFETIME AND HITLESS KEY ROLLOVERA MACsec key chain can have multiple pre-shared keys, each configured with a key id and an optional lifetime. A key lifetime specifies at which time the key expires. In the absence of a lifetime configuration, the default lifetime is unlimited. When a lifetime is configured, MKA rolls over to the next configured pre-shared key in the key chain after the lifetime is expired. Time zone of the key can be local or UTC. The default time zone is UTC.
key chain [name] macsec
key 01
key-string [Hex string]
lifetime [time] [local]
The hex string must be 32 Hex characters or 64 characters depending on the Cryptographic algorithm selected (example: AES-128-CMAC or AES-256-CMAC, respectively)
Keys roll over to the next key within the same key chain by configuring a second key (key 02) in the key chain and configuring a lifetime for the first key. When the first key (key 01) lifetime expires, it automatically rolls over to the next key in the list. If the same key is configured on both sides of the link at the same time, then the key rollover is hitless (that is, the key rolls over without traffic interruption).
page 15Cisco White Paper
WAN MACsec Enhancements
CONFIGURABLE 128/256-BIT ENCRYPTION AND HITLESS ROLLOVERTwo encryption cipher suites must be selected for MACsec to operate: a cipher suite that encrypts data packets and another cipher suite that encrypts MKA control packets.
It is recommended that cipher suites be paired, allowing for consistent cryptography (pair 128 bit ciphers for data and MKA or 256 bit ciphers for data and MKA).
Cipher suite selection for MACsec data packets encryption is as follows:
• MACsec cipher suite to encrypt data packets is configured as part of MKA policy. There can be more than one cipher suite in the list.
• A key server uses the first cipher suite from the list.
• All non-key servers must use the same cipher suite as the key server. The cipher suites can be configured in any order in the non-key server. If the same cipher suite is not available in the list, the session establishment fails.
• In the absence of a MACsec cipher suite configuration, all the peers have a default list, which includes all the cipher suites supported by the local hardware. For ASR1001-X, the cipher suite list is GCM-AES-128, GCM-AES-256.
• If a MACsec cipher suite is not configured by an admin, the default cipher suite of GCM-AES-128 (Galois/Counter Mode of Advanced Encryption Standard cipher with 128-bit key) is used.
CLI SYNTAX FOR CONFIGURING ENCRYPTION ALGORITHM FOR DATA PACKETSmka policy POLICY1
macsec-cipher-suite [gcm-aes-128 | gcm-aes-256]
Cryptographic algorithm selection for MKA control protocol packets encryption is as follows:
• The cryptographic algorithm to encrypt MKA control protocol packets is configured as part of key chain. There can be only one cryptographic algorithm configured per key chain.
• A key server uses the configured MKA cryptographic algorithm from the key chain used.
• All non-key servers must use the same cryptographic algorithm as the key server.
If an admin does not configure an MKA cryptographic algorithm, a default cryptographic algorithm of AES-CMAC-128 (Cipher-based message authentication code with 128-bit Advanced Encryption Standard) is used.
CLI SYNTAX TO CONFIGURE ENCRYPTION ALGORITHM FOR MKA CONTROL PACKETS
key chain [name] macsec
key 01
key-string [Hex string]
cryptographic-algorithm [aes-256-cmac | aes-256-cmac]
page 16Cisco White Paper
Configuration Guidelines
Configuration GuidelinesSolution deployment prerequisites:
• Cisco ASR 1001-X with 15.S(01)S or later software. On-board 1 GE and 10 GE ports of ASR 1001-X support WAN MACsec.
• Layer 2 transparent Ethernet Services-MetroE Circuit (EVCs) Service provider network should provide MACsec Layer 2 control protocol transparency such as EAPoL. Note: The WAN MACsec solution in ASR1001-X depends on MKA (as defined in 802.1X-2010) for advertis-ing capabilities, exchange keying material, cipher suite negotiations, etc. MKA uses EAPoL protocol.
There is no special license needed for WAN MACsec, but it is available as part of regular crypto RTU license.
Figure 6 Network topology
MPLS Cloud
Carrier Ethernet ServiceEoMPLS/VPLS
PE3
PE4
PE1 PE2ASR 1001-X CE1
Traffic Generatorin CE1 VLAN
ASR 1001-X CE3
ASR 1001-X CE2
E-LINE (P2P)E-LAN (P2MP)
MACsec Encrypted Link
60
25
F
page 17Cisco White Paper
Configuration Guidelines
USE CASE 1: POINT TO POINT E-LINE SERVICE
Point-to-Point SA Configuration—CE to CE (Port-based E-Line Service)
EnterpriseNetwork
Remote Site
• MACsec enabled interface • Physical • Sub-interface (802.1Q)
EnterpriseNetwork
CentralCampus/DC
MKA Keying(802.1X-2010)
Carrier EthernetService
E-LINE (P2P)
MKA Session
MACsec Flow
MKA Key
MACsec Interface
CE2 CE1
60
27
F
Ethernet service:
• Point-to-point pseudo wire service (no MAC address lookup)
• Port-mode
This guide uses the following conventions for commands that you enter at the command-line interface (CLI).
Commands to enter at a CLI prompt: configure terminal
Commands that specify a value for a variable: ntp server 10.10.48.17
Commands with variables that you must de�ne: class-map [highest class name]
Commands at a CLI or script prompt: Router# enable
Long commands that line wrap are underlined. Enter them as one command:
police rate 10000 pps burst 10000 packets conform-action
Noteworthy parts of system output (or of device con�guration �les) are highlighted: interface Vlan64 ip address 10.5.204.5 255.255.255.0
How to Read Commands
page 18Cisco White Paper
Configuration Guidelines
CE1 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
ip address 10.3.1.1 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
CE2 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
ip address 10.3.1.2 255.255.255.0
mka pre-shared-key key-chain KEY1
macsec
Point to Point SA Configuration—Hub and Spoke, VLAN-based E-Line Service (P2P)—Only MACsec sub-interfaces
EnterpriseNetwork
EnterpriseNetwork
Branch Site
EnterpriseNetwork
Branch Site
• MACsec enabled interface • Physical • Sub-interface (802.1Q)
Port-mode wouldrequire 2 physical
interfaces
MKA Keying(802.1X-2010)
Carrier EthernetService
E-LINE (P2P)
CE2 CE1
CE3 VLAN20
VLAN10
CentralCampus/DC
60
28
F
Ethernet service:
• Point-to-point PW service (no MAC address lookup)
• Port-mode or 802.1Q offering
page 19Cisco White Paper
Configuration Guidelines
CE1 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
macsec dot1q-in-clear 1
macsec replay-protection-window-size 1000
interface GigabitEthernet0/0/4.1
encapsulation dot1Q 10
ip address 10.3.1.1 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
interface GigabitEthernet0/0/4.2
encapsulation dot1Q 20
ip address 10.3.2.1 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
CE2 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
macsec dot1q-in-clear 1
macsec replay-protection-window-size 1000
interface GigabitEthernet0/0/4.1
encapsulation dot1Q 10
ip address 10.3.1.1 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
page 20Cisco White Paper
Configuration Guidelines
CE3 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
macsec dot1q-in-clear 1
macsec replay-protection-window-size 1000
interface GigabitEthernet0/0/4.1
encapsulation dot1Q 10
ip address 10.3.1.1 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
Point-to-Point SA Configuration—Mix of MACsec and Non-MACsec Spokes, VLAN-based E-Line Service (P2P)
EnterpriseNetwork
Branch Site
EnterpriseNetwork
Branch Site
CentralCampus/DC
• MACsec enabled interface • Physical • Sub-interface (802.1Q)
MKA Keying(802.1X-2010)
Carrier EthernetService
E-LINE (P2P)
CE2 CE1
CE3
CE4
EnterpriseNetwork
60
29
F
Ethernet service
• Point-to-point PW service (no MAC address lookup)
• Port-mode or 802.1Q offering
page 21Cisco White Paper
Configuration Guidelines
CE1 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
macsec dot1q-in-clear 1
macsec access-control should-secure
interface GigabitEthernet0/0/4.1
encapsulation dot1Q 10
ip address 10.3.1.1 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
interface GigabitEthernet0/0/4.2
encapsulation dot1Q 20
ip address 10.3.2.1 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
interface GigabitEthernet0/0/4.3
encapsulation dot1Q 30
ip address 10.3.3.1 255.255.255.0
CE2 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
macsec dot1q-in-clear 1
macsec access-control should-secure
interface GigabitEthernet0/0/4.1
encapsulation dot1Q 10
ip address 10.3.1.2 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
page 22Cisco White Paper
Configuration Guidelines
CE3 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
macsec dot1q-in-clear 1
interface GigabitEthernet0/0/4.1
encapsulation dot1Q 20
ip address 10.3.2.2 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
CE4 Configurationinterface GigabitEthernet0/0/4.1
encapsulation dot1Q 30
ip address 10.3.3.2 255.255.255.0
USE CASE 2: E-LAN SERVICE (VPLS SERVICE)
Point-to-Point SA Configuration—Hub and Spoke
Branch Site
EnterpriseNetwork
Branch Site
EnterpriseNetwork
EnterpriseNetwork
CentralCampus/DC
MKA Keying(802.1X-2010)
Carrier EthernetService
E-LAN (multi-point)
CE2 CE1
CE3
60
31
F
Ethernet Service
• Multipoint service (typically VPLS)
• Port mode or 802.1Q offering
page 23Cisco White Paper
Configuration Guidelines
CE1 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
macsec dot1q-in-clear 1
macsec replay-protection-window-size 1000
eapol destination-address broadcast
interface GigabitEthernet0/0/4.1
encapsulation dot1Q 10
ip address 10.3.1.1 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
CE2/CE3 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
macsec dot1q-in-clear 1
interface GigabitEthernet0/0/4.1
encapsulation dot1Q 10
ip address 10.3.1.2 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
macsec replay-protection-window-size 1000
eapol destination-address broadcast
page 24Cisco White Paper
Configuration Guidelines
Point to Point SA Configuration—Hub and Spoke, Spoke to Spoke
Figure 7 Point to Point SA Configuration—Hub and Spoke, Spoke to Spoke
Branch Site
EnterpriseNetwork
Branch Site
EnterpriseNetwork
EnterpriseNetwork
CentralCampus/DC
MKA Keying(802.1X-2010)
Carrier EthernetService
E-LAN (multi-point)
CE2 CE1
CE3
• MACsec enabled interface • Physical • Sub-interface (802.1Q)
60
32
F
Multiple VLAN-based E-LAN Services (P2MP)—Multiple VLAN-based E-LAN Services (P2MP)
Metro EthernetNetwork
P2MPEVCsVLAN 10
VLAN 10
VLAN 10
VLAN 20
VLA
N 2
0 VLA
N 20
CE1
CE2
CE3
CE4 CE5 60
33
F
Ethernet Service
• Multipoint service (typically VPLS)
• Port mode or 802.1Q offering
page 25Cisco White Paper
Configuration Guidelines
CE1 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
macsec dot1q-in-clear 1
macsec replay-protection-window-size 1000
interface GigabitEthernet0/0/4.1
encapsulation dot1Q 10
ip address 10.3.1.1 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
interface GigabitEthernet0/0/4.2
encapsulation dot1Q 20
ip address 10.3.2.1 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
CE2/CE3 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
macsec dot1q-in-clear 1
macsec replay-protection-window-size 1000
interface GigabitEthernet0/0/4.1
encapsulation dot1Q 10
ip address 10.3.1.2 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
page 26Cisco White Paper
Configuration Guidelines
CE4/CE5 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
macsec dot1q-in-clear 1
macsec replay-protection-window-size 1000
interface GigabitEthernet0/0/4.2
encapsulation dot1Q 20
ip address 10.3.2.2 255.255.255.0
mka pre-shared-key key-chain KEY1 macsec
USE CASE 3: MACSEC + SGT TRANSPORT
Branch Site
EnterpriseNetwork
EnterpriseNetwork
CentralCampus/DC
Classification
• MACsec enabled interface • Physical • Sub-interface (802.1Q)
MKA Keying(802.1X-REV)
SGT Transport
ISE Directory
Carrier EthernetService
E-LINE (P2P)
CE CE
MKA Session
MACsec Flow
MKA Key
MACsec Interface
60
34
F
CE1 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
ip address 10.3.1.1 255.255.255.0
cts manual
propagate sgt
mka pre-shared-key key-chain KEY1 macsec
page 27Cisco White Paper
Configuration Guidelines
CE2 Configurationkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface GigabitEthernet0/0/4
ip address 10.3.1.2 255.255.255.0
cts manual
propagate sgt
mka pre-shared-key key-chain KEY1 macsec
page 28Cisco White Paper
Configurable MKA, Key Chain & MACsec Parameters
Configurable MKA, Key Chain & MACsec ParametersMKA global policy configurable parameters:
• key-server priority
◦ 0 to 64
◦ Default: 0
• macsec-cipher-suite
◦ macsec-cipher-suite gcm-aes-128
◦ macsec-cipher-suite gcm-aes-256
◦ Default: macsec-cipher-suite gcm-aes-128
• confidentiality-offset
◦ 0, 30, 50
◦ Default: 0
Keychain global configurable parameters:
• key
◦ Key ID
• cryptographic-algorithm
◦ cryptographic-algorithm aes-128-cmac
◦ cryptographic-algorithm aes-256-cmac
◦ Default: cryptographic-algorithm aes-128-cmac
• keystring
◦ Hex Characters
◦ Default: NA
• lifetime
◦ hh:mm:ss
◦ Local time in local time zone
◦ Default: unlimited
page 29Cisco White Paper
Configurable MKA, Key Chain & MACsec Parameters
MACsec-interface-configurable commands and parameters:
• macsec replay-protection-window-size
◦ 0-x
◦ Default: 64
• macsec-access-control
◦ must-secure
◦ should-secure
◦ Default: must-secure
• macsec-dot1q-in-clear
◦ 0, 1
◦ Default: 0
• macsec
• eapol destination-address
◦ H.H.H (any mac address)
◦ bridge-group-address
◦ lldp-multicast-address
◦ broadcast
◦ Default: (01:80:C2:00:00:03)
page 30Cisco White Paper
Performing Maintenance Tasks
Performing Maintenance Tasks
PERFORMING MAINTENANCE TASKS (WITHOUT IMPACTING TRAFFIC)
Changing a Pre-Shared Key (CAK Rollover): Sample ConfigurationKeys can be configured to automatically roll over to the next key by configuring a lifetime on both routers.
Fromkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
Tokey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
lifetime local 10:30:00 Oct 30 2014 11:30:00 Oct 30 2014
key 02
key-string 11145678901234567890123456789012
Changing a Key Chain (Keychain Rollover): Sample Configuration
Fromkey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
interface TenGigabitEthernet0/0/0.10
mka pre-shared-key key-chain KEY1
Tokey chain KEY1 macsec
key 01
key-string 12345678901234567890123456789012
key chain KEY2 macsec
key 01
key-string abcdef0987654321abcdef0987654321
interface TenGigabitEthernet0/0/0.10
mka pre-shared-key key-chain KEY2
page 31Cisco White Paper
Performing Maintenance Tasks
Changing a Key Server Role: Sample ConfigurationYou can force a router to become a key server by configuring a lower priority than other peer routers that par-ticipate in the same session. It is recommended that you configure a key server priority so that the key server selection is deterministic. For example, in a hub-and-spoke scenario, the ideal place for a key server is the hub site router.
Hub Site (Key Server)mka policy POLICY1
key-server priority 0
interface TenGigabitEthernet0/0/0.10
mka pre-shared-key key-chain KEY1
mka policy POLICY1
Spoke Sites (Non-Key Servers)mka policy POLICY1
key-server priority 1
interface TenGigabitEthernet0/0/0.10
mka pre-shared-key key-chain KEY1
mka policy POLICY1
Changing MACsec Cipher-Suite to Encrypt Data Traffic with 128/256 bit Encryptionmka policy POLICY1
macsec-cipher-suite gcm-aes-128
# alternative configuration using AES 256
# macsec-cipher-suite gcm-aes-256
interface GigabitEthernet0/0/1.10
mka policy POLICY1
Changing MKA Cipher-Suite to Encrypt Control Traffic with 128/256 bit Encryptionkey chain KEY3 macsec
key 01
key-string abcdef0987654321abcdef0987654321
cryptographic-algorithm aes-128-cmac
# alternative configuration using AES 256
# cryptographic-algorithm aes-256-cmac
interface TenGigabitEthernet0/0/0.10
mka pre-shared-key key-chain KEY3
page 32Cisco White Paper
Performing Maintenance Tasks
Changing EAPoL Destination MAC AddressThe EAPoL destination MAC address can be changed from physical interface configuration mode or sub-interface configuration mode and is automatically inherited by the sub-interfaces if configured at the physical-interface level. If you need to override the inherited value, configure it at the sub interface mode. The default EAPoL desti-nation MAC address is 01:80:c2:00:00:03.
interface TenGigabitEthernet0/0/0
eapol destination-address [ip address | bridge-group-address | lldp-multicast-address]
Changing EAPoL EtherTypeMEF standards define a set of MAC addresses that carrier Ethernet transit switches consume for their own control plane processing requirements. Current MACsec and MKA implementations negotiate MKA keys using an EAPoL packet. Because these EAPoL MAC addresses are part of the set of MAC addresses defined by the MEF to be consumed by the provider switches, customers deploying MACsec over a public carrier Ethernet transport cannot use MACsec across these providers without changing the default MACsec behavior using EAPoL MAC addresses.
To mitigate this problem, Cisco introduced the ability for the operator deploying WAN MACsec to change the EAPoL destination address and/or EtherType to an address that is not consumed by the provider equipment.
The EAPoL destination EtherType can be changed to an alternate value from the default EtherType of 88E5, to avoid being consumed by a provider bridge.
interface TenGigabitEthernet0/0/0
eapol eth-type 876F
Changing Confidentiality Offsetmka policy POLICY1
confidentiality-offset 30
interface GigabitEthernet0/0/1.10
mka policy POLICY1
PERFORMING MAINTENANCE TASKS (TRAFFIC IMPACTING)
Changing a Replay-Protection-Window SizeReplay protection window can be changed from physical interface configuration mode or sub-interface configu-ration mode and is automatically inherited by the sub interfaces if configured at the physical interface level. If you need to override the inherited value, configure it at the sub interface mode. The default replay protection window size is 64.
interface TenGigabitEthernet0/0/0
macsec replay-protection window-size 10
interface TenGigabitEthernet0/0/0.10
macsec replay-protection window-size 5
page 33Cisco White Paper
Performing Maintenance Tasks
Enable/Disable VLAN (dot1q) tag in the clear optionThe macsec dot1q-in-clear command can only be configured on physical interface, and the setting is automati-cally inherited by the sub-interfaces.
interface GigabitEthernet0/0/1
macsec dot1q-in-clear 1
Enable/Disable Unencrypted Packets Processing (Access Control Setting)The macsec access-control must-secure/should-secure command can only be configured on physical inter-face, and the setting is automatically inherited by the sub-interfaces.
interface GigabitEthernet0/0/1
macsec [access-control must-secure | should-secure]
page 34Cisco White Paper
Scalability
ScalabilityTable 3 Number of supported MACsec peers per port in ASR 1001-X
1 Gb 10 Gb
Max # of SA 16 (receive)+16 (transmit) 64 (receive)+64 (transmit)
Max # of SC 8 (receive)+8 (transmit) 32 (receive)+32 (transmit)
Max # of point-to-point 8 32
Max # of sites in P2MP setup 8 32
Max # of group CA4 (with 2 members in each group)
16 (with 2 members in each group)
Although MACsec is supported up to line-rate on each interface, the forwarding capability may be limited by the maximum system forwarding capability.
page 35Cisco White Paper
Monitoring and Troubleshooting
Monitoring and TroubleshootingYou can use the following show commands to monitor MACsec session details:
• show macsec summary
• show macsec statistics interface [interface name]
• show macsec status interface [interface name]
MONITORING MACSEC SESSIONS: SAMPLE OUTPUTR2#show macsec summary
MACsec Capable Interface Extension
---------------------------------------------
TenGigabitEthernet0/0/1 One tag-in-clear
GigabitEthernet0/0/1 One tag-in-clear
MACsec Enabled Interface Receive SC VLAN
-----------------------------------------------------
GigabitEthernet0/0/1.10 : 8 10
R2#show macsec status int gi0/0/1.10
Capabilities:
Validate Frames: Strict
Ciphers Supported: GCM-AES-128 GCM-AES-256
Include SCI: Yes
Cipher: GCM-AES-128
Confidentiality Offset: 0
Transmit SC:
SCI: 0022BDEF43830014
Transmitting: TRUE
Transmit SA:
Next PN: 1712
Receive SC:
Receiving: TRUE
Receive SA:
In Use: TRUE
Next PN: 1731
page 36Cisco White Paper
Monitoring and Troubleshooting
You can use the following show commands to monitor MKA session details:
• show mka sessions
• show mka sessions detail
• show mka sessions interface [interface] port [port] detail
• show mka policy [MKA policy name]
MONITORING MKA SESSIONS: SAMPLE OUTPUT
Figure 8 Show MKA sessions, 1 MKA session with 5 MACsec peers
You can use the following debug commands to collect debug information:
• debug mka [events | errors | packets] Use: Troubleshooting MKA session bring-up issues
• debug mka linksec-interface Use: Troubleshooting MKA keep-alive issues
• debug platform software macsec [error | info | detail | none] Use: MACsec info/error debugging
page 37Cisco White Paper
Monitoring and Troubleshooting
Example syslog messages%MKA-5-SESSION_START: MKA Session started
%MKA-5-SESSION_SECURED: MKA Session was secured
%MKA-4-SESSION_UNSECURED: MKA Session was stopped
%MKA-5-SESSION_STOP: MKA Session stopped by MKA
%MKA-5-CAK_REKEY: MKA Session is beginning a CAK Rekey
%MKA-6-CAK_REKEY_SUCCESS: MKA Session CAK rekey is successful
%MKA-6-SAK_REKEY: MKA Session is beginning a SAK Rekey
%MKA-4-SAK_TRANSMIT: Installing new TxSA
%MKA-6-SAK_REKEY_SUCCESS: MKA Session successfully completed a SAK Rekey
%MKA-4-MKA_MACSEC_CIPHER_MISMATCH: Lower strength MKA-cipher than macsec-cipher
%MACSEC-6-RX_SC_EXCEED: RX SCI %x : TX SCI %x : vport %d : secy vport %d
%MACSEC-6-TX_SA_PN_EXPIRE: (TX SCI %x : AN %d) TX SA AN about to expire
page 38Cisco White Paper
Network Design Considerations When Leveraging MACsec over Ethernet Service Offerings
Network Design Considerations When Leveraging MACsec over Ethernet Service OfferingsOne critical aspect of using Ethernet encryption is its ability to create flat network topologies, specifically for connecting core routers. This is not recommended. Flat Ethernet transport networks can cause many problems and shortfalls when you are designing a core backbone, and you should take caution as to how the network is designed. Ethernet has an inherent broadcast element, so you might view this as an advantage (because 10s or 100s of routers can have a single point of access into this Ethernet cloud) and leverage MACsec for encryp-tion. But a problem is created with N-1 concept (N being the number of routers are attached to the flat Ethernet domain).
Figure 9 Router peering model for E-LAN (example 1)
EthernetTransport
Physical View Logical ViewPE1 PE2
PE4 PE3
PE1
Routers areAttached to“Flat” BridgeDomain
PE2
PE4 PE3
FlatEthernetBridgeDomain
60
35
F
In Figure 10, each PE router has a single physical attachment into the bridge domain. Each router creates a rout-ing adjacency (an IGP or BGP) with every other router on the network, so applying N-1, each router establishes three routing adjacencies. Now, consider 100 routers attached to the same bridge domain, and again, applying N-1 routing adjacencies. Each PE now contains 99 IGP/BGP routing adjacencies. This flat network model poses many significant challenges, including:
• The N-1 maintenance of routing protocol adjacencies.
• The inability to efficiently prune routers from a given multicast tree.
The lack of state for QoS on SLAs for bandwidth being sent per peer is because the bandwidth is shared at the access links (example: 99 sites can send to 1, creating a major over-subscription factor for the access link).
page 39Cisco White Paper
Network Design Considerations When Leveraging MACsec over Ethernet Service Offerings
Figure 10 Router peering model view for E-LAN (example 2)
PE1 PE2
PE4 PE3
EthernetVirtualCircuitService
Ethernet Sub-interfacewith 802.1Q Support
802.1Q Trunk
EthernetTransport
PE1 PE2
PE4 PE3
Appears as Routersare Peering OverSecure Ethernet Wire
Ethernet Sub-interfacewith 802.1Q Support
Physical View Logical View
60
36
F
For core link interconnect, the recommended approach (Figure 11) is to leverage a point-to-point Ethernet ser-vice that allows the deployment of a peering model and is supported with MACsec link encryption. This peering approach offers several key advantages over the flat multipoint model, including:
• Much better scaling from a routing peer perspective.
• Deterministic QoS, because an SLA can be applied per Ethernet VC, quantifying the amount of bandwidth per logical connection.
• Deterministic traffic shaping per logical connection, eliminating the overrun of a particular site (based on how the logical connections are sized).
As previously discussed, the Ethernet encryption concept offers advantages over IPsec and overlay networks, but from an IP design perspective, there are scaling vulnerabilities lurking if the designer is not cautious.
Assuming the customer/agency is leveraging these various Ethernet services along with MACsec in order to gain the benefits of a more simplistic deployment, a hybrid approach can be taken. The benefits of the hybrid ap-proach include:
• A hierarchical network design that allows massive scale for routing.
• Leveraging hierarchy to segment core from aggregation.
• Leverage both point-to-point and multipoint Ethernet services.
• Use point-to-point Ethernet services in the core, while leveraging multipoint service at the access/aggrega-tion back-haul.
page 40Cisco White Paper
Network Design Considerations When Leveraging MACsec over Ethernet Service Offerings
Figure 11 Flat Ethernet E-LAN multipoint service
Regional POP
EthernetTransport
60
30
F
In Figure 12, there is a mix of small and large sites, most regionally located, sharing the same flat Ethernet ser-vice. This design poses many of the challenges mentioned above when leveraging a flat Ethernet service, and in this example, there are 24 total routers sharing the Ethernet service, so each router contains 23 routing adjacen-cies. In addition, there is no deterministic nature for how traffic is sent to a given site, and none of the routers on the shared Ethernet domain will ever have the awareness of senders and receivers of traffic, so it is easy to over-run the access circuit from CPE to Cloud.
One approach to overcoming these challenges, and building the foundation for a massively scalable network de-sign, is to take a hierarchical approach.
page 41Cisco White Paper
Network Design Considerations When Leveraging MACsec over Ethernet Service Offerings
Figure 12 Hybrid Ethernet E-LAN multipoint service/Ethernet point to point service
Regional POP
Ethernet “Point-to-Point” Service (E-LINE)
Ethernet “Multipoint” Service (E-LAN)
60
22
F
The hybrid model demonstrated in Figure 13 is leveraging the same routers and locations as in Figure 12, but the hybrid model applies a hierarchical design while leveraging both point-to-point and multipoint Ethernet service capabilities. The locations are split into regional PoPs, which dedicate a bridge domain and core router. Multi-point Ethernet service is leveraged for the back-haul of remote-site/branch locations, and a point-to-point E-Line service is used for core router interconnects. This approach allows deterministic QoS within the core and sim-plicity of multipoint for back-haul, while limiting the number of routing adjacencies needing to be maintained on each router. In Figure 12, in contrast, the peering model is N-1.
In the hybrid model, the maximum routing adjacency is seven (versus 23 in the flat model), but more importantly, this design offers massive scalability and can leverage common core technologies such as MPLS traffic engi-neering for bandwidth management, L2/L3 VPN services, and rapid convergence to name a few.
When you are leveraging Ethernet and MACsec, understanding the impact the Ethernet service can have on the overall design is critical to choosing the appropriate transport service.
page 42Cisco White Paper
WAN MACsec Best Practices
WAN MACsec Best Practices • Ensure basic Layer 2 Ethernet connectivity is established and verified before attempting to enable MACsec.
(Example: a basic ping between the CEs).
• When configuring for the first time, ensure that you have out-of-band connectivity to the remote site in order to avoid locking yourself out after enabling MACsec if the session fails to establish.
• It is recommended that you configure access-control should-secure while enabling MACsec for the first time and subsequently remove the CLI to change to default “Must secure” after the session establishment is successful, unless it is needed for migration cases as mentioned in “Point-to-Point SA Configuration—Mix of MACsec and Non-MACsec Spokes, VLAN-based E-Line Service (P2P).”
• It is recommended that you configure an interface MTU adjusting for MACsec overhead, ~32 bytes. Although MACsec encryption/decryption occurs at the PHY level and MTU is not an issue for the source or destination router, it may be an issue for the intermediate SP router. Configuring an MTU value at the interface allows for MTU negotiation that includes MACsec overhead.
page 43Cisco White Paper
Caveats and Restrictions
Caveats and Restrictions • macsec dot1q-in-clear and macsec access-control [must-secure | should-secure] can only be config-
ured on the main interface, and the setting is automatically inherited by the sub-interfaces. Due to hardware restriction, this behavior cannot be changed.
• mka policy, macsec replay-protection-window and eapol destination-address can be configured on the main and/or sub-interface and the value is automatically inherited by the sub-interfaces when configured on the main interface. Explicit configuration on the sub-interface overrides the inherited value or policy for that sub-interface.
• mka pre-shared-key key-chain [name of key-chain] For port based sessions, pre-shared-key should be configured on main interface only. For VLAN based sessions, pre-shared-key should be configured on sub-interface(s) only. If the pre-shared-key is configured on the main interface, it will not be inherited to the sub-interface(s).
FEATURE LIMITATIONSMACsec with EtherChannel (link bundling) is not supported.
• MACsec configuration on EtherChannel is not supported.
• MACsec-configured interfaces cannot be part of EtherChannel.
page 44Cisco White Paper
Glossary
GlossaryCA connectivity association
CAK connectivity association key
CE router customer edge router
CPE customer premise equipment
DCI data center interconnect
EAPoL Extensible Authentication Protocol over LAN
EP-LAN Ethernet private local-area network
EPL Ethernet private line
EVC Ethernet virtual circuit
EVP-LAN Ethernet virtual private local-area network
EVPL Ethernet virtual private line
GE gigabit Ethernet
GRE generic routing encapsulation
IP Internet protocol
IPsec Internet protocol security
L2TP Layer 2 tunneling protocol
LLDP link layer discovery protocol
MACsec media access control security
MAN metropolitan-area network
MEF Metro Ethernet Forum
MetroE Metro Ethernet
MKA MACsec key agreement
MPLS multiprotocol layer switching
MTU maximum transmission unit
NIST National Institute of Standards and Technology
P router provider router, sometimes referred to as P
P2MP point to multipoint
PE router provider edge router, sometimes referred to as PE
PoP point of presence for a service provider
QoS quality of service
SAK security association key
SA security association
page 45Cisco White Paper
Glossary
SC secure channel
SGT security group tag
UDP user datagram protocol
VRF virtual routing and forwarding
WAN wide-area network
Americas HeadquartersCisco Systems, Inc.San Jose, CA
Asia Pacific HeadquartersCisco Systems (USA) Pte. Ltd.Singapore
Europe HeadquartersCisco Systems International BV Amsterdam,The Netherlands
Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco Website at www.cisco.com/go/offices.
ALL DESIGNS, SPECIFICATIONS, STATEMENTS, INFORMATION, AND RECOMMENDATIONS (COLLECTIVELY, “DESIGNS”) IN THIS MANUAL ARE PRESENTED “AS IS,” WITH ALL FAULTS. CISCO AND ITS SUPPLIERS DISCLAIM ALL WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THE DESIGNS, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. THE DESIGNS ARE SUBJECT TO CHANGE WITHOUT NOTICE. USERS ARE SOLELY RESPONSIBLE FOR THEIR APPLICATION OF THE DESIGNS. THE DESIGNS DO NOT CONSTITUTE THE TECHNICAL OR OTHER PROFESSIONAL ADVICE OF CISCO, ITS SUPPLIERS OR PARTNERS. USERS SHOULD CONSULT THEIR OWN TECHNICAL ADVISORS BEFORE IMPLEMENTING THE DESIGNS. RESULTS MAY VARY DEPENDING ON FACTORS NOT TESTED BY CISCO.
Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.
© 2016 Cisco Systems, Inc. All rights reserved.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)
Cisco White Paper
Please use the feedback form to send comments and suggestions about this guide.
B-00940C-1 09/16
Recommended