Cavium, Inc. Confidential Information - Do Not Copy 1

White Paper

ABOUT IPSEC Internet Protocol Security (IPsec) is a network protocol suite that authenticates and encrypts data packets sent over a network. IPsec includes various protocols for establishing mutual authentication between hosts or gateways at the beginning of the session and negotiation of cryptographic keys to use during the session. IPsec can protect data flows between a pair of hosts (host-to-host), between a pair of gateways (network-to-network), or between a gateway and a host (network-to-host). LiquidIO Smart NICs efficiently secures and accelerates IPsec Protocol with all the various supported functions. IPsec uses the following protocols to perform various functions. a. Authentication Headers (AH) provides connectionless data integrity and data origin authentication for IP datagrams and provides protection against replay attacks.

b. Encapsulating Security Payloads (ESP) provides confidentiality, data- origin authentication, connectionless integrity, an anti-replay service (a form of partial sequence integrity), and limited traffic-flow confidentiality. IPsec can be implemented in a host-to-host transport mode, as well as in a network tunnelling mode and with LiquidIO, all these modes are accelerated to secure hosts within and across data centers. • Tunnel Mode: In tunnel mode, the entire IP packet is encrypted and authenticated. It is then encapsulated into a new IP packet with a new IP header. Tunnel mode is used to create virtual private networks for network-to-network communications (e.g. between routers to link sites), host-to-network communications (e.g. remote user access) and host-to-host communications (e.g. private chat).

[Fig 1: Tunnel Mode Packet Format] [2]

• Transport Mode: In transport mode, only the payload of the IP packet is usually encrypted or authenticated. The routing is intact, since the IP header is neither modified nor encrypted; however, when the authentication header is used, the IP addresses cannot be modified by network address translation, as this always invalidates the hash value.

[ Fig 2: Transport Mode Packet Format] [2]

• Optimizing server hardware utilization by offloading compute intensive security data processing to LiquidIO Smart NIC adapters

• 10/25GbE network capabilities and IPsec offloads into single PCIe adapter

• Comprehensive IPsec algorithms support

LiquidIO® IPsec Architecture

Cavium, Inc. Confidential Information - Do Not Copy 2

LiquidIO® IPsec Architecture White Paper

ABOUT LIQUIDIO SMART NIC The LiquidIO® Smart NIC family of intelligent adapters provide high performance and programmable server adapter solutions for various data centre deployments. LiquidIO Smart NIC adapters offer standard software packages supporting various offload solutions including IPsec offload, while enabling customers to customize the product for specific use cases. LiquidIO Smart NICs feature MIPS-based general-purpose processing cores. These are configured to run in the Cavium Simple Executive (SE) environment. Simple Executive is Cavium’s bare-metal application environment that leverages the underlying processing architecture to accelerate packet processing in a run-to-completion model.

LINUX IPSEC INTEGRATION WITH LIQUIDIOLiquidIO IPsec architecture utilizes Simple Executive environment for offloading IPsec data path while using Linux OS for configuration and management of IPsec control path.

[Fig 3: LiquidIO IPsec Block Diagram]

IPSEC CONTROL PATH IPsec control path consists of various components in user and kernel space. In user-space, Key Exchange Tools such as Internet Key Exchange (IKE), provides automated cryptographic key exchange and management mechanisms for IPsec tunnel establishment. And in kernel space, transform(NETXFRM), Inter-component Messaging (ICMSG) and LiquidIO Driver modules provide IPsec offloading capabilities.

IKE service in user space interacts with Linux Kernel IPsec implementation using NETLINK sockets. Using existing NETLINK interface (XFRM), kernel-mode IPsec requests IKE in user-space to create new Security Associations(SAs) or start negotiation and upon successful IKE negotiation the SA/Policy information is pushed into the kernel SA and Policy database (namely Security Association Database(SAD)/Security Policy Database (SPD)).

LiquidIO ModulesThe following modules update IPsec SAs and Policies from the control path to the SAD/SPD of LiquidIO firmware. LiquidIO NETXFRM Module This module intercepts NETLINK messages that are used to configure IPsec SAs and Policies in kernel XFRM framework. These messages are intercepted before they are given to original Linux handler functions of IPsec. The intercepted messages are then used to configure IPsec SAs and Policies in LiquidIO NIC firmware. LiquidIO ICMSG Module Inter-component Message Module acts as an interface between the LiquidIO NETXFRM module and LiquidIO Host Driver. This module receives the Intercepted/Parsed IPsec messages from NETXFRM module, builds commands and sends the commands to LiquidIO firmware through Host driver. Also, it handles the communication between NETXFRM module and LiquidIO Firmware in synchronous manner. IPSEC DATA PATH When LiquidIO NIC receives packets from wire (Outbound/Inbound) or host (Outbound), SA lookup is done in the firmware for both Outbound and Inbound packets. Tunnel mode traffic handling is different from that of transport mode because in tunnel mode packets are forwarded to x86 Host (slowpath) when there is route cache miss or unhandled packets are received.

Cavium, Inc. Confidential Information - Do Not Copy 3

LiquidIO® IPsec Architecture White Paper

Tunnel Mode: 1. Outbound Packets: Route cache lookup is performed to retrieve the MAC address of tunnel destination endpoint. If a route cache entry is found, packets are encrypted and forwarded out on wire in the firmware itself (fastpath). Otherwise the packets will be sent to host for encryption and forwarding (slowpath) until the route cache is updated.

2. Inbound Packets: Packets are decrypted in firmware. Route cache lookup is performed to retrieve the MAC address of destination IP in the inner IP header. If a route cache entry is found, packets are forwarded out on wire in the firmware itself (fastpath). Otherwise, packets are sent to host for forwarding (slowpath) until the route cache is updated.

Transport Mode:

1. Outbound Packets: Packets are received from x86 host. Upon successful SA lookup,

a. For TCP packets larger than MTU size, TSO is done.

b. For all TCP/UDP packets, L4 checksum calculation is done.

c. Packets are encrypted. If encrypted packets exceed MTU size, post-fragmentation is done.

d. IPsec packets are sent out on wire.

Cavium, Inc. Confidential Information - Do Not Copy 4

LiquidIO® IPsec Architecture White Paper

2. Inbound Packets: Packets are received from wire. Upon successful SA lookup, packets are decrypted and sent to Host.

Sysctl Parameters:

When IPsec policy and Security Association (SA) are present, disable_xfrm and disable_policy flags are used to disable IPsec processing in the x86 kernel network stack. disable_xfrm is used to bypass IPsec processing on outbound packets. disable_policy is used to bypass IPsec processing on inbound packets. disable_xfrm and disable_policy flags can be configured per interface, therefore, only incoming and outgoing traffic on the specific network interface will be affected.

Note: * means the default behaviour

SOFTWARE DEVELOPMENT KIT (SDK) LiquidIO Smart NICs support a feature-rich SDK that includes GNU tool-chain (compiler, debugger, and profiling tool). Developers use the LiquidIO SDK to develop various specialized network and security offloads available with LiquidIO Smart NICs including OVS and IPsec. The SDK is flexible, allowing customers and partners to develop additional custom and proprietary application-specific offloads for their data center needs or to add new features to Cavium’s production-quality software packages. SDK also includes out-of-box applications in binary and source packages. Source packages can be modified for customizations or for adding extra proprietary features to the existing packages. Well-documented APIs are available as part of SDK to develop custom applications.

CONCLUSION Cavium IPsec software provides the ability to accelerate the IPsec processing by using LiquidIO Smart NICs. Regardless of the IPsec mode, IPsec encryption/decryption processing can be offloaded to LiquidIO Smart NICs. LiquidIO Smart NICs is the best Cloud VPN gateway solution by accelerating complex packet processing and preserving CPU cycles, allowing more VMs or Applications to be deployed on the server.

REFERENCES1. https://tools.ietf.org/html/rfc43012. http://images.techhive.com/images/idge/imported/article/nww/2007/04/06fig02-100299775-orig.jpg

disable_xfrm/

disable_policy

Transport Inbound

Transport

Outbound

Tunnel

Inbound

Tunnel

Outbound

0/0Rejects

decrypted packets*

Do IPsec processing

Rejects decrypted packets*

Do IPsec processing*

1/0Rejects

decrypted packets*

No IPsec processing

Rejects decrypted packets*

No IPsec processing

0/1Accepts

decrypted packet

Do IPsec processing*

Accepts decrypted

packet

Do IPsec processing*

1/1Accepts

decrypted packet

No IPsec processing

Accepts decrypted

packet

No IPsec processing

LiquidIO® IPsec Architecture White Paper

Corporate Headquarters Cavium, Inc. 2315 N. First Street San Jose, CA 95131 408-943-7100

Follow us:

2018 Cavium, Inc. All Rights reserved. NITROX and OCTEON are registered trademarks of Cavium, Inc. All rights reserved worldwide. QLogic Corporation is a wholly owned subsidiary of Cavium, Inc. Cavium, QLogic are registered trademarks or trademarks of Cavium Incorporated, registered in the United States and other countries. All other brand and product names are registered trademarks or trademarks of their respective owners.

This document is provided for informational purposes only and may contain errors. Cavium reserves the right, without notice, to make changes to this document or in product design or specifications. Cavium disclaims any warranty of any kind, expressed or implied, and does not guarantee that any results or performance described in the document will be achieved by you. All statements regarding Cavium’s future direction and intent are subject to change or withdrawal without notice and represent goals and objectives only.

Security Offloads* • IP Security (IPsec) IPsec Offload* • Encryption Algorithms: NULL, DES-CBC, 3DES-CBC, AES (128, 192, 256)-CBC & AES (128, 192, 256)-CTR • Authentication Algorithms: NULL, HMAC-MD5, HMAC-AES- XCBC, HMAC-SHA1, HMAC-SHA (256, 384, 512) • AEAD Algorithms: AES (128, 192, 256)-GCM • IKE Compliance: v1 and v2 Stateless Offloads* • TCP segmentation offload (TSO) • Large send offload (LSO) • Receive side scaling (RSS) • Large receive offload (LRO) Overlay Network Offloads* • Virtual Extensible LAN (VXLAN) • Network Virtualization using Generic Routing Encapsulation (NVGRE) • Generic Network Virtualization Encapsulation (GENEVE)

* Any offload functionality is fully programmable by the customers.

Operating Systems/Distributions • CentOS, Ubuntu®

I/O Virtualization • SR-IOV • 2 physical functions (PFs) • 126 virtual functions (VFs) Ethernet Frame • Jumbo Frame Support Bus Interface • PCIe® Gen3 x8, Gen2 x8 (electrical) Host Interrupts • MSI/MSI-X On-board Memory • 4/8GB DDR4 +ECC

Compliance IEEE Specifications • IEEE Std 802.3ae 10 Gigabit Ethernet • IEEE 802.3ad Link Aggregation and failover • IEEE 802.1Q.1p VLAN tags and priority • IEEE 802.1Qbb (PFC) • IEEE 802.1Qaz (ETS and DCBX) Boot Support • Preboot execution environment (PXE) • Unified extensible firmware interface (UEFI) Ports • SFP+(10GbE), 10GBase-T and SFP28 (25GbE) • Dual SATA ports (Gen3)

FEATURE SET