Embed Size (px)
DESCRIPTION
RoCEv2 is an extension of the original RoCE specification announced in 2010 that brought the benefits of Remote Direct Memory Access (RDMA) I/O architecture to Ethernet-based networks. RoCEv2 addresses the needs of today’s evolving enterprise data centers by enabling routing across Layer 3 networks. Extending RoCE to allow Layer 3 routing provides better traffic isolation and enables hyperscale data center deployments. Watch the video presentation: http://insidehpc.com/2014/09/slidecast-ibta-releases-updated-specification-rocev2/
Citation preview
InfiniBand Trade Association
NEWS UNDER EMBARGO UNTIL
TUESDAY, SEPTEMBER 16 at
6:00AM PACIFIC TIME
RoCEv2 Specification Release
• Extends capabilities of original RoCE
specification announced in 2010
• Addresses needs of evolving enterprise data
centers moving to hyperscale networking
environments
IBTA Announces
RoCEv2 Specification
© InfiniBand Trade Association 2
Global member organization dedicated to developing,
maintaining and furthering the InfiniBand specification
• Architecture definition
– RDMA software architecture
– InfiniBand, up to 56Gb/s and 168Gb/s per port
– RDMA over Converged Ethernet (RoCE)
• Compliance and interoperability testing of commercial products
• Markets and promotes InfiniBand/RoCE
– Online, marketing and public relations engagements
– IBTA-sponsored technical events and resources
InfiniBand Trade Association
(IBTA)
© InfiniBand Trade Association 3
• Remote Direct Memory Access
• The movement of data between servers’ user
space without CPU involvement
– Traditional data movement utilized TCP/IP, many
copies, and significant CPU overhead
– RDMA utilizes hardware offloads to move data faster
with less overhead
What is RDMA?
4 © InfiniBand Trade Association
Without Remote Direct Memory Access
5 © InfiniBand Trade Association
Server - Initiator
Sockets
NIC Driver Buffer
Transport
Protocol Driver Buffer
NIC Driver Buffer
Transport
Protocol Driver
Sockets
Application
Server - Target
NIC Buffer NIC Buffer
Application
Buffer
Buffer
Buffer
Buffer Buffer
With Remote Direct Memory Access
6 © InfiniBand Trade Association
Server - Initiator
Sockets
NIC Driver Buffer
Transport
Protocol Driver Buffer
NIC Driver Buffer
Transport
Protocol Driver
Sockets
Application
Server - Target
RNIC Buffer RNIC Buffer
Application
Buffer
Buffer
Buffer
Buffer Buffer
Essential for evolving enterprise data center and hyperscale networking infrastructure
• I/O is central to achieving highest performance
• Efficient computing reduces power, cooling and space requirements
• OS bypass enables fastest access to remote data
• Scalable storage to meet growing demand
• Delivers direct access to data over the WAN
Benefits of RDMA
• Low latency and CPU overhead
• High network utilization
• Efficient data transfer
• Support for message passing, sockets and storage protocols
• Supported by all major operating systems
Why RDMA?
© InfiniBand Trade Association 7
• RDMA transport over Ethernet – Efficient, light-weight transport, layered directly over Ethernet
– Takes advantage of PFC (Priority Flow Control) in DCB Ethernet
– IBTA standard, supported in OFED 1.5.1
• Lowest latency in the Ethernet industry – 1.3µs end-to-end RDMA latency
– Enables faster application completion, better server utilization and higher scalability
• Tremendous support momentum by ecosystem – Cloud service providers, DB Vendors, Financial ISVs, Server &
Storage OEMs
– Entire Ethernet management ecosystem is available
RoCE
8 © InfiniBand Trade Association
• TCP wasn’t designed for RDMA
• RoCE supports lowest latency
• Adapter power comparable to plain NICs
Most Efficient RDMA over Ethernet
© InfiniBand Trade Association 9
• Extends RoCE functionality & scalability
• L3 routing through the use of UDP headers
• Transparent to applications and underlying network infrastructures – InfiniBand Architecture followed OSI model closely
– RoCEv2 only modified third layer
– Frames generated and consumed in the NIC (below the API)
• Enables standard network mechanisms for forwarding, management, monitoring, metering, accounting, firewalling, snooping, and multicast
RoCEv2 Specification Details
© InfiniBand Trade Association 10
Protocol Stacks, Side by Side
11 © InfiniBand Trade Association
Frame Format
12 © InfiniBand Trade Association
RDMA Use in the Data Center
© InfiniBand Trade Association
Scale up with Single Ethernet L2 Domain (from end node perspective)
L2 L2 L2
Plain L2 or TRILL or PBB or Virtualization Overlays
13
RDMA Use in the Data Center
© InfiniBand Trade Association
L2 L2 L2
14
RoCE Use Case – Microsoft Azure
© InfiniBand Trade Association 15
Microsoft Keynote
Albert Greenberg SDN in Azure Infrastructure
RoCE Use Case – Dell Fluid Cache SAN
© InfiniBand Trade Association 16
RoCE Use Cases – Zadara iSER
© InfiniBand Trade Association 17
RoCE Use Case – Applied Micro ARM
© InfiniBand Trade Association 18
Thank You
IBTA Members
© InfiniBand Trade Association 20
3M
Advanced Photonics, Inc.
Agilent Technologies, Inc.
Amphenol Interconnect Products
Annapurna Labs (US) Inc.
Anritsu
AppliedMicro
Bay Microsystems
Broadcom
Bull SAS
Cisco
ConnPro Industries Inc.
Cray, Inc.
Emulex
FCI USA Inc.
Finisar Corporation
Foxconn
Fujikura America, Inc.
Fujitsu Limited
Hewlett-Packard
Hitachi
IBM
Intel Corporation
Lamprey Networks, Inc.
Lawrence Livermore National Laboratory (LLNL)
Lorom Cable & System Design
LUXSHARE-ICT Co., Ltd.
Mellanox Technologies, Inc.
Mercury Systems
Meritec
Microsoft
Molex Inc.
NetApp
Netronome
Oracle
QLogic
SAE Magnetics (HK) Ltd.
Samtec
Semtech
Silicon Graphics, Inc.
Software Forge, Inc.
Sumitomo Electric Industries
TE Connectivity
Tektronix
The Siemon Company
Time Interconnect Ltd.
UNH InterOperability Lab
Unisys Corporation
Volex Inc.
Woodward McCoach, Inc.
Yamaichi Electronics USA
BOLD = Steering Committee member
