Single Consistent Scalable TERALYNX Architecture...Single Consistent Scalable TERALYNX Architecture 7 Scalable TERALYNX Product Family 10 Summary 12 Innovium, born in Cloud Era, delivers

Single Consistent Scalable TERALYNX Architecture

WHITEPAPER

© Copyright 2019, Innovium, Inc. All rights reserved.

Rise of Edge Computing and Edge Data Centers 6

Single Consistent Scalable TERALYNX Architecture 7

Scalable TERALYNX Product Family 10

Summary 12

Innovium, born in Cloud Era, delivers Industry leading Data Center networking 3

Overview

• Production TERALYNX 7 for Modern Data Centers

• Introducing TERALYNX 5 for ToR, Enterprise and Edge


Innovium, born in Cloud Era, Delivers Industry leading Data Center Networking

There has been massive secular growth of data centersin the recent past, driven by economics, agility andflexibility. Cloud companies such as AWS, Microsoft,Facebook, Alibaba, Tencent are experiencing massiveyear-over-year growth. Cisco Cloud Index highlights thehuge growth being experienced in enterprise and serviceprovider cloud data centers. Further, data centers arealso expanding for private/hybrid cloud and edgedeployments.

Growth of Data Centers Data centers have many common traits:

§ Data centers typically have large scale with thousands

of servers or more

§ They run highly distributed applications and IP based

storage

§ They host applications & workloads which are dynamic

and unpredictable

§ All have AI workloads that are growing and becoming

more pervasive

Figure 1. Data Center Network Trends & Requirements

Trends Requirements

Larger Scale

Highly-DistributedApplications + Storage

Dynamic, UnpredictableWorkloads

AI Everywhere Low Latency

Software Programmable

Deep Application Analytics

High Radix


These trends in the data center translate to the following keyrequirements in the network.

§ High bandwidth & radix: Network traffic is exploding inthese large-scale data centers, with massive growth ineast-west traffic driven by distributed micro-services basedapplications, Artificial Intelligence & Machine learning,NVMe-based IP storage and video. Customers need highbandwidth, high-radix and high-performance networkingsolutions.

§ Deep Application Analytics: As the scale of data-centerincreases, network operators have a tremendous need forreal-time telemetry, event driven analytics, andautomation to trouble shoot problems and simplifyoperations.

§ Software Programmable Infrastructure: Data centercustomers are looking for ways to future proof networkhardware infrastructure using software programmability tosupport new network capabilities and protocols.

These customers were using legacy architectures & productsoriginally developed to address enterprise and service providernetworks. Further, with Moore’s Law slowing down, it was hardfor existing solutions to scale performance higher. Innovium wasborn in the cloud era with an intense focus on the data centerand not encumbered by legacy architectures. It assembled astrong, world-class data center network switch teamthat had delivered multiple generations of switching siliconpervasively deployed in data centers all over the world. Fueledby customer-focused innovation, the team continued itsexcellent engineering track record and delivered productionready TERALYNX 7 switch silicon in record time to address thegrowing data center networking challenges and requirements.

TERALYNX 7 for Modern Data Centers

Figure 2. Innovium TERALYNX 7 Switch Silicon Solutions

§ Low Latency: With increasing use of distributedapplications and networked storage, latency in each hop ofthe network is critical. Customers are looking for thelowest latencies in network infrastructure to boostperformance.

1st Gen TERALYNX 7 Silicon in Production

• Robust silicon, shipping in volume• 15+ Systems: Cisco & ODM products• Multiple Cloud deployments• Extensive PAM4 ecosystem validation

Proven & Comprehensive Software & SDK

• Multiple NOS ported on platforms, including Cisco NXOS and SONiC

• Comprehensive feature-set, including compelling telemetry

• Highly efficient warm-boot and fast-boot

1RU, 32-port 400G

4RU, 100/400G Switch

Hyperscale


TERALYNX 7: Innovium’s market leading TERALYNX 7programmable switch silicon delivers customers industryleading performance of up to 12.8Tbps with breakthroughtelemetry, low-latency, comprehensive features and highestport radix for 10G – 400G connectivity. It has made excellentprogress in the marketplace.

Hardware: Innovium’s TERALYNX 7 switch silicon is highlyrobust and is in production. It has been selected by leadingdata-center switch OEM, ODM and cloud providers for a rangeof deployment scenarios enabling superior infrastructure andROI. Multiple TERALYNX 7 based switches are available inconfigurations that include 1RU, 32x100/200/400G, 2RU,64x100/200G and 4RU, fixed & modular 128x100G. Innoviumand our ecosystem partners have jointly conducted extensiveinteroperability tests of TERALYNX 7 with an expansive set ofNRZ and PAM-4 DAC and ACC cables, optics modules, re-timer/gearbox, and network test equipment for 100G to 400Gto enable rapid time to deployment for customers.

Software: Innovium has proven and comprehensive softwareand SDK, with feature-set that includes rich tunneling, robustRoCE and unmatched real-time telemetry. Multiple OEM, Cloudand open-source NOS including Cisco NXOS & SONiC have beenseamlessly ported to TERALYNX leveraging Innovium’s softwareand SDK. The software supports standards-based CRUD APIs andis optimized for high performance, including warm and fastboot.


Lot of data is being created and collected at the edge ofnetworks that needs processing. These include environmentssuch as modern retail, industrial & factory automation,autonomous cars, 5G mobile devices and smart cities. Dataprocessing and analysis is being brought close to the source ofthat data to reduce

latency, minimize bandwidth cost for uploading that data to adata center, adhere to security/compliance regulations andmeet real-time expectations. This has led to emergence of edgecomputing, which is growing very rapidly.

Rise of Edge Computing and Edge Data Centers

Figure 3. Emergence of Edge Computing

The industry recognizes the benefits of cloud and data centertechnologies – cost-effective standardized server-storage-network infrastructure, cloud-native applications, agile DevOpsprocesses and automation. Edge computing has embracedcloud and data center technologies including networking, but ata smaller scale with focus on performance, power and cost.

Most enterprises today have standardized on hybrid cloudstrategy, where they augment their internal data centers withpublic cloud data centers. Many are also adopting the samesoftware and management tools

used in public cloud data centers, with solutions like AWSOutpost, Microsoft Azure Stack and Google Anthos.

As we see above, IT investments are increasingly moving to datacenters. Data centers exists in public cloud, privatecloud/enterprises and at the edge, including in 5G deployments.Networking requirements for all these data centers areconsistent, except for requirements for bandwidth/radix, powerand cost. The size and scale of the network varies based on thetype of data center and # of servers they host. All thesecustomers want a consistent architecture with same featuresand functionality.

Large-Scale Cloud Data Centers

Some apps moving

Edge Cloud

5G

Edge Computing

Use Cases

5G RAN

Retail

Autonomous Vehicles

Smart Cities& Stadiums

IoT & Factory Automation

Enterprise/ Private Cloud

Single Consistent Scalable TERALYNX Architecture


Innovium’s grounds-up TERALYNX architecture has beenenhanced to scale from 1Tbps to 51.2Tbps+ to meet thecustomer’s modern data center feature consistentrequirements ranging from edge to enterprise to the largestcloud data center operators. Ultra area-efficient and modulararchitecture with grounds-up data center optimizations enablesTERALYNX v2 architecture to deliver scalable performance andconsistent capabilities one process node earlier with time tomarket and cost advantage for customers.

Architectural Advantages

innovations, with over 70 patents filed or in process, span avariety of areas including data structures, algorithms andR&D flow.

2. Patented TERALYNX architecture is ultra silicon-areaoptimized. It helps deliver lower power, smaller silicon areawith larger on-chip buffers and lower cost. It also enablesInnovium to design the switch in one process node earlier,delivering time to market advantage and lower cost tocustomers. While slowing down of Moore’s law has mademulti-die solution more prevalent, that has not addressedthe higher power and costs associated with multiple large-die tape-outs in leading edge process nodes such as 7nmand 5nm.

3. The TERALYNX architecture is modular and extremelyscalable. It helps deliver products that scale from 1Tbps to51.2Tbps+ with the same consistent features andfunctionality, with no trade-offs and same software.

Figure 4. TERALYNX Architectural Advantages

Innovium TERALYNX architecture has three key fundamentaladvantages.

1. It has been architected from ground-up with a clean sheetdesign to address the most demanding needs of datacenter customers. That helps TERALYNX deliver acomprehensive feature set, lowest latency and superiortelemetry. Breakthrough patented

Ground-Up Design

• Customer-focused innovation• 70+ Patents• 2X+ better Latency

Silicon Area Optimized

• Highest performance with all features

• Higher power efficiency• Single die vs alternatives

Modular & Scalable

• Design principle• Delivers 12.8T with

full features• Scalable Product line


TERALYNX ArchitectureTERALYNX architecture consists of the following keycomponents:

§ Programmable feature-rich INNOFLEX™ ForwardingPipeline: A programmable forwarding pipeline architecturethat delivers comprehensive data center feature set withdeterministic low latency, and ability to support newprotocols through standards-based programming.

§ Highly Scalable TERASCALE™ Fabric: A scalable and low-latency switch & buffer fabric architecture that deliverscomprehensive QoS, superior traffic management, largesmart packet buffers and scales from 1 to 51.2Tbps+performance. TERASCALE has shown to deliver up to 2xperformance and buffering in the same silicon area vsalternatives.

§ Advanced FLASHLIGHT™ Telemetry & Analytics: A grounds-up telemetry architecture that delivers

extensive real-time visibility and actionable granular networkanalytics to troubleshoot and resolve network congestionproblems helping customers move towards higher-quality,autonomous networks.

Further, the architecture is flexible so that it works across single-chip and multi-chip system designs. All products in theTERALYNX family support 10/25G NRZ & 50G PAM4 SerDes toenable 10-400GbE connectivity. Also, all products in theTERALYNX family runs on the same set of APIs in the softwaredevelopment kit (SDK).

Comparison with Alternate Switch Silicon Architectures

Innovium’s TERALYNX architecture scales from 1Tbps to51.2Tbps+ with consistent features and functionality with nofeature trade-offs. However, alternate merchant siliconsolutions have had to resort to different architectures to meetcustomer needs as shown in figure 6. There are performanceoptimized architectures that deliver performance, but cannotdeliver features and functionality. Conversely, there are featureoptimized architectures that deliver features and functionality,but not performance requirements. Hence, alternate solutionsfall short on delivering consistent architecture, features andfunctionality for products with different performancerequirements.

Features (Deep Analytics, Software Programmable, DC Features)

Perf

orm

ance

(Rad

ix, T

hro

ugh

pu

t)

BandwidthFocused

Alternatives

FeatureFocused

Alternatives

Additional BenefitsUnique analytics

Superior Perf/$ and Perf/WLower latency

Scales from 1 to 51.2Tbps+

TERALYNXArchitecture

Figure 5. TERALYNX v2 Architecture

Figure 6. TERALYNX Architecture compared to Alternate Solutions


Customer BenefitsAs described above, the TERALYNX architecture providescustomers a single scalable architecture with these key benefits.

§ Most scalable and modular architecture that scales from1Tbps to 51.2Tbps+

§ Best performance/$ and performance/W for all products oneprocess node earlier

§ Operational and development efficiency for customers fromconsistent features and functionality across products withdifferent performance points, with no trade-offs and samesoftware APIs.

§ Superior FLASHLIGHT telemetry capabilities for autonomousnetworks

§ Lowest latency to deliver best application performance

§ 10-400G connectivity across entire TERALYNX family

§ Ease of programming and software leverage across products


Scalable TERALYNX Product Family

With the introduction of TERALYNX 5, Innovium extendsmarket leading TERALYNX architecture and product line todeliver breakthrough capabilities for TOR, Enterprise, Edgeand 5G data center applications.

It is an optimized data center switch silicon for 1.2 – 6.4Tbpsperformance, with up to 128x 10/25G NRZ & 50G PAM4SerDes, 10GbE to 400GbE ports, largest on-chip buffers,powerful analytics, and industry’s best performance per $ andperformance per Watt. It is the most advanced 10/25G NRZswitch silicon in the market, future-proofed with PAM4support, enabling seamless transition from NRZ to PAM4technology. TERALYNX 5 builds upon the patented,production-proven Innovium’s flagship 12.8Tbps TERALYNX 7products and extends Innovium’s product lines from ToR todata center core with a single consistent architecture for datacenters across public, private, hybrid cloud, enterprise andedge. TERALYNX 5 is fully feature & software compatible withTERALYNX 7.

Data analytics, AI, compute intensive applications and Flash-based storage are driving need for higher bandwidth fromservers and storage. Ethernet NICs are delivering multi-100Ginterfaces with PCI Express Gen 4 and PAM4 support.Customers are looking for cost and power-efficient ToR switchsilicon solutions to address their higher bandwidth needs withconsistent features and functionality as higher tiers of thenetwork.

Introducing TERALYNX 5 for ToR, Enterprise and Edge Data Centers

With TERALYNX 5, Innovium supports 10G – 400G connectivitywith NRZ & PAM4 technology, all the way to theserver/storage, enabling seamless insertion into existingnetwork infrastructure.

TERALYNX 5 provides breakthrough capabilities for ToR,Enterprise, Edge and 5G deployment scenarios. They include

§ Largest on-chip buffers in a switch of its class to deliverbest-in-class network quality

§ Robust RoCE for lowest latency and rich QoS necessary fordistributed storage and AI applications

§ Leading table sizes & ACLs needed at the access layer andedge

§ Lower power needed in these space constraineddeployments

§ IEEE 1588 timing synchronization required in 5G and otherdata center applications

Figure 7. TERALYNX 5: Most Advanced 1.2 -6.4Tbps switch silicon


§ FLASHLIGHT delivers actionable, real-time telemetry datacorrelated to applications, which is essential to monitor,troubleshoot and simplify network operations

§ Breakthrough cost that delivers 2X+ performance per $ vsalternatives

§ 10/25G NRZ & 50G PAM4 SerDes, needed to support 10-400GbE ports

§ SW programmability for support of new protocols, achievedwithout impact to throughput or latency suffered byalternatives

In summary, TERALYNX 5 is a highly disruptive product thatbrings a combination of NRZ & PAM4 connectivity to the massmarket with most advanced features at industry’s bestperformance/$ and performance/W. As shown in figure 8, it isideal for multiple use cases: ToRs, enterprise, edge and 5G datacenters

Figure 8. TERALYNX 5 Use Cases: ToR, Enterprise, Edge & 5G

Enterprise, Edge & 5G DeploymentsLarge-Scale Data Center

Rack Solution POD Solution

25G ToR 100G ToR

IP Storage / NVMeOF

ToR

25G ToR 100G ToR

100/200/400G Switch

ToR Solution

Innovium offers a single consistent scalable architecture for all data center switching solutions from top to bottom with unmatchedcapabilities and value. All TERALYNX products based on this architecture, deliver software programmability, advanced telemetry,low-latency and comprehensive rich data-center feature-set. While TERALYNX 7 is ideal for the most demanding customers thatneed high bandwidth & high radix switching solutions, TERALYNX 5 is ideal for ToR, Enterprise, Edge and 5G switching solutions.TERALYNX switches have been selected and being deployed by market-leading OEM and cloud providers for hyperscale, enterpriseand edge applications.

Summary


Innovium, Inc.6001 America Center Drive,San Jose, CA [email protected]

Documents

Single Consistent Scalable TERALYNX Architecture...Single Consistent Scalable TERALYNX Architecture 7 Scalable TERALYNX Product Family 10 Summary 12 Innovium, born in Cloud Era, delivers