NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas

NETWORK-ON-CHIP (NOC):

A New SoC Paradigm

Dr. Konstantinos Tatas

PRESENTATION OUTLINE

Introduction

Part A

Motivation – SoC Communication

Current Solutions

NoC Concept

Part B

Work@MicroLab

Summary

THE MANY CORES ERA

Source: International Roadmap for Semiconductors 2007 edition (http://www.itrs.net/)

THE GROWING GAP: COMPUTATION VS. COMMUNICATION

Taken From ITRS, 2001

2:1

9:1

Design complexity - high IP reuse Efficient high performance interconnectScalability of communication architecture

GROWING CHIP DENSITY

1998ASIC - 0.35 m

2012SoC - 22nm

Memory, I/O

P

Variety of dedicated interfaces Poor separation between computation and communication. Design Complexity Unpredictable performance

TRADITIONAL SOC NIGHTMARE

The architecture is tightly coupled

DMA CPU DSP

MemCtrl. Bridge

MPEGI oo

The “Board-on-a-Chip” Approach

C

System Bus

Control Wires Peripheral Bus

COMPUTATIONAL DEMANDS OF FUTURE MULTIMEDIA APPLICATIONS - MEMORY BANDWIDTH SCALES PROPORTIONAL

Source: K. Uchiyama., VLSI Circuit Digest of Technical Papers, p 6, 2008.

BW~

20

12

-20

15

tod

ay

K. Uchiyama., “Power-Efficient Heterogeneous Parallelism for Digital Convergence”, VLSI Circuit Digest of Technical Papers, IEEE p 6-9, June 2008Jian Li, “3D Integration opportunities and challenges”, ISCAS 2008 tutorial on 3D

SHARED ADDRESS SPACE COMMUNICATIONS

SYSTEM BUS

CROSS-BAR

MULTI-STAGES NETWORK ON CHIP

AN NOC EXAMPLE

•Source: ossum, Intel @ MPSoC’07

Regular topologies: general-purposed on-chip multiprocessors

Custom topologies:

NOC TOPOLOGIES

NOC VS. “OFF-CHIP” NETWORKS

What is Different?

Routers on Planar Grid Topology

Short Point-To-Point Links between routers

Unique VLSI Cost Sensitivity:

Area-Routers and Links Power

M odule

M odule M odule

M odule M odule

M odule M odule

M odule

M odule

M odule

M odule

M odule

No legacy protocols to be compliant with …No software simple and hardware

efficient protocolsDifferent operating env. (no dynamic

changes and failures)






Custom Network Design – You design what you need!






M odule

M odule M odule

M odule M odule

M odule M odule

M odule

M odule

M odule

M odule

M odule

Replace

M odule

M odule M odule

M odule M odule

M odule M odule

M odule

M odule

M odule

M odule

M odule

Example1: Replace modules





M odule

M odule M odule

M odule M odule

M odule M odule

M odule

M odule

M odule

M odule

M odule

Example2: Adapt Links

Adapt Links

M odule

M odule M odule

M odule M odule

M odule M odule

M odule

M odule

M odule

M odule

M odule


•Compare the cost of:NoCNon-Segmented Bus (NS-Bus) Segmented Bus (S-Bus) Point-To-Point (PTP)

NOC COST SCALABILITY VS. ALTERNATIVES

Bus NoC

Longer connections

higher parasitic

capacitance

Performance does not

downgrade with network

scaling

Arbitration grows and

becomes a bottleneck

Arbitration and routing are

distributed

Bandwidth is limited and

shared by all cores

Aggregated bandwidth

scales with network size

Latency is wire-speed

once arbitration granted

control

Multiple hops increase

latency

Well-known and simple

concepts

Further study needed

WHY NOC?

Communication infrastructure

Communication paradigm selection

Application mapping optimization

Programming model

Physical design

Design automation/tool-flow integration

WHICH ARE THE MAIN CHALLENGES?

Circuit Switching A real or virtual circuit establishes a direct connection between source and destination.

Packet Switching Each packet of a message is routed independently. The destination address has to be provided with each packet.

Store and Forward Packet Switching The entire packet is stored and then forwarded at each switch.

Cut Through Packet Switching The flits of a packet are pipelined through the network. The packet is not completely bu ered in each switch.ff

Virtual Cut Through Packet Switching The entire packet is stored in a switch only when the header flit is blocked due to congestion.

Wormhole Switching is cut through switching and all flits are blocked on the spot when the header flit is blocked.

BASIC SWITCHING TECHNIQUES

Phases:Circuit SetupTransmissionTear Down

Disadvantages:Exclusive allocation of resourcesLong setup phase

Advantages:High performance - throughput and latencyLow power consumptionLow overhead during transmission phasePredictable transmission

CIRCUIT SWITCHING (ARE THEY NOC?)

PACKET SWITCHING VS CIRCUIT SWITCHING

NOC ROUTER

NoC-based MPSoC

• nodes– Processing Elements (PEs),

such as CPUs, custom IPs, DSPs, etc.

– storage elements (embedded memory blocks),

• Routers• Links• Network Interfaces (NIs)• Often a switch together

with its host node memory is referred to as a tile.

NoC Topologies

• Regular/irregular

• Direct/indirect– each node has a direct point-to-point link to a

subset of other nodes in the system, called neighboring nodes

2D Mesh

IP IP IP

IP IP IP

IP IP IP

R R R

R R R

R R R

•simplest and most popular topology for NoCs.

•Every switch, except those at the edges, is connected to four neighboring switches and one node.

2D Torus

IP

R

IP

R

IP

R

IP

R

IP

R

IP

R

IP

R

IP

R

IP

R

IP

R

IP

R

IP

R

IP

R

IP

R

IP

R

IP

R•layout of a regular mesh except that nodes at the edges are connected to switches at the opposite edge via wrap-around routing channels.

•Every switch has five ports

• The limitation of this topology affects the long end-around connections

Octagon

IP

IPIP

IPIP

IP

IPIP

RR

R

R

R

R

RR

•well-established direct topology found in NoCs.

•ring of 8 nodes connected by 12 bi-directional links.

•links provide two-hop communication between any pair of nodes in the ring

•simple algorithms for fast yet efficient shortest-path routing.

•In case a platform consists of more than eight nodes, the octagon is extended to multidimensional space

Fat-tree and butterfly fat-tree• nodes are connected to an

architecture's external switch • switches have point-to-point links

to other switches. • processing units and memory

modules are assigned to the leafs of the trees,

• switches are placed at the vertices,

• communication involves climbing up and down some part of the tree.

• A pair of coordinates is used to label each node, ($l$, $p$), where $l$ denotes a node's level and $p$ gives its position within this level.

R

IP IP

R

IP IP

R

IP IP

R

IP IP

R

R R

IPIPIP IPIPIP IPIPIP IPIPIP

R

R R R R

R

Polygon• widely accepted

topology• packets travel in a loop

from one router to the next.

• We can add chords to the circle

• if chords are inserted only between opposite routers, the topology is called a spidergon.

R

IPIP

R

R R

R R

IPIP

IP IP

Star

• central router in the middle of the star, • computational resources, or subnetworks, in the spikes of the star. • The capacity requirements of the central router are quite large, • significant possibility of congestion in the middle of the star

IP

IP

IPIP

IP

R COREIP

R R

R R

IP

IP

IP

Flow Control

• intra-switch

• switch-to-switch– Buffered– Bufferless

• end-to-end

ACK/NACK• handshaking protocol • When a sender puts data on

the link, it activates a VALID signal.

• When the receiver is ready to consume the valid data, it activates the corresponding ACK signal.

• If the data is corrupt or there is no buffer space to store them, a NACK signal is activated instead.

• Upon receipt of a NACK, the sender starts resending flits starting from the not acknowledged one

• inherently supports fault tolerance,

• additional buffer space required to keep sent flits in case retransmission is required.

Stall/go

• requires just two control wires

• one going forward, signifying data availability,

• one going backward and signaling either a condition of buffers filled ("STALL") or of buffers free ("GO")

Credit-based

• transmitter has a "credit" counter • initialized to the value of empty buffer slots of the receiver • decrements it every time a flit is sent. • The credit counter must be updated in case the receiver consumes or

forwards a flit and therefore increases its buffer space. • a credit value that is sent back to the transmitter to be added to the current

value of the credit counter. • transmitter stalls when the credit value is zero and • resumes when its value increases again.

NI Design• logic required to connect the nodes to the NoC. • NIs can differ significantly depending on the nature of the

node• Using a NI allows IPs and communication infrastructure

to be designed independently• One end of a NI is connected to a router using the

selected flow control protocol • the other to the node IP • Since most IPs are designed to communicate through a

bus, the NI uses a bus interface • NI is not simply a protocol adapter from a processor bus

to a router port. • Ideally, the NI must offer the processing cores the view

of a shared memory system, and the network itself should be transparent.

NI services

• adaptation services – packetization/depacketization– protocol conversion and clock domain crossing. – absolute minimum services required of the NI so that data can

be sent and received on the NoC• transaction reordering services, • error and flow control services

– error detection and/or correction – request retransmission when required

• route computation services– Source routing

• upper layer services– Cache coherence

Typical NoC Packet Format

• Header– routing and network control information. – In the case of distributed routing the information required is the destination and source

addresses – in the case of source routing the complete routing information is written– In the case of variable packet size a length field is required

• Payload• Tail

– sequence number– error control fields such as hamming code or CRC fields

Header Tail

Packet type

SA DA

Routing

OR

SAHOP

0HOP

1 . . . HOPN

Length SNError

Control

Payload

Control Address Data

Source vs Distributed Routing

• In source routing the entire routing path is computed at the source and appended to the packet. – The routers do not make any routing decisions,

• in distributed routing, the routing path is decided in a hop-by-hop basis at each router even for deterministic routing algorithms. – The only information required to be found in the packet is the

destination address.

• The advantage of source routing is that it requires simple routers and can easily support irregular architectures. Its disadvantage is that it does not provide adaptiveness and requires more complex NIs and packets.

Source vs Distributed Routing

R R R R

R R R R

R R R R

R R R R

(0,1) (1,1) (2,1)

(2,2)

(0,1) (1,1) (2,1) (2,2)

E E S PE . . . SR principle

. . . (2,2)

DA

DR principle

Documents

NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas