An Overview of Chiplet Technology for the AMD EPYC and

An Overview of Chiplet Technology

for the AMD EPYC™ and Ryzen™

Processor Families

Gabriel Loh

August 24th, 2021

IEEE SCV – Industry Spotlight

2 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight

Insatiable Performance Demands

1995 2000 2005 2010 2015 2020

1 exaFLOP

1 petaFLOP

0.1

1

10

100

1000

10000

Apr-18 Oct-18 May-19 Dec-19 Jun-20 Jan-21

Para

mte

rs (

Billions)

Number of ML Model Parameters

Switched

Transformers

GPT-3

T-NLG

T5Megatron

GPT-2

BERT-LargeGPT

[1]

[1] Based on published parameter counts of leading training models. [2] https://openai.com/blog/ai-and-compute

Required compute doubling every ~3.4 months [2]


Technology Challenges

2004 2006 2008 2010 2012 2014 2016 2018 2020

65nm45nm

32nm22nm

14nm10/7nm

90nm


Technology Challenges

Also…

◢Non-recurring costs per SoC

◢ Engineering design

◢ Verification

◢ Mask sets

Cost Per Yielded mm2 for a 250mm2 Die

-

1.0

2.0

3.0

4.0

5.0

6.0

45nm 32nm 28nm 20nm 14nm 10nm 7nm 5nm

Norm

aliz

ed

cost p

er

yie

lded

mm

2


Monolithic Die Manufacturing

Wafer





Yielded Processors


High-level Chiplets Concept

Wafer





More Yielded Processors


Chiplet Cost

Silicon cost is non-linear

with die area


Chiplet Overheads

◢ Inter-chiplet communication interfaces

◢Per-die functionality


Chiplet Overheads

◢Architectural design effort, partitioning

> X

◢ Inter-chiplet communication interfaces

◢Per-die functionality


Case Study:

AMD EPYC™ Processors


1st-gen AMD EPYC™ Product Targets

◢Up to 32 “Zen” CPU cores

◢Eight DDR4 memory channels

◢128 lanes PCIe® gen4 I/O

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

DD

RD

DR

DD

RD

DR

DD

RD

DR

DD

RD

DR

I/O I/O I/O I/O

I/O I/O I/O I/O

Monolithic 32-core Chip

777mm2 total area in 14nm


1st-gen AMD EPYC™ MCM Organization

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

DDRDDR

DDR

DDRDDR

DDRDDRDDR

I/O

I/O

I/O

I/O

I/O

I/O

I/O

I/O

◢213mm2 per chiplet (14nm)

◢4x → 852mm2 total

+10% silicon area

0.59x cost


1st-gen 14nm chiplet

Moving to 7nm Technology

2x device density

+25% performance

or

50% power at 1x perf.

64-core possibility!


1st-gen 14nm chiplet


2x device density

Co

res, C

ach

eI/O

, D

DR



2x device density

Co

res, C

ach

eI/O

, D

DR

Cost Per Yielded mm2 for a 250mm2 Die

-

1.0

2.0

3.0

4.0

5.0

6.0

45nm 32nm 28nm 20nm 14nm 10nm 7nm 5nm

Norm

aliz

ed

cost p

er

yie

lded

mm

2


Technology-optimized Chiplet Organization

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

Four “Zen”

Cores+L3

DDRDDR

DDR

DDRDDR

DDRDDRDDR

I/O

I/O

I/O

I/O

I/O

I/O

I/O

I/O



Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

DD

RD

DR

DD

RD

DR

DD

RD

DR

DD

RD

DR

I/O

I/O

I/O

I/O

I/O

I/O

I/O

I/O Move cores and

cache to 7nm

Ke

ep

in

ma

ture

12

nm

pro

ce

ss

86% of die area benefits



Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

DD

RD

DR

DD

RD

DR

DD

RD

DR

DD

RD

DR

I/O

I/O

I/O

I/O

I/O

I/O

I/O

I/O

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3

Four “Zen 2”

Cores+L3


Packaging Challenges

◢Chiplets can help address silicon challenges,

but can also introduce other new challenges

to overcome

◢Example: package routing already fully

utilized

◢ Package layers in 1st-gen processor used for

power delivery to CCD

CC

DC

CD

CC

DC

CD

CC

DC

CD

CC

DC

CD

IOD


Solution: Silicon-Package Co-design

◢Metal resources below

chiplet previously

consumed by power

distribution

◢Coordinated re-design of

both L3 VDDM LDOs and

package metal/routing

resources enabled solution

◢Solution achieved VDDM

IR drop within 10mV

◢Take-away: advanced

chiplet design requires an

increasingly vertical and

multi-domain co-design

approach

Packag

eR

DL


2nd-gen AMD EPYC™ Benefits

0.0

0.5

1.0

1.5

2.0

64 Cores 48 Cores 32 Cores 24 Cores 16 Cores

Norm

aliz

ed D

ie C

ost

Great cost benefit vs. monolithic

Makes 64 cores possible

Two tape-outs for full stack

Linear cost with core count

Full memory and IO

7nm CCD + 12nm IOD Hypothetical Monolithic 7nm


Additional Advantages of Technology-optimized Chiplet Organization

Rem

ote

Requ

est

Local Request

Remote Request

90ns memory latency

141ns memory latency

Average latency 128ns

(25% local, 75% remote)


Additional Advantages of Technology-optimized Chiplet Organization

I/O PHY I/O PHY

DD

R

DD

R

DD

R

DD

R

DD

R

DD

R

DD

R

DD

R

I/O PHY I/O PHY

IS

PCIe

PCIe

IS

PCIe

PCIe

I/O PHY I/O PHY

I/O PHY I/O PHY

MC

MC

MC

MC

MC

MC

MC

MC

Data Fabric switch: 2 FCLK

(best-case, low load)

Repeater: 1 FCLK

1

2

3

4

1 ~94ns

2 ~97ns

3 ~104ns

4 ~114ns

@ FCLK = 1.46 GHz

Improvement in Worst-Best case latency

from 51ns to 20ns

Despite mandatory hop, local

latency only increases 4ns (90ns→94ns)


Case Study:

AMD Ryzen™ Processors

29 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight29 An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight

Leveraging Technology Across Markets

CC

DC

CD

CC

DC

CD

CC

DC

CD

CC

DC

CD

DD

RD

DR

DD

RD

DR I/O I/O

I/O I/O

DD

RD

DR

DD

RD

DR

I/O I/OI/O I/O

IOD

I/O I/O

DD

RD

DR

CC

D

CC

D

CCD Reuse

Direct IOD IP leverage

Up to 16-core desktop


Chiplet Benefits for AMD Ryzen™ Processors

I/O I/O

DD

RD

DR

CC

D

CC

D

0.0

0.5

1.0

1.5

2.0

2.5

16 Cores 8 Cores

No

rma

lized

Die

Co

st

Chiplet (CCD+cIOD) Hypothetical Monolithic 7nm


Further Silicon Leverage

CC

D

CC

D

DD

RD

DR

I/O I/O

PCIe

SATA

USB

I/O I/O

DD

RD

DR

Silicon Reuse

Address Platform Need

Harvest Silicon


Silicon Reuse + Package Customization

DDRDDR

I/O

I/O

DDRDDR

I/O

I/O

DDRDDR

I/O

I/O

DDRDDR

I/O

I/O

I/OI/O

I/OI/O

DDR DDR

DDRDDR

I/O

I/O

I/O

I/O

DDR

DDR


Summary

Server

IOD cIOD

CCD

CCD

CCD

CCD

See our ISCA 2021 paper for

additional details and insights


Thanks to the multiple AMD teams

across the globe

Their dedication and hard work are

what truly breathes life into all of

our products


Disclaimer

The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions, and

typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but

not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product

differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. Any computer system has

risks of security vulnerabilities that cannot be completely prevented or mitigated. AMD assumes no obligation to update or otherwise correct

or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content

hereof without obligation of AMD to notify any person of such revisions or changes.

THIS INFORMATION IS PROVIDED “AS IS.” AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE

CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS, OR OMISSIONS THAT MAY

APPEAR IN THIS INFORMATION. AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF NON-INFRINGEMENT,

MERCHANTABILITY, OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR

ANY RELIANCE, DIRECT, INDIRECT, SPECIAL, OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY

INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

© 2021 Advanced Micro Devices, Inc. All rights reserved.

AMD, the AMD Arrow logo, EPYC, Ryzen, Threadripper, Infinity Fabric, and combinations thereof are trademarks of Advanced Micro

Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective

companies.

Documents

An Overview of Chiplet Technology for the AMD EPYC and