©2015 Kingston Technology Corporation. All rights reserved.

All trademarks and registered trademarks are the property of their respective owners.

Kingston Technology Server Architecture and Kingston Memory Solutions Ingram Micro

May 2015

Mike Mohney

Senior Technology Manager, TRG

Kingston Server Upgrades

• Introduction to Kingston

• Kingston Solutions and Memory Roadmap

• Memory Configurations for Intel server platforms

2

Kingston Snapshot

3

• Founded in 1987, 27+ years

• Privately Held Company

• Financially Secure

• For 8 years , revenue exceeding $4B

• 4,000 Employees Worldwide

• Worldwide Manufacturing & Testing

• Focused Business: Memory Modules, Flash Products & Tech Solutions

• Delivering over 1M units per day

• Products distributed in 125 Countries and available in over 30,000 locations worldwide; widely accepted brand

Source: Kingston Technology

• Manufacturing Centers are all ISO 9001:2008 certified

• 2014 worldwide capacity: Over 30 million units per month

• Over 50 SMT manufacturing lines in 4 global facilities, producing both DRAM & Flash products

Kingston Global Infrastructure

* 4 Kingston Technology Manufacturing Facilities Worldwide

** Panram and OSE are subcontractors to Kingston

** ** * * * *

Source: Kingston Technology, June 2014 4

Kingston Investments & Expertise

Kingston manages all six processes required to build modules, cards, and drives.

Wafer Production

Wafer Testing

Wafer Processing/Chip Production

Component Testing

Module, Card & Drive Production

Module, Card & Drive Testing

5

Strategic Alliances: Server Compatibility Check

Kingston works with many facets of technology to ensure server memory compatibility.

Industry Leading OS, Software, Hardware

PC OEM’s Motherboard Manufacturers

6

Intel and Kingston

Since 1998, Intel and Kingston have jointly developed memory technology validation testing processes for PC133, RDRAM, DDR, DDR2, FB-DIMM, DDR3, LRDIMM, and DDR4.

DDR3 DRAM chip validation testing on new Advantest tester (DDR4 capable)

Checking early FB-DIMM prototype against specifications

7

Kingston Performance Upgrades

• Introduction to Kingston

• Kingston Solutions and Memory Roadmap

• Memory Configurations for Intel server platforms

8

Reliability: The Challenges of Memory Testing

Kingston Quality Standard: Zero bad cells 1 bad cell = defective module

1GB Over 8.5 billion cells

2GB Over 17 billion cells

4GB Over 34 billion cells

8GB Over 68 billion cells

16GB Over 136 billion cells

Memory Capacity Number of Cells

9

Semiconductor Reliability

Early-Life Failures

Typically the first few months of usage

This curve of semiconductor failures over time is called a ‘bathtub’ curve

End-of-Life Failures

Nu

mb

er O

f Fi

eld

Fai

lure

s (i

n u

nit

s)

Time

Ship out from Factory to user

Failure curve for lower-quality

semiconductor products

Useful Life Of Semiconductor

A typical useful life would be 20 years or more

10

Kingston Production Testing and Dynamic Server Burn-In

11

100% Testing

• No spot testing! Kingston memory is 100% tested prior to leaving our factories, as we have always done

Dynamic Server Memory Burn-In

• Shrinking DRAM lithography increases sensitivity to temperature, forcing higher error rates and exposing weak memory cells

• Kingston’s advanced burn-in process screens these by simulating 3 months of heavy server use at higher temperatures under full system environment and frequency stress

Proprietary Production Testers

• Kingston’s test engineers designed the first automated production testers in the memory industry

• New patented automated testers build upon our test engineering legacy to meet the more stringent demands of evolving memory technology

35 Quality Control Checkpoints

• Kingston uses AQL sampling methods to re-inspect and re-test every lot to ensure our quality standards are being met

Kingston holds 29 patents related to memory testing

FAST PAGE MODE EDO

PC66 PC100 PC133 RDRAM

PC600 PC700 PC800

DDR200 DDR266 DDR333 DDR400 RDRAM PC1066

DDR2-400 DDR2-533 DDR2-667

1987- 1990

1997- 1999 2000

2001-2003

2004-2005

2006

2007- 2009

DDR3-800 DDR3-1066 DDR3-1333

2015

DDR2-800 FB-DIMMs

DDR4-2933 DDR3L 1.35V DDR3-1600 DDR3L-1600 DDR3-1866 LRDIMM

DDR4-2400

2017 2010- 2013

DDR4-2666

2016

Technology Data Rate Module Classification Peak Bandwidth

DDR3 (1.5V) DDR3L (1.35V) DDR3U (1.25V)

800 DDR3-800 / PC3-6400 6400 MB/s or 6.4 GB/s

1066 DDR3-1066 / PC3-8500 DDR3L-1066 / PC3L-8500

8500 MB/s or 8.5 GB/s

1333 DDR3-1333 / PC3-10600 DDR3L-1333 / PC3L-10600

10600 MB/s or 10.6 GB/s

1600 DDR3-1600 / PC3-12800 DDR3L-1600 / PC3L-12800

12800 MB/s or 12.8 GB/s

1866 DDR3-1866 / PC3-14900 14900 MB/s or 14.9 GB/s

DDR4 (1.2V)

2133 DDR4-2133 / PC4-2133 17000 MB/s or 17 GB/s

2400 DDR4-2400 / PC4-2400 19200 MB/s or 19.2 GB/s

2666 DDR4-2666 / PC4-2666 21300 MB/s or 21.3 GB/s

2933 DDR4-2933 / PC4-2933 23400 MB/s or 23.4 GB/s

12

DDR4-2133

2014

Memory Roadmap (1987 – 2017)

Form Factors 288-pin DIMM 260-pin SODIMM

Performance 1600MT/s, 1866MT/s, 2133MT/s, 2400MT/s, 2666MT/s, 2933MT/s

4000MT/s (possible extension) 4266MT/s (possible extension)

Power 1.2V with improved refreshes, ½K page for x4 12V for NVDIMM (4-pins)

DRAM Densities/DIMM Capacities 4Gb, 8Gb, 16Gb (x4, x8, x16) Future DIMM capacities of 64GB, 128GB, 256GB

New Features 3D chip stacking support (up to 8 stacked dies) Point-to-Point Memory Channel addressing More stable than DDR3 with improved error correction code

13

New Module Key Location

Slightly thicker for

improved performance

Curved edge for better

socket connection

“Low Voltage” (LV)

DDR3L = Low Voltage 1.35V JEDEC (industry spec)

• Modules are classified as PC3L-xxx

Reducing memory power consumption 10% by dropping module voltage requirement, lowering energy costs to data centers

Kingston DDR3L modules are dual-voltage supporting 1.35V / 1.5V

Supported on current and legacy server platforms

• Intel SNB/IVB/HSW (Xeon E3+ Series)

• Intel WSM-EP/SNB-EP/IVB-EP/HSW-EP (Xeon 5600/E5 v1/v2 Series)

• Intel WSM-EX/IVB-EX (Xeon E7/v2 Series)

• AMD Magny-Cours+ (Opteron 6100/6200/6300 Series)

• AMD Lisbon+ (Opteron 4100/4200/4300 Series)

Supported on mobile/desktop platforms from Intel and AMD

• Skylake (DDR3L Only), Broadwell, Haswell, & Ivybridge (Core i5 / Core i7)

• AMD A-Series

DDR3U (1.25V)

Ultra Low Voltage

JEDEC Spec for Mobile, Server

14

Low Voltage spec for DDR4 deemed

not possible by DRAM

semiconductors

Load-Reduced DIMMs for Servers

JEDEC Industry Standard Specification

LRDIMMs overcome the impact of Ranks (Loads) upon memory speed

• Parallel memory bus impacted by memory module Ranks

• Enable greater memory capacity and speeds while reducing “load”

• Require special buffer chips

Eight-Rank modules now possible with less loading on memory bus

• Will enable faster 32GB, 64GB, 128GB LRDIMMs

Supported on 2012+ Intel and AMD servers

Included in DDR4 spec with new, more efficient design – Q1 2015

Kingston launched Q1 2013

15 Source: Intel IDF, September 2011

KVR13LL9Q4/32 KVR16LL11Q4/32 KVR18L13Q4/32

KVR21L15Q4/32 KVR21L15Q4/64

DDR4 LRDIMMs

DDR3 LRDIMMs

Coming soon

16 Source IDT: www.idt.com/document/whp/ddr4-lrdimms-both-memory-capacity-and-speed

Load-Reduced DIMMs for DDR4 – New Design, Reduced Latency

Non-Volatile DIMM (NVDIMM)

What Is It?

A non-volatile memory module directly accessed by the memory controller (approximately

10x faster than PCIe interface). Silicon based memory technology is reaching its

perfomance and efficiently limit.

17 Source: Google Images

SK Hynix 16GB DDR4 NVDIMM Diablo/SanDisk ULLtra DIMM Viking ArxCis-NV DDR3 NVDIMM

Types of NVDIMM standards

NVDIMM-F – An all FLASH memory module accessible by the memory controller.

NVDIMM-N – A DRAM memory module made persistent by the use of NAND flash silicon.

NVDIMM-N2 – Similar to NVDIMM-N but with system accessible Flash.

NVDIMM-M – MRAM based memory module accessible by the memory controller.

NVDIMM-x – New technology. PRAM? RRAM? HMC?

Kingston Memory Product Lines

ValueRAM

o Generic / JEDEC Specification

o Ideal for system builders or small

datacenters

System Specific

o Also known as “Branded”

o Guaranteed compatibility

o Ideal for users who want to upgrade

OEM machines

OEM / Private Label

o Customized in box solutions

o Contracted supply agreements

HyperX

o Enthusiast gaming market

o Overclocking

o Brand focused

18

System Specific Memory

Guaranteed Compatible

• Testing conducted in the system in which the memory will be installed

Importance of Branded Memory

• Different chips provide varying performance levels

• Chip Compatibility can vary

• Possible constraints in module height or width

• Presence Detect Configuration

19

ValueRAM Product Decoder

20

• Kingston ValueRAM is built to industry standard specifications outlined by J.E.D.E.C., the Joint Electron Device Engineering Council, on which Kingston holds a seat. Intel, AMD, and other computing chipset architects adhere to these standards for industry agreement and cross-platform compatibility.

• J.E.D.E.C. provides the specifications for Pin Count, Form Factor, Speed, Power, and Memory Technology Types

Intel Certified intended specifically for customers who are using Intel branded server/workstation motherboards or systems

• Qualified and supported by Intel

• Controlled builds (BOMs)

• PCNs with 45-90 Day Notice

Server Premier intended for server builders with fully controlled BOM requirements (PCB, chip brand/die revision, etc) regardless of brand of motherboard used

• Fully BOM controlled

• PCNs with 45-90 Day Notice

• Easily manage what is ordered

• No price premium

ValueRAM Server Focused Programs

21

ValueRAM Qualifications

22

In addition to the strategic partnerships Kingston has with ODMs for submitting memory for qualification, we also work with independent memory test labs AVL and CMTL Labs for specific platform, motherboard, or system qualification. Test certificates are available on our website.

http://www.kingston.com/us/memory/valueram/server

Example:

• Optimizing Memory:

• DDR4 is NOT compatible with DDR3 based systems (module will not fit)

• Install memory in kits of identical modules according to memory architecture

• Configure all processors the same way for most bandwidth

• Do Not Mix Module Types within a server.

• Mixing DDR3 (1.5V) and DDR3L (1.35V) will default the memory to 1.5V.

23

Module Types For Server Platforms

ECC Unbuffered DIMM Registered DIMM Load Reduced DIMM

Registered DIMM Load Reduced DIMM

288-pin DIMM

240-pin DIMM

ECC Unbuffered DIMM

DRAM Chip Types Used in Servers

1 2 3 4

1 2 3 4 5 6 7 8

Registered DIMMs

LRDIMMs

Registered DIMMs

LRDIMMs

Unbuffered ECC DIMMs

Unbuffered DIMMs

SODIMMs

Single Bit Error

Detection and

Correction

x4

x8

Multi Bit Error

Detection and

Correction

24

64-bit Memory Loads (aka Ranks)

• Ranks are virtual modules on a single DIMM, designed to save space on the motherboard

• Only one Rank is active at a time, the other ranks on the DIMM go into standby saving power

• 2R and 4R DIMMs consume less power than 1R DIMMs

• Memory lanes on the motherboard become congested with more memory loads, so server systems are limited by the number of ranks that can be used, as the memory slows down with more ranks populated, just line cars in heavy traffic

Single Rank (1R)

Quad Rank (4R)

Dual Rank (2R)

25

64bits

64bits 64bits

64bits 64bits 64bits 64bits

Why Add Channels and Increase The Memory Speed?

26

0

20

40

60

80

100

120

Single Channel(1 DIMM per CPU)2 x 16GB = 32GB

Dual Channel(2 DIMMs per CPU)

4 x 16GB = 64GB

Quad Channel(4 DIMMs per CPU)8 x 16GB = 128GB

DDR3 & DDR4 Aggregate Server Memory Bandwidth

DDR3-800

DDR3-1066

DDR3-1333

DDR3-1600

DDR3-1866

DDR4-2133

Benchmark by SiSoftware SANDRA 2013.05.19.44 on Intel DDR3 and DDR4 platforms

GB/s

Memory Supported Memory Configuration

Xeon 5500 Series DDR3-1333 Triple-Channel = K3 kits

Xeon 5600 Series DDR3-1333 / DDR3L Triple-Channel = K3 kits

Xeon E5 Series DDR3-1600 / DDR3L-1333 Quad-channel = K4 kits

Triple-channel = K3 kits

Xeon E5 v2 Series DDR3-1866 / DDR3L-1600 Quad-channel = K4 kits

Triple-channel = K3 kits

Xeon E5 v3 Series DDR4-2133 Quad-channel = K4 kits

Xeon 6500 / 7500 / E7 Series DDR3-1066

DDR3L-1066 on E7 Series Dual-channel = K2 / K4 kits

Xeon E7 v2 Series DDR3-1600 / DDR3L-1600 Quad-channel = K4 kits

Xeon W3500 / 3600 / E3

Xeon E3 v2 / v3 Series

DDR3-1066 / DDR3L (W3600)

DDR3/L-1600 / DDR3-1866 Dual-channel = K2 / K4 kits

Opteron 8400 / 2400 Series DDR2-800 Dual-channel = K2 kits

Opteron 6100 - 6300 Series DDR3 / DDR3L 1333-1866* Quad-channel = K4 kits

Opteron 4100 - 4300 Series DDR3 / DDR3L 1333-1866* Dual-channel = K2 kits

Memory Configuration Options by Processor

27

* Subject to motherboard compatibility

Kingston Performance Upgrades

• Introduction to Kingston

• Kingston Solutions and Memory Roadmap

• Memory Configurations for Intel server platforms

28

Microarchitecture Roadmap

1600 1866 2133 2400

29

Picture sources: Google images

HP ProLiant Gen 8 / Gen 9 ®

Dell PowerEdge ® IBM System xSeries M4 ®

Cisco UCS M3 ®

Intel Xeon E5 / E5 v2 / E5 v3 “EP” Series Server Memory Configuration

30

Intel Xeon E5 Series Summary

Source: Intel public marketing 31

Intel Xeon E5-2600/4600 v1/v2/v3 Architecture Quad Channel – up to 12 DIMMs per CPU (24 per 2-CPU System, 48 per 4-CPU System)

32

Intel Xeon E5-2400 v1/v2 Architecture

Triple Channel – up to 6 DIMMs per CPU (12 per 2-CPU System)

33

1.2V

Module Type

(No Mixing in a Server)

1 DIMM per

Channel

1 DPC

2 DIMMs per

Channel

2 DPC

3 DIMMs per

Channel

3DPC

Registered DIMM (RDIMM) 2133 1866 1600

Load Reduced DIMM (LRDIMM) 2133 2133 1600

34

Intel Xeon E5-2600 v3 DDR4 Memory Speeds Haswell-EP

Processor Model Determines Maximum

Memory Speed

2133

1866

1600

Ch 1Ch 2Ch 4Ch 3Ch 1Ch 2Ch 4Ch 3

DDR4

Server

Intel S2600WT

Balanced

Configuration

(128GB)

123123321321 123123321321

2 DIMMs

Per Channel

Memory Bandwidth

98.357 GB/s

31.41%

Ch 1Ch 2Ch 4Ch 3Ch 1Ch 2Ch 4Ch 3

Unbalanced

Configuration

(128GB)

123123321321 123123321321

Memory Bandwidth

67.464 GB/s

3DPC 1DPC 1DPC 3DPC 3DPC 1DPC 1DPC 3DPC

2DPC 2DPC 2DPC 2DPC 2DPC 2DPC 2DPC 2DPC

E5-2650v3

CPU1

E5-2650v3

CPU2

E5-2650v3

CPU1

E5-2650v3

CPU2

(30.89 GB/s)

Optimizing Intel Xeon E5 v3 Servers

35

16 x KVR21R15S4/8

8GB DDR4-2133 RDIMMs 1Rx4 1.2v

Benchmark by SiSoftware SANDRA 2013.05.19.44 on Intel S2600WT

128GB

Performance Gains for Intel DDR4 Servers

Source: http://www.intel.com/performance 36

2133 2133

1600

1866

1600

1066 1000

1200

1400

1600

1800

2000

2200

1 DPC 2 DPC 3 DPC

Xeon E5 v3

Xeon E5 v2

35%

25%

13%

Bandwidth comparison using 32GB DDR3 and DDR4 Load Reduced DIMMs

Me

mo

ry S

pe

ed

768GB 512GB 256GB

Intel Xeon E5 v2 Processors

http://www.intel.com/content/www/us/en/processors/xeon/xeon-e5-brief.html

Processor Model Determines Maximum

Memory Speed

37

1.5V

Module Type

(No Mixing in a Server)

System

Configuration

Module Ranks

1R/2R=Single/Dual

4R = Quad

1 DIMM per

Channel

1 DPC

2 DIMMs per

Channel

2 DPC

3 DIMMs per

Channel

3DPC

Registered DIMM

(RDIMM)

1 Socket Per

Channel

1R/2R 1866

4R 1066

2 or 3 Sockets

Per Channel 1R/2R 1866 1600 1066

4R 1066 800 NO QR

1DPC 2DPC 3DPC

38

1.35V

Module Type

(No Mixing in a Server)

System

Configuration

Module Ranks

1R/2R=Single/Dual

4R = Quad

1 DIMM per

Channel

1 DPC

2 DIMMs per

Channel

2 DPC

3 DIMMs per

Channel

3DPC

Registered DIMM

(RDIMM)

1 Socket Per

Channel

1R/2R 1600

4R 800

2 or 3 Sockets

Per Channel 1R/2R 1333 1333 800

4R 800 800 NO QR

Intel Xeon E5-2600 v2/4600 v2 DDR3 RDIMM Memory Speeds Ivybridge-EP

1.5V / 1.35V

Module Type

(No Mixing in a Server)

Module Ranks

4R = Quad

1 DIMM per

Channel

1 DPC

2 DIMMs per

Channel

2 DPC

3 DIMMs per

Channel

3DPC

Load Reduced DIMM

(LRDIMM)

4R at 1.5V 1866 1600 1066

4R at 1.35V 1600 1600 1066

39

1.5V 1.35V

Module Type

(No Mixing in a Server) System Configuration

1 DIMM per

Channel

1 DPC

2 DIMMs

per Channel

2 DPC

3 DIMMs

per Channel

3DPC

1 DIMM per

Channel

1 DPC

2 DIMMs per

Channel

2 DPC

3 DIMMs per

Channel

3DPC

ECC Unbuffered

DIMM (ECC UDIMM)

1 Socket Per

Channel

1866 1600

2 or 3 Sockets

Per Channel 1866 1600 NO

UDIMM 1333 1333 NO

UDIMM

1DPC 2DPC 3DPC

Intel Xeon E5-2600 v2/4600 v2 DDR3 LRDIMM & ECC UDIMM Memory Speeds

Ivybridge-EP

Optimizing Intel Xeon E5 v1/v2 Servers

40

16 x KVR16LR11D4/16

16GB DDR3-1600 RDIMMs 2Rx4 1.5v

Benchmark by SiSoftware SANDRA 2013.05.19.44 on Intel S2600GZ

256GB

Unbalanced

Balanced

34.88%

Kingston Summary

• Top buyer of memory chips/only the best chips

• Ranked #1 in DRAM

• Ranked #1 in USB drives

• Listed on America’s largest private companies

• Worldwide presence/financially solid

• Channel leadership – OEM, channel, Retail, eTail

• Established relationships with technology leaders

• Value-Add to Flash products

• Major investments in technology

• Most extensive testing in the industry

• Guaranteed compatibility/certifications

• Managing the quality process – from wafer to module

41

Questions?

42

43