PRISM: Zooming in Persistent RAM Storage Behavior

University of PittsburghJuyoung Jung

PRISM: Zooming inPersistent RAM Storage Behavior

Juyoung Jung and Dr. Sangyeun Cho

Dept. of Computer ScienceUniversity of Pittsburgh

University of PittsburghJuyoung Jung University of PittsburghJuyoung Jung

Contents

Introduction Background PRISM Preliminary Results Summary


Technical Challenge

HDD

- Slow and - Power hungry

Fundamental solution?

Alternative storage medium


Technical Challenge

+ much faster+ better energy+ smaller …

- erase-before-write- write cycles- unbalanced rw- scalability …

NAND flash based SSD


Emerging Persistent RAMs Technologies

Latency Programenergy

Endurance Density

Read Write Erase

NAND flash 25us 200us 1.5ms 10nJ 104~106 4~5F2

PCM 20ns 100ns N/A 100pJ 108~109 5F2

STT-RAM 10ns 10ns N/A 0.02pJ 1015 4F2

RRAM 10ns 20ns N/A 2pJ 106 6F2

FeRAM 75ns 50ns N/A 2pJ 1013 6F2

PRAMs Win !


Technological maturity

Latency

Energy consumption

Read/write imbalance

Write endurance


Our Contribution

There is little work to evaluate Persistent RAM based Storage Device (PSD) Research environment not well prepared

Present an efficient tool to study the impact of PRAM storage device built with totally different physical properties


PRISM

PeRsIstent RAM Storage Monitor

Study Persistent RAM (PRAM) storage behavior Potentials of new byte addressable storage device PRAMs’ challenges as storage media

Guide PSD (PRAM Storage Device) design Measure detailed storage activities (SW/HW)


PRISM

HDD or SSD PSD

Block devices Non-Block devices


PRISMtest.exe

read(file1)write(buf,file1)

data values

metadata impact resource conflictsaccess frequency

OS virt. addr

cell wearing stat etc.

Low-level storage behavior

PRAM phys. addr

exploit parallelism

wear leveling

bit masking

address mapping


PRISM Implementation Approach

PSD

User Applications

Virtual File System

Individual FS (EXT3, HFS, TMPFS, etc.)

Buffer/Page Cache

Block I/O Layer

HDD/SSD device driver

SW

HW

I/O scheduler

File system

Interconnect

Slots, chipsHolistic A

pproachEmulation

Simulation

InstrumentationTechnique

Tracers

Tracers


PRISM Components

Frontend Tracer

Backend PSDSim

User Apps

PRISM


Application

Linux VFS

In-memory TMPFS file system

File I/O requests

Write dataTracer

Module

User-defined Tracer

Modules

MetadataTracer

Module

Proc-based Logger

Raw data gathered

write

write

write

access timebyte-granulerequest info

……

InvokingWR tracer

Frontend Tracer


PRISM Components

Frontend Tracer

Backend PSDSim

User Apps

PRISM


Backend Event-driven PSDSim

Various performance results

PRAM ReferenceStatistics Analyzer

Wear leveling Emulator

PRAMStorage

ControllerPRAM

Energy Model

Raw trace data Storage configuration

Persistent RAM storage simulator (PSDSim)

the used PRAM technologystorage capacity

# of packages, dies, planesfixed/variable page sizewear-leveling scheme …


Preliminary Results of Case Study

Objective: enhancing PRAM write endurance Experimental Environment

Components SpecificationCPU Intel Core Duo 1.66 GHz (32-bit)L2 Cache 2 MBMain Memory 1 GB SDRAMOperating System Linux 2.6.24File System Tmpfs 10 GB capacityWorkload TPC-C 2-hours measurement


Possible Questions from Designers

How our workloads generate writes to storage?1. Temporal behavior of write request pattern2. The most dominant data size written3. Virtual address space used by OS4. File metadata update pattern

What about hardware resource access pattern?1. Resource utilized efficiently2. Do we need consider wear-leveling?


TPC-C File update patternTemporal behavior of writes

Populating 9 relational tables

(Wall-clock time)

writes are very bursty

proper queue size


File Data Written Size4KB written size dominant

Not cause whole block update


Virtual Page Reuse Pattern

OS VMM allocates pages from low memory

HIGHMEM


Plane 7Plane1Plane1Plane1

Example Hardware Organization

PKG 0

PKG 1

PKG 2

PKG 3

1GB PSDDie 1Die 0

256MB package 0

Plane 0

Page 0Page 1Page 2Page 3

Page N-1Page N

8 planes in 128MB Die 1

16MB plane 7


Hardware Resource Utilization

pkg0 pkg1 pkg2 pkg30

500

1000

1500

2000

2500

3000

3500

4000

Acc

ess

Cou

nt

Die1

Die0


Storage-wide plane resource usage

0

100

200

300

400

500

600

plane0 plane1 plane2 plane3 plane4 plane5 plane6 plane7

1258

Serious unbalanced resource utilization

Acc

ess

Cou

nt

Die0 Die1 Die0 Die1 Die0 Die1 Die0 Die1Package 0 Package 1 Package 2 Package 3

Q. plane-level usage fair ?


Wear-leveling Effect

Heavily biased

resource usage

Before Wear-leveling After RR Wear-leveling

Balancedresource usage


PRISM Overhead

PRISM’s I/O performance impact We ran IOzone benchmark with 200MB (HDD-friendly)

sequential file write operations

3.5x slowdown

Raw tmpfs PRISM HDD0

0.5

1

1.5

2

2.5

3

3.5

4

Spee

d no

rmal

ized

to P

RISM

Raw tmpfs PRISM0

0.5

1

1.5

2

2.5

3

3.5

4

Nor

mal

ized

Late

ncy


PRISM Overhead

PRISM’s I/O performance impact We ran IOzone benchmark with 200MB (HDD-friendly)

sequential file write operations

11.5x faster

Raw tmpfs PRISM HDD0

0.5

1

1.5

2

2.5

3

3.5

4

Spee

d no

rmal

ized

to P

RISM

PRISM HDD0

2

4

6

8

10

12

14

Spee

d no

rmal

ized

to H

DD


PRISM Summary

Versatile tool to study PRAM storage system Study interactions between storage level interface

(programs/OS) and PRAM device accesses Run a realistic workload fast Extendible with user-defined tracer easily Explore internal hardware organizations in detail


Thank you !

Documents

PRISM: Zooming in Persistent RAM Storage Behavior