Oct 15, 2006 PETASCALE DATA STORAGE INSTITUTE The Drive to Petascale Computing Faster computers need more data, faster. --

Oct 15, 2006 http://www.pdsi-scidac.org/

PETASCALE DATA STORAGE INSTITUTEThe Drive to Petascale ComputingFaster computers need more data, faster.

-- 2001: 10 TF

-- 2005: 100 TF

-- 2008: 1 PF

-- 2011: 10 PF

2015: 100 PF --

PDSI Thrusts:• Data Capture• Education & Dissemination• Innovation

Everything Must Scale with Compute• Checkpoint at Terabytes/sec• Petabyte files• Billions of files• Revisit programming for Input/Output• Data center automation• Acceleration for search

Computing Speed

Parallel I/O

Network Speed

Memory

ArchivalStorage

TFLOP/s

GigaBytes/sec

Gigabits/sec

110 3 10 5

5

50

500

5,000

2.5

25

250

TeraBytes2,500

.5

5

50

550

5005,000

5

500

50

.5

‘00

‘04‘08

2012Year

10 2

DiskPetaBytes

.05.55

50

MetadataInserts/sec

200

200

20,000

2,000

500

GigaBytes/sec

ApplicationPerformance


Steeped in Terascale Experience

Pheonix& XFS

Lightning & PanFS

Q & PFS

MPP2 & Lustre

Red Storm & Lustre

Roadrunner & PanFS

Jaguar & Lustre

Blue Mountain & XFSSeaborg & GPFS

PETASCALE DATA STORAGE INSTITUTE


Storage Manager

pNFS server

pNFS

HPC Apps

Driver

1. SBC (blocks)2. OSD (objects)3. NFS (files)

Layout grant &revoke


Peta-BytesTera-B/secGiga-filesMega-CPUs

Tera-BytesGiga-B/secMega-filesKilo-CPUs

Education & Dissemination

Innovation

Data Capture

Education• Workshops• Tutorials• Course materials

Outreach• Storage-research-list • Collaboration w/ other Scidacs

IT Automation• Instrumentation• Visualization• Machine Learning• Diagnosis• Adaptation

App Workloads• INCITE resources• Trace & replay tools (e.g. BLAST, CCSM, Calore, EVH1, MCNP, GYRO, Sierra, QCDand other Scidacs)

API Standards• POSIX API• Rich metadata• Compute-in-disk• Archive API• Quality of Storage

Scaling Further• Global/WAN access• Federated security• Metadata at scale• Para-virtualization

NFSv4extended

w/ layouts

•••

Storage bricks

pNFS

MDS

Mechanical

tier

I/O replies &requests

•••

Automation Agents

supervisor

Monitoringinfo

Configurationsettings

Managerialtier

Goal specifications & complaints

Statistics &predictionsAdministrator

HPC NFS• Parallel NFS• Secure NFS• IETF Standard

Strategic Plan

Failure Data• Capture & publish• Computer Failure Data Repository(e.g. LANL’s outages by root cause)


Carnegie Mellon UniversityGarth Gibson (PI)

University of California, Santa Cruz Darrell Long (co-PI)

University of Michigan, Ann ArborPeter Honeyman (co-PI)

Los Alamos National Laboratory Gary Grider (co-PI)

Lawrence Berkeley National LaboratoryBill Kramer (co-PI)

Oak Ridge National LaboratoryPhilip Roth (co-PI)

Pacific Northwest National LaboratoryEvan Felix (co-PI)

Sandia National LaboratoryLee Ward (co-PI)


Participating Organizations


Programming for StorageThe Need for Training Programmers for Storage

HPC IT managers work for users who program apps Often performance of apps/workflows dependent on storage Many times best solutions would be to change the program Reality is app specialists intolerant of requests to reprogram

for better storage performance That is, reprogramming for storage performance often

doesn’t get done

Approach: Create tools, training to help a priori Give programmers libraries, performance debugging tools

that avoid or detect poor storage patterns Give tutorials, case studies, help pages showing weak

programming approaches and how to improve them



Example from BioInformaticsPseudo code example from IT manager -- single thread

for( I=0, I<1000, I++){for( J=0, J<1000, J++){

buf = compute (I,J);f = open( “file_foo”);lseek(f, offset(I,J));write(f, buf, lengthof(buff))close(f);

}} Buf turns out to be small, unaligned, fixed length Obvious fixes:

– Open/close outside both loops– Malloc sizeof 1000000*lengthof(buff), copy into it in memory,

one write at end


Documents

Oct 15, 2006 PETASCALE DATA STORAGE INSTITUTE The Drive to Petascale Computing Faster computers need more data, faster. --