HP’s The Machine - Instituto de Informática - UFRGS - Instituto de … · 2015-11-05 · UFC) Universidade Federal de Campina Grande (UFCG) Universidade Federal de Pernambuco (UFPE)

© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

HP’s The MachineRoque Scheer, HP Brazil R&D

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2

Disclaimer, Acknowledgements

The views in this talk are my opinions and not necessarily those of HP

Thanks toDejan Milojicic, Greg Astfalk, Alvin AuYoung, Cullen Bash, Dhruva Chakrabarti, Al Davis, Paolo Faraboschi, Gary Gostin, Richard Lewington, Terence Kelly, Kim Keeton, Pat Knebel, Hideaki Kimura, Mike Krause, Naveen Muralimanohar, Indrajit Roy, Rob Schreiber, Mike Tan, Haris Volos… and probably many more


HP Brasil – Centro de Pesquisa e Desenvolvimento

6mil m²de área instalada

em Porto Alegre, em 4prédios, com 435 assentos e

2 laboratórios

400Mide investimento

R$

apenas nos últimos 5anos

+1.100colaboradores

no ecosistema de P&D.

16 anosde operação

Desde 2003 sediado no TecnoPUC com participação na

concepção do Parque


Ecossistema de P&D da HP no BrasilInovação aberta e pesquisa e desenvolvimento avançado

Instituto Atlântico (IA)Universidade Federal do Ceará (UFC)

Universidade Federal de Campina Grande (UFCG)

Universidade Federal de Pernambuco (UFPE)Centro de Estudos e Sistemas Avançados do Recife (CESAR)

Universidade Federal do Rio de Janeiro (UFRJ/COPPE)

Universidade de São Paulo (USP)Instituto de Pesquisa EldoradoCentro de Tecnologia da Informação Renato Archer (CTI)Flextronic Instituto de Tecnologia (FIT)Centro de Pesquisa e Desenvolvimento em Telecomunicações (CPqD)

Centro Internacional de Tecnologia de Software (CITS)

Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS)Universidade Federal do Rio Grande do Sul (UFRGS)Universidade FEEVALEInstituto de Pesquisa Eldorado

Legend: Universities / R&D Centers

Universidade Federal de Minas Gerais (UFMG)


HP Labs: history of innovation

1975Standard for Interface Bus

1966Light Emitting Diode (LED)

1968Programmable Desktop Calculator

1989 Digital Data Storage Drive

1980 64-channel Ultrasound

1986 Commercialized RISC chips

2005 Virus Throttle

1999 Molecular Logic Gate

2003 Smart Cooling

1986 3D graphics workstations

1972 Pocket Scientific Calculator

1984 Inkjet Printer

1980 Office Laser Printer

1967Cesium-beam atomic clock

1994 64-bit architecture

2001 Utility Data Center

2002 Rewritable DVD for standard players

2008 Memristor discovered

1966 2011MagCloud

2012StoreAll

OpenFlow switches

2013Moonshot

2010ePrint

StoreOnce

3D PhotonEngine

Threat Central

2014Location Aware

SureStart


Innovation horizons

HP Labs Business Units

1 2 3 5 6 10 204

Applied Research 2 – 5 years

Advanced Development Up to 2 years

Exploratory Research 5 – 20+ years

R e v o l u t i o n a r y Evolutionary

The future


By 2020

… for 8Billion (4)

Next wave: cyber physical age

Pervasive Connectivity

Explosion of Information

Smart Device Expansion

Internet of Things

(1) IDC “Worldwide Internet of Things (IoT) 2013-2020 forecast” October 2013. (2) IDC "The Digital Universe of Opportunities: Rich Data and the Increasing Value of the Internet of Things" April 2014 (3) Global Smart Meter Forecasts, 2012-2020. Smart Grid Insights (Zypryme), November 2013 (4) http://en.wikipedia.org

200Billion

(1)

IoT “Things”

30Billion

(2)

ConnectedDevices

(3)

1Billion Smart

Meters

Internet of People

44 ZB


Architecture has not changed for 60 years


Traditional computer architecturePerformance Wall:

• Multi-core introduced due to single-thread performance “wall.”

Storage Hierarchy:

• HDD/SSD layer is significant performance bottleneck.

• Prevents data getting closer to compute.

Data Movement:

• Too slow for real-time access to shared memory.

Memory Wall:

• DRAM reaching a technology scaling wall.


Architecture of the future: The Machine

Special purpose SoCs

Photonics

Massive memory pool

Photons IonsElectrons


Application-focused silicon

• Less general-purpose, more workload focused

• Dramatic reduction in power, cost, and space

• SoC vendors bring their own differentiated features and opportunities to disrupt markets

Traditional Server Motherboard

StorageCtrlr

Mgmt Network

ManagementLogic Video

Southbridge

ProductionNetwork

NIC(s)

VGAConsole

ProcessorProcessor

ECC Memory ECC Memory

HDDs

System on a Chip (SoC)-based Server

StorageCtrlr

Mgmt ProductionNetwork

NIC(s)

Processor

ECC Memory

Storage

MgmtInterface

Custom Accelerators

SoC

Special purpose SoCs


Communication fire hose for memristor stores

Photonics technology

Designed for multiple use cases• Low cost

• Compact form factor

• Easy to integrate on circuit boards

• No calibration required

• Extensible to higher bandwidth

Orders of magnitude lower energy per bit• 1-2 pJ per bit

Short term: short range, low cost VCSEL

Long term: micro-ring resonator

(low cost, long distance, integrated on silicon)

Photonics


Memristors: ions to store information

Ions (charged atoms) are much better behaved than electronsHeavier, and can be pushed by an electrical fieldStay where you park them, even in a very small box (4F2)

A bit can be represented by the location of an ion in a box

MΩ

1nm

++

10

kΩ


Non-volatile memory (NVM)

Persistently stores dataAccess latencies comparable to DRAMByte addressable (load/store) rather than block addressable (read/write)

Flash-backed DRAM

2D and 3D Flash

Phase-Change Memory

Spin-Transfer Torque MRAM

Resistive RAM(e.g., Memristor)

ns μs

Latency

Haris Volos, et al. "Aerie: Flexible File-System Interfaces to

Storage-Class Memory," Proc. EuroSys 2014.

Massive memory pool


Disruptive technologies

Perf

orm

ance

Time

Existingtechnology(DRAM)

Disruptivetechnology(NVRAM)


SRAM

DRAM

Hard Disk

On-chip caches

Main memory

Disk

Disk cache Flash SSD

Memory hierarchy today

Spee

d

Cost

per

bit

Capacity

Massive memory pool


Collapsing the hierarchy

CPUs

DIMM DDR

HDD DISK

High Capacity DDR Tier

Cold Storage HDD tier

Intelligent Flash SSD Tier

CPUs

High Bandwidth Tier

2.5D

Performance + Capacity NVM Tier

CPUs

3D DRAM or NVM

Extreme Bandwidth Tier

Massive memory pool


X

XX

NVRAM

global global


Traditional file systems

ExamplesExt2/3/4, XFS, BTRFS, ZFS, LFS

Separate storage address spaceData is copied between storage and DRAM

Block-level abstraction leads to inefficiencies

Use of page cache leads to extra copiesTrue even for memory-mapped I/O

Software layers add overhead

Subramanya R Dulloor, et al. "System Software for Persistent Memory," Proc. EuroSys 2014.

Storage: disks, SSDs

Traditional FS

Applications

Page Cache

Block Device

mmap file IO

VFS


Non-volatile memory aware file systems

ExamplesMicrosoft BPFS

Intel PMFS

Low overhead access to persistent memoryNo page cache (a.k.a. DAX)

Direct access with mmap

Leverage hardware support for consistencyPM

Traditional FS

Applications

Page Cache

Block Device

mmap file IO

NVM FSmmu

mappings

mmap

VFS

file IO

Subramanya R Dulloor, et al. "System Software for Persistent Memory," Proc. EuroSys 2014.


Do we need a separate durable data representation?

• Conventional durability techniques

– Separate object and persistent formats

– Translation code

– Programmability and performance issues

• In-memory durability

– Enabled by NVRAM (memristors, PCM, etc.)

– In-memory objects are durable throughout

– Byte-addressability simplifies programmability

– Low load/store latencies offer high performance

In-memory objects

File orDatabase

Serialize

Deserialize

CPU

CACHES

DRAM NVRAM


Why can’t I just write my program, and have all my data be persistent?

The NVM programming problem

• Consider a simple banking program ( just two accounts):

double accounts[2];

• Between which I want to transfer money. Naïve implementation:

transfer(int from, int to, double amount)

accounts[from] -= amount;

accounts[to] += amount;

What if I crash here?What if I crash here?

Crashes cause corruption, which prevents us from merely restarting the computation


Manual solution

• Need code that plays back undo log on restart

• Getting this to work with threads and locks is very hard

• Really want to optimize it

• Very unlikely application programmers will get it right!

persistent double accounts[2];transfer(int from, int to, double amount) <save old value of accounts[from] in undo log>;<flush log entry to NVRAM>

accounts[from] -= amount;<save old value of accounts[to] in undo log>;<flush log entry to NVRAM>

accounts[to] += amount;<flush all other persistent stores to NVRAM><clear and flush log>


Provide a construct that atomically updates NVRAM

Our solution: consistent sections

• Ensures that updates in __atomic block are either completely visible after crash or not at all

• If updates in __atomic block are visible, then so are prior updates to persistent memory

persistent double accounts[2];transfer(int from, int to, double amount) __atomic

accounts[from] -= amount;accounts[to] += amount;


Atlas programming model

• Programmer distinguishes persistent and transient data

• Persistent data lives in a “persistent region”• Directly mappable into process address space (without DRAM buffers)

• Accessed via CPU loads and stores

• Programmer writes ordinary multithreaded code• Automatic durability support at a fine granularity, complete with recovery code

• Supports consistency of durable data derived from concurrency constructs

• Protection against failures• Process crash: works with existing architecture

• Tolerating kernel panics and power failures requires NVRAM and CPU cache flushes

D. Chakrabarti, H. Boehm and K. Bhandari. Atlas: Leveraging Locks for Non-volatile Memory Consistency. Proc. OOPSLA, 2014.


Tools for supporting software development activities

• The Machine Architectural Simulator

– Simulates the hardware at the device/register level

– Used to develop and debug system software (OS, Firmware)

– Detailed hardware simulation results in slow performance

• The Machine Emulator

– Emulates the architecture at a higher level using Linux virtualization

– Used to develop and test user-space applications and tools

– Good relative performance

• NVM Latency Emulator

– Emulates the impact of memory bandwidth and latency on applications

– Used to evaluate the impact of NVRAM performance on software execution


Summary Everything changes…Hardware

• Memory controllerArchitecture

• Coherence/sharing model

• Consistency model

• Error handling, RAS

Software

• OS, memory management

• Compilers and runtime

• Algorithms and data structures

• Storage hierarchy

• Applications

• Security and ProtectionMagnetic

Universal Memory

Registers

Cache

CPU

Load/Store Direct

Access

Block Indirect Access

Non-volatilePooledStorage classMemory speed

Universal memory is comingComputing shifts to a persistent world


Summary, Continued

The Machine provides new computing architectureSpecialized SoCs + massive shared NVM pool + photonic interconnects

Many opportunities for OS and software innovation

Where to look for more informationhttp://www.hpl.hp.com/research/systems-research/themachine/

HP Discover 2014 talks on The Machine• HP Labs Director Martin Fink's announcement: https://www.youtube.com/watch?v=Gxn5ru7klUQ

• Kim Keeton’s talk on technologies: https://www.youtube.com/watch?v=J6_xg3mHnng

Paolo Faraboschi’s keynote at HPCA/PPoPP/CGO

http://www.hpl.hp.com/research/systems-research/themachine/

https://www.youtube.com/watch?v=Gxn5ru7klUQ

https://www.youtube.com/watch?v=J6_xg3mHnng


We are hiring aggressively !

• Systems software for peta-scale NVM systems: OSes; data management; programming models; runtimes and compiler/language support; security; manageability; RAS; QoS; system modelling and workload characterization

• Analytics at peta-scale: frameworks for scalable big data analytics; machine learning; graph analytics; visualization

• Networking and mobility: enterprise, data-center and cloud networks; software defined networking; mobile cloud architectures, systems, platforms and services; mobile sensing and context awareness.

Visit: http://jobs.hp.com

Open Positions in Porto Alegre and São Paulo.

http://jobs.hp.com/

© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Q&A

Documents

HP’s The Machine - Instituto de Informática - UFRGS - Instituto de … · 2015-11-05 · UFC) Universidade Federal de Campina Grande (UFCG) Universidade Federal de Pernambuco (UFPE)