Copyright © 2000,2003, Storage Networking Industry Association

1 Copyright © 2000,2003, Storage Networking Industry Association

Obligatory rubric

• Copyright © 2000,2003 Storage Networking Industry Association (SNIA).

• SNIA-TC members may use this material freely; all others must conform to usage notes in the speaker noted for this slide.

• See SNIA-SSM-colors.ppt for printing and color/graphics conventions.

Copyright © 2000,2003, Storage Networking Industry Association

SNIA shared-storage modelA work in progress …

An architectural overview

This revision:

• 2001-06-05 last content update

• 2003-04-13 last graphics update


Contents

• Purpose

• The SNIA storage model Layers, functions, and services Networks and interfaces

• Applying the SNIA storage model Common storage architectures

• Conclusions


Purpose

• Present a simple model for shared storage architectures

• Use it to describe common examples graphically

• Expose, for each one: What services are provided, where Where interoperability is required [future] Pros and cons of the architecture


Benefits

• A common “architecture vocabulary”

• Reference comparisons between common solutions

• Help to align the industry Customers can better structure their choices Vendors can better explain their differences


What the model is and is not

• It is not: A specification, an architecture, a design, a product,

a recommendation, or an installation

• It is: A framework that captures the functional layers and

properties of a storage system


Classic storage model

Application

Storage domain:“anything goes!”

Network?Appliance?

Data mover?

Array?

NAS?

JBOD?


File/record layer

Block layer

Storage devices (disks, …)

Database(dbms)

File system(FS)

The SNIA shared storage modelS

tora

ge

do

ma

in

Ser

vice

s

Network

Host

DeviceBlock aggregation

Application


File/record layer

The SNIA storage model:File/record layer

Database(dbms)

File system(FS)


The SNIA storage model:File/record layer — functions

• Aka “access methods” File system, database

• Primary responsibility: packing many smaller things into a few larger ones Fine-grain naming, space allocation

• Secondary responsibilities Caching for performance Coherency in distributed systems


The SNIA storage model:Block layer

Block layer


Block aggregation


The SNIA storage model:Block layer — functions

• Storage devices – storing data disk drives, tape drives, solid-state disks, …

• Block aggregation – address mapping in-SN aggregation, or “virtualization” slicing & concatenation, striping local & remote mirroring, RAID-n

• Examples volume managers disk array LUs

• Secondary responsibilities caching


File/record layer

Database(dbms)

File system(FS)

Block layer

The SNIA storage model:Access path examples


Block aggregation

Note: all 8 possible paths can be used!

Application


Block layer

• Block-mapping functions:what can be done

• Functional decomposition:where it can be done

• Sample architectures

Block layer


Block aggregation


Block layerWhat can be done

• Space management making a large store from many small ones packing many small stores into one large one

• Striping for performance (load balancing, throughput)

• Redundancy full (local & remote mirroring, RAID-1, -10, …) partial (RAID-3, -4, -5, …) point-in-time copy


Block layerWhere it can be done

• Host-side logical volume managers device drivers, HBAs

• SN-based HBAs, specialized SN appliances

• Device-based array controllers (e.g., RAID) disk controllers (e.g., sparing)

Storage devices

Block aggregation

Network

Host

Device


Block layerHow it is done

• Building blocks input: vector of blocks output: vector of blocks

• Result: building blocks can be stacked enables the 3 layer model for the

block layer layers can be nested on one another could be extended to more layers

……

…

Block aggregation


Device block-aggregation

Network block-aggregation

Host block-aggregation

Block layerSample architectures

ApplicationH

ost.

with

LVM

(+ s

oftw

are

RA

ID?)

1. Direct-attach

Disk array

SN

Hos

tw

ith L

VM

2. SN-attach

File

/rec

ord

lay

erB

lock

lay

er

3. SN aggregation

Aggregationappliance

Hos

t,no

LVM

Storage devices


File/record layer

• Byte-mapping functions:what can be done

• Functional decomposition:where it can be done

• Sample architectures

File/record layerDatabase(dbms)

File system(fs)


File/record layerWhat can be done

• Database management systems tuples → tables tables → table-spaces table-spaces → volume

• File systems files → volume

• New types http caches: a kind of distributed file system?


device

File/record layerWhere it can be done

• Host-side file systems and databases NFS, CIFS, etc. are client-server

splits inside the file system

• SN-based NAS head

• Device-based NAS functions in array box

NAS

Ho

st w

ith

NF

S/C

IFS

clie

nt

Ho

st w

ith

lo

cal

FS

/db

ms





File/record layer

Hos

t. w

ith L

VMan

d so

ftwar

e R

AID

1. Direct-attach

File

/re

cord

lay

er

Blo

ckla

ye

r

NASserver

4. NAS server

Hos

t. w

ith L

VM

Disk array

2. SN-attach

Host3. NAS head

NAS head

Host

LAN

Application

SN


File/record layerDatabase(dbms)

File system(fs)

The SNIA storage modelA layered view

IV. Application

III. File/record layerIIIa. DatabaseIIIb. File system

II. Block aggregationIIa. HostIIb. NetworkIIc. Device

I. Storage devices

Storage devices

Block aggregation

Application

Network

Host

Device

IV

III

IIc

IIb

IIa

I


Sto

rage

do

ma

in

File/record layer

Block layer

The SNIA storage modelServices subsystem

Application


Block aggregation

Database(dbms)

File system(fs)

Ser

vice

sD

isco

very

, mon

itorin

gR

esou

rce

mgm

t, c

onfig

urat

ion

Sec

urity

, bi

lling

Red

unda

ncy

mgm

t (b

acku

p, …

)H

igh

avai

labi

lity

(fai

l-ove

r, …

)C

apac

ity p

lann

ing


Services

• Operations off the critical path naming, discovery, monitoring, configuration, security, billing,

redundancy management (backup, …), high availability management (fail-over, …), capacity planning, …

strong ties into system-wide management services

• Vital for successful operation and a major opportunity for SNIA … … but not discussed further in this presentation





Caching… can be added to almost any layer

Hos

t. w

ith L

VMan

d so

ftwar

e R

AID

File

/re

cord

lay

er

Blo

ckla

ye

r

Host

NAS head

Host

Hos

t. w

ith L

VM

Disk array

SN

NAS head

LAN

Ideally, caching only affects performance.

But: coherency implications do affect management (protocols, interfaces)

Application

Cache

Cache

Cache

Cache

Cacheappliance

Cache





ClusteringInter-box aggregation

Hos

t. w

ith L

VMan

d so

ftwar

e R

AID

File

/re

cord

lay

er

Blo

ckla

ye

r

Host

NAS head

Host

Hos

t. w

ith L

VM

Disk array

SN

NAS head

LAN

Cluster FS

Multi-node LVM

Purposes:• load spreading across peers (scalability)

• alternate paths (high availability, scalability)

Application


Q: Data versus storage?A: Putting information into containers

user: data (“learning my preferences”)application: container(“user keystroke history”)

application: data (“user keystroke history file”)file system: container (“byte vector”)

file system: data (“a named file”)volume system: container (“blocks in volume”)

volume system: data (“replicated, striped layout”)disk array: container (“blocks in LU”)


SharingContent sharing or resource sharing?

• Content sharing (“logical”, “data”) contents accessed and understood by multiple

clients• e.g., file system, Oracle Parallel Server dbms

some of the hard issues:• coherency• heterogeneous data formats

• Resource sharing (“container”, “physical”) e.g., disk array where hosts access disjoint LUs





SharingContent sharing and resource sharing

Hos

tw

ith L

VM

File

/rec

ord

lay

erB

lock

lay

er

Hos

tw

ith L

VM

SN

NAS system

HostHost

LAN

Application

Resource sharing

Array is shared,but LUs are disjoint

Data sharing

NAS system is shared – and so are the files


Storage device

Block layer

File layer

Operating System

Application

Networks and interfacesare pervasive in the model

Network or interface

Network

Bus

API

Full benefits come only fromcommon, open interfaces






Networks and interfacesComposition and scaling

Open interfaces allow:1. vertical composition2. horizontal scaling3. supplier independence

Network

Bus

API

API

Network

Bus

API


Networks and interfacesOpen interfaces require …

• Well defined: functions (what they do) interface protocols (data formats) access protocols

(system call, RPC, flow control, …)

• That are: published supported by multiple products=> standards (which is where SNIA comes in)

Network

Bus

API


? NAS

Q: “SAN” versus “NAS”?A: a poorly-formed question

• Q: hardware: FibreChannel vs Ethernet vs InfiniBand?

• Q: API: blocks vs files (aka “NAS”)vs objects (OSD)?

• Q: protocol: FCP vs TCP/IP vs … ?

• A: (to all the above) it depends …

• Storage network (SN): any (mostly) dedicated network,

installed (mostly) for storage traffic whatever the hardware, API, or protocol

“SAN”

FC E’net

iSCSI

Fil

es

Blo

cks

FC E’net

Fil

es

Blo

cks FS/block com

bo


Some common storage architectures

Mapping the SNIA model onto some current implementations





Direct-attach block storage

Disk array

Hos

t,no

LVM

File

/rec

ord

lay

erB

lock

lay

er

Hos

t. w

ith L

VMan

d so

ftwar

e R

AID

Hos

t w

ith L

VM

Application • Direct-attach

• Multi-attach box

E.g., SCSI





SN-attached block storage

Disk array

Hos

t,no

LVM

File

/rec

ord

lay

erB

lock

lay

er

SN

“SN” = any networkused for storage access.E.g., Fibre Channel,Ethernet, …

Hos

t with

LVM

Application





SN-attached block storagewith metadata server

Disk array

Hos

t with

LVM

File

/rec

ord

lay

erB

lock

lay

er

SNBlock-aggregation

metadata server

Hos

t,no

LVM

Aggregation map(metadata) issupplied by anexternal server

Application





Block storage aggregationin a storage network appliance

Disk array

SN

File

/rec

ord

lay

erB

lock

lay

er


Hos

tw

ith L

VM Hos

t,no

LVM

Application aka “SNvirtualization”

Functions: LVM, caching





Multi-site block storage

Disk array

SNAggregation

appliance

Hos

t. w

ith L

VM

Application

Disk array


Hos

t. w

ith L

VM

Application

SN appliance

WAN

WAN

Host-to-host

Functions: point-in-timecopy, caching, local & remote mirroring, …

WAN

Device-to-device

File

/re

cord

laye

rB

lock

laye

r





File server

NAS system

HostHost

File

/re

cord

lay

er

Blo

ckla

ye

r

LAN-attached“NAS” system

May do SN/blockaggregation, etc. insidein the NAS system box

Application

LAN

privateSN?





File server controller(“NAS head”)

NAS head

Disk array

File

/re

cord

lay

er

Blo

ckla

ye

r

No storage in thefile server controller boxHostHost

LAN

Application

SN





NAS/file server metadata manager (“asymmetric”)

Disk array

File

/re

cord

lay

er

Blo

ckla

ye

r

Hosts get file meta-data from FS/NAS controller, thenaccess the data directly

Hos

t. w

ith L

VM

Filemetadata

Blockaccesses

FS controller canalso be NAS server

HostLAN

SN

Application

File system metadata





Object-based Storage Device (OSD), CMU NASD

OSDdevice

File

/re

cord

lay

er

Blo

ckla

ye

r

Host

Object metadata

Host

Application

LAN

Securitymetadata

Filemetadata


Summary & conclusions


File/record layer

Database(dbms)

File system(FS)

The SNIA shared storage modelS

tora

ge

do

ma

in

Block layer


Se

rvic

es

Dis

cove

ry, m

onito

ring

Res

ourc

e m

gmt,

con

figur

atio

nS

ecur

ity,

billi

ngR

edun

danc

y m

gmt

(bac

kup,

…)

Hig

h av

aila

bilit

y (f

ail-o

ver,

…)

Cap

acity

pla

nnin

g

Application

Network

Host

DeviceBlock aggregation





Block layerSample architectures

Hos

t. w

ith L

VMan

d so

ftwar

e R

AID

1. Direct-attach

SN

Hos

t. w

ith L

VM

Disk array

2. SN-attach3. SN aggregation


Hos

t,no

LVM

File

/rec

ord

lay

erB

lock

lay

er

Application





File/record layerSample architectures

Hos

t. w

ith L

VMan

d so

ftwar

e R

AID

1. Direct-attach

File

/re

cord

lay

er

Blo

ckla

ye

r

NASserver

4. NAS serverHost

3. NAS head

NAS head

Host

LANH

ost.

with

LVM

Disk array

SN

2. SN-attachApplication


Uses for the model

• Vendors place products in the space of architectures clarify product differences

• Customers understand vendor offerings better

• The industry basis for common definitions, communication,

understanding, interoperability


Conclusions

• The SNIA shared storage model is both simple and useful to highlight similarities and differences as a basis for comparisons

• Still a work in progress data movers, tape drives, … better comparisons … suggestions?

• The SNIA-TC welcomes input: <[email protected]>


Authors

• John Wilkes, Hewlett-Packard (project lead)• Harald Skardal, NetApps• David Black, EMC• Wayne Rickard (SNIA TC chairperson), Gadzoox

• Co-conspirators: the rest of the SNIA Technical Council Dave Anderson, Seagate Technology Jim Carlson, IBM Garth Gibson, CMU/Panasas Kevin Phaup, HighGround Systems David Thiel, Compaq

Documents

Copyright © 2000,2003, Storage Networking Industry Association