Embed Size (px)
Citation preview
2
Storage Is Supposed To Be Getting Cheaper!• Disk cost is dropping
rapidly• $250 buys:
▫ 1994: 2 GB▫ 1999: 20 GB▫ 2004: 200 GB▫ 2009: 2000 GB
• But enterprise storage costs keep rising!
Where Is The Cost?• Hardware and software
make up a small percentage of total enterprise storage spending…
• …and hard disk drive capacity makes up a small percentage of that!
• Data center/environmental, administrative personnel, maintenance, and data protection are much bigger
• The biggest opportunity is inefficiency, but this has always been hard to tackle
3
Hardware/Software
Main-ten-ance
Network
Data Center
Operations and Admin-istration
Cost of
Capi-tal
Overhead from Utiliza-tion
4
Over-Allocation and Under-Utilization
Raw
Dis
k C
ap
aci
ty P
urc
hase
d
Usa
ble
Pro
tect
ed S
tora
ge
Capaci
ty
Allo
cate
d t
o
Serv
ers
Use
d b
y
File
s
Request
ed
C
apaci
ty
Requir
ed
Capaci
ty
Conventional storage
provisioning is grossly
inefficient
5
Thin Provisioning Simplified!
Actually Used
Allocated but unused
Used
Free for allocation
Traditional storage provisioning
Thin storage provisioning
Thin Provisioning: Potentially Problematic• Storage is commonly over-allocated to
servers• Some arrays can “thinly” provision just the
capacity that actually contains data▫500 GB request for new project, but only 2
GB of initial data is written – array only allocates 2 GB and expands as data is written
• What’s not to love?▫Oops – we provisioned a petabyte and ran
out of storage▫Chunk sizes and formatting conflicts▫Can it thin unprovision?▫Can it replicate to and from thin provisioned
volumes?
7
Are You Solving a Technical or Business Issue?
Ever Play the “Telephone” Game?
Application
File/Record Layer
File System
Database
BlockAggregation
Host
Network
Device
Storage Devices
IV
III
IIc
IIb
IIa
I
SNIA Shared Storage Model
Each layer obscures the ones above and below it
9
File System
Storage
It’s (Relatively) Easy to Allocate on Write
As applications write data
Capacity is allocated
File system write requests pass through to storage systemsso they can wait to allocate as requested
10
File System
Storage
But What About De-Allocate on Delete?
Data is deleted
Capacity is freed up
Most file systems don’t send a consistent “de-allocate” message to storage
so many thin systems get fatter over time
11
Two Approaches To Thin
12
Server Smarts: Metadata Monitoring• File system/VM combos can handle thin
provisioning on their own▫ ZFS, Veritas Volume Manager, VMware VMFS
• Arrays can “watch” an operating system allocate and de-allocate storage▫ Perilous! Known file systems and volume formats only!▫ Data Robotics Drobo supports FAT32, NTFS, HFS+
File System
Storage
Drobo watches the file
allocation table for deletes
13
Storage Smarts: Zero Page Reclaim•Storage arrays watch for “pages”
containing all zeros and simply don’t write them▫IBM XIV, 3PAR, NetApp (with dedupe), HDS,
EMC V-Max•Some storage vendors rely on utilities to
reclaim▫NetApp SnapDrive for Windows 5.0▫Compellent Free Space Recovery▫Veritas Storage Foundation Thin
Reclamation•Can also force it with sdelete
14
Zero Page Reclaim: Pros and Cons
•Pro:▫Straightforward to implement in storage▫Some implementation: VMware
eagerzeroedthick•Con:
▫Requires application/OS/file system to actually have written all zeroes - most just ignore unused space rather than zeroing
▫Most implementations are page-based▫Drives more I/O▫VMware thin/thick don’t work
15
The Lingo: WRITE_SAME•Facilitates zero page reclaim
▫“Write this block 1,000,000 times”•Pro:
▫Conserves I/O operations▫Popular with array vendors▫Exists and is even implemented (a little)
•Con:▫Depends on file system layer intelligence▫Still introduces extra I/O▫Could be very, very bad in a thin-unaware
array
16
The Bridge: Veritas Thin API• Thin Reclamation API can
communicate de-allocation to arrays by zeroing using WRITE_SAME/UNMAP▫ Introduced in 5.0 (UNIX)
and 5.1 (Windows)▫ Supports 3PAR, EMC
CLARiiON CX4, HDS USPV/VM, HP XP20k/24k, IBM XIV
▫ Will also support Compellent, EMC Symmetrix DMX, Fujitsu Eternus, HP EVA, HDS AMS, IBM DS8k, NetApp
• SmartMove copies only allocated blocks▫ Supports any/all storage
systems▫ Works with thin-capable
arrays▫ Speeds up migrations in
all cases
17
What About TRIM?•TRIM (ATA) and TRIM/UNMAP/PUNCH (SCSI)
can inform storage that a block is no longer needed
•Designed for SSD architecture:▫Cells grouped into 4 kB pages and 512 kB
blocks▫Only empty pages can be written to▫Writing to empty pages is quick!▫Writing to used pages requires a block erase▫Read-erase-write is slow(er)
•OS support for TRIM:▫Windows 7 & Server 2008 R2▫Linux 2.6.33, Open Solaris, FreeBSD 9
TRIM Isn’t For Thin•Not really a thin-provisioning command
but could play one on TV▫NetApp proposed a hole punching standard
to INCITS T10 committee▫HDS and EMC prefer UNMAP bit▫A similar NetApp approach uses NFS and a
Windows file system redirect
More Obstacles!
20
Granularity (Page Sizes)
Large page – no thin provisioning
Small page – thin even with fragmentation
21
Processing and Scheduling
Intensive Ineffective
22
Fragmentation Kills Thin Provisioning
Fragmented filesystem spansthin pages
Defragmentedfile system allowsthin provisioning
The Performance Crunch•How high can we drive utilization without
killing performance?
Stephen’s Dream•Thin provisioning could be awesome, provided
it is integrated at all levels of the stack▫Smart applications that don’t spew data
everywhere▫Smart file systems and volume managers that
communicate what is and isn’t used▫Smart virtualization layers that don’t obscure
usage▫Smart storage systems that act on all of this
information with granularity and without falling over dead
▫Smart monitoring systems to tie everything together and head off disaster
Thank You!
Stephen [email protected]/sfoskett+1(508)451-9532
FoskettServices.comblog.fosketts.net
GestaltIT.com
25
