© 2007 Hitachi Data Systems
Hitachi Dynamic Provisioning Best Practiceswith Applications
23 January 2008
Steve BurrSolution Architect, Global Solution ServicesHitachi Data Systems
2
Presentation Goal
• Discuss best ways of using Hitachi Dynamic Provisioning software with an emphasis on real applications
– Microsoft Exchange 2003, Microsoft Exchange 2007 – SQL Server 2005– Oracle 10g, Oracle 11g RAC
• Effective use of over provisioning
3
Hitachi Dynamic Provisioning (HDP)
Host Servers only see Virtual Volumes
Dynamic Provisioning
Pools
Dynamic Provisioning
Virtual Volumes
(DP-VOLs)
LDEV LDEV LDEV LDEV LDEV LDEV LDEV LDEV
Disk Drives/Drive Groups
LDEVs(Pool-VOLs)
Actual storage capacity in Dynamic Provisioning
pool is assigned when host writes data
to new area of LUN
Hitachi Dynamic Provisioning
Volume shows“Virtual Volume”which doesn’t
have actual storage capacity
Write Data
4
Hitachi Dynamic Provisioning Three Potential Benefits
Thin provisioning plus Hitachi Virtualization, Tiered Storage, Volume Migration
• Easier management– Simpler planning (reactive strategic management)– Avoid or defer tough decisions about LDEV size– Control, change and optimize: capacity, performance, reliability , cost tier
• Naturally balances performance– With large pools there are more spindles than static provisioning– May scale performance by growing pool
• Over provisioning capacity– Simplify capacity planning– Reduce need for “urgent change”– Reduce need for – “disk X on server Y has run out.
We’ll have to re-allocate the whole estate.”
5
Capacity Leveling: Example
An example• Three Database Applications• Initial estimates: 500GB
1500GB Used 1500GB Free
• Actual use: 100GB, 200GB, 800GB
2000GB Used 1100GB Used400GB Free
N N N
PVV V
N N N
?
V VV
6
Performance Leveling: Example
Small database
Largedatabase
Low I/O Easy Easy
High I/O Problem area
Good but needs design
Pool
ExampleMicrosoft Exchange 2007 design:
5,000 Mailboxes400 MB/mailbox (2TB)Workload = 1 IOPS/mailbox2 Servers
Design = 8*RAID-10(2+2) AG
But Find that some groups give hotspots so
redesign
ThenDecide to roll out BlackBerrysNew IOPS = 3 IOPS/mailboxBut 2TB is unchanged?
7
Over Provisioning
Allocating more storage space to users than they’ll initially need
No over provisioning (rightsizing)• No growth benefit but performance and management benefits still applyMild over provisioning• Giving headroom for expansionAggressive over-provisioning• Greatest benefits• More than you’d want to buy, “Every LUN is 2TB”• Needs care and safeguardsAggressive but risk averse, over provisioning• Over provision aggressively but• Combine storage, application and OS techniques to remove risk• Great benefits with safeguards• Particularly good for early deployments, uncontrolled environments and test
environments
8
How Dynamic Provisioning will perform with applications:
Will depend on • platform factors (particularly file system)• application factors (particularly storage configuration)• how it is managed
9
File system factor 1 – Metadata
• Metadata: management information• Includes: times, names, directories, pointers to data, maps, label
– inodes, MFT, FAT– Where is it: start, end, multiplexed, how much touched?– When created
• Space is not wasted, just prematurely allocated• Still have management benefits and performance benefits
Metadata Rounded to page size
10
File system metadata
JFS (VxFs) Top only
HFS Write by 10MBTop only
OCFS2 Top onlyWrite by 2GB (2%)
Linux
Write every 128MB (30%)
Write by 52MBTop onlyUnder examinationWrite every 64MB (65%)Top onlyTop only
Windows 2003 NTFS
With Max AUAt 4K page
With Max AU
XFSExt2, Ext3
UFSVxFSZFSJFSJFS2VxFS
AIX
Solaris
HP-UX
11
File system factor 2 – reclaim policy
• If it starts efficient, what happens when it’s used?• May depend on how you use it.• Where does “next free block” come from?• Free space management:
– Table, bitmap or linked list of blocks– One or multiple lists, multiple levels
MRU – most recently used
LRU – least recently used
Examples…
12
Metadata and reclaim policiesExamples of pool use
0
100
200
300
400
500
600
Used Pre-allocated
0
100
200
300
400
500
600
Used MRU
0
100
200
300
400
500
600
Used Allocated
0
100
200
300
400
500
600
LRU Used
pre-allocateUFS
LRU reclaimNTFS
MRU reclaimASM
“Leaky”OCFS2
Is this the only good one?
Pool Used versus “Space Used”
13
Is reclaim always an issue?
• No – many storage requirements are quite static• Needs to be addressed:
if application creates and recreates files and file system doesn’t reclaim well
• Workarounds:– Don’t over provision– Use static provisioning– Administration methods…
14
Controlling space
Dynamic Partition Expansion:1.Create and map over-provisioned volume2.But don’t allocate all to partition3.When top of partition is reached
OS is forced to reclaim4.If total space gets low5.Use OS to expand partition6.Can be scripted7.Generally easier than hardware add, lun resize etc.8.Recommended if you have no detailed guidelines9.Helps with metadata and reclaim issues
echo select disk 51 > diskpart.screcho select partition 1 >> diskpart.screcho extend size=600 >> diskpart.screcho detail disk >> diskpart.scrdiskpart /s diskpart.scr
ALTER DISKGROUP VS1DG RESIZE DISK DISKVS1 SIZE 738M;
15
Application testing
• Construct lab with: application, test tools andHitachi Dynamic Provisioning software
• Test: volume manager, file system then application
• Monitor “allocated pool” versus “used space”whilst performing typical actions:
– Initial Creation and Configuration– Addition, Append, Modify, Deletion– Force Fragmentation– Backup and manage
• Analyze and produce guidelines
16
Exchange 2003 and 2007 tests
• Microsoft loadsim and loadgen
• Realistic, unlike JetStress
• 10 days: 4.8m transactions, 1.3m messages read, 1.25m delivered, 350K deleted, appointments, attachments, dlists
• Set zero e-mail retention policy
17
Exchange tests
Exchange Data Use with Dynamic Provisioning
10200
10400
10600
10800
11000
11200
11400
9:36 14:24 19:12 0:00 4:48 9:36 14:24 19:12
daily 0200 mdbspace reclaim
After days of testing.Gap normal.No increase over time.
18
Exchange tests
Exchange Data Use with Dynamic Provisioning
0
5000
10000
15000
20000
25000
30000
35000
18/05 20/05 22/05 24/05 26/05 28/05 30/05 01/06
Don’t defragment
mdb!
19
Exchange tests
Exchange Log Use with Dynamic Provisioning
010,00020,00030,00040,00050,00060,00070,00080,00090,000
18/05 20/05 22/05 24/05 26/05 28/05 30/05 01/06
Defragmentation did not help
Partition limit
Online Backup with truncate using Hitachi Protection Manager with VSS
20
Exchange best practice
• Exchange 2003 and 2007 similar
• Good for Mail Stores (.mdb)– Mailbox management safe
• Storage Group Log files (.log)– Unlimited on NTFS, so either:
• don’t over provision, • use dynamic partition expansion• or use static provisioning
• Do not defragment
21
SQL Server Tests
• No equivalent to Loadsim; used simple sql scripts• Create database, table, insert and drop rows:
7M records added/updated, 8M deleted, 3/4GB. • Investigate initial allocation and growth• Three data areas:
– Database (.mdf)– Online Log (.ldf)– Full and Log backups (.bak .trn)
22
SQL Server Tests
• SQL Server Space Allocation Policy• Perfect for Hitachi Dynamic Provisioning• Reduces riskCREATE DATABASE trusqlONPRIMARY ( NAME = trusql1,
FILENAME = 'S:\V\B906\trusql1.mdf',SIZE = 420MB,MAXSIZE = 42000,FILEGROWTH = 42)
LOG ON ( NAME = trulog1,FILENAME = 'S:\V\B907\trulog1.ldf',SIZE = 42MB,MAXSIZE = 10000,FILEGROWTH = 42)
23
SQL Server Tests
SQL Server DATA (Large Initial Size)
0100200300400500600700800
0 500 1000 1500 2000
MB
HDP WIN
420MB +Initial FS overhead
Insignificant over use
Soon keeps in step.
Alignment different.
SQL Server does format space
24
SQL Server Tests
SQL Server DATA (42MB INCREMENT)
500
600
700
800
900
1000
1100
MB
HDP WIN
Still no deviationAvg 95MB
25
SQL Server Tests
SQL Server DATA
0
500
1000
1500
2000
2500
3000
0 2000 4000 6000 8000 10000 12000 14000 16000
Records Added
MB
HDP WIN MODEL
Remains parallel
26
SQL Server Tests
SQL Server LOG
0500
10001500200025003000350040004500
0 2000 4000 6000 8000 10000 12000 14000 16000
Records Added
MB
HDP WIN MODEL
Required steady state.
27
SQL Server Best Practice
• Database (.mdf)– Good, base size on planned initial use– Increment base on days growth Could be as low as 42MB
• Online logs (.ldf)– Good, base size on planned initial use– Increment base on days growth
• Log file backups (.trn)– Unconstrained, either:
• don’t over provision• use dynamic partition expansion• use static provisioning
• None of this is significantly different from normal SQL Server Management
28
Oracle Tests
• Ported SQL Server scripts; added Oracle Physical SchemaeAutomatic Storage Management (ASM)• ASM is both a volume manager and a specialized file system.
Can dynamically add/remove/expand storage objects.• ASM does not conflict with: Hitachi storage, virtualization or
Hitachi Dynamic Provisioning. They complement. Some overlap – which gives more options.
• Hitachi/Oracle ASM recommendations for layoutOCFS2• Oracle Cluster File System – version 2• Shared file system for RAC
29
Tablespace configuration
CREATE TABLESPACE "VS1“;DEFAULT: SIZE 100M AUTOEXTEND ON MAXSIZE UNLIMITED
CREATE SMALLFILE TABLESPACE "VS1" DATAFILE '+VS1DG' SIZE 42M AUTOEXTEND ON NEXT 42M MAXSIZE 1GEXTENT MANAGEMENT LOCAL SEGMENT SPACE MANAGEMENT AUTO LOGGING DEFAULT NOCOMPRESS;
CREATE TABLE TESTVS1 (COL1 number(10) NOT NULL, COL3 number(10) NOT NULL)TABLESPACE “VS1" STORAGE (INITIAL 1M);
30
ASM: Tablespaces
0
100
200
300
400
500
600
700
800
900
MB
DG Pool TableChanged tablespace to compression on
31
ASM: Tablespaces
Multiple Tables per TableSpace
0
50
100
150
200
250
300
350
400
450
500
MB
DG Pool Table DGMultiple TableSpaces per DiskGroup and drop/re-create TableSpaces
0
50
100
150
200
250
300
350
400
450
500
DG Pool Table DG
32
ASM: Archive Redo Logs
0.0
0.5
1.0
1.5
2.0
2.5
3.0
GB
Pool Archive Redo
33
Oracle without ASM: factors
ASM
Switch, Cache, Virtualize, Dynamic Provision
Pools
VMLVLV
A
VM raw
A
FS
A
VMLVLV
VM FS
A A
CFS
A
DP-Volumes
Raw
A
DG
Far too many choices:Database version (20)Base OS (17+)OS version (8+)Volume Manager (0-2)File System (0-22)Oracle configuration parameters (30+)
34
Oracle Cluster File System – OFCS2
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
GB
Pool Table DG Difference
35
Oracle guidelines
• External redundancy only • Create tablespace
– minimize initial allocation• Create table
– minimize initial allocation
• NEXT=growth for a period – day (or week/month)– Ideally multiples or integer divisor of 42MB page.
36
Replication products
• ShadowImage and Volume Migrator today. Internal and external storage. Copy-On-Write, TrueCopy and Universal Replicator in development.
• Works well with Hitachi Dynamic Provisioning. Some rules, but they’re obvious.
• Useful configuration: HDP to HDP.• Tested with Hitachi Protection Manager and it works well
37
General recommendations
• Avoid tools which write all the disk: – low level unix media format or check– volume level copy tools (dd)
• Don’t software RAID-1 mirror or RAID-5
• Use file level copy• Defragmentation – not recommended
38
Hitachi Dynamic Provisioning Tools
• Raid Manager:raidvchkdsp –g HDP –v aouGroup PairVol Port# TID LU Seq# LDEV# Used(MB) LU_CAP(MB) U(%) T(%) PID
HDP MDF CL6-B-1 18 35 10044 b906 2814 93750 1 70 9
HDP LDF CL6-B-1 18 36 10044 b907 4032 93750 1 70 9
HDP AUX CL6-B-1 18 37 10044 b908 8442 93750 1 70 9
• Hitachi Device Manager– Create, delete, increase Pools– Create DP-VOLs, V-VOL Groups– Alerts
• Hitachi Tuning Manager– Trending– Trended alerts– Reports– Performance and capacity
Pool 1% used Pool 9
PoolThreshold
Max size as seen by OS.
PoolAllocated to Device
Device Details
39
Performance with Hitachi Dynamic Provisioning
• Generally all good news• Both modeling and testing seems to show that there is little or no
difference in back-end requirement.• But can benefit from leveling• Small reduction in peak front-end performance but is less often a
bottleneck• Your Hitachi Representative has modeling tools to assist
40
Thank you
• Call to action:– Hitachi Dynamic Provisioning is easy to understand and use.– If you have a Universal Storage Platform V or Universal Storage Platform
VM, evaluate it– Have a go at quantifying the cost savings
• More information:– “Guidelines for the Use of Hitachi Dynamic Provisioning Software with the
Microsoft® Windows Operating System and Microsoft SQL Server 2005, Microsoft Exchange 2003 and Microsoft Exchange 2007”
– “Guidelines for the use of Hitachi Dynamic Provisioning Softwarewith Oracle® Databases and Automatic Storage Management”
– “Oracle Database 10g Automatic Storage Management Best Practices with Hitachi Replication Software on the Hitachi Universal Storage Platform™Family of Products”, co-authored by Oracle and Hitachi Data Systems
41
Upcoming WebTech Sessions:
We have several additional WebTech’s planned this year
06 February: Virtualizing Storage to Gain Maximum Benefit from a VMware Environment
20 February: Replication Management19 March: Hot Spot Management with Hitachi Tiered Storage Manager 23 April: Creating Tuning Manager Reports
We will be adding additional sessions based upon participant requests.
• Please go to www.hds.com/webtech next week for the a link to the recording, presentation, and Q & A in addition to registering for upcoming WebTechs.
Questions/Discussion
42
Thank You
43