Stephen Archbold. Working with SQL Server for 6+ years Former Production DBA for 24/7 High volume...
Preview:
Citation preview
- Slide 1
- Stephen Archbold
- Slide 2
- Working with SQL Server for 6+ years Former Production DBA for
24/7 High volume operation Currently SQL Server consultant at
Prodata Specialising in Performance Tuning and Consolidation Blog
at http://blogs.Prodata.ie and
http://simplesql.blogspot.comhttp://blogs.Prodata.iehttp://simplesql.blogspot.com
Get me on twitter @StephenArchbold About me
- Slide 3
- Data Filegroup/File Fundamentals Storage Design Patterns OLTP
Data Warehousing Fast Track style Data Warehousing on a SAN What
other go faster buttons have we got Case Study The unruly fact
table How do we make the changes Agenda
- Slide 4
- General Filegroup Recommended Practices. Separate for: Nothing
but system tables in Primary I/O patterns Different volatility Data
Age If using Multiple Files in a Filegroup Files must be equally
sized Files must be equally full SQL does not redistribute data
when adding more files Filegroup/File Fundamentals
- Slide 5
- Transactional processing is all about speed You want to get the
transaction recorded and the user out as quick as possible Metric
for throughput becomes less about MB/Sec, and more about
transactions and I/Os per second Pattern 1 - OLTP
- Slide 6
- Solid State Disk becoming more commonplace These thrive on
Random I/O As the databases can be small, file/filegroup layout can
suffer Faster disk brings different challenges Challenges of
OLTP
- Slide 7
- File2.NDF PRIMARY Filegroup File MyDB.MDF File1.NDF File1.NDF
Transactions Reference Ref.NDF Volatile Volatile.NDF
PAGELATCH!
- Slide 8
- Facts and Figures
- Slide 9
- Behind the scenes Wait Type% SOS_SCHEDULER_YIELD 55
PAGEIOLATCH_EX 17 PAGELATCH_SH 15 ASYNC_IO_COMPLETION 5
PAGELATCH_UP 5 Wait Type% SOS_SCHEDULER_YIELD 66 PAGEIOLATCH_EX 12
PAGELATCH_SH 10 ASYNC_IO_COMPLETION 7 SLEEP_BPOOL_FLUSH 2 Single
FileTwo Files
- Slide 10
- Resolving in memory contention lies with the file layout This
is actually nothing new, TempDB has been tuned this way for years!
Keep in mind, files are written to in a round robin fashion What
can we take away from this?
- Slide 11
- Data Warehousing
- Slide 12
- Large Volume Star Schema Need to optimize for sequential
throughput Scanning Entire Table Not Shared Storage Pattern 2 Fast
Track Scenario
- Slide 13
- Filegroup File / LUN Large Partitioned Fact Table Enclosure 1
MyFact_part1.NDF MyFact_part2.NDF Controller 1 HBA 1 Partition 1
Partition 2 Enclosure 2 MyFact_part5.NDF MyFact_part6.NDF
MyFact_Part7.NDF Myfact_Part8.NDF Controller 1 Controller 2 HBA 2
Partition 5 Partition 6 Partition 7 Partition 8 CPU
MyFact_Part3.NDF Myfact_Part4.NDF Controller 2 Partition 3
Partition 4 CPU Enclosure MyFact_part9.NDF MyFact_part10.NDF
MyFact_Part11.NDF Myfact_Part12.NDF Controller 1 Controller 2 HBA 3
Partition 9 Partition 10 Partition 11 Partition 12 CPU
- Slide 14
- Pros Easy to figure out your needs Simple, cheap and fast In
depth guidance available from Microsoft Cons Not recommended for
pinpoint queries Only really for processing entire data sets Need
VERY understanding Infrastructure team Fast Track Pros and
Cons
- Slide 15
- Large Volume Star Schema Cannot optimize for sequential
throughput Shared Storage More mixed workload Pattern 3
Datawarehouse on SAN
- Slide 16
- We need Read Ahead Enterprise edition is capable of issuing a
request for 512KB on a single read ahead request (Standard youre
stuck at 64K) It can issue several of these (outstanding I/O) at a
time, up to 4MB But you may not even be close to 512KB Goal Large
Request Size
- Slide 17
- Run something like: And watch this guy: How close are you to
the 512k Nirvana
- Slide 18
- #1 killer of read ahead Read ahead size will be reduced if
pages being requested arent in logical order Being a diligent type,
you rebuild your indexes Because SQL is awesome, it does this using
parallelism! So whats the catch? If Read Ahead is your goal, MAXDOP
1 to rebuild your indexes! Fragmentation - Party Foul Champion
- Slide 19
- Enclosure 1 PRIMARY MyDB.MDF Filegroup File / LUN
- Slide 20
- Partition 1 Partition 2 Partition 3 Partition 4 Partition 5
Partition 6 Partition 7 Partition 8 Enclosure 1 Filegroup File /
LUN MyDB.MDF Primary Dimensions Volatile Dimensions.NDF Staging.NDF
Facts.NDF Facts Fact.NDF Large Fact Partition1.NDF Partition2.NDF
Partition3.NDF Partition4.NDF Partition5.NDF Partition6.NDF
Partition7.NDF Partition8.NDF
- Slide 21
- How does Analysis Services pull in data? Getting data out of
your Data Warehouse for Analysis Services
- Slide 22
- On read heavy workloads and Enterprise Edition, Compression If
storing multiple Tables in a Filegroup: -E For Data Warehouses -
This allocates 64 extents (4MB) per table, per file, rather than
the standard 1 (64K) If using multiple Files in a Filegroup -T1117
For all - This ensures that if auto growth occurs on one file, it
occurs on all others. Ensures round robin remains in place In
General on dedicated SQL servers Evaluate -T834 Requires Lock Pages
in memory enabled This enables large page allocations for the
Buffer Pool (2Mb 16Mb) Can cause problems if memory is fragmented
by other apps Do we have any go faster buttons?
- Slide 23
- 3 TeraByte Data Warehouse Table scan was topping out at 300
mb/sec Storage was capable of 1.7 GB/sec Table partitioning was in
place All tables were in a single Filegroup Had to get creative on
enhancing throughput Case Study The Unruly Fact Table
- Slide 24
- 16 core server, Hyper Threaded to 32 cores 128 GB of Memory
SQLIO fully sequential, storage gives 2.2 GB/Sec 32 range scans
started up to simulate workload Page compression was enabled, the
T834 trace flag was enabled MAXDOP of 1 on the server to ensure #
of threads were controlled Test Conditions
- Slide 25
- Facts and Figures
- Slide 26
- Other metrics ScenarioTime SecsAvg IO (K)Avg MB/SecMax MB/Sec
Baseline70181575622 Post Index Rebuild with MAXDOP 168181615668
Single FG for large table(s)565119421196 Multiple FG one per
partition425051,1501,281
- Slide 27
- Thankfully easy - Index rebuilds! For non partitioned tables,
drop and re-create on the new Filegroup For partitioned tables
Alter the partition scheme to point to the new FileGroup For heaps,
create a Clustered Index on the table on the new filegroup, then
drop it! How do we make the changes
- Slide 28
- File and Filegroup considerations can yield huge gains Know
your workload and optimise for it If you have a Hybrid workload,
then have a Hybrid architecture! Dont neglect your SQL Settings
Code changes and indexes arent the only way to save the day!
Summary
- Slide 29
- Paul Randal Multi file/filegroup testing on Fusion IO
http://www.sqlskills.com/blogs/paul/benchmarking-multiple-data-files-on-ssds-plus-the-latest-fusion-io-
driver/
http://www.sqlskills.com/blogs/paul/benchmarking-multiple-data-files-on-ssds-plus-the-latest-fusion-io-
driver/ Fast Track Configuration Guide
http://msdn.microsoft.com/en-us/library/gg605238.aspx Resolving
Latch contention
http://www.microsoft.com/en-us/download/details.aspx?id=26665
Maximizing Table Scan Speed on SQL 2008 R2
http://henkvandervalk.com/maximizing-sql-server-2008-r2-table-scan-speed-from-dsi-solid-state-storage
Specifying storage requirements (Find that sweet spot!)
http://blogs.prodata.ie/post/How-to-Specify-SQL-Storage-Requirements-to-your-SAN-Dude.aspx
Fragmentation in Data Warehouses
http://sqlbits.com/Sessions/Event9/The_Art_of_War-Fast_Track_Data_Warehouse_and_Fragmentation
Partial Database Availability and Piecemeal restores
http://technet.microsoft.com/en-US/sqlserver/gg545009.aspx Useful
links
- Slide 30
- Thank you!