Sizing your content databases

Understanding the limits

Randy Williams

• Enterprise Trainer & Evangelist• Based in San Diego, CA• SharePoint MVP alum (2009 – 2011)• Speaker at many global conferences

• 20+ years in IT● Developer, consultant, trainer, author

• Columnist: SharePoint Pro magazine• randy.williams@avepoint.com• @tweetraw

Agenda

Understanding new limits

Remote BLOB storage (RBS)

SummaryQ&A

1Achieving

larger capacities

2 3 4

Agenda

Understanding new limits

1

The SharePoint storage dilemma

• Documents, databases, and BLOBs• Storage growth

SharePoint

SQL Server 2008/R2

Content Database

Content Database

Content DatabaseYear 0

ContentYear 1

CollaborationYear 2

Content MgtYear3

Records Mgt

Active Content Actual Content

Previously supported limits

1 TB

200 GBGeneral use scenarios

Large, single-site repositories and archives (records center)

100 GB site collection *

* A larger site collection is supported if itis the only site collection in the database

Revised limits (July ‘11)

No explicit

limit

200 GBGeneral use scenarios

Document archive scenario:caveats apply

No explicit size – limit by scenario, database size,

item count

4 TBAll scenarios:caveatsapply

Site collection

Understanding scenarios

• SharePoint is multi-purpose• Scenario primarily refers to needs and

usage patterns● Read/write centric● Concurrent users● Average/peak loads● Recovery objectives

• Isolate different usage patterns toseparate databases

Common scenarios

Record Center• Long term retention• Low volatility – few write

operations• Very limited reads

Team Site• Day to day collaboration

w/ shorter retention• Heavy read and write

operations

Larger databases Smaller databases

What are the 4TB-level caveats?

• A larger db requires faster storage● Between 0.25 – 2.0 IOPS/GB● 4TB DB : 1000 IOPS minimum

• Plans developed for DR/HA• Capacity planning/perf testing• Recognize added complexity

● Skilled architects and proactive admins• 60M total item limit per db

http://technet.microsoft.com/en-us/library/cc262787.aspx

What are the >4TB caveats?

• All 4TB caveats, plus• Document Center or Record Center only• In any given month

● <5% of content accessed● <1% of content modified

• No alerts, user workflow, item-level security, et al

http://technet.microsoft.com/en-us/library/cc262787.aspx

Why is 200GB still a good target?

• Support operations are much easier• Better performance

● The larger the db, the slower it gets• Easier to meet backup and recovery

objectives● Most recoveries begin with a db restore● Can you meet your recovery objectives?

• Patching / upgrading is faster200 GB

Book giveaway

• What is the maximum number of items that can be stored in a single content database?

Why are larger DBs slower?

• Read queries take longer● More rows to filter, group and sort

• Write queries take longer• Increased lock contention

● Causes more blocking• More data, but data cache same size• DB maintenance takes longer

● reindex● dbcc checkdb

What happens as size increases?

http://technet.microsoft.com/en-us/library/hh395916.aspx

SIZE AFFECTS PERFORMANCEDemo

Agenda

Achieving larger

capacities

2

Achieving storage performance

• Storage array (RAID 1+0)● 10 300GB SAS drives, 15k RPM● 1.5 TB effective space● ~1500 IOPS = 1.0 IOPS/GB

• Set of drives (RAID 1+0)● 4 750GB SATA drives, 10k RPM● 1.5 TB effective space● ~300 IOPS = 0.2 IOPS/GB

• Go with higher quality storage● SAS > SATA ; SAN > DAS

Scaling storage

• Multiple storage arrays (RAID 1+0)• Break out into multiple LUNs• Add additional data files to DB, one per

array• Advice

● Many smaller drives > fewer larger ones

● RAID 1+0 > RAID 5

Data

\SP_DocCenter_1.mdf

\SP_DocCenter_2.ndf

\SP_DocCenter_3.ndf

\SP_DocCenter_4.ndf

\SP_DocCenter.ldf Log

F:

G:

H:

I:

J:

Additional performance guidance

• How many data files?● Advice varies – between 0.25 to 1 per physical CPU● Each on a different spindle/LUN

• Adjust database growth settings● Use 50-100MB for each data file● Use 20-40MB for log

• Enable instant file initialization• Optimize tempdb

● Use multiple data files● Pre-size to 25% of largest db● RAID 1+0

http://slidesha.re/pwVlJM

DB SETTINGS AFFECT PERFORMANCE

Demo (if time permits)

Achieving Recovery Objectives

• Built-in SharePoint backup is incapable of working with large capacities● Site collection backup limit : 15GB● Practical database backup limit : 200GB

• Look at your backup/recovery objectives● Most recoveries involve a database restore

• Deploy SP1 – site recycle bin• Look for third-party solutions

http://slidesha.re/rlv3u1

Agenda

Remote BLOB storage (RBS)

3

Remote BLOB Storage (RBS)

• Storing document (BLOB) outside database

• Cannot be used to scale beyond database limits● Effective size = DB size + BLOB store

• Can externalize based on document size• Built in RBS support with SQL Server

2008 (FILESTREAM provider)

Overview of BLOB externalization

Web Front-end

Upload

File System

Po

inter

(stub

)

SQL Server

Metadata

File

RB

S

BLOB

Externalized BLOB is transparent to both SharePoint and its users

Advantages of externalizing BLOBs

• Reduce storage costs• Increase performance

● Read & write of BLOBs● All other database activities

• Access to features of BLOB storage platform

• Efficient content restructure● Shallow copy in SP1

Advantages of keeping BLOBs in SQL

• One storage container to● Maintain● Monitor● Recover

• Tier I storage● Performance relative to lower tiers of storage

benefits all content access

• SQL caching● Performance of reads/writes of small documents● SQL caching benefits reads

RBS Guidance

• Consider using in document-heavy databases• Trade off

● Storage cost & performance benefits versus● More complex architecture (support, DR, HA)

• Consider third party providers● More full-featured solutions

• In general● Do not externalize documents <1MB ● Ideal number varies widely

For more, see bit.ly/v4iMMm

Book giveaway

• What is the SQL Server command that checks for database corruption?

Agenda

4Summary

Q&A

In review

• 4TB is the new supported limit for all scenarios

• No limit for record/document centers• Keys to achieving larger sizes

● Storage performance planning/testing● DR/HA planning/testing

• RBS offers benefits but does not extend these limits

randy.williams@avepoint.comhttp://linkd.in/plEEb1@tweetraw

Questions?

Victory Lap- social event "SharePoint Victory Lap" Social Event for

SPSLA will be at: 5:30pm to 8pm at Di Piazzas (5205 E. Pacific Coast Hwy, 90804)

We want your feedback!Use this QR code or visit:http://sps.la/feedback

Silver Sponsors: