View
218
Download
1
Category
Tags:
Preview:
Citation preview
1© Copyright 2012 EMC Corporation. All rights reserved.
Building the Right Infrastructure for Microsoft
SharePoint
2© Copyright 2012 EMC Corporation. All rights reserved.
EMC and Microsoft Alliance• EMC and Microsoft have been engaged as
strategic partners for over 16 years
• Shared view of importance of information
• Joint sharing of intellectual property
• EMC participates in Microsoft Technology Adopter Programs (TAPs) and Rapid Deployment Programs (RDP)
• CEO-sponsored worldwide go-to-market alliance
• Over 300 published technical solutions
• Over 100 data sheets on Microsoft technologies
• EMC has Hundreds of Microsoft Certified Professionals (MCPs)
• Microsoft is one of EMC’s Top 5 customers
16+ Microsoft Gold/Silver Competencies
21x Time Partner of Year
2,700+ consultants worldwide
Cooperative Support Agreement
Equipment in Redmond labs
Leading Microsoft consultancies
3© Copyright 2012 EMC Corporation. All rights reserved.
Affordable. Simple. Efficient. Powerful.
VNXe3100 VNX7500VNX5700VNXe3300 VNX5100 VNX5500VNX5300
4© Copyright 2012 EMC Corporation. All rights reserved.
Optimized for MicrosoftComplete Integration Across the Microsoft Stack
BusinessProductivity
ApplicationPlatform
CoreInfrastructure
On-Premise / Cloud
• ESI/SCOM/SCVMM Packs• SMB3.0 Support • Thin Provisioning • ODX Copy Offload
• FAST Cache for 9x increase in TPS
• VFCache for 87% reduction in RT
• Combined for 11x increase in TPS
• Manage Storage from SharePoint UI
• Shrink SQL ContentDB by 90-95% via RBS
• 30-40% reduced search RT using FAST Cache
5© Copyright 2012 EMC Corporation. All rights reserved.
Challenge: Relentless Growth of Unstructured Data
People / Business Processes:
1. Disconnected information silos
2. Information not available to those who need it in real-time
3. Governance
IT Related:
4. TCO / Asset Utilization
5. Backup Windows / Recovery Processes
6. Version Control, user mapping, permissionsSource : IDC, The Digital Universe
Decade
6© Copyright 2012 EMC Corporation. All rights reserved.
Why SharePoint? Why Now?
Why SharePoint?– Fastest growing platform in MSFT’s
history– $2B in 2011, 20,000 seats/DAY!– Accessible, extensible, unified
content platform
Why now?– 67% of customers report rolling out
SharePoint to the entire organization
– New release (Wave 15) coming in November
Gartner Magic Quadrant for Enterprise Content Management
7© Copyright 2012 EMC Corporation. All rights reserved.
Infrastructure Challenges with Current Sharepoint Deployments1. Unstructured Data Takeover
– Primary document types stored in SharePoint are PDF, Word, PowerPoint, and large Excel spreadsheets
– Documents typically well over 1MB– Files take over valuable server disk space and resources
2. Avalanche of large media– SharePoint often stores large files like media images and videos– SQL not optimized for file level access in general and streaming in
specific – Applications will not return control until file is fully loaded– File size and concurrency will increase the chance of browser timeouts
8© Copyright 2012 EMC Corporation. All rights reserved.
Infrastructure Challenges with Current Sharepoint Deployments3. Old and unused files hogging valuable SQL storage / CPU
– No means to intelligently tier off / archive older stale content– 60-80% of SQL BLOB corpus rarely used or not used at all– Equates to over 50% of the SQL Content DB Corpus if not externalized!
4. Lack of scalability – Proliferation of server resources to address performance bottlenecks– Poor economic scaling of SQL server licenses– Native backup / recovery tools too slow for sizable documents
▪ Only 600GB backed up in six hours using native tools▪ Only several TB’s backed up in six hours using MSFT DPM
– Lack of consistency in recovery of farm, ContentDB, and BLOB Store– Lack of tools to load-balance server resources
9© Copyright 2012 EMC Corporation. All rights reserved.
Building a SharePoint Architecture for Scale
Legacy ECM
File Shares
Future content growth
Today Centralize & Take Control Consolidate & Optimize
5%
95%
BLOBs
Metadata
10© Copyright 2012 EMC Corporation. All rights reserved.
Metalogix StoragePoint Librarian
ContosoFinance
AP
ARHR
Employees
Training Materials
Handbooks and Policies
Invoices
Purchase Orders
File Share LibrarianContent stored on file
system.
ContosoFinance
AP
ARHR
Employees
Training Materials
Handbooks and Policies
Invoices
Purchase Orders
SQL Content DB
List Items and
MetadataContent stays on file share
StoragePoint CatalogsFile share
StoragePoint Creates
SharePoint Structure and Items
11© Copyright 2012 EMC Corporation. All rights reserved.
The Challenge with SharePoint BLOBs
10 x 100GB SharePoint Content Databases
1TB of SQL Content Database Storage
PPT
XLS
DOC TIFF
90-95% unstructured, rest metadata
“Typically, as much as 80 percent of data for an enterprise-scale deployment of SharePoint Foundation consists of file-based data streams that are stored as BLOB data. These BLOB objects comprise data associated with SharePoint files. However, maintaining large quantities of BLOB data in a SQL Server database is a suboptimal use of SQL Server resources. You can achieve equal benefit at lower cost with equivalent efficiency by using an external data store to contain BLOB data.”
Source: http://msdn.microsoft.com/en-us/library/bb802976.aspx
• Performance for large files, especially write intensive • Large file upload time can be improved by as much
as 50% with RBS
• Can’t easily & economically scale
• Asset Utilization SQL Servers
• BLOBs don’t benefit from SQL query engine. Why are they there?
• Long backups / long restores
• Lack of tiered storage = Poor TCO
12© Copyright 2012 EMC Corporation. All rights reserved.
Metalogix StoragePoint RBS ArchitectureStoragePointWindows
SharePoint Services
.NET
SOAP
WebDAV
REST
SPRequest
SharePoint Object Model
Content Database
Metadata
Shared Services
Profile Management
Compression
Encryption
Logging
Notifications
Endpoint
Manager
Adapter
Adapter
Adapter
Filters (Size and Type)
Monitoring
Migration
Caching
Adapter Framework
EBS
RBSAtmos
VNX
Isilon
13© Copyright 2012 EMC Corporation. All rights reserved.
Metalogix StoragePoint Data Flow
SQL Server
RBSProvider
SP Web Application
Service
Document
BLOB write handed off to RBS Provider
Invoice #
Date
Amount
Vendor
Metadata
Document uploaded
Metadata uploaded
Meta
data
saved
to
SQ
L S
erv
er
Is there a storage profile?
BLOB
writ
ten
to
SQL
Serv
er
Determine Endpoint
Synchronous Asynchronous
BLOB Migrator Agent
YES
NO
Share Object Store / Cloud
14© Copyright 2012 EMC Corporation. All rights reserved.
StoragePoint – Using Aging Rules
VNX/NL-SAS VNX/NL-SAS Compressed Isilon
Active Inactive Archive
Invo
ice
Date
> 1 y
ear
Invo
ice
Date
> 3 y
ears
Older Content to Lower $/GB Tiers
Met
adat
a
BLOB
• On-Premise• Private Cloud• Public Cloud
15© Copyright 2012 EMC Corporation. All rights reserved.
StoragePoint - Advanced Filters
Storage Profile
FileSystem (Sync)
Filter: File Type = PDF, TIF, TIFF
Share1
FileSystem (Sync)Filter: List = HR DOCS
Share2
FileSystem (Sync)Filter: Site (Web) = Finance
BLOB Store 3
16© Copyright 2012 EMC Corporation. All rights reserved.
EMC / Metalogix Reference Architecture
Metadata5-10%
BLOBs90-95%
FC Network
CIFS Network
Share NL SAS
Share SAS
Share Deep Compressed
SQL Search DB
StoragePoint Rule moves BLOB from Tier 1 to Tier 2 At Invoice Date + 6 months
StoragePoint Rule moves BLOB from Tier 2 to Tier 3 At Invoice Date + 1 year
VNX FAST Cache enables faster indexing/crawl/search
Reference Architectures: http://www.emc.com/collateral/hardware/technical-documentation/h8185-sharepoint-vnx-metalogix-psg.pdf http://powerlink.emc.com/km/live1/en_US/Offering_Technical/Presentation/remote-blob-storage-sharepoint-tp.pptx
• Flash 1st 75% Cost Reduction via VNX FAST and smart use of SSDs and NL-SAS
• FAST Cache Up to 30-40% reduction in response times for Search
• FAST VP 80% less performance tuning for SQL
• Lower SQL Licensing Costs due to server virtualization and RBS
tempDBSearch Query
VNX FAST VP to reduce performance tuning
17© Copyright 2012 EMC Corporation. All rights reserved.
Applying StoragePoint Advanced Filters
Metadata FC Network
CIFS Network
Share FLR
Share NL SAS
LUNFAST Cache
Use Advanced Filters to move HR data to where retention /
compliance is ensured
Encrypt sensitive data before it is moved to storage endpoint
(up to 256-bit)
LUNFAST VP
• Set retention periods at file system creation • Default/minimum/maximum
• Automatic File Lock and Delete
VNX File Level Retention:
Archive / Big Data
Active Data
18© Copyright 2012 EMC Corporation. All rights reserved.
Business Continuity
Virtualizing SharePoint is Essential
Consolidation
Performance
Availability
Maintenance
Load Balancing
Achieve 2-10x consolidation ratio, especially for larger deploymentsLower SQL Licensing by consolidating vCPU onto physical CPU
Improved front end performance with more, smaller WFEs rather than few large WFEs
VM based protection for SharePoint provides homogeneous high availability (WSFC, VMware HA)
Simplified DR management (Geo-Clustering, vCenter Site Recovery Manager)
Live migration of virtual machines (Hyper-V Live Migration, VMware vMotion)
Maximized overall performance with balanced cross farm HW utilization (SCVMM PRO , VMware DRS)
19© Copyright 2012 EMC Corporation. All rights reserved.
Three Sources of Savings
1. Better Asset Utilization– File Share typically not well utilized – VNX Utilization ~70% on SAS / NL-SAS and up to 90% on SSD
2. More Efficiency via Flash 1st – Results in fewer drives, lower energy bills, and lower $/GB– Also lower $/IOPS via FAST Suite
3. Lower SQL licensing via RBS and virtualization– Reduce database size and increase VM density per physical server
20© Copyright 2012 EMC Corporation. All rights reserved.
FLASH 1st Data StrategyHot data on FAST Flash SSDs—cold data on dense disks
“Hot”high activity As data ages, activity falls,
triggering automatic movement to high-capacity disk drives for lowest cost
Highly active data is stored on Flash SSDs
for fastest response time
Movement Trigger
FlashSSD
“Cold”Low Activity
Data
A
ctiv
ity
High- capacity
HDD
Data Age
21© Copyright 2012 EMC Corporation. All rights reserved.
VNX5500
210 15K DRIVES
32 TB Usable. 50% Y/Y Growth. 60 Days Data Shelf-life.
FLASH 1st Example
22© Copyright 2012 EMC Corporation. All rights reserved.
VNX5300
20200GB SSDs
16600GB 15K
SAS
243TB
NL-SASVNX5500
210 15K DRIVES
32 TB Usable. 50% Y/Y Growth. 60 Days Data Shelf-life.
FLASH 1st Example
60 DRIVES
FAST Cache & FAST VP
23© Copyright 2012 EMC Corporation. All rights reserved.
32 TB Usable. 50% Y/Y Growth. 60 Days Data Shelf-life.
FLASH 1st Example
Footprint Power IOPS
33K
3,938W45RUVNX5300
20200GB SSDs
16600GB 15K
SAS
243TB
NL-SAS
60 DRIVES
FAST Cache & FAST VP18RU
990W
74K
5%FLASH
24© Copyright 2012 EMC Corporation. All rights reserved.
VNX FAST SuiteA FLASH 1st strategy enabler
FAST Cache continuously ensures that the hottest data is served from high-performance Flash SSDs
FAST VP* optimizes storage pools automatically, ensuring that active data is being served from SSDs, while cold data is moved to lower-cost disk tiers
Together they deliver a fully automated FLASH 1st storage strategy for optimal performance at the lowest cost attainable
Monitor and tune the whole system with the complementary Unisphere Quality of Service Manager and Unisphere Analyzer
* Not available for VNX5100
Real-time caching with FAST Cache
Scheduled optimizationwith FAST VP
FlashFlashSSD
15KHDD
High-
capacity HDD
25© Copyright 2012 EMC Corporation. All rights reserved.
VNX FAST Cache boosts SQL2012
Data Disk RAID FC
OLTP DB Files
600K SAS 5 ON
TempDBs 600K SAS 1/0 OFF
DB Logs 600K SAS 1/0 OFF
FAST Cache: 14 x 100GB SSDsWorking Set < FAST Cache
Results:• 5x Improvement of IOPS and TPS• 30 Minute Cache Warming
IOP
STP
S
26© Copyright 2012 EMC Corporation. All rights reserved.
Lower $/GB and $ / IOPS with RBS
Average
$/GB SAS–15K$2/GB
LOWEST
$/IOSSD
$.99/IOMetadata
5-10%
BLOBs90-95%M
eta
log
ixS
tora
geP
oin
t
Example:• Avg. Capacity VNX5300 = 50TB• Without RBS on SAS-15K: $100K• With RBS and 5% SSD and 95% NL-SAS: $40-50K
LOWEST
$/GB NL-SAS$.43/GB
No RBS With RBS
27© Copyright 2012 EMC Corporation. All rights reserved.
Lower SQL Licensing Costs
SQL Server is rarely CPU constrained– For SharePoint the performance
bottleneck tends to be the WFE– Idle CPU = Wasted licensing $$
SQL2012 Server Enterprise Licensing:
– Physical: per physical core – Virtual: per vCPU (all you can eat
per physical core)– No cost difference between a
physical core and vCPU
Virtualization = greater VM density and lower SQL licensing costs
– Savings determined by vCPU : Core Ratio (e.g. 4:1 ratio =75% savings)
RBS = greater VM density and lower SQL licensing costs
– Lower CPU load on SQL servers
Have to weigh against performance / CPU utilization
– SQL ContentDB recommended CPU limit 50%
– Perfmon helps determine the status quo
28© Copyright 2012 EMC Corporation. All rights reserved.
RBS + VNX = More efficient SQL IO
Overall more efficient handling of IO– Reduction in overall IO by 20-30%– IO coalescing on VNX
Most IO handled by VNX CIFS share– 70-80% of backend IO handled by
CIFS share– RBS reduces SQL CPU cycles by only
storing the varbinary file attribute in SQL
– Actual BLOB is stored in more efficient file system
Non-Externalized Externalixed0.00
100.00
200.00
300.00
400.00
500.00
600.00
BLOB StoreContentDB
Source: http://powerlink.emc.com/km/live1/en_US/Offering_Technical/Presentation/remote-blob-storage-sharepoint-tp.pptx
29© Copyright 2012 EMC Corporation. All rights reserved.
Results• Average page load time improved by 4x after RBS • Backup times from SQL and SharePoint reduced by a
factor of 13x• >90% moved from expensive SQL drives to NL-SAS
Challenge• Internally facing communications site, which included
large amounts of rapidly changing large objects• Page load times and backup / recovery times
Large Government Contractor EMC VNX and Metalogix StoragePoint Improves Operational Agility and Performance
“The dashboard reports showed that I was now saving >90% of my storage on the expensive SQL drives.”
Derek MartinCloud Solutions Architect
30© Copyright 2012 EMC Corporation. All rights reserved.
SharePoint Performance Considerations
Performance for SharePoint highly use case dependent– Focus on Enterprise Content Management*
SharePoint workload for content management– Can be characterized by 60/20/10/10 for Browse/Modify/Search/Upload– File size, distribution, and concurrent access impact performance
Symptoms vs. Remedy – Symptoms often occur during individual tasks (e.g. file upload by user)– Sizing however needs to factor in system level workloads for the entire farm– In addition need to size system for blended workload (e.g. Exchange, SQL,
SharePoint)
31© Copyright 2012 EMC Corporation. All rights reserved.
SharePoint Performance Considerations
The storage subsystem is rarely the bottleneck– Generally web front end (WFE) CPU represents the bottleneck – VNX FAST Suite enables very efficient configurations (majority NL-SAS, 2-5% SSDs)– Search functions are the workloads that happen to benefit the most
▪ Search critical as SharePoint environments and document repositories grow ▪ Microsoft Test Report for Very Large-Scale Document Repositories validates VNX performance
– VNX typically deployed with mixed workloads (e.g. Exchange, SQL, and SharePoint)
Case for RBS: SQL Database not efficient at handling files IO– Each write represents an update of the table and the log– Non-externalized storage formats use the SQL buffer pool when the data pages are
accessed– Large files and high concurrency can lead to buffer overruns compounding the issue– Other processes such as index rebuilds can also compete for the same server
resources
32© Copyright 2012 EMC Corporation. All rights reserved.
Best Use of VNX FAST Suite for SharePoint
SharePoint Component
FAST VP FAST Cache Workload
Search Index - ++• R:W Ratio of 90:10; 32K block read, 64K write; Small working set.• Search Index re-uses the same blocks on disk as working space to process
list items during indexing. Storage IO quiet between crawls.• Highly changing, throw-away data.
Search Query + +• R:W Ratio of 70:30; 32K read, 64K write; Small working set.• Query improves due to random large block reads/writes being serviced by
FAST Cache, but Query storage is large and costly in FAST Cache. • Highly-read data with small burst write changes.
tempdb + +• R:W Ratio of 50:50; 8K read, 32K write.• TempDB (database page reuse) is ideally suited to FAST Cache algorithms. • Same blocks are re-used on disk and performance of TEMPDB directly
affects SharePoint performance request.
Content DB + +• R:W Ratio of 95:5; 16K (read and write); Working set reduced via RBS.• Low IOPs requirements does not require FAST Cache.• DB Index operations see an improvement.
BLOB Store - - • Low IOPs requirements, large-block I/O.
33© Copyright 2012 EMC Corporation. All rights reserved.
Recovery Options / Tradeoffs
What are the recovery objectives?– Recovery Level: Database, Database + BLOBs, Database + BLOBs + Farm– Recovery Time: how long does it take to recover – Recovery Point: How much can I afford to lose vs. how much am I willing to spend– Distance: Local vs. Remote– Granularity: Item level vs. entire corpus – Management Overhead: how much time do I need to spend on backup / recovery
Challenges with native tools:– Limited in scale, scope, and performance (e.g. STSADM, Central Administration, Recycle Bin)– Configuration data not backed up– Low backup / recovery rate: rated at 600GB in 6 hours (TB’s with DPM)– Manual re-creation of configuration – STSADM / Central Administration not recommended above 85GB for site collections or Content DB
larger than 100GB
34© Copyright 2012 EMC Corporation. All rights reserved.
Availability & Protection Methods / Tradeoffs
Requirements (cumulative)
Backup / Recovery Replication High Availability
Database (incl. BLOBS) SQL native backupSTSADM / Central Admin.Avamar
SQL Availability Groups SQL Failover Clustering
DB+ Remote BLOB Store VNX Snaps / Replication Manager
SQL Failover Clustering (DB Only)
Item-Level Recovery Recycle BinAvamar / Ontrack (not RBS aware)
VNX Snaps / Replication Manager / Selective Restore Manager or Ontrack
SQL Failover Clustering (DB Only)
Farm Configuration / VM / OS VNX Snaps / Replication Manager
Hypervisor virtual machine clustering
Site-to-Site RecoverPoint / Cluster Enabler / SRM
35© Copyright 2012 EMC Corporation. All rights reserved.
Replication Manager / Item-Level Restore
RM supports recovery of Content DB and Remote BLOB Store via script
Order of restore:– Single ContentDB BLOB Store – StoragePoint DB– Singe ContentDB
For item-level restore:– Point Selective Restore Manager or
Ontrack PowerControls to mounted RM database files
– Browse to content and restore the required file(s)
or
Source: http://powerlink.emc.com/km/live1/en_US/Offering_Technical/Presentation/remote-blob-storage-sharepoint-tp.pptx
or
36© Copyright 2012 EMC Corporation. All rights reserved.
Replication Manager Protection
Data protected Start time End time Duration
SharePoint farm and search 7:45:26 7:49:04 3 minutes 38 seconds
StoragePoint 7:57:08 7:57:09 1 second
BLOB stores 7:57:21 7:58:12 51 seconds
Total time to protect entire farm content: 4 minutes 30 seconds
Data restored Start time End time Duration
BLOB store 8:48:02 8:48:18 16 seconds
StoragePoint 8:52:45 8:54:07 1 minute 22 seconds
Content Database 08:59:47 9:03:22 3 minutes 35 seconds
Total time to restore a Content Database: 5 minutes 13 seconds
Full farm protection
250 GB Single Content Database Restore
37© Copyright 2012 EMC Corporation. All rights reserved.
SharePoint 2010 Disaster Recovery:Virtualized Farm (Full Replication)
SharePoint Farm ASecondary SitePrimary Site
BLOB Store
VMs RecoverPoint
VMs
VMs
VMs
• Resource/Protection Group level granularity
• Active/Active (Sync distances) or Active/Passive (Async distances)
Failover automation: VMware Site Recovery
Manager (SRM) Protection Groups for all server roles
EMC Cluster Enabler with Hyper-V failover clustering
Databases
BLOB Store
VMs
VMs
VMs
VMs
Databases
WFE
Application Servers
Crawl
SQL Cluster
38© Copyright 2012 EMC Corporation. All rights reserved.
Automated DRConsistency Group (RP) VNX Volumes Grouping SRM CE
Web Front Ends Boot + Query (optional) Protection Group Cluster Group
Query Servers Boot + Query Component Protection Group Cluster Group
Index Servers Boot + Index Component Protection Group Cluster Group
Application Servers Boot + Application Volumes Protection Group Cluster Group
SQL Server(s)
Boot + Pagefile (optional)
Protection Group Cluster Group
SQL System Databases
Configuration Databases
Search Databases
Content Databases
RBS BLOB Store
Designing DR consistency for SharePoint
Search consistencyExternalized Content consistency
39© Copyright 2012 EMC Corporation. All rights reserved.
Provisioning SharePoint - ESI Open / browse local or remote
SharePoint farm
Provision SharePoint Web Application:
– Create disks– Create content database– Create Web Application – Attach the content database
Create content database only (no WFE)
– Web Application can be created later using SharePoint admin and the database created in ESI
Users can select any SQL server that is part of the farm and any instance in
that server.
leave unchecked
40© Copyright 2012 EMC Corporation. All rights reserved.
Health Monitoring in the Private CloudEMC System Center Operations Manager (SCOM) Management Packs:
• Array discovery & health monitoring
• Full hierarchical storage schema and relationship view
• Dashboards incl. physical view (SPs, DAEs, NICs, HBAs, Drives) and logical view (RAID Groups, Volumes/Pools, LUNs, Protocols…)
• Performance Monitoring• End to end application service
view• Map / visualize relationships
between storage and SharePoint farm
EMC System Center Orchestrator (SCO) Management Packs:
• Focal point for automation of heterogeneous datacenter management
• Service Manager for workflows and pre-defined run books• 21 storage activities and
counting• Example Drive failure:
1. Gather SP Collects 2. Post on pre-defined share 3. Send alerts to admin
41© Copyright 2012 EMC Corporation. All rights reserved.
Leveraging VSPEX
Storage
Network
x86 Server
Hypervisor
Application
*not committed
Proven Infrastructure
Partn
er B
undle
ISV
43© Copyright 2012 EMC Corporation. All rights reserved.
RBS Options & Feature ComparisonFeature RBS FILESTREAM StoragePoint SourceOne AvePoint
Catalogue Files into native SharePoint ribbon X √ X XBLOB Externalization (RBS) √ √ √* √
Offload BLOBs to remote storage platforms X
√ √ √BLOB store must be local to SQL Server
Outbound BLOB cache for off-premises (i.e. Cloud) operations X √ X X
Define multiple storage endpoints per content database X √ √ √
Filter endpoints by file type or scope (i.e. list or content type) X √ X X
Take endpoints offline (and/or send notifications) if they fill up or enter an error state
X √ √ X
Use SharePoint filename or folder on offloaded BLOBs X √ X X
Promote SharePoint properties to endpoint X √ X XSecurely delete (shred) BLOBs X √ X XReport and Health Monitor Dashboard X √ √ X
Administration User Interface X√
√ √Central Admin
PowerShell Support X √ √
Compression X √ X X
Encryption X√
X XTransmission and "at rest"
Tiered Storage (HSM) support X √ X X
Orphaned BLOB Garbage Collection √ √√ √ √
Basic Policy-based Basic
Recommended