Embed Size (px)
Citation preview
Exchange Deployment Planning Services
Exchange Server 2010Planning and Sizing
Exchange 2010 Planning Goals
The Exchange 2010 Planning module has the following goals: Introduce planning tasks and
recommended configurations for Exchange 2010 Server
View typical role placement scenarios for Exchange 2010 Server
Ideal audience for this workshop Messaging SME Networking SME Security SME
Exchange 2010 Planning Audience
Exchange 2010 PlanningAgenda
In this module focus on the following: Overview of performance can
scalability testing and direction Guideline and ratios Role specific details Toolkit for planning and sizing
Exchange 2010 Planning
After this module you should have: Basic planning knowledge for
Exchange 2010 Basic understanding of Exchange 2010
Server role placement
Overview of Testing and Guidance Process
Research• Understand
hardware• Business
requirements to technical requirements
• Understand Exchange
Evaluation• Solution sizing,
configuration, validation
• Hardware selection
Implementation• Migrate & grow
AgendaA quick tour through the sizing process
• Exchange sizing and performance tuning can be complex− Use planning & sizing toolset to
simplify− Take advantage of hardware
advances to get the most out of Exchange
Understand Hardware
Driving Factors In The Technology Landscape• Major changes in server hardware technology in
the last decade have influenced changes in Exchange architecture
• Processor advances− 64-bit: massive amounts of addressable memory− Core density: cores per processor continues to grow
• Storage advances− SAS/SATA/SSD− Topologies: SCSI, FC, iSCSI, DAS− Growth in drive capacity (and areal density)
• Power consumption & efficiency• Virtualization• Cloud computing
Processor Advances
• Overall available megacycles per processor (socket) increasing rapidly− Per-core megacycles
constant or decreasing to maintain power requirements
• Processor technology improvements make GHz comparison somewhat meaningless
Storage Advances
Hardware• Since 2003, disk
capacity has grown dramatically− 2TB desktop class SATA
(and midline SAS) disks available, larger sizes available shortly
• Sequential throughput increasing linearly based on areal density− 2010 SATA =~ 250MB/sec
• Random I/O performance not expected to improve substantially− 15k RPM is the ceiling
Workload• Mailbox sizes rapidly
increasing (1-10GB desired)
• Knowledge workers (and IT) want everything online and instantly searchable− Reduced mailbox
management effort− Data accessible from
everywhere (incl. mobile client)
− Increased knowledge worker productivity
• Average message size increasing
Storage Terminology
• Three classes of drive types− Enterprise (ENT)
− Dual port SAS interface− 10K, 15K RPM− 146, 300, 400, 450, 600GB + − Large Form Factor (LFF) & Small Form Factor (SFF)
− Midline (MDL)− SAS – dual port− SATA – single port− 7200 RPM + LFF & SFF− 500GB, 750GB, 1TB, 2TB +
− Entry (ETY) – Not suitable for Exchange
Business Requirements
Business Requirements
• Information retention (size and duration)− Limited by Restore SLA − Regulatory requirements− Growth, mergers etc.
• Backup/restore strategy • Site level disaster recovery
− RPO/RTO
• Administrative model • Consolidation and virtualization• Power consumption
Understand Exchange
Scale Out vs. Scale Up
• Scale out is a strategic choice made by the product group
• Scale out provides the following at low cost:− Large mailboxes− High availability− Rich feature set
• Scaling up increases risk that an outage or failure affects more users
• Scaling up usually costs more, and can force feature decisions due to hardware choices− Consider all factors in the equation, particularly
storage
Scale Up Options
• Multiple Role Servers (“brick” deployments)− Likely the best option for big hardware (> 2
socket) – best hardware utilization overall− Be aware of recommendations for max processor
& memory
• Virtualization− Evaluate whether potential added complexity
and monitoring challenges make this a win
• Single role− Product not engineered for single role high scale
(> 2 socket)
Extreme caution necessary – validate carefully in a test lab
Supported vs. Recommended• Supported usually means well tested• Support statements define strict
boundaries• Recommendations define the “best
case” or the state that we want our customers to achieve
• Understand risks of going outside of recommendations or support boundaries
Processor Core Scalability• Single Role Servers
− Recommend a 2-socket platform− 4-core processors = 8 total cores− 6-core processors = 12 total cores
− Expect diminishing returns moving to 16+ cores on >= 4 socket platform
− Known issues updating memory across cores− Not NUMA aware or optimized for scale around data locality− Code can take longer to execute; transaction costs rise
• Multiple Role Servers− Recommend 24 cores maximum for high-scale “Enterprise
Multiple Role Server”− Multiple processes from different roles help us scale better
• Hyperthreading− Disable on production Exchange servers− Causes monitoring and capacity planning challenges
Role Ratio Guidelines
• Processor core ratios− CAS : Mailbox
− = 3 : 4− HUB : Mailbox
− = 1 : 7 (no A/V on Hub)− = 1 : 5 (with A/V Hub)
− GC : Mailbox− = 1 : 4 (32-bit GC)− = 1 : 8 (64-bit GC)
Processor and Memory Configuration
Recommended Configuration
Role Maximum Processor Cores Optimal Memory
Hub & Edge Transport 12 cores 1GB per core(or 8GB minimum)
Client Access Server 12 cores 2GB per core(or 8GB minimum)
Mailbox 12 cores 4GB plus 3-30MB per mailbox
Unified Messaging 12 cores 2GB per core(or 4GB minimum)
Multiple Role Server 24 cores 8GB plus 3-30MB per mailbox
Network Load Balancing
• Exchange 2010 requires load balanced CAS for internal connections− Consider HA needs− Size for connection count spikes
• Windows NLB− Not recommended above 8 nodes
• Hardware Load Balancer− Recommended for larger environments− Multiple Role Server HA scenarios
Sizing The Mailbox Role• Proper sizing is key across
all resources:
• Storage & memory are most critical – ensure proper sizing for performance, capacity, reliability
• In depth detail available at http://tinyurl.com/262cpg9
Mailbox RoleSizing Rules Of
Thumb
• 2-socket platform best for performance and TCO
• User profile determines resource requirements for IOPS, memory, CPU
• Don’t forget about high availability
Resource Key Considerations
Storage I/O and capacity requirements
Memory Database cache requirements (reduces I/O)
CPU Required for RPC operations, content indexing, mailbox assistants, replication operations
Network Log replication and RPC operations consume bandwidth
Mailbox Storage Sizing• Storage must be sized for
− Performance (IOPS)− Capacity (GB)
• Performance sizing based on user profile (message throughput)
• Capacity sized based on user mailbox size− See TechNet for details on
required overhead (whitespace, dumpster, etc.)
• Design will either be performance- or capacity-bound
• IOPS guidance based on production observations and internal testing
Estimated IOPS Per-MailboxMessages
Sent+Received
per mailbox per day
(~75KB average message size)
Database cache
per mailbox
(MB)
Estimated IOPS: Single
database copy
Estimated IOPS: Multiple
database copies
50 3 .060 .050
100 6 .120 .100
150 9 .180 .150
200 12 .240 .200
250 15 .300 .250
300 18 .360 .300
350 21 .420 .350
400 24 .480 .400
450 27 .540 .450
500 30 .600 .500
Mailbox Memory Sizing• Services need base memory for
ongoing operations: − Basic overhead for servicing user
requests− Content indexing− Mailbox assistants
• Store process needs per-user memory for database cache, based on user profile− Properly sized database cache
memory required for IOPS reduction
• Deep checkpoint depth + 32KB pages allow E2010 to benefit from larger memory configurations than E2K7
Mailbox Role Cache Memory Sizing
Messages Sent+Receive
d per mailbox
per day (~75KB average message size)
Database cache per
mailbox (MB)
50 3
100 6
150 9
200 12
250 15
300 18
350 21
400 24
450 27
500 30
Mailbox Memory Sizing• Cache size defaults based on
installed RAM− Size per-mailbox memory, then
map to fit in default cache− Remaining memory reserved for
base service requirements• Nehalem platform has new rules
for memory configuration− Haven’t seen a need to optimize
for memory speed, so optimize for memory size
• For example:− 4000 users with the 200 profile
(12MB per mailbox): 4000*12MB = 48GB
− 48GB fits in 53.6GB default cache
− Deploy 64GB server
Default Mailbox Database Cache Sizes
Server Installed Physical Memory
Database Cache Size
(Mailbox Role Only)
Database Cache Size
(Multi-role)
2GB 512MBNot
supported
4GB 1GBNot
supported
8GB 3.6GB 2GB
16GB 10.4GB 8GB
24GB 17.6GB 14GB
32GB 24.4GB 20GB
48GB 39.2GB 32GB
64GB 53.6GB 44GB
96GB 82.4GB 68GB
128GB 111.2GB 92GB
Mailbox CPU Sizing• Proper CPU sizing is critical: sizing
of other roles depends on it• Megacycle values provided are
based on a particular reference platform, newer CPUs differ− Megacycle adjustment based
on SPECint may be required• Sizing process & calculation can
get somewhat complex− Use calculator tools to simplify
this process− See TechNet guidance for
details on megacycle adjustments
• Recommend disabling hyperthreading− May cause capacity planning &
monitoring challenges
Estimated Per-Mailbox CPU Consumption
Messages Sent+Receiv
ed per mailbox
per day (~75KB average message size)
Megacycles for active or stand-
alone mailbox(increase by 10% for each passive copy)
Megacycles for
passive mailbox
50 1 .15
100 2 .3
150 3 .45
200 4 .6
250 5 .75
300 6 .9
350 7 1.05
400 8 1.2
450 9 1.35
500 10 1.5
Sizing The Client Access Server Role • CPU and memory are
key for CAS:Client Access Server
RoleSizing Rules Of
Thumb
• 2-socket platform best for performance and TCO
• CPU is typically the bottleneck, memory sizing is key as well
• 3 CAS CPU cores for every 4 Mailbox CPU cores (servicing active users)
• Load balancing is important for performance and high availability
• 2GB RAM per CPU core is optimal
Resource Key Considerations
CPU Required for handling client workload transactions, content conversion, garbage collection
Memory Memory required for ongoing transaction processing
Network All clients connect to CAS, network bandwidth and latency are important for client experience, load balancing likely required
Storage Utilized for content conversion, logging
Client Access Server Workload Sizing• Workload impact on CAS
server is variable depending on user profiles & mix of workloads
• CPU & memory scale guidance for CAS based on assumptions of a mixed-protocol heavy information worker profile− Consider other workloads
and adjust− Remember all MAPI
traffic now affects CAS• Use Windows Server 2008
R2 for best CAS scale− Major improvements in
rpcproxy (Outlook Anywhere), potentially scaling to 15k Outlook Anywhere users on 8-core CAS
CAS Workload Relative Cost Comparison
WorkloadCPU Cost (MHz/user
)
Network Cost (Kbytes/sec/user
)
Outlook 0.35 0.37
Outlook Anywhere
0.80 0.44
Exchange ActiveSync(delta from
Outlook)
1.60 1.04
Exchange Web Services
(Microsoft Entourage)
0.71 0.54
Outlook Web App 0.86 0.88
IMAP4* 0.86 0.14
POP3* 0.33 0.79
OWA SP1 Performance
• Major SP1 investment on improving OWA performance− Pre-fetch: get data to the client before
they ask for it− Async: make long running operations
happen in the background− Defer: wait to download scripts when
possible
OWA Perf Improvement Results
100ms/5Mbps 200ms/512Kbps 300ms/50Kbps
Scenario E2010(ms)SP1(ms
)E2010(m
s) SP1(ms
)E2010(ms)
SP1(ms)
Logon (PLT1) 3,720 3,499 16,099 14,557 156,363133,98
1
Logon (PLT2) 2,406 1,981 3,116 2,429 8,191 6,071
Delete a message 307 139 548 122 1,035 131
Preview a conversation
1,130 79 1,398 82 3,143 92
Mark as read or unread
257 35 467 22 1,125 30
Move or copy 340 123 622 145 1,029 150
Empty a tree folder 185 134 375 142 792 138
Attach 2.13 MB file 16,067 < 150 < 150 454,019 <150
Sizing The Hub Transport & Edge Role• CPU and memory are key for transport:
• Size storage capacity for queue requirements
• Use battery-backed write cache disk controller− Disk I/O can be a bottleneck on an un-
tuned Hub− Log I/O becomes virtually free with a
BBWC controller
Transport RolesSizing Rules Of
Thumb
• 2-socket platform best for performance and TCO
• CPU is typically the bottleneck, memory sizing is key as well
• 1 Transport CPU core for every 7 Mailbox CPU cores (no A/V)or1 Transport CPU core for every 5 Mailbox CPU cores (with A/V)
• 1GB RAM per CPU core is optimal
Resource Key Considerations
CPU CPU required for message processing, hygiene activities, custom agents
Memory Database cache requirements (reduces I/O), messages in queue represented in memory for perf
Storage Low to moderate I/O requirement for relay & delivery activity, queuing needs more I/O + capacity
Network Bandwidth utilized to relay/deliver messages – scales with message volume, latency can cause queuing
Transport Write Cache Observations• Battery-backed
write cache not absolutely necessary
• If workload is low enough, sequential nature of log I/O & ESE write behavior may allow running w/o BBWC
• Lab test can confirm if HW will sustain desired load
Effect of disabling BBWC on transport log I/O:Disk latencies rise significantly, IOPS drop significantly, throughput remains stable – ESE is doing fewer larger writes and keeping up with required I/O, so in this case BBWC not required
Sizing The Unified Messaging Role • CPU and network are key for UM:
• Scale out UM servers based on concurrent call requirements
• Size CPU based on requirements for Voice Mail Preview
Unified Messaging Role
Sizing Rules Of Thumb
• 2-socket platform best for performance and TCO
• 2GB RAM per core is optimal
• CPU is typically the bottleneck, particularly when Voice Mail Preview is being used
• Default 100 concurrent calls per server (inbound or outbound)
• Voice Mail Preview is CPU intensive: ~1 message/min/core
Resource Key Considerations
CPU CPU used for media operations and Voice Mail Preview transcription – UM is typically “CPU heavy”
Network Network bandwidth used for calls as well as communication with Mailbox role. Minimize latency for best user experience
Memory Memory required for ongoing transaction processing
Storage UM doesn’t have significant storage requirements
Sizing Multi-Role Hub/CAS Servers• Potentially optimal hardware
utilization− Server consolidation –
minimize physical servers• Simplified sizing
− Hub and CAS roles are relatively well balanced given resource requirements
− Virtualization – simplify server configuration with Hyper-V 4-core VMs (8-core physical = 1 4-core Mailbox, 1 4-core Hub/CAS)
Multi-Role Hub/CASSizing Rules Of Thumb
• 2-socket platform best for performance and TCO
• CPU is typically the bottleneck, memory sizing is key as well
• 1 Hub/CAS CPU core for every 1 Mailbox CPU core
• 2GB RAM per CPU core is optimal
8 core root
CAS/HUB
Mailbox
16 core root
CAS/HUB
Mailbox
CAS/HUB
Mailbox
24 core root
CAS/HUB
Mailbox
CAS/HUB
Mailbox
CAS/HUB
Mailbox
Sizing Multi-Role “Brick” Servers • Mailbox, CAS, and Hub Transport roles
recommended− UM supported, but not recommended
• Excellent solution for high core configurations
• Half of cores for Mailbox, half for CAS+Hub
• Use 8-24 cores− 8GB RAM plus 3-30MB/mailbox
recommended (follow mailbox database cache sizing guidance)
• Typical deployment scenarios:− Simple unit of scale (brick) model
− Each multi-role server represents a building block
− Servers with on-board SATA storage (10-16 disks) are optimal
− Small organization/branch office – server consolidation− Minimize the number of physical servers,
operating system instances, and Exchange server instances to manage
Multi-Role “Brick” Servers
Sizing Rules Of Thumb
• Recommend maximum 4-socket platform for multi-role deployment
• Use 8GB RAM plus 3-30MB per mailbox (see Mailbox role sizing details)
Sizing Virtualized Server Roles• Exchange isn’t virtualization “aware”
– VM is just a different hardware platform
• TechNet is the best source of support guidance and best practiceshttp://tinyurl.com/26k4g5j http://tinyurl.com/5abmlh
• Server Virtualization Validation Program (SVVP) Support Policy Wizard helps to determine supported configurationshttp://tinyurl.com/lyko6t
• Be aware of major support limitations− Root clustering + DAG− Unified Messaging role− Snapshots & differencing disks
Virtualized Server Roles
Sizing Rules Of Thumb
• Size for physical resources, add ~12% CPU overhead for hypervisor
• Avoid resource oversubscription
• Don’t co-locate Mailbox databases on a root server
• CAS+Hub combination can make scale calculations easy
Capacity Planning Tools
38
Capacity Planning Tools
• Profiling− EPA 2010
• Sizing− Mailbox Server Role Requirements
Calculator
• Validation− Jetstress 2010− Exchange Load Generator 2010
“Loadgen”
Exchange Profile Analyzer 2010• Generates statistical profile of user actions
− Messages sent and received/day− Rule counts− Item size and counts
• Inputs− Crawls mailboxes with MAPI (previously DAV) − OWA log analysis tool and “summarizer”
• Accuracy somewhat dependent on how users manage their mailbox
• Updated version expected Q3 CY10• Download existing version from
http://go.microsoft.com/fwlink/?LinkId=80471
Mailbox Role Requirements Calculator• Follows Product Group
recommendations on:− Storage− Memory− Mailbox sizing
• Goal of the calculator is to output:− I/O requirements − Capacity requirements − LUN design
• Available today via the Exchange team blog: http://msexchangeteam.com/
Jetstress 2010
• Exchange I/O simulator− Uses Jet (ESE) database engine
• Analyzes server I/O performance for Exchange requirements
• What can Jetstress be used for?− Storage performance validation− Storage reliability testing− End-to-end testing of storage components
• What can’t Jetstress be used for?− Validation of client experience− Integration testing with third party software solutions
• Download from http://www.microsoft.com/downloads/en/details.aspx?FamilyID=13267027-8120-48ed-931b-29eb0aa52aa6
Jetstress 2010What’s new
• Updated with Exchange 2010 Mailbox I/O Profile− This profile is not yet final and is subject to change
between now and Exchange 2010 release
• Database duplication is now multi-cast− Dramatically reduces the time to prepare databases for
testing
• Now using MSExchange Database I/O counters for I/O measurement− Allows placing databases and logs on the same volume
• Log replication I/O is simulated based on Exchange 2010 HA architecture
• Background Database Maintenance (Checksum) is now simulated
Exchange Load Generator 2010• The only supported multi-protocol load generator
for Exchange− Replaces Loadsim and ESP
• Windows UI interface as well as a command-line interface
• Both task-based and scripted simulation modes• Consumed both internally at Microsoft and
externally• Existing modules include: Outlook 2003/2007
(online and cached), POP, IMAP, SMTP, OWA, ActiveSync
• Download from http://www.microsoft.com/downloads/en/details.aspx?FamilyID=cf464be7-7e52-48cd-b852-ccfc915b29ef
Exchange Load Generator 2010What’s new• Requires Windows® Vista, Windows® 7 or
Windows 2008 OS (SP2/R2)• No longer requires Exchange Management
Tools• ActiveSync Module• Dynamic mail generator
− No need for message files, available in 5 languages, supports attachments
• NSPI connections• UI enhancements
Tools Process Flow
Exchange Profile Analyzer
Performance Monitor
Exchange Load Generator
Mailbox Role Calculator
Exchange Jetstress
UserProfil
e
Mailflow & Other Stats
IOPS
UserProfil
e
Exchange Tested Solutions
Exchange Tested Solutions
• A program designed to showcase well-designed, well-tested, and cost effective Exchange 2010 solutions
• Includes latest and greatest server and storage products from our partners
• Highlights key Exchange 2010 features alongside key IHV enabling technology
• Can be used to accelerate hardware purchasing decisions
• Targeting Q4CY10 for release to TechNet
Exchange Solution Reviewed Program• ESRP encourages storage vendors to
properly test storage solutions with Exchange workloads
• Results are published in the form of solution whitepapers for a particular user count & profile
• Overall test process validated by Microsoft prior to posting link to completed whitepaper
• Currently 24 Exchange 2010 solutions posted from Dell, EMC, HDS, HP, IBM, NetApp, and Xiotech
• http://tinyurl.com/yhlmbhg
Migrate & Grow
My System Is Stable…Now What?• Monitor, measure, analyze
− Consider System Center Operations Manager to monitor performance
− Keep an eye on user profile – changes may require additional hardware
− Watch performance data trends for capacity planning
• Keeping the system balanced is key to optimal hardware utilization− Balance heavy users across databases− Ensure that DAG active copies are balanced across
mailbox servers− CAS workloads should be load balanced across array− Storage should be balanced for capacity and I/O
throughput
• Consider eliminating hard boundaries− Cloud services may help reduce cost/complexity –
evaluate TCO
Architectural Design Session
Design Session
End of Exchange 2010 Planning and Sizing Module
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after
the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
