Upload
derick-gregory
View
220
Download
0
Tags:
Embed Size (px)
Citation preview
Identify & Fix Performance Problems with Visual Studio 2012 UltimateBenjamin DayBenjamin Day Consulting, Inc.benday.com/blog | @benday
About Me
Cambridge, MAConsultant, Coach, & TrainerMicrosoft MVP for Visual Studio ALMTeam Foundation Server, Software Testing, Scrum, Software ArchitectureScrum.org Classes
Professional Scrum Developer (PSD)Professional Scrum Foundations (PSF)
www.benday.com, [email protected], @benday
Have you heard of Scrum.org?
http://www.pluralsight.com/training/Courses/TableOfContents/alm-fundamentals
http://pluralsight.com/training/Courses/TableOfContents/alm-for-developers
Thanks.
To my content reviewersJerri Chiu, MicrosoftDennis Bass, Microsoft
The Agenda
Basics of Web Performance TestsLoad TestsLoad Test RigsPerformance Explorer
Existing application Identify performance problemsGo from bad performance to good
The Purpose of Load Testing
Identify the capabilities of your applicationPossible goals
ExploreVerify Find the limitsCrush
The Obligatory “Eat Your Vegetables” Slide.
Load test throughout the development cycle.
Catch problems earlyEstablish performance baselineWatch for performance trends
Cheaper to fix early
Plan for performance
Best Practice
Cheaper to Fix Early
Early Less Early Nearly Done Production0
10
20
30
40
50
60
Time
Cost
to fi
x
Dose Of Reality
You know you should work that way.You probably don’t.
This talk is for you.
You have problems.How do you figure out what’s going wrong?
Inspiration for this talk
Real customer engagementSpent years developing a web applicationReleased it Seriously grumpy customersHuge hardware~20 simultaneous usersFixed it
4000+ simultaneous usersReduced hardware
The Tools
Visual Studio Ultimate
• Web Performance Tests• Load Tests• Load Test Rigs
Visual Studio Premium
• Profiling Tools• Performance Wizard
Tour of the application.
Disaster Strikes.
The application is deployed to production.Users are losing their minds.
Intermittent errors on the site.Site is slow.
Your boss is seriously angry.“Fix it!”
The Problem.
“No Repro”“Works on my box.”You don’t know what the errors are.The site seems to work fine.
Wild guess: it dies under load.
You need to create some load.
The Plan.
1. Script user activity with Web Performance Tests2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance Wizard6. Fix the code
Web Performance Tests.
The Plan.
1. Script user activity with Web Performance Tests
2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance Wizard6. Fix the code
What is a Web Performance Test?
Test type in Visual Studio 2012 UltimateSimulate a user using an ASP.NET application
HTTP trafficBuilding block of Load Tests
Tip: Web Tests & User Stories
Web Tests should simulate a User StoryExample:
As an administrator, I need to search for a person by username so that I can deactivate that person’s account.As a customer, I want to order a pizza and pay with a credit card.
Helps to organize your suite
Create a basic Web Test
Data-driven Web Tests
Run web test once per row in data sourceSimulate multiple users doing *similar* actionsData source types
OLE DB, CSV, XMLParameterization
Better tests, better coverage
Web Tests in the underperforming app
Load Tests.
The Plan.
1. Script user activity with Web Performance Tests2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance Wizard6. Fix the code
What are Load Tests?
Mix of Web Performance and Unit Tests
Simulates a mix of users doing a mix of things
Simulate Network speedsDifferent browsersVarying user activity loads
Why would you put a Unit Test in a Load Test?
Helpful for testing WCFCumbersome to simulate WCF via Web Test
(You’d have to re-invent the wheel.)Unit tests can re-use the WCF libraries
(Done and done.)
Important for Silverlight and SOAhttp://wcfloadtest.codeplex.com/http://sqlloadtest.codeplex.com/
(Hooray!)
“Hey Kids,Let’s create some Load Tests!”
Running load tests.
The Plan.
1. Script user activity with Web Performance Tests2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance Wizard6. Fix the code
Two ways to run load tests.
Generate load from Visual Studio 2012Uses only 1 core/processorLimited to 250 simulated users
Generate load from a Test RigFar better testSimulate lots more users
Load Test Rig & The Application
ControllerOrchestratesLicensingPerfMon Counters
Agent (1..n)Simulates users
Controller
Agent 1
Web Server
Database
Agent 2
Agent n
Coordinates & Gathers Information
Generates Load
Application Under Test
Load Test Rig: Manage Test Controller
Load Test Rig: Test Settings
Load Test Rig: Remote Execution
The Plan.
1. Script user activity with Web Performance Tests2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance Wizard6. Fix the code
Run Load Tests on the Rig
Before: Load Test Summary
Before: Load Test Graphs
The Core Problem.
‘sp_who2’ says lots of open connections.
After: Load Test Summary
After: Load Test Graphs
Performance Profiling
The Plan.
1. Script user activity with Web Performance Tests2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance
Wizard6. Fix the code
Profiling Tools
Load Tests help you find issues in a system“Surface” issues
Profiling helps you find issues in the code
Unit Tests, Web Tests, Load TestsRepeatableGood for targeted tuning
System under “natural” loadNot repeatableLooking for rough patterns
Profiling Methods•Gathers information at intervals (clock cycles)•No code modifications•Use this for first pass explorationsSampling•Modifies your code•Gathers detailed timing and count information
Instrumentation
•Multi-threaded code profilingConcurrency
•Object creation•Garbage collection.NET Memory
•Interaction between your application and SQL Server via ADO.NET
Tier Interaction
Load Test Performance Sessions Via Wizard
Must be only Web Performance TestsCan’t profile via Load Test Rig
Let’s go profile some stuff.
The Plan.
1. Script user activity with Web Performance Tests2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance Wizard6. Fix the code
Any last questions?
For More Information: Articles
Uses Visual Studio 2012
http://tinyurl.com/8sdjjak
For More Information: Articles
Uses Visual Studio 2010
http://tinyurl.com/3xjqgvz
http://tinyurl.com/2ulvvvr
For More Information: Microsoft Links
Visual Studio Performance Testing Quick Reference Guidehttp://vsptqrg.codeplex.com/ Content Index for Web Tests & Load Testshttp://tinyurl.com/ye97e52
For More Information: Microsoft Bloggers
VS Team Test Blog Site http://blogs.msdn.com/b/vstsqualitytools/Ed Glas - http://blogs.msdn.com/edglas/Bill Barnett - http://blogs.msdn.com/billbar/Sean Lumley - http://blogs.msdn.com/slumley/Dennis Stone - http://blogs.msdn.com/densto/Mike Taute - http://blogs.msdn.com/mtaute/Neelesh Kamkolkar - http://blogs.msdn.com/b/nkamkolkar/
Thank you.
http://www.benday.com | [email protected]
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to
be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS
PRESENTATION.
Miscellaneous tips.
Include Load Tests In Your Nightly Build
Create a Test List (*.vsmdi)Include the Test List in the buildCaptures trend data at 24 hours intervalsCompare the load test data with Excel
Static Code Analysis / FxCop
Help you to find issues in your codeShow you IDispose errors
Rules are configurableUse your judgment about which are relevant for your team
Avoid Static Methods & CA1822
Static Code Analysis Rule CA1822 is evil.
Sacrifices maintainability for perfomance.
Blog post: “Static Methods Are A Code Smell”http://bit.ly/cnEsuV
Only optimize known performance problems.
You’d be surprised by what *ISN’T* a performance problemDon’t spend a ton of time coding fancy solutions to performance problems you *THINK* you might haveFind and fix *REAL* problems
Tip: WCF Message Size
Data structure formatting can make a huge differenceExample: Banks and the States they operate inOption 1: Dictionary<string, string>
Key = Bank IdValue = State Abbreviation
Option 2: Dictionary<string, string[]>Key = State AbbreviationValue = Array of Bank Id’s
Option 2 was approximately 90% smaller
Tip: Add Performance Counters
Create Performance Counters (perfmon) in your appDo this early in the development cycle
More directed profiling of your app during Load Tests
Good for managing your app when it’s in production(Operations people love this.)
Using Performance Sessions on x64
You can’t profile x64 IIS processes from the IDE
Option #1: Profile from the command lineStart ASP.NET profiling using VSPerfAspNetCmd.exe Run the web test
Option #2: Run the ASP.NET process in x86 mode
How To Run ASP.NET in x86 Mode
Set “Enable 32-bit Applications” to true on your application’s AppPool
Resources
Connect. Share. Discuss.
http://northamerica.msteched.com
Learning
Microsoft Certification & Training Resources
www.microsoft.com/learning
TechNet
Resources for IT Professionals
http://microsoft.com/technet
Resources for Developers
http://microsoft.com/msdn
Complete an evaluation on CommNet and enter to win!
MS Tag
Scan the Tagto evaluate thissession now onmyTechEd Mobile
Required Slide *delete this box when your slide is finalized
Your MS Tag will be inserted here during the final scrub.
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to
be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS
PRESENTATION.