View
219
Download
0
Tags:
Embed Size (px)
Citation preview
QA Lab ManagementMeeting new Business Challenges with Dynamic Quality Assurance
Mike AzocarSr. Developer Technical SpecialistMicrosoft [email protected]
“we have a changed requirement, what do I need to test?”
Have you heard any of these?
“my testers are spending too long testing the same thing”
“tooling is expensive (time, licenses, people)”
“developers and testers work in silos and don’t communicate/speak the same language”
“when is my software ready to ship?”
“the developers say the defects are useless”
What if you could…
reduce the time it takes to determine the root cause of a bug
enable users to easily run acceptance tests and track their results
reduce the time it takes to verify the status of a reported bug fix
reduce the time it takes to replicate a bug uncovered by user actions
reduce the time it takes to isolate differences between the test and production environment
rele
ase
pla
nnin
g set test strategydefine done, doneestablish environmentsreview configurationscreate plansgenerate data
multiple iterations
What tester do
inception construction release
rele
ase
ite
rati
on regression
testingrelease doneness testingrelease sign off
itera
tion p
lannin
g add stories to plandefine acceptanceselect regression tests
itera
tion e
xecu
tion author tests
run testsfile bugsverify fixesautomate tests
itera
tion
retr
osp
ect
iveupdate master
planidentify product debtidentify test debtselect test for automation
Generalist SpecialistManual Testing
Some scripting
Creates scripts to set up lab, create data
Strong scripting skills
Some coding skills
Strong coding
Develops fully automated testing procedures
Expert coding skills
Where does testing happen?
Black Box Testing
White Box Testing
API Testing
70% of testing happens here
Majority of test tools target here
Visual Studio 2010 Test Capabilities
coded ui test
unit testingweb performance test
load test
test runner
test case management
lab management
data collectors (video, action log, event log etc)
team foundation server with reporting(bugs, test cases, requirements, user stories, source control, build)
generalist
specialist
rele
ase
pla
nnin
g set test strategydefine done, doneestablish environmentsreview configurationscreate plansgenerate data
itera
tion p
lannin
g add stories to plandefine acceptanceselect regression tests
itera
tion e
xecu
tion author tests
run testsfile bugsverify fixesautomate tests
itera
tion
retr
osp
ect
iveupdate master
planidentify product debtidentify test debtselect test for automation re
lease
ite
rati
on regression
testingrelease doneness testingrelease sign off
What testers do
inception construction release
Virtual Machines
Hyper-V Host
Visual Studio 2010 Lab Management
Automated Build-
Deploy-Test Workflow
Reduces Hardware Utilization
Remote Diagnostic Collection
Virtual Environment Setup/Tear
down
• Built on Hyper-V• Integrated with TFS
2010• Enabled by SCVMM
• Integrated management• Multi-hypervisor
management• P2V and V2V• Template deployment• Optimize resource usage
• Managed by Microsoft Test Manager
Library Share 1
Stored VM 1
Stored VM 2
Stored VM 3
Build Controller
Test Controller
Visual Studio Lab Management 2010 Leveraging virtualization for ALM
Physical Environment / ESX
Build Agent
Test Agent
Machine 1
Machine 2
Machine 3
• Lab Management
• Build Management
• Test Case Management
• Project Management
• Source Control
• Work Item Tracking
• Reporting
TFS 2010
Visual Studio 2010Microsoft Test Manager
Hyper-V Host 1
Build Agent
Lab AgentTest
Agent
Virtual Machine 1
Virtual Machine 2
Virtual Machine 3
Windows Vista
SCVMMVM Template Library
Sysprepped “gold images”• Windows Server 2003• Windows Server 2008•…etc.• Stored in SCVMM Library
Templates and Virtual Machines
Windows 7
Windows Server 2008
Test AgentBuild
AgentLab Agent
Windows Server 2003
Reporting on Testing Progress
Build progress overtime is tracked
Recent build status is reported to the
team
Reporting on Testing Progress
Are we getting enough coverage?
How many lines of code are we churning?
Reporting on Testing Progress
Workflow run for Deployment and
Lab Restore snapshots create new ones
after build
Test against Lab Environments Automatically
Failing Tests are Flagged for further
review
Amount of Code Covered by Test is
show
Amount of code checked in per
build
Your Creativity Unleashed
− Save time, focus on high value tasks− Fast Forward for Manual Testing− High quality bugs with single click
− Embrace Change− Rapid Setup/Teardown of Environments− Test prioritization to assess impact of code
check ins− Traceability and Visibility
− Rich Modern Experience− Elegant Task focused UI− Combined manual & fast forward for manual testing− Plan, Test, Track flow
Simplicity through Integration− TFS is THE collaboration hub
− Break silos between roles− Share Virtual environments between
team members
− Go Live Readiness− Performance Testing Early and Often− Highly scalable (Also 64 bit support) − Standard languages for scripts
− Quality Insight− Cross subject area reporting− Real time feedback
Quality code ensured− No more “no repro”
− Integrated, Extensible Diagnostics− Rich Bugs, fix first time, every time− Intelli-trace
− Automation to reduce Regression− Coded UI Test− Familiar language & toolset support
− Eliminate wasted cycles − Test in production like environments− Efficient, predictable build hand off
Call to Action
Download Visual Studio and Team Foundation Server RTM virtual machine for Windows Virtual PC and try Test Professional today! http://bit.ly/VSTFSVPC
Check out the Online MSDN Documentation – Testing http://bit.ly/TestApp
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Visual Studio, the Visual Studio logo, and [list other trademarks referenced] are trademarks of the Microsoft group of companies.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond
to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.