Agile Test Automation: Truth, Oxymoron or Lie?
George WilsonOriginal Software
Agenda
Why Agile?
ProcessTech-nology
Data
Quick Poll
Are you currently using Agile methodologies?
Yes – for all projectsYes – for some projectsThinking about itNot at all
AGILE!
Why Agile?
Process
GAP
Traditional
Parallel Teamwork
Dev Test Business
Why Test AutomationMay Fail
Extensive ramp-up timeLower on-going value
Skills Acquisition
Build Script Library
Automation partially available
Script Maintenance Cycles
Benefit ‘Black-outs’Shelf-ware Warning
Benefit
Effort
Time
Effo
rt
4-8+ months
Programmatic Test Tools
Falling Back to Manual Testing
Less challengingMore effort
Skills Acquisition
Build Script Library
Automation partially available
Script Maintenance Cycles
Benefit ‘Black-outs’Shelf-ware Warning Benefit
Effort
Time
Effo
rt
4-8+ months
Programmatic Test Tools
Fall Back to Manual Testing
Tech-nologyAgileEco-system
Agile Eco-System
OrganisationCooperationWork allocationTraceabilityMetrics
Agile Testing Success
Build tests in parallelFirst time manualIssuesAutomationData
Automation is Essential
Manual is slowFaster feedbackConsistencyEmpower testersConfidenceAvoid toxic debtDocumentation
Inhibitors
Rapid changeAttitudeInitial investmentSweat and painFear
Traditional Automation Pyramid
Application
Automation
High Risk
Manual
Evolving Automation
Automation
Too Late, too slow
Sp 1 Sp 2 Sp 3 Sp 4 Build regression
Not a separate sprint.
Sp 1 Sp 2 Sp 3 Sp 4
Regn Regn Regn
Not overflowing the sprint.
Regression Deliverable
Des
Code
FT
RT
Sprint 1 Sprint 2 Sprint 3
ReF
Des
Code
FT
RT
ReF
Des
Code
FT
RT
ReF
Design
Code, TDD
Refactor
Functional Test
Regression Test
Regression in each sprintEasy, FlexibleAvoid toxic debt – bugs and scripts
ManualAuto
Table effectsUI – Db validationData Rules
Start: ManualTesting
Progress: AutomatedTesting
Test Cycles
Evolutionary Testing
Unit TestingIntegrationSimulation
Tibco, Software AG, Web Methods, Websphere…
OracleSQLServerDB2
SOA & Middleware
Code-freeTestStubSimulateDrive
Testers view…
Stories
Test Cases
Manual Tests
Defects
Auto Regrs’n Adapt
Heal
Data
Explore
Daily Regression
Cross platformMulti actionBranchingError recovery
Agile AQM model
Control – Electronic Whiteboard
Testing is all about data
Input Process Output
Data Process ResultData Process ResultData
Data
Setup & Tear-Down
Raise order Despatch Invoicing GL processing
Incremental Roll-back
Automated Test Processes
Testing the Database
Pol Last Claim Claim Curr Tot Claim Name
1203301 00.11.2006 Y GBP 1266.61 Jackson
1203302 10.11.2006 C EUR 505.20 Henderson
Wilks1203303 01.01.2006 N GPB
1203301 12.01.2006 420.00
1203301 08.11.2006 761.42
1203302 10.11.2006 505.20GBP Pounds SterlingEUR EuroUSD US Dollars
Rules based validation
Re-use is key
ProcessKnowledgeResultsData
Automation meaningless without re-use
Agile Test Automation
Why Agile?
ProcessTech-nology
Data
Agile Test Automation
Essential
Truth?
Oxymoron?
Lie?
Essential
User InterfaceFast build for UICode free
Middleware & ServicesStub and simulateTest component interaction
DatabasePlan dataManage data
Dilbert says...
Recommended
