Principles of TestingG RameshGopalaswamy_Ramesh@Yahoo.Com
Understanding the role of testing vis--vis other life cycle activities w.r.t qualityGetting an appreciation of the basic principles of testingBasis Material for this session: Software Testing Principles and Practices, Srinivasan Desikan and Gopalaswamy Ramesh, Pearson Education, 2006-2007
Phases of software development (not necessarily sequential!)Requirements gatheringDesignDevelopmentTestingDeployment and maintenance
Quality during various phasesNot sustainable!High Rework CostsIdeal State!Risky!
The car is completeYou just have to paint it!
This car can accelerateFrom 0 to 60 in 5 secsYeah, but it accelerates when I press the brake!
Principle #1A product should meet the customers needsCustomers need not be external customersTesters are customers for developersA software should be testable
Testing should find out defects before customer finds them out!
Principle #1:ImplicationsPractice of incorporating customer supplied test cases into battery of testsIncluding customers and / or customer facing into the testing functionsConcept of Alpha / Beta testing
Principle #2Remember the triangle problem?Remember the last class 6 character problem?
Principle #2:ImplicationsChoose your tests intelligentlyChoose tests that have a higher probability of uncovering defectsBoundary Values, Choice of regression tests
Principle #3: ImplicationsDifferent types of testing interspersed during product cycleDifferent methods of integrationConcept of Regression Testing
Principle #4:The Cat and the Saint
Principle #4:ImplicationsWhite Box TestingStarting from code Black Box TestingStarting from external functionalityIntegration TestingStuff works together!System / Performance TestingWorks in real life conditionsRegression TestingMakes sure changes dont affect existing stuffDomain / adhoc testingBringing domain expertise into testing
Principle #5:ImplicationsA test should document the Input dataexpected results the test processThe behavior has to be externally corroboratedNo Turing Machine can verify itself!
Principle #6:The Pesticide Paradox
Principle #6:ImplicationsNeed to redefine and refine tests as we move forwardAs we pass old tests, new defects will surfaceRegression Tests have to be retired and new regression tests have to be designed
Principle #7:The Rags and The Convoy
Principle #6:ImplicationsLook for side effectsLook for spaghetti code or ragsTie maintenance and testing functions closely
Principle #8:Cop on the Bridge
Prevention and Cure:Quality Assurance Vs Quality Control Quality AssuranceProcess orientedDefect Prevention oriented ProactiveStaff functionQuality ControlProduct orientedDefect detection / correction orientedReactive Line function
Principle #8 leads to Principle #9:
Principle #9:End of the Pendulum
Principle #8 and #9:ImplicationsAchieve a balance between (Defect Prevention) Quality Assurance and (Defect Detection) Quality ControlView QA and QC as supplementary and not adversary job functionsDistribute QA / QC functions throughout for early detection (Principle #1)
Principle #10:The Men in BlackThe IBM experiment (Peopleware, DeMarco & Lister)Issues of pride we discussed last class
Principle #10:ImplicationsInstilling pride in testersEstablishing an identity for testersShowing them a career path
Principle #11:Automation Syndrome
Putting it all together#1: Goal of testing is to find defects BEFORE customers find them#2: Program testing can only show presence of bugs, never their absence#3: Test early and often#4: Understand the why and not just the what / how#5: Test the tests first#6: Tests develop immunity discover new tests!#7: Focus on convoys of defects#8: Testing encompasses defect prevention#9: Strike a fine balance between defect prevention and defect detection#10: Plan the automation initiatives#11: Instill pride in testers
Todays Agenda Recap
Understanding the role of testing vis--vis other life cycle activities w.r.t qualityGetting an appreciation of the basic principles of testing