1. Pairwise Tes*ng Strategic Test Case Design www.xboso6.com

2. XBOSo& Quality and Tes4ng Process Improvement and So&ware Tes4ng Experts San Francisco, Beijing, Oslo, Amsterdam 3. Pairwise Tes*ng The Problem: Many defects only occur when a combina4on of inputs or events occur that interact with each other. But to test every combina4on or permuta4on is impossible. The Solu4on: Pairwise tes4ng normally begins by selec4ng values for the systems input variables. These individual values are o&en selected using domain equivalence par44oning. The values are then permuted to achieve coverage of all the pairings. 3 4. Pairwise Tes*ng The reasoning behind all-pairs tes4ng: the simplest bugs in a program are generally triggered by a single input parameter. The next simplest category of bugs consists of those dependent on interac4ons between pairs of parameters, which can be caught with all-pairs tes4ng. Bugs involving interac4ons between three or more parameters are progressively less common, while at the same 4me being progressively more expensive to nd by exhaus4ve tes4ng, which has as its limit the exhaus4ve tes4ng of all possible inputs. 4 5. Pairwise Tes*ng Problem: Test System S X YZInputs X = A,B Y = C,D System S Z = E,F There are 23 = 8 possible combina4ons 5 6. Pairwise Tes*ng 8 Test CasesYou can get rid of Test Case 2 (T2)Test ID X Y Z {A, -, E} is covered in T1T1 A C E {A, D, -} is covered in T4 { -, D, E} is covered in T6T2 A D E T3 A C F Problem:T4 A D F How many test cases can you get rid of? List the Test cases you would keep.T5 B C E Time yourself.T6 B D E T7 B C F T8 B D F 6 7. Pairwise Tes*ng 8 Test CasesPairwise testing4 Test CasesTest ID X Y Z Test ID X Y Z T1 A C E T1 A C E T2 A D E T4 A D F T3 A C F T6 B D E T4 A D F T7 B C F T5 B C E T6 B D E T7 B C F T8 B D F 7 8. Example How many combinations are there for twelve inputs? 8 - Hexawise.com 9. Pairwise Tes*ng - Hexawise.com 9 10. Pairwise Tes*ng - Hexawise.com 10 11. Pairwise Tes*ng 11 12. Pairwise Tes*ng Tools To generate the 20 test cases in the previous slide from the possible 3.7M test cases would be extremely 4me consuming. Just a few of the many SW tools to generate the tes4ng pairs. Hexawise AllPairs by James Bach, free PICT (Pairwise Independent Combinatorial Tes4ng) tool Automa4c Ecient Test Generator or AETG 12 13. Limita*ons of Pairwise Tes*ng Common tes*ng problems are made worse: Selec*ng the wrong input: The problems of equivalence par44oning are made worse with a pair of wrong inputs. The Oracle Problem: Is a defect properly revealed and do we recognized it when it is revealed? The input values to the system under test are not discrete. Problems of Pairwise tes*ng: High probability combina4ons get too lihle ahen4on. Not knowing the interdependencies between variables. 13 14. Pairwise Tes*ng Works well when: Number of possible parameter values is large You can eec4vely separate parameter values into equivalence classes. 14 15. Pairwise Tes*ng: Cau*on Pairwise tes4ng is a tool, it cannot replace a skilled experienced tester. As a tool, it is eec4ve only as much as the tester is eec4ve. Responsibility for good tes4ng remains with the tester, not the tool. 15 16. Pairwise Tes*ng Conclusion: When used properly, pairwise test set genera4on is an important technique that can help you produce beher so&ware systems. 16