Software testing

  • View

  • Download

Embed Size (px)




  • 1. Software Testing Strategies based onChapter 131

2. Software TestingTesting is the process of exercising a program with the specific intent of finding errors prior to delivery o the end user. errors requirements conformance performance an indication of quality 2 3. Who Tests the Software?developer Understands the system but, will test "gently" and, is driven by "delivery "3independent tester Must learn about the system, but, will attempt to break it and, is driven by quality 4. Software Testing Goals Types of tests Levels of tests Test measures Test plan 5. Goals Verification: Have we built the software right? Bug-free, meets specsValidation: Have we built the right software? Meets customers needsAre Verification and Validation different or variation of the same idea My argument: meeting specs should equal meeting customer needs... not generally true (my kitchen, satellite systems) 6. Software Testing Goals Types of tests Levels of tests Test measures Test planmost of this discussion focuses on verification (more specifically bug testing) 7. Types of tests Black box White box 8. Black box tests inputoutput interfa ce1. Does it perform the specified functions? 2.Does it handle obvious errors in input? 3.Ariane5 lousy error handling 4.Classic ints vs floats, yards vs meters Black box should catch these if there is adequate test coverage 9. Example: ordered list of ints L=create() L.insert(5) L.insert(-1) L.insert(-1) p=L.getFirst() print (p) L.delete(p) p=L.getFirst() print(p) p=L.getNext(p) print(p) p=L.getNext(p)class OrdInts create getFirst getNext insert delete print-1 -1 5 error 10. Black box tests Advantage: black box testerdeveloper is unbiased byimplementation details. e.g., Use Case testing, just work through all the Use Cases Disadvantage: black box tester is uninformed about implementation details unnecessary tests test same thing in different way insufficient tests can miss the extremes, especially if actual usefollows a different pattern 11. black box tests CodeInputx x xxchoose good distribution of input hope good distribution of code tested 12. unnecessary tests InputCodelarge range of input may exercise a small part of code e.g., operator test of satellite control stations, run through each input and output light/key option. Testing same functions, 13. insufficient tests InputCodea small range of input may exercise a large range of code but can you know this without knowing the code? Did we miss the 20% 14. sufficient tests InputCodecomplex codea small range of input may exercise a small but important/error-prone region 15. White box tests Based on code test 1test 2 16. Example: ordered list of ints class ordInts { public: private: int vals[1000]; int maxElements=1000; } 17. Example: ordered list of ints bool testMax() { L=create(); num=maxElements; for (int i=0; i -5 and x < 5 then What would be the equivalence classes? 26. Black-Box TestingComparison Testing Used only in situations in which the reliability of software isabsolutely critical (e.g., human-rated systems) Separate software engineering teams develop independentversions of an application using the same specification Each version can be tested with the same test data to ensure that all provide identical output Then all versions are executed in parallel with real-time comparison of results to ensure consistency26 27. Levels of tests UnitwhiteIntegration System System integrationblack 28. Testing measures (white box) Code coverage individual modules Path coverage sequence diagrams Code coverage based on complexity test of the risks, trickypart of code (e.g., Unix you are not expected to understand this code) 29. Levels of Testing Unit testing Integration testing Validation testing Focus is on software requirements System testing Focus is on system integration Alpha/Beta testing Focus is on customer usage Recovery testing forces the software to fail in a variety of ways and verifies that recovery is properly performed Security testing verifies that protection mechanisms built into a system will, in fact, protect it from improper penetration Stress testing executes a system in a manner that demands resources in abnormal quantity, frequency, or volume Performance Testing test the run-time performance of software within the context of an integrated system29 30. Unit Testing Tests the smallest individually executable code units. Usually done by programmers. Test cases might be selected based on code, specification, intuition, etc. Tools: Test driver/harness Code coverage analyzer Automatic test case generator 31. Integration Testing Tests interactions between two or more units or components. Usually done by programmers. Emphasizes interfaces. Issues: In what order are units combined? How do you assure the compatibility and correctness of externally-supplied components? 32. Integration Testing How are units integrated? What are the implications of this order? Top-down => need stubs; top-level tested repeatedly. Bottom-up => need drivers; bottom-levels tested repeatedly. Critical units first => stubs & drivers needed; critical units testedrepeatedly. 33. Integration Testing Potential Problems: Inadequate unit testing. Inadequate planning & organization for integration testing. Inadequate documentation and testing of externally-supplied components. 34. Stages of Testing Testing in the Large System Testing End-to-End Testing Operations Readiness Testing Beta Testing Load Testing Stress Testing Performance Testing Reliability Testing Regression Testing 35. System TestingTest the functionality of the entire system. Usually done by professional testers. 36. Realities of System Testing Not all problems will be found no matter how thorough orsystematic the testing. Testing resources (staff, time, tools, labs) are limited. Specifications are frequently unclear/ambiguous and changing (and not necessarily complete and up-to-date). Systems are almost always too large to permit test cases to be selected based on code characteristics.