Testing Strategy

Big Picture Planning for Software Testing

Why Plan Your Tests?

Testing can take a significant amount of (often unplanned) time

There is more to be tested than possibleHaphazard testing is likely to miss errorsHaphazard testing is likely to duplicate

effortHaphazard testing is likely to generate

confusion

The Testing Keyword: “Systematic”

Plan what you will test, when, and how.Design tests with a purpose (diagnosis)Use TRs as part of your test planStart small, but test the “all”Test logic, function, behavior, and

requirements.Test your deployment.Test your documentation.

Types (Levels) of Testing

Unit: Tests individual modules and classes, generally with knowledge of program logic (white-box or glass-box testing).

Integration: Tests collaborating classes or modules, also with knowledge of program structure.

Validation: Tests requirements, functions and behavior (black-box or functional testing)

System: Tests application in context.

Things We Can Do to Make Testing Easier

Specify requirements quantifiably: e.g., GUI response time should be less than 1 sec

State test objectives explicitly: e.g., this test verifies data structure initialization

Create a user profile: determine expected pattern of use (and then also test for unexpected)

Build bullet-proof software: expect exceptions Use TRs to think deeply about the program logic Let someone else review your test cases for

completeness Instrument your classes to show their state Use unit testing frameworks where available

Common Errors

In Computation Misunderstood operator precedence Mixed data types (casting) Incorrect initialization Precision problems Incorrect equations (or derivation of the same)

In Logic (control flow) Comparison errors (type mismatch, wrong operator) Misunderstood precedence Missed equality Improper loop initialization or termination Improper loop maintenance Boundary values misunderstood or just missed (off-by-one) Data structures or their operations misunderstood Overly complicated Boolean expressions (e.g., double negation)

In Error Handling Error description is meaningless Error reported is not error encountered Error not caught Error handling not tested

Top-Down Versus Bottom-Up Testing

Top-Down Advantages:

Main control structure exercised early Early demonstration of capability builds confidence in app

Disadvantages: Requires stub code May degenerate into “big bang” testing approach

Bottom-Up Advantages:

Easier to comprehend Helps create solid program foundation Supported by testing frameworks

Disadvantages: “Glue” code required to test cooperating modules or classes

Additional Points of Emphasis

Testing and quality (pg 451 7/e, pg 356 6/e) Testing and the programmer’s “conflict of

interest” (pg 451 7/e, pg 357 6/e) Other types of testing: regression and smoke

testing (pg 462-463 7/e, 369-370 6/e) Verification &Validation (pg 451 7/e, pg 356 6/e) Still other types of testing: recovery, security,

etc. (pgs 470-472 7/e, pgs 377-378 6/e) Comments on Debugging (pgs 473-478 7/e, pgs

379-382 6/e)