TESTINGTEST PLAN

16 October 2013

You must have your system on a server for demo 2

Reminder

Types of Testing: Purpose

Functional testing Usability testing Conformance testing Performance testing Acceptance testing Reliability testing …

Other classifications

ScopeUnit, component, system, regression, …

Access to codeBlack box vs. white box(Note that black box testing still assumes

knowledge of coding and development in general)

What are you trying to test?

Most common actions? Most likely problem areas?

Risk-based testing

Risks Identify criteria of concern: availability,

quality, performance, … Risk of it not being met

likelihoodconsequences

If I’m testing code for a grocery store, what is the impact of the code not being highly available?

Four Parts of Testing

Model Select test cases Execute test cases Measure

Questions to Ask for Each Test

How will this test find a defect? What kind of defect? How powerful is this test against

that type of defect? Are there more powerful ones?

Basic Software Model

capabilities

environment

User interfaces

APIs

Operating system

Files

Input

Output

Storage

Processing

Test Case Selection

EnvironmentsWhat happens if a file changes out from under you?Consider all error cases from system calls

○ (e.g., you can’t get memory)Test on different platforms: software and hardwareTest on different versions and with different languages

CapabilitiesInputs (boundary conditions, equivalence classes)Outputs (can I generate a bad output?)States (reverse state exploration)Processing

From the User Interface: Inputs Error messages Default values Character sets and data types Overflow input buffers Input interactions Repeated inputs

How easy is it to find bugs in Word?

Test Coverage Metrics Statement coverage

basic block coverage Decision coverage

Each Boolean expression Condition coverage

Each entity in Boolean expressions Path coverage

loops

Advantages of different models?

Unit Test What do they do?

Regression testing frameworkIncrementally build test suitesTypically language specific

What is available?JUnit most well knownSUnit was the first (Smalltalk)xxUnit where xx represents every language

known to CS○ (even Wikipedia lists 75 languages)

How important is unit test? 91: Patriot Missile

incoming missile missed rounding error

93: Intel Pentium chip incorrect calculations table too small

96: Ariane V video auto-destruct data conversion – 64 bit float to 16 bit integer

98: Mars Climate Orbiter crashes into Mars imperial units instead of metric

Regression Test Automated Run with every build Issues

TimeMany are GUI based (sample)

ToolsWide range of capabilities and qualityAvoid pixel-based toolsRandom vs. scripted testing

Informal version: rerun tests