S

CS 5380 Software Engineering

Chapter 8 Testing

Validation vs Verification

Validation Are we building the right product?

Verification Are we building the product right?

Chapter 7 - Design and Implementation2

Purpose of Testing

Demonstration of fulfillment of requirements

Identification of defects

Chapter 7 - Design and Implementation3

Meeting Requirements(Validation)

At least one test per requirement xxxxxxxxx

Often more than one test Several valid cases xxxxxxxxx

Chapter 7 - Design and Implementation4

Identification of Defects

Identifying undesirable results System crash Incorrect computation Data corruption

May need many tests

Chapter 7 - Design and Implementation5

Testing is Never Final

Dijkstra: Testing can show presence of errors, not their absence

“There is always one more bug”

Chapter 7 - Design and Implementation6

How Much Testing?- Factors Driving Testing

Software purpose Safety drives high requirements

User expectations Some tolerance

Marketing “We lose money every day the product is not

delivered”

Chapter 7 - Design and Implementation7

Inspection vs Testing

Both address same issues Does the software meet requirements? Does the software fail in some situations?

Advantage of Inspection Identification of multiple errors at one time possible Identification of errors in incomplete software Looks beyond defects

Inefficient code, poor code structure, reuse

Advantage of testing Can be automated Detailed scenarios evaluated

Chapter 7 - Design and Implementation8

Inspection

Gives insight into the method, and hence potential flaws

Does not give us automation, regression

May be difficult to comprehend all cases

Complements other methods

Good review point is before check-in

Chapter 7 - Design and Implementation9

Testing Stages

Development Done during development, by programmers Often on development systems

Release Testing complete system before delivery Generally on a separate test system Formal test plans

User Test by users, on their systems Real world application Beta testing

Chapter 7 - Design and Implementation10

Development TestingLevels

Unit Object, function level

Component Interaction of several objects Focus on component interactions

System Entire system Focus on interactions

Chapter 7 - Design and Implementation11

Unit Testing

Object

Automated Setup Perform operation Evaluate Results

Execution Not UI

Chapter 7 - Design and Implementation12

State Diagram Testing

System or object for which you have a state diagram

Test all transitions State, event

Chapter 7 - Design and Implementation13

Activity Diagram Testing

Diagram clearly shows alternatives to be tested.

Chapter 7 - Design and Implementation14

Chapter 7 - Design and Implementation15

Component Testing

Testing of composite components

Several objects

Interfaces to consider Parameter (object) Procedural (object) Shared memory Message passing

Chapter 7 - Design and Implementation16

Component Examples

Objects

Fast city grocers

Web browser

Chapter 7 - Design and Implementation17

System Testing

Testing all components that come together Internal objects External objects – GUI, db External systems – credit card

Chapter 7 - Design and Implementation18

Regression Testing

Testing that past cases should still work

Automation is key to effective regression testing

Chapter 7 - Design and Implementation19

Release testing

Similar to system testing, but Separate team (not development) Goal: more about meeting requirements than finding

bugs

Goal: verify to the customer that the software is ready

Chapter 7 - Design and Implementation20

Scenario Testing

Complete scenarios with data

May be followed by user

May be automated.

Chapter 7 - Design and Implementation21

Performance Testing

Response times

Concurrency response

Identify degradation issues

Failure due to unexpected combination of events

Stress the system beyond normal use

Chapter 7 - Design and Implementation22

User Testing

Alpha – users with developers

Beta – Users in production mode Generally to identify bugs

Acceptance testing Customer acceptance Acceptance plan needed

Chapter 7 - Design and Implementation23

Release Cutoff

Software released when number of bugs/type tolerable

Unusual case causes software to crash Unexplained loss of data – single record lost Data corrupt (verified in customer data) Single minor function (.1% of users use this) fails Software lacks feature (50% of users want it)

Chapter 7 - Design and Implementation24

Extent of Testing

Depends on Purpose Health/Safety Critical – insulin pump Credit card retention Web ordering

Depends on User Expections

Depends on Marketing pressures New customer delivery needed Customer testimonials/references Features for existing customers

Existing customer response to errors

Chapter 7 - Design and Implementation25

Testers

Who does test? Object level?

Programmer other than developer Component level?

Programmers/ Test bed specialists Integration level?

User domain / Programmers Full scenario

Customers / Beta

Chapter 7 - Design and Implementation26

Experience with Testers

Search for people that test well

Most people will use a system for 1 hour and say it has 2 bugs

Another tester might use the same software for 10 minutes and find 10 bugs

Chapter 7 - Design and Implementation27