21
System Testing • There are several steps in testing the system: – Function testing – Performance testing – Acceptance testing – Installation testing

System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Embed Size (px)

Citation preview

Page 1: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

System Testing

• There are several steps in testing the system:– Function testing– Performance testing– Acceptance testing– Installation testing

Page 2: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Function Testing

• Tests functions performed by the system• Checks that the integrated system performs its

functions as specified in the requirements• e.g. a function test for a bank account package

verifies that the package can correctly: – Credit a deposit– Enter a withdrawal– Calculate interest– Print the balance

Page 3: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Performance Testing

• Once convinced that the functions work as specified, compare the integrated components with the non-functional system requirements

• e.g. performance test on the bank account package evaluates:– Speed with which calculations are made

– Precision of the computation

– Security precautions required

– Response time to user inquiry

Page 4: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Performance Testing

• At this point, the system operates as designers intended.– This is a verified system.

• Next, the system is compared with the customer’s expectations by reviewing the requirements definition.– If satisfied that the system built meets the

requirements, then we have a validated system.

Page 5: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Acceptance Testing

• The customers test the system, making sure that it meets their understanding of the requirements, which may be different from the developer’s.

• This assures the customers that the system they requested is the system that was built for them.

• Sometimes it is run in its actual environment but often is run at a test facility different from the target location.

Page 6: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Installation Testing

• This allows users to exercise system functions and document additional problem that result from being at the actual site.

• e.g. a naval system may be designed, built and tested at the developer’s site, which is configured as a ship might be, but is not on an actual ship.– Once development site tests are complete, an additional

set of installation tests may be run with the system on board each type of ship that will eventually use a system.

Page 7: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Function Testing

• Previous tests concentrated on components and their interactions.

• This first step in system testing ignores the system structure and focuses on functionality.

• Approach is more closed box than open box– As it need not be known what component is being

executed, rather what the system is to do.

• Each function can be associated with those system components that accomplish it.– For some functions, the parts may comprise the entire

system.

Page 8: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Performance Testing

• Addresses the non-functional requirements.• System performance is measured against

performance objectives set by the customer as expressed in the non-functional requirements.

• Performance testing is designed and administered by a test team, and the results are provided to the customer.

Page 9: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Performance Testing

• Function testing– May demonstrate that a test system can calculate the

trajectory of a rocket, based on the rocket’s thrust, weather conditions, and related sensor and system information.

• Performance testing examines how well the calculation is done– the speed of the response to user commands– accuracy of the result– accessibility of the data

Page 10: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Reliability, Availability and Maintainability

• Reliability– Explores how the software functions consistently and

correctly over long periods of time

• Availability– Explores if the software is available when we need it

• Maintainability– When the software product fails, it examines how

quickly and easily repairs are performed

Page 11: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Reliability

• Reliability is the probability that a system will operate without failure under given conditions for a given time interval.– It is expressed on a scale from 0 to 1. – Highly reliable close to 1– Unreliable close to 0

Page 12: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Availability

• Availability is the probability that a system is functioning completely at a given instant in time, assuming that the required external resources are also available.– It is expressed on a scale from 0 to 1.– Completely up and running 1– Unusable 0

Page 13: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Maintainability

• Maintainability is the probability, for a given condition, that any maintenance activity can be carried out within a stated time interval and using stated procedures and resources.– It is expressed on a scale from 0 to 1.

• Software maintenance can still be done when the system is up, which is different from hardware maintenance where the system is unavailable.

Page 14: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Measures

• To derive these measures, attributes of failure data is examined.

• Capture failure data (i-1) failures

• Record inter-failure times t1, t2, …, ti-1

• Mean Time To Failure (MTTF)– MTTF = (t1+ t2+ …+ ti-1) / (i-1)

Page 15: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Measures

• Suppose each underlying fault has been fixed and the system is again running.

• Ti denotes the yet-to-be observed, next time to failure (a random variable).

• Mean Time To Repair (MTTR) tells us the average time it takes to fix a faulty software component.

• Mean Time Between Failures (MTBF)– MTBF = MTTF + MTTR

Page 16: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Measures• Reliability-

– As the system becomes more reliable, its mean time to failure should increase

– R = MTTF / (1 + MTTF)

• Availability – Can be measured so as to maximise MTBF (or MTTF?)– A = MTBF / ( 1 + MTBF ) ?– Or A = MTTF/ (MTTF + MTTR)?– Or A = MTTF/MTBF?

• Maintainability– When a system is maintainable, MTTR is minimised– M = 1 / (1 + MTTR)

Page 17: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Acceptance Testing

• Once convinced that the system meets all the requirements specified, the customers and users asked to test.

• Customer leads testing and defines the test cases.– Purpose is to enable the customers and users to

determine if the system we built really meets their needs and expectations.

– Tests are written conducted and evaluated by the customers, with assistance from the developers to only answer technical questions.

Page 18: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Types of Acceptance Tests

• Benchmark test– Customer prepares a set of test cases that represent typical

conditions under which the system will operate when actually installed.

– Commonly used when customer has special requirements.

• Pilot test– Installs the system on an experimental basis, where users

test all the functions of the system as though system was actually installed permanently

– Customer often prepares a suggested list of functions that each user tries to incorporate in typical daily procedures

Page 19: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Types of Pilot Tests

• Alpha test– In-house pilot test that is run before being

released to the customer for the real pilot test

• Beta test– Customer’s pilot test that is run at the

customer’s site with a small subset of customers’ potential users

Page 20: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Parallel testing

• May be performed if a new system is replacing an existing one or is part of a phased development.

• The new system operates in parallel with the previous system. – Users become gradually accustomed to the new system but

continue to use the old system to duplicate the new– This gradual transition allows users to compare and contrast

the new system with the old– Allows sceptical users to build their confidence in the new

system by comparing the results obtained in both and verifying that the new system is just as effective and efficient as the old.

Page 21: System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing

Installation Testing

• This test focuses on two things:– Completeness of installed system – Any functional or non-functional characteristics that may

be affected by site conditions

• If the acceptance is performed on-site and conditions under which it was tested do not change, then the installation test may not be needed.

• When the customer is satisfied with the results, testing is complete and the system is formally delivered.