11 Functional System Testing

8/8/2019 11 Functional System Testing

http://slidepdf.com/reader/full/11-functional-system-testing 1/25

Functional System Testing

Written by

Adam Carmi



Outline Goal of testing

Test cases, test suites and test data

What is f unctional system testing?

Coverage

Functional testing techniques:

± Functional analysis

± Equivalence partitioning

± Boundary value analysis



The goal of software testing The pr ocess of uncovering evidence of def ects in

sof tware systems

± Does not include efforts associated with tracking down bugs and f ixing them

No amount of testing will impr ove the quality of acomputer pr ogram

± T

he mo

re testing we do

of

a system, the mo

reconvinced we might be of its correctness

± Testing cannot in general pr ove a system works 100%correctly



Test cases The basic component of testing is a Test Case

In its most general form: (inputs, expected-result)

± inputs include system state, user commands and data

values to be pr ocessed

± expected result includes visible/audible inter f ace

changes or changes in the system state

Test cases are organized into Test Suites

± f unctionality, security, per formance, «



Test case execution

A running of the sof tware (under test) that

pr ovides the inputs specif ied in the test case

and o bser ving the results and comparing

them to those specif ied by the test case

± If the actual result varies f r om the expected

result, then a f ailure has been detected



Test data An eff ective test strategy requires caref ul acquisition and

preparation of test data prior to testing

± Testing can suff er if test data is poor

Test data concerns: ± Depth: quantity and size of data

± Breadth: variance of data values and data types

± Scope: completeness, relevance and accuracy of data

R esult of a query should be valid for the specif ic purpose of the

query, and not due to a missing or inappr o priate value ± Conditions: data should ref lect specif ic ³conditions´ in the domain

Data that would otherwise arrive af ter per forming specif ic o perationsover time

Test data and test results are expensive to construct



Example: Test data for TVRS

Name: test1.db

Description: Violation records designed for validating violation lookup

Violation ID Off ender¶s f irst name Off ender¶s last name Issuing policeman ID

243567 R achel Josef 8700342

237812 Dan Levi 6386541

264683 Dan Porat 1346329

255245 Dina Josef 8245731

000345 longFirst N longLast N 8700342

« « « «



Specification vs. implementation The basic appr oaches to testing sof tware are based on its

specification and implementation

White box testing ± test cases and data are constructed based on the code that implements the sof tware

± quality and correctness of computations is validated

± will n ot be f urther discussed in this tutorial

Black box testing ± test cases and data are constructed

based so

lelyo

n the sof

tware¶s specif icati

on

Gray box testing ± test cases are constructed to directlytarget various modules and layers of the system (requiresarchitectural insight)



Functional System Testing

Testing of a completed application to determine

that it pr ovides all of the behaviors required of it

± Testing of completed increments that pr ovide somedegree of end-user f unctionality

Search for def ects that are variances between the

actual o peration of the system and the

requirements for the system

System is treated as a black/gray box



How much testing is adequate?

Completely validating IEEE 754 f loating- point division requires 264 test-cases!

float divide(float x, float y)

Fr om practical and economic perspectives,

exhaustive testing is usually n

ot p

ossible ± Which sof tware pieces should we test?

± Which test cases should we choose?



Coverage

Coverage is a measure of how completely a test

suite exercises the capabilities of a piece of

sof tware ± ³Each line of code should be executed at least once´

± ³One test case should be constructed f r om each

specif ied requirement´

It is necessary to use testing techniques thatnarr ow down the number of test cases allowing the

br oadest testing coverage with the least effort



Technique:F

unctional Analysis

Analyze the expected behavior of the system

according to its f unctional specif ication

Generate a test pr ocedure for each of the possibleusage scenarios

± Corresponds to use case scenarios

± Analyze how a change in one part of the system aff ects

other parts

± ³Grand tour´ test cases: the result of one test case

pr oduces the data that is the input to the next test case



Example:

Functional Analysis I

Use Case: Remove Traff ic Violation

1. Super visor calls for deletion of the chosen Traff ic Violation2. TVRS pr ompts Super visor for conf irmation

3. Super visor conf irms

4. TVRS requests Off endersDB to delete the Traff ic Violation

f r om the off ender¶s record

5. Off endersDB appr oves that the Traff ic Violation has beendeleted

6. TVRS allows Super visor to look up a new Traff ic Violation as

described in the ³Lookup Traff ic Violation´ UC



Example:

Functional Analysis II

Test case ID: 134543

Pre-conditions: 1. TVRS initialized with test1.db database

2. Violation 243567 displayed in the ³Lookup Violation´ dialog

R elated use cases: ³Lookup Traff ic Violation´, ³R emove Traff ic Violation´

Expected resultAction

Conf irmation dialog is displayedPress ³Delete´ button

³Lookup Violation´ dialog is displayedPress the ³Yes´ button

A message dialog stating that violation

³243567´ is not stored in TVRS

Enter ³243567´ at ³Violation ID´ text f ield

and press the ³Search´ buttonTest results

Actual results:Passed

Failed

Def ect diagnosis:



Example:

Functional Analysis III

Verif y eff ects

of

change

Filled when

the test case

is executed

How do we know that

violation 243567 is

stored in the system?

Can a tester

diagnose the cause

of a def ect?

In addition, a query

could be run on the

Off enders database



Technique: Equivalence

Partitioning

Identif ies ranges of input and initial conditionsthat are expected to pr oduce the same result

A gr oup of test cases form an equivalence class if : ± They test the same f eature/scenario

± If one test reveals a f ault, the other ones (pr o bably) willtoo

± If

a test do

es no

t rev

eal af ault, the

other

ones(pr o bably) will not either

It is adequate to use only a single representative of the equivalence class



Example: Equivalence Partitioning I

Input value specif ication for ³Lookup Violation´ form:

Valid valuesField name

[0-9]{0, 9}Violation ID

[a-zA-Z]{0, 10}Off ender¶s f irst name

[a-zA-Z]{0, 10}Off ender¶s last name



Example: Equivalence Partitioning II

Field Valid

equivalent

classes

Valid

representative

values

Invalid

equivalent

classes

Invalid

representative

values

Violation ID Known violation 00243567 ID < 0 or ID >

999999999

-1, 1234567890

Unknown violation 32456720 Non numeric ID 23ab@

Empty ³´

Off ender¶s

f irst name

Unknown violation David Character# > 10 Hasalongname

Single known

violation

R achel Invalid character ad0@am

Many known

violations

Dan

Empty ³´

« « « «



Example: Equivalence Partitioning III

The number of test cases to choose f r om is reduced to

(3 + 3) × (4 + 2) × (4 + 2) = 216

The actual number can be f urther limited ± Single invalid f ield per test case (3 × 4 × 4 + 7 = 55)

± Importance of use case

± R esources available

± Most f requent input

± Lif e-critical sof tware

± Inf easible test cases

± R andomly

± ...



Technique: Boundar y Value

Analysis Based on experience / heuristics

± Testing boundary conditions of equivalence classes is

more eff ective Choose input boundary values as equivalence

classes representatives

Choose inputs that invoke output boundary values

Examples: ± (0, 10] validate using 0, 1, 2, 9, 10, 11

± R ead up to 5 elements validate reading 0, 1, 4, 5, 6 elements



BVA as an equivalence

partitioning extension

Choose one (or more) arbitrary value(s) in

each equivalence class

Choose valid values exactly on lower and

upper boundaries of equivalence class

Choose invalid values immediately below

and above each boundary (if applicable)



General purpose test-suite

construction technique May be used to o btain reasonable coverage with little

effort ± Use cauti ously!

± Unsuitable whenvalues

of di

ff erent

f ields are related

1. While test cases can be added

Each new test case should include as many un-included valid non-boundary

equivalence class representatives as possible

2. While test cases can be added

Each new test case should include as many un-included valid boundary

equivalence class representatives as possible3. While test cases can be added

Each new test case should include a single invalid equivalence class

representative that has not been included before

4. Manually replace/remove redundant or inf easible test-cases



Example:

Countr y

Club I

Day Sunday - Thursday Friday - Saturday

Gueststatus

Visitor Member Student Visitor Member Student

Age

(years)

Admission f ee

[0, 16) 25 10 20 35 10 30

[16, 60) 50 25 45 70 25 65

[60, 120] 35 15 30 50 15 45

Specif ication



Example:

Countr y

Club II

Field Valid

equivalent

classes

Valid

representative

values

Invalid

equivalent

classes

Invalid

representative

values

Day Sun - Thu Mon, Sun, Thu

Fri - Sat Fri, Sat

Guest

status

Visitor Visitor

Member Member

Student Student

Age [0, 16) 2, 0, 15 Non-numeric

value

14@a

[16, 60) 34, 16, 59 Age < 0 or

Age > 120

-1, 121

[60, 120] 100, 60, 120

A combo box is used

for choosing the day

and guest status



Example:

Countr y

Club III

Test case ID Day Guest status Age Result

1 Mon Visitor 2 25

2 Fri Member 34 25

3 Mon Student 100 30

4 Sun Visitor 0 25

5 Sat Member 16 10

6 Thu Student 60 30

7 Sun Member 15 10

8 Sat Student 120 45

9 Thu Visitor 59 50

10 Mon Member 14@a Invalid age

11 Fri Student -1 Invalid age

12 Fri Visitor 121 Invalid age

valid

valid

(boundary)

invalid

Documents

11 Functional System Testing