Download ppt - unit 4 software testing

Narasimha Rao.PNarasimha Rao.P

Compiled with reference from:Compiled with reference from: Software Testing Techniques: Boris BeizerSoftware Testing Techniques: Boris Beizer Craft of Software Testing: Brain MarrickCraft of Software Testing: Brain Marrick

Unit 4 – Domain TestingUnit 4 – Domain Testing

ref boris beizer 2

Domain TestingWe will see in this part of Unit 3:

• Domains & Paths

• Nice & Ugly Domains

• Domain Testing

• Domains & Interfaces Testing

• Domains & Testability

U3

ref boris beizer 3

Transaction-Flow TestingContents of the unit

• Domains• Model, as a set, paths• Predicates• closure, dimensionality, assumptions

• Nice & ugly domains• specified & implemented domains• Nice domains – properties, testing implications• Ugly domains, simplification by programmers & testers

• Domain testing• Bugs & testing for those bugs• Testing procedure• Variations, tools, effectiveness

• Domains & Interface Testing• Range, closure, span• Interface range, domain compatibility• Finding the values

• Domains & Testability• Linearizing, coordinate transformations. Canonical program form

U3

ref boris beizer4

Domain Testing - Domains & Paths U3

Domain Testing ModelDomain Testing Model

INPUT CLASSIFY DO CASE 1 OUTPUT

DO CASE 2

DO CASE 3

DO CASE n

D1

D3

(x,y,z,…)

(1, 2, 3, 4, 5,…)

{ Outcome }

D2

Dn

ref boris beizer5

Domain Testing U3

Domain TestingDomain Testing

Views Programs as input data classifiers

Verifies if the classification is correct

ref boris beizer6



Two ViewsTwo Views

• Based on SpecsBased on Specs

• Functional TestingFunctional Testing

• Based on Implementation informationBased on Implementation information

• Structural Technique

ref boris beizer7



• Variables• Simple combinations of two variables• Numbers from - to +

• Input variables as numbers

• Loop-free Programs

ref boris beizer8



• Structural Knowledge is not needed. Only Specs.

• For each Input Case,

• A Hypothetical path for Functional testing

• An Actual path for Structural testing

ref boris beizer9


DomainDomain (simplified) (simplified)

• A Single Connected Set of numbers

• No arbitrary discrete sets

• Defined by the boundaries

• One or more boundaries

• Specified by predicates

• Bugs likely at the boundaries

• One or more variables

D1

ref boris beizer10


PredicatesPredicates

• Interpretation

• Structural Testing - CFG

• Functional Testing - DFG

• Specifies the Domain Boundary

• n Predicates in Sequence 22n domains

• Or, just two domains A .AND. B .AND. C

ref boris beizer11


PathsPaths

• At least One PATH thru the Program

• More paths for disconnected domains

ref boris beizer12


Summary:Summary:

• Domain for a loop-free program corresponds to a set of numbers defined over the input vector

• For every domain, there is at least one path thru the routine, along which that domain’s processing is done

• The set of interpreted predicates traversed on that path (ie., the path’s predicate expression) defines the domain’s boundaries.

ref boris beizer13

Domain Testing - Domains & Paths - Domain Closure U3

Boundary: Closed / Open

MIN MAX

D1D2

D3X < MAXX >= MIN

MIN MAXD1 D2 D3

X >= MIN X <= MAX

Closed

Closed Open

ref boris beizer14

Domain Testing - Domains & Paths - Domain ClosureBoundary: Closed / Open

MIN MAXD1 D2 D3

X > MIN X < MAXOpen

ref boris beizer15

Domain Testing - Domains & Paths - Domain Dimensionality U3

• One dimension per variable

• At least one predicate• Slices thru previously defined Domain

• Slicing Boundary

• N-spaces are cut by Hyperplanes

ref boris beizer16

Domain Testing - Domains & Paths – Bug Assumptions U3

• Processing is OK. Domain definition may be wrong.

Boundaries are wrong.

Predicates are wrong.

• Once input vector is set on the right path, it’s correctly processed.

• More bugs causing domain errors …

ref boris beizer17

Domain Testing - Domains & Paths – Bug Assumptions.. U3

• Double-zero representation

• Floating point zero check

• Contradictory domains

• Ambiguous domains defined

undefined

ref boris beizer18

Domain Testing - Domains & Paths – Bug Assumptions.. U3

• Over-specified domains

• Boundary errors

Boundary closure

Shifted

X > 3 .AND. X < 2 .AND. Y > 3

ref boris beizer19

Domain Testing - Domains & Paths - Bug Assumptions U3

• Boundary errors….

Tilted

Missing

Extra boundary

ref boris beizer20


• Closure Reversal

• Faulty Logic

Simplification of compound predicates

X >= 3

ref boris beizer21


Strategy for domain testing in 2-dimStrategy for domain testing in 2-dim

ref boris beizer22

Domain Testing - Domains & Paths - Restrictions U3

In General…

1.1. Coincidental CorrectnessCoincidental Correctness

DT cannot detectExample

• Representative outcomeRepresentative outcome

Partition testing

Input Equivalence

D1

D2

F1(x) : +1

F2(x) : -1

ref boris beizer23



INPUT CLASSIFY DO CASE 1 OUTPUT

DO CASE 2

DO CASE 3

DO CASE n

D1

D3

(x, y, z, …)(1, 2, 3, 4, 5,…)

{ Outcome }D2

Dn

Function f1

f2

f3

f4

ref boris beizer24

Domain Testing - Domains & Paths - - Restrictions U3

3.3. Simple boundaries & Compound predicatesSimple boundaries & Compound predicates

X0 16

X >=0 .AND. X <= 16

Y = 1 .AND. X >=0 .AND. X <= 16

X0 16

Y

D1

D1

ref boris beizer25

Domain Testing - Domains & Paths - - RestrictionsU3

Compound predicates….Compound predicates….

• Impact of Impact of .OR..OR.

• Concave, Disconnected

• Adjacent domains with same function

• Example

A B C + D E FA B C + D E F

• Eliminate compound predicates

ref boris beizer26


4. Functional Homogeneity of Bugs

• Functional form still Retained

a x + b y >= c

• Bugs are only in a, b, c

ref boris beizer27

Domain Testing - Domains & Paths - - RestrictionsU3

5. Linear Vector Space

• Linear boundary predicate, Interpreted• Simple relational operators

• Conversion to linear vector space

• 2-d Polar co-ordinates

• Polynomials

• Problems with Non-linear Boundaries

ref boris beizer28


6. Loop-free Software

Predicate for each iteration

Loop over the entire transaction

Definite loop

ref boris beizer29

Domain Testing - Domains & Paths – Nice & Ugly domains U3

Nice DomainsNice Domains

RequirementsRequirements

• Bugs ill-defined domains

Before DTBefore DT

• Analyze specs

• Make the Boundary Specs Consistent & Complete

ref boris beizer30

Domain Testing - Domains & Paths – Nice domains U3

Implemented Domains

• Complete, Consistent & Process all inputs

Specified Domains

• Incomplete, Inconsistent

Programmer’s / Designer’s Effort

ref boris beizer31


Nice Domains

• Linear, Complete, Systematic, Orthogonal, Consistently Closed, & Convex

U1 U2

V1

V2

D11 D12

D21 D22

ref boris beizer32


Nice Domains

Advantages

• Ease of DT

• Fewer Bug Occurrences

ref boris beizer33


Nice Domains

Boundaries are

1. Linear

1. Complete

2. Systematic

1. Orthogonal

5. Closure consistency

6. Convex

7. Simply connected

ref boris beizer34


1. Linear Boundaries

Interpreted linear inequalities

n-dim Hyperplane:

n+1 Points

n+1 + 1 Test Cases

Non-Linear

• Transform

ref boris beizer35


2. Complete Boundaries

• Span the total number space (-, +)

• One set of Tests

Incomplete…

• Reasons

• Tests

ref boris beizer36

Domain Testing - Domains & Paths - Nice domains U3

3.3. Systematic BoundariesSystematic Boundaries

• Linear Inequalities differing by a constant

fj (X) ≥ kj or, fj (X) ≥ g (j, c) g (j, c) = j + k * c

• Parallel lines• Identical Sectors in a Circle

• DTDT• Test a domain Tests for other Domains

ref boris beizer37


4.4. Orthogonal BoundariesOrthogonal Boundaries

• Two boundaries or, boundary sets

• Parallel to axes

• DTDT

• Each Set Independently

• # Tests O (n)

Uj

Vj

ref boris beizer38


Orthogonal BoundariesOrthogonal Boundaries

• Tilted sets• transformation• Test cases: O(n)

• Concentric circles with radial lines

• Rectangular coordinates• Polar

r ≥ aj .AND. r < aj+1 .AND.

≥ j .AND. < j+1

ref boris beizer39


Non-Orthogonal BoundariesNon-Orthogonal Boundaries

• Test Intersections O ( n2 ) + O (n)

ref boris beizer40


5.5. Closure ConsistencyClosure Consistency

• A Simple pattern in all boundary closures

• Example

• Same relational operator for systematic boundaries

D1 D2D3 D4

ref boris beizer41


6.6. Convex DomainConvex Domain

• Line joining any two points lies with in the domain

• DT• n on-points & 1 off-pt

ConcaveConcave

• “ But, However, Except, Or … “ in Specs• Handle with special care

ref boris beizer42


7.7. Simply Connected DomainSimply Connected Domain

• In a single piece

• 2 complementary domains

• D1: Convex D2: Concave, not-Connected

• Programmer / Designer

• Convex part First

D1D2

D1D2

D3

ref boris beizer43

Domain Testing - Domains & Paths – Ugly Domains U3

Generally,Generally,

From Bad Specs

Programmer / Designer Simplifies => Ugly to Good

possibility of Introduction of more bugs ?!?

ref boris beizer44


CausesCauses

1.1. Non-linear BoundariesNon-linear Boundaries

1.1. Ambiguities & ContradictionsAmbiguities & Contradictions

1.1. Simplifying the topologySimplifying the topology

1.1. Rectifying boundary closuresRectifying boundary closures

ref boris beizer45


1.1. Non-linear BoundariesNon-linear Boundaries

• Transform

ref boris beizer46


2.2. AmbiguitiesAmbiguities

• Holes in input vector space.

• Missing boundary

• Detected by Specification languages & tools.

ref boris beizer47


2.2. Contradictions..Contradictions..

• Overlapping of

• Domain Specs

• Closure Specs

D1 D2

D3

ref boris beizer48


3.3. Simplifying the Topology….Simplifying the Topology….

• Complexity !

• Concavity, Holes, Disconnectedness

ref boris beizer49



• Smoothing out concavity

• Filling in Holes

ref boris beizer50



• Joining the pieces

Correct:

• Connect disconnected boundary segments

• Extend boundaries to infinity

ref boris beizer51


4.4. Rectifying Boundary Closures.Rectifying Boundary Closures.

• make closures in one direction for parallel boundaries with closures in both directions

• Force a Bounding Hyperplane to belong to the Domain.

Consistent Direction Inclusion / Exclusion Consistency

ref boris beizer52

Domain Testing - Domains & Paths – Domain Testing U3

General DT StrategyGeneral DT Strategy

1. Select test points near the boundaries.

2. Define test strategy for each possible bug related to boundary

3. Test points for a domain useful to test its adjacent domain.

4. Run the tests. By post test analysis determine if any boundaries are faulty & if so how?

5. Run enough tests to verify every boundary of every domain

ref boris beizer53


DT for Specific Domain BugsDT for Specific Domain Bugs

Generally,

• Interior point

• Exterior point

• Epsilon neighborhood

• Extreme point

• On point

ref boris beizer54


DT for Specific Domain BugsDT for Specific Domain Bugs

Domain D1

Epsilon neighborhood

Boundary point

Extremepoint

On Points

Off Points

ref boris beizer55


1.1. 1-d Domains1-d Domains

2.2. 2-d Domains 2-d Domains

3.3. Equality & inequality PredicatesEquality & inequality Predicates

4.4. Random TestingRandom Testing

5.5. Testing n-dimensional DomainsTesting n-dimensional Domains

ref boris beizer56


Testing 1-d Domains : Testing 1-d Domains : Bugs with open boundariesBugs with open boundaries

AABB

xx

AABB

xx

AABB

xxx1x1

AABB

xx x1x1

Closure Bug

Shift left Bug

Shift Right Bug

ref boris beizer57


AABB

xxMissing Boundary AABB

xxxx

Extra Boundary AABB

x1x1xx

CCxx xx

Testing 1-d Domains : Testing 1-d Domains : Bugs with open boundariesBugs with open boundaries

Bugs with Closed Boundaries : Similar to the aboveBugs with Closed Boundaries : Similar to the above

ref boris beizer58


2.2. Testing 2-d DomainsTesting 2-d Domains

• Closure bug

• Boundary Shift : up / down

• Tilted Boundary

• Extra Boundary

• Missing Boundary

ref boris beizer59


Strategy for domain testing in 2-dimStrategy for domain testing in 2-dim

2 n : Domains share tests 3 n : no sharing of tests by domains

ref boris beizer60


3.3. Equality & Inequality PredicatesEquality & Inequality Predicates

• An Equality predicate defines a line in 2-d

c’c’cccc

AA

BB

aa

dd

bb

To avoid bugs

ref boris beizer61


4.4. Random TestingRandom Testing

• A Point in the center : verifies computation

ref boris beizer62


5.5. Testing n-Dimensional Domains (strategy)Testing n-Dimensional Domains (strategy)

n-dimensions, p boundary segments

• (n+1)*p test cases : n on points & 1 off point

• Extreme pt shared : 2 * p points

• Equalities over m-dimensions create a subspace of n-m dimensions

• Orthogonal domains with consistent boundary closures, orthogonal to the axes & complete boundaries

• Independent testing

ref boris beizer63


Procedure for solving for valuesProcedure for solving for values

Simple procedure. Need tools.

1. Identify input variables

2. Identify variables which appear in domain-defining predicates, such as control-flow predicates

ref boris beizer64


ProcedureProcedure

3. Interpret all domain predicates in terms of input variables:• Transform non-linear to linear• Find data flow path

4. Predicate expression with p # predicates. Find # domains : < 2 p

3. Solve inequalities for extreme points

4. Use extreme points to solve for nearby on points …

ref boris beizer65


EffectivenessEffectiveness

• Cost effective

• Bugs on boundaries, extreme points

• Hardware logic testing – tool intensive

ref boris beizer66


Domains & Interface TestingDomains & Interface Testing

• Domain

• Range

Function / RoutineClassify

DomainVariable Range

ref boris beizer67



• Span compatibility

Routine 2Routine 1Domain

VariableRange for

Routine2

Domain for

Routine2

Range for

Routine1

ref boris beizer68



• Closure compatibility

ref boris beizer69


Interface Range/Domain Compatibility TestingInterface Range/Domain Compatibility Testing

• Test each var independently

• Find an inverse function

• Build a super domain

ref boris beizer70


Finding the values / solving inequalitiesFinding the values / solving inequalities

ref boris beizer71


Domains and TestabilityDomains and Testability

• Linearizing transformationsLinearizing transformations

• Polynomial• Rectangular to polar• Generic & rational => Taylor series

ref boris beizer72



Perform transformations for better testing.Perform transformations for better testing.

• Co-ordinate transformationsCo-ordinate transformations

ref boris beizer73



• Canonical Program FormCanonical Program Form

ref boris beizer74

Domain Testing – Questions from the previous year’s exams U3

1. Explain Nice & Ugly Domains.

2. What is domain testing? Discuss applications of domain testing

3. Explain the domain boundary bugs for two dimensional domains

4. What is the purpose of Domain Testing? Give its schematic representation

5. Define the following concepts.1. Domains 2. Domain closure 3. Domain

dimensionality 4. Bug assumptions for Domain Testing

6. Explain simple domain boundaries & compound predicates.

7. Classify what can go wrong with boundaries, then define a test strategy for each case in domain testing.

ref boris beizer75

Domain Testing U-3

To Unit 5…Paths, Path Products & Regular Expressions