Narasimha Rao.PNarasimha Rao.P
Compiled with reference from:Compiled with reference from: Software Testing Techniques: Boris BeizerSoftware Testing Techniques: Boris Beizer Craft of Software Testing: Brain MarrickCraft of Software Testing: Brain Marrick
Unit 4 – Domain TestingUnit 4 – Domain Testing
ref boris beizer 2
Domain TestingWe will see in this part of Unit 3:
• Domains & Paths
• Nice & Ugly Domains
• Domain Testing
• Domains & Interfaces Testing
• Domains & Testability
U3
ref boris beizer 3
Transaction-Flow TestingContents of the unit
• Domains• Model, as a set, paths• Predicates• closure, dimensionality, assumptions
• Nice & ugly domains• specified & implemented domains• Nice domains – properties, testing implications• Ugly domains, simplification by programmers & testers
• Domain testing• Bugs & testing for those bugs• Testing procedure• Variations, tools, effectiveness
• Domains & Interface Testing• Range, closure, span• Interface range, domain compatibility• Finding the values
• Domains & Testability• Linearizing, coordinate transformations. Canonical program form
U3
ref boris beizer4
Domain Testing - Domains & Paths U3
Domain Testing ModelDomain Testing Model
INPUT CLASSIFY DO CASE 1 OUTPUT
DO CASE 2
DO CASE 3
DO CASE n
D1
D3
(x,y,z,…)
(1, 2, 3, 4, 5,…)
{ Outcome }
D2
Dn
ref boris beizer5
Domain Testing U3
Domain TestingDomain Testing
Views Programs as input data classifiers
Verifies if the classification is correct
ref boris beizer6
Domain Testing - Domains & Paths U3
Domain Testing ModelDomain Testing Model
Two ViewsTwo Views
• Based on SpecsBased on Specs
• Functional TestingFunctional Testing
• Based on Implementation informationBased on Implementation information
• Structural Technique
ref boris beizer7
Domain Testing - Domains & Paths U3
Domain Testing ModelDomain Testing Model
• Variables• Simple combinations of two variables• Numbers from - to +
• Input variables as numbers
• Loop-free Programs
ref boris beizer8
Domain Testing - Domains & Paths U3
Domain Testing ModelDomain Testing Model
• Structural Knowledge is not needed. Only Specs.
• For each Input Case,
• A Hypothetical path for Functional testing
• An Actual path for Structural testing
ref boris beizer9
Domain Testing - Domains & Paths U3
DomainDomain (simplified) (simplified)
• A Single Connected Set of numbers
• No arbitrary discrete sets
• Defined by the boundaries
• One or more boundaries
• Specified by predicates
• Bugs likely at the boundaries
• One or more variables
D1
ref boris beizer10
Domain Testing - Domains & Paths U3
PredicatesPredicates
• Interpretation
• Structural Testing - CFG
• Functional Testing - DFG
• Specifies the Domain Boundary
• n Predicates in Sequence 22n domains
• Or, just two domains A .AND. B .AND. C
ref boris beizer11
Domain Testing - Domains & Paths U3
PathsPaths
• At least One PATH thru the Program
• More paths for disconnected domains
ref boris beizer12
Domain Testing - Domains & Paths U3
Summary:Summary:
• Domain for a loop-free program corresponds to a set of numbers defined over the input vector
• For every domain, there is at least one path thru the routine, along which that domain’s processing is done
• The set of interpreted predicates traversed on that path (ie., the path’s predicate expression) defines the domain’s boundaries.
ref boris beizer13
Domain Testing - Domains & Paths - Domain Closure U3
Boundary: Closed / Open
MIN MAX
D1D2
D3X < MAXX >= MIN
MIN MAXD1 D2 D3
X >= MIN X <= MAX
Closed
Closed Open
ref boris beizer14
Domain Testing - Domains & Paths - Domain ClosureBoundary: Closed / Open
MIN MAXD1 D2 D3
X > MIN X < MAXOpen
ref boris beizer15
Domain Testing - Domains & Paths - Domain Dimensionality U3
• One dimension per variable
• At least one predicate• Slices thru previously defined Domain
• Slicing Boundary
• N-spaces are cut by Hyperplanes
ref boris beizer16
Domain Testing - Domains & Paths – Bug Assumptions U3
• Processing is OK. Domain definition may be wrong.
Boundaries are wrong.
Predicates are wrong.
• Once input vector is set on the right path, it’s correctly processed.
• More bugs causing domain errors …
ref boris beizer17
Domain Testing - Domains & Paths – Bug Assumptions.. U3
• Double-zero representation
• Floating point zero check
• Contradictory domains
• Ambiguous domains defined
undefined
ref boris beizer18
Domain Testing - Domains & Paths – Bug Assumptions.. U3
• Over-specified domains
• Boundary errors
Boundary closure
Shifted
X > 3 .AND. X < 2 .AND. Y > 3
ref boris beizer19
Domain Testing - Domains & Paths - Bug Assumptions U3
• Boundary errors….
Tilted
Missing
Extra boundary
ref boris beizer20
Domain Testing - Domains & Paths - Bug Assumptions U3
• Closure Reversal
• Faulty Logic
Simplification of compound predicates
X >= 3
ref boris beizer21
Domain Testing - Domains & Paths - Bug Assumptions U3
Strategy for domain testing in 2-dimStrategy for domain testing in 2-dim
ref boris beizer22
Domain Testing - Domains & Paths - Restrictions U3
In General…
1.1. Coincidental CorrectnessCoincidental Correctness
DT cannot detectExample
• Representative outcomeRepresentative outcome
Partition testing
Input Equivalence
D1
D2
F1(x) : +1
F2(x) : -1
ref boris beizer23
Domain Testing - Domains & Paths U3
Domain Testing ModelDomain Testing Model
INPUT CLASSIFY DO CASE 1 OUTPUT
DO CASE 2
DO CASE 3
DO CASE n
D1
D3
(x, y, z, …)(1, 2, 3, 4, 5,…)
{ Outcome }D2
Dn
Function f1
f2
f3
f4
ref boris beizer24
Domain Testing - Domains & Paths - - Restrictions U3
3.3. Simple boundaries & Compound predicatesSimple boundaries & Compound predicates
X0 16
X >=0 .AND. X <= 16
Y = 1 .AND. X >=0 .AND. X <= 16
X0 16
Y
D1
D1
ref boris beizer25
Domain Testing - Domains & Paths - - RestrictionsU3
Compound predicates….Compound predicates….
• Impact of Impact of .OR..OR.
• Concave, Disconnected
• Adjacent domains with same function
• Example
A B C + D E FA B C + D E F
• Eliminate compound predicates
ref boris beizer26
Domain Testing - Domains & Paths - Restrictions U3
4. Functional Homogeneity of Bugs
• Functional form still Retained
a x + b y >= c
• Bugs are only in a, b, c
ref boris beizer27
Domain Testing - Domains & Paths - - RestrictionsU3
5. Linear Vector Space
• Linear boundary predicate, Interpreted• Simple relational operators
• Conversion to linear vector space
• 2-d Polar co-ordinates
• Polynomials
• Problems with Non-linear Boundaries
ref boris beizer28
Domain Testing - Domains & Paths - Restrictions U3
6. Loop-free Software
Predicate for each iteration
Loop over the entire transaction
Definite loop
ref boris beizer29
Domain Testing - Domains & Paths – Nice & Ugly domains U3
Nice DomainsNice Domains
RequirementsRequirements
• Bugs ill-defined domains
Before DTBefore DT
• Analyze specs
• Make the Boundary Specs Consistent & Complete
ref boris beizer30
Domain Testing - Domains & Paths – Nice domains U3
Implemented Domains
• Complete, Consistent & Process all inputs
Specified Domains
• Incomplete, Inconsistent
Programmer’s / Designer’s Effort
ref boris beizer31
Domain Testing - Domains & Paths – Nice domains U3
Nice Domains
• Linear, Complete, Systematic, Orthogonal, Consistently Closed, & Convex
U1 U2
V1
V2
D11 D12
D21 D22
ref boris beizer32
Domain Testing - Domains & Paths – Nice domains U3
Nice Domains
Advantages
• Ease of DT
• Fewer Bug Occurrences
ref boris beizer33
Domain Testing - Domains & Paths – Nice domains U3
Nice Domains
Boundaries are
1. Linear
1. Complete
2. Systematic
1. Orthogonal
5. Closure consistency
6. Convex
7. Simply connected
ref boris beizer34
Domain Testing - Domains & Paths – Nice domains U3
1. Linear Boundaries
Interpreted linear inequalities
n-dim Hyperplane:
n+1 Points
n+1 + 1 Test Cases
Non-Linear
• Transform
ref boris beizer35
Domain Testing - Domains & Paths – Nice domains U3
2. Complete Boundaries
• Span the total number space (-, +)
• One set of Tests
Incomplete…
• Reasons
• Tests
ref boris beizer36
Domain Testing - Domains & Paths - Nice domains U3
3.3. Systematic BoundariesSystematic Boundaries
• Linear Inequalities differing by a constant
fj (X) ≥ kj or, fj (X) ≥ g (j, c) g (j, c) = j + k * c
• Parallel lines• Identical Sectors in a Circle
• DTDT• Test a domain Tests for other Domains
ref boris beizer37
Domain Testing - Domains & Paths - Nice domains U3
4.4. Orthogonal BoundariesOrthogonal Boundaries
• Two boundaries or, boundary sets
• Parallel to axes
• DTDT
• Each Set Independently
• # Tests O (n)
Uj
Vj
ref boris beizer38
Domain Testing - Domains & Paths - Nice domains U3
Orthogonal BoundariesOrthogonal Boundaries
• Tilted sets• transformation• Test cases: O(n)
• Concentric circles with radial lines
• Rectangular coordinates• Polar
r ≥ aj .AND. r < aj+1 .AND.
≥ j .AND. < j+1
ref boris beizer39
Domain Testing - Domains & Paths - Nice domains U3
Non-Orthogonal BoundariesNon-Orthogonal Boundaries
• Test Intersections O ( n2 ) + O (n)
ref boris beizer40
Domain Testing - Domains & Paths - Nice domains U3
5.5. Closure ConsistencyClosure Consistency
• A Simple pattern in all boundary closures
• Example
• Same relational operator for systematic boundaries
D1 D2D3 D4
ref boris beizer41
Domain Testing - Domains & Paths - Nice domains U3
6.6. Convex DomainConvex Domain
• Line joining any two points lies with in the domain
• DT• n on-points & 1 off-pt
ConcaveConcave
• “ But, However, Except, Or … “ in Specs• Handle with special care
ref boris beizer42
Domain Testing - Domains & Paths - Nice domains U3
7.7. Simply Connected DomainSimply Connected Domain
• In a single piece
• 2 complementary domains
• D1: Convex D2: Concave, not-Connected
• Programmer / Designer
• Convex part First
D1D2
D1D2
D3
ref boris beizer43
Domain Testing - Domains & Paths – Ugly Domains U3
Generally,Generally,
From Bad Specs
Programmer / Designer Simplifies => Ugly to Good
possibility of Introduction of more bugs ?!?
ref boris beizer44
Domain Testing - Domains & Paths – Ugly Domains U3
CausesCauses
1.1. Non-linear BoundariesNon-linear Boundaries
1.1. Ambiguities & ContradictionsAmbiguities & Contradictions
1.1. Simplifying the topologySimplifying the topology
1.1. Rectifying boundary closuresRectifying boundary closures
ref boris beizer45
Domain Testing - Domains & Paths – Ugly Domains U3
1.1. Non-linear BoundariesNon-linear Boundaries
• Transform
ref boris beizer46
Domain Testing - Domains & Paths – Ugly Domains U3
2.2. AmbiguitiesAmbiguities
• Holes in input vector space.
• Missing boundary
• Detected by Specification languages & tools.
ref boris beizer47
Domain Testing - Domains & Paths – Ugly Domains U3
2.2. Contradictions..Contradictions..
• Overlapping of
• Domain Specs
• Closure Specs
D1 D2
D3
ref boris beizer48
Domain Testing - Domains & Paths – Ugly Domains U3
3.3. Simplifying the Topology….Simplifying the Topology….
• Complexity !
• Concavity, Holes, Disconnectedness
ref boris beizer49
Domain Testing - Domains & Paths – Ugly Domains U3
3.3. Simplifying the Topology….Simplifying the Topology….
• Smoothing out concavity
• Filling in Holes
ref boris beizer50
Domain Testing - Domains & Paths – Ugly Domains U3
3.3. Simplifying the Topology….Simplifying the Topology….
• Joining the pieces
Correct:
• Connect disconnected boundary segments
• Extend boundaries to infinity
ref boris beizer51
Domain Testing - Domains & Paths – Ugly Domains U3
4.4. Rectifying Boundary Closures.Rectifying Boundary Closures.
• make closures in one direction for parallel boundaries with closures in both directions
• Force a Bounding Hyperplane to belong to the Domain.
Consistent Direction Inclusion / Exclusion Consistency
ref boris beizer52
Domain Testing - Domains & Paths – Domain Testing U3
General DT StrategyGeneral DT Strategy
1. Select test points near the boundaries.
2. Define test strategy for each possible bug related to boundary
3. Test points for a domain useful to test its adjacent domain.
4. Run the tests. By post test analysis determine if any boundaries are faulty & if so how?
5. Run enough tests to verify every boundary of every domain
ref boris beizer53
Domain Testing - Domains & Paths – Domain Testing U3
DT for Specific Domain BugsDT for Specific Domain Bugs
Generally,
• Interior point
• Exterior point
• Epsilon neighborhood
• Extreme point
• On point
ref boris beizer54
Domain Testing - Domains & Paths – Domain Testing U3
DT for Specific Domain BugsDT for Specific Domain Bugs
Domain D1
Epsilon neighborhood
Boundary point
Extremepoint
On Points
Off Points
ref boris beizer55
Domain Testing - Domains & Paths – Domain Testing U3
1.1. 1-d Domains1-d Domains
2.2. 2-d Domains 2-d Domains
3.3. Equality & inequality PredicatesEquality & inequality Predicates
4.4. Random TestingRandom Testing
5.5. Testing n-dimensional DomainsTesting n-dimensional Domains
ref boris beizer56
Domain Testing - Domains & Paths – Domain Testing U3
Testing 1-d Domains : Testing 1-d Domains : Bugs with open boundariesBugs with open boundaries
AABB
xx
AABB
xx
AABB
xxx1x1
AABB
xx x1x1
Closure Bug
Shift left Bug
Shift Right Bug
ref boris beizer57
Domain Testing - Domains & Paths – Domain Testing U3
AABB
xxMissing Boundary AABB
xxxx
Extra Boundary AABB
x1x1xx
CCxx xx
Testing 1-d Domains : Testing 1-d Domains : Bugs with open boundariesBugs with open boundaries
Bugs with Closed Boundaries : Similar to the aboveBugs with Closed Boundaries : Similar to the above
ref boris beizer58
Domain Testing - Domains & Paths – Domain Testing U3
2.2. Testing 2-d DomainsTesting 2-d Domains
• Closure bug
• Boundary Shift : up / down
• Tilted Boundary
• Extra Boundary
• Missing Boundary
ref boris beizer59
Domain Testing - Domains & Paths – Domain Testing U3
Strategy for domain testing in 2-dimStrategy for domain testing in 2-dim
2 n : Domains share tests 3 n : no sharing of tests by domains
ref boris beizer60
Domain Testing - Domains & Paths – Domain Testing U3
3.3. Equality & Inequality PredicatesEquality & Inequality Predicates
• An Equality predicate defines a line in 2-d
c’c’cccc
AA
BB
aa
dd
bb
To avoid bugs
ref boris beizer61
Domain Testing - Domains & Paths – Domain Testing U3
4.4. Random TestingRandom Testing
• A Point in the center : verifies computation
ref boris beizer62
Domain Testing - Domains & Paths – Domain Testing U3
5.5. Testing n-Dimensional Domains (strategy)Testing n-Dimensional Domains (strategy)
n-dimensions, p boundary segments
• (n+1)*p test cases : n on points & 1 off point
• Extreme pt shared : 2 * p points
• Equalities over m-dimensions create a subspace of n-m dimensions
• Orthogonal domains with consistent boundary closures, orthogonal to the axes & complete boundaries
• Independent testing
ref boris beizer63
Domain Testing - Domains & Paths – Domain Testing U3
Procedure for solving for valuesProcedure for solving for values
Simple procedure. Need tools.
1. Identify input variables
2. Identify variables which appear in domain-defining predicates, such as control-flow predicates
ref boris beizer64
Domain Testing - Domains & Paths – Domain Testing U3
ProcedureProcedure
3. Interpret all domain predicates in terms of input variables:• Transform non-linear to linear• Find data flow path
4. Predicate expression with p # predicates. Find # domains : < 2 p
3. Solve inequalities for extreme points
4. Use extreme points to solve for nearby on points …
ref boris beizer65
Domain Testing - Domains & Paths – Domain Testing U3
EffectivenessEffectiveness
• Cost effective
• Bugs on boundaries, extreme points
• Hardware logic testing – tool intensive
ref boris beizer66
Domain Testing - Domains & Paths – Domain Testing U3
Domains & Interface TestingDomains & Interface Testing
• Domain
• Range
Function / RoutineClassify
DomainVariable Range
ref boris beizer67
Domain Testing - Domains & Paths – Domain Testing U3
Domains & Interface TestingDomains & Interface Testing
• Span compatibility
Routine 2Routine 1Domain
VariableRange for
Routine2
Domain for
Routine2
Range for
Routine1
ref boris beizer68
Domain Testing - Domains & Paths – Domain Testing U3
Domains & Interface TestingDomains & Interface Testing
• Closure compatibility
ref boris beizer69
Domain Testing - Domains & Paths – Domain Testing U3
Interface Range/Domain Compatibility TestingInterface Range/Domain Compatibility Testing
• Test each var independently
• Find an inverse function
• Build a super domain
ref boris beizer70
Domain Testing - Domains & Paths – Domain Testing U3
Finding the values / solving inequalitiesFinding the values / solving inequalities
ref boris beizer71
Domain Testing - Domains & Paths – Domain Testing U3
Domains and TestabilityDomains and Testability
• Linearizing transformationsLinearizing transformations
• Polynomial• Rectangular to polar• Generic & rational => Taylor series
ref boris beizer72
Domain Testing - Domains & Paths – Domain Testing U3
Domains and TestabilityDomains and Testability
Perform transformations for better testing.Perform transformations for better testing.
• Co-ordinate transformationsCo-ordinate transformations
ref boris beizer73
Domain Testing - Domains & Paths – Domain Testing U3
Domains and TestabilityDomains and Testability
• Canonical Program FormCanonical Program Form
ref boris beizer74
Domain Testing – Questions from the previous year’s exams U3
1. Explain Nice & Ugly Domains.
2. What is domain testing? Discuss applications of domain testing
3. Explain the domain boundary bugs for two dimensional domains
4. What is the purpose of Domain Testing? Give its schematic representation
5. Define the following concepts.1. Domains 2. Domain closure 3. Domain
dimensionality 4. Bug assumptions for Domain Testing
6. Explain simple domain boundaries & compound predicates.
7. Classify what can go wrong with boundaries, then define a test strategy for each case in domain testing.
ref boris beizer75
Domain Testing U-3
To Unit 5…Paths, Path Products & Regular Expressions