20-MidtermReview

ECE453/SE465/CS447/ECE653/CS647: Midterm Review

Lin TanFebruary 28, 2014

20-MidtermReview - February 28, 2014

Admin.

✤ Lin’s additional office hours will be 1:30-2:20 and 4:30-5:20 Wed March 5th in DC 2536.!

✤ Midterm from 2012 and its solutions have been posted on LEARN.


Software Testing Instructor: Lin Tan

OS X Mavericks Goto Fail Bug

!3

if ((err = SSLHashSHA1.update(&hashCtx,&serverRandom)) != 0) goto fail; if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; goto fail; if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0) goto fail; err = sslRawVerify(...); fail: return err;

http://opensource.apple.com/source/Security/Security-55471/libsecurity_ssl/lib/sslKeyExchange.c - contains the bug!http://www.opensource.apple.com/source/Security/Security-55179.13/libsecurity_ssl/lib/sslKeyExchange.c - doesn’t contain the bug!



Tools that can detect the bug

!4

Clang -Wunreachable-code or -Weverything!gcc -Werror!!Visual Studio -Wall!PC-Lint:warning 539: Did not expect positive indentation !PVS-Studio:V640: Logic does not match formatting!…!

http://www.msuiche.net/2014/02/22/sslverifysignedserverkeyexchange-a-k-a-the-goto-epicfail-bug/



Graph Coverage

• You should be able to do the following with graphs: • Define graphs, paths • Create TRs for structural and dataflow criteria • Structural: NC, EC, EPC, PPC, CPC • Dataflow: ADC, AUC, ADUPC • Use the subsumption chart to evaluate test sets • Create graphs from source code • Understand concerns in handling multiple methods • Use specifications (e.g., sequencing constraints) to create

graphs • Different approaches to check/test specifications, and their

pros and cons

!5



Strengths and Weaknesses of Graph Coverage:• Must create graph • Node coverage is usually easy, but cycles make

it hard to get good coverage in general. • Incomplete node or edge coverage point to

deficiencies in a test set.

!6



Summary• Summarizing Structural Coverage:

• Generic; hence broadly applicable • Uses no domain knowledge

• Summarizing Dataflow Coverage: • Definitions and uses are useful but hard to reason.

• Miscellaneous other notes: • Control-flow graphs are manageable for single methods, but

not generally more than that. • Use call graphs to represent multiple methods, hiding details

of each method. • When we want to test du-paths across multiple design

elements, use first-use/last-def heuristic. • Testing based on specifications: sequencing constraints can

be hard to find, and hard to check/test.

!7



Midterm

• Open-book exam !

• See course website for time and location! • https://ece.uwaterloo.ca/~lintan/courses/testing/

exams.html

!8



Coverage

• Lecture 1 - 17 (Syntax-based Testing)

!9



Main Topics

• Fault, Error, Failure • Graph Coverage

• Structural Coverage • Data Flow Coverage • Control Flow Graphs • For Design Elements • For Specifications

• Syntax-based Testing

!10



Fault, Error, Failure

!11

public static int numZero (int[] x) { //Effects: if x==null throw NullPointerException"// else return the number of occurrences of 0 in x"

int count = 0;"for (int i =1; i <x.length; i++) {"

if (x[i]==0) {"count++;"

}"}"return count;"

}

x = [2,7,0], fault executed, error, no failurex = [0,7,2], fault executed, error, failureState of the program: x, i, count, PC

Wrong State: x = [2,7,0] i =1 count =0 PC=first iteration of if

Expected State: x = [2,7,0] i =0 count =0 PC=first iteration of if

Fix: for (int i =0, i<x.length; i++)



Graph

!12



ADC, AUC, ADUPC Example (2)

• ADC requires: • AUC requires: • ADUPC requires:

!13

{[n0,n1]}{[n0,n1], [n0, n2,n3,n5]}

{[n0,n1], [n0,n2,n3,n5], [n0,n1,n3,n5]} List all du-paths even if they are a subpath of another. !



Subsumption• Criteria Subsumption: A test criterion C1

subsumes C2 if and only if every set of test cases that satisfies criterion C1 also satisfies C2 !

• Must be true for every set of test cases

!14

Edge Coverage

EC

Node Coverage

NC

subsumes


Introduction to Software Testing (Ch 2) © Ammann & Offutt

Graph Coverage Criteria Subsumption

!15

Simple Round Trip Coverage

SRTC

Prime Path Coverage

PPC

Complete Path Coverage

CPC

Node Coverage NC

Edge Coverage EC

Edge-Pair Coverage EPC

Complete Round Trip Coverage

CRTC

All-defs Coverage

ADC

All-DU-Paths Coverage ADUPC

All-uses Coverage

AUC



Paths

• Path • Subpath • Test Path • Simple Path • Prime Path • Visit • Tour

• direct, with sidetrips, with detours

• Du-tour

!16



Definitions & Uses

• Definitions. Here are some Java statements which correspond to definitions.

• x = 5: x occurs on the left-hand side of an assignment statement;

• foo(T x) {...} : implicit definition for x at the start of a method;

• bar(x): during a call to bar, x might be defined if x is a C++ reference parameter.

• (subsumed by others): x is an input to the program. !

• Uses. The book lists a number of cases of uses, but it boils down to x occurs in an expression that the program evaluates.

!17



Last-defs & First-uses

!18

last-defs are 2, 3 the first-use is 12


Introduction to Software Testing (Ch 2) © Ammann & Offutt© Ammann & Offutt 11

1 // Program to compute the quadratic root for two numbers 2 import java.lang.Math; 3 4 class Quadratic5 { 6 private static float Root1, Root2; 78 public static void main (String[] argv) 9 { 10 int X, Y, Z;11 boolean ok;12 int controlFlag = Integer.parseInt (argv[0]); 13 if (controlFlag == 1) 14 { 15 X = Integer.parseInt (argv[1]); 16 Y = Integer.parseInt (argv[2]); 17 Z = Integer.parseInt (argv[3]); 18 } 19 else20 { 21 X = 10; 22 Y = 9; 23 Z = 12; 24 }

25 ok = Root (X, Y, Z);26 if (ok) 27 System.out.println 28 (“Quadratic: ” + Root1 + Root2); 29 else30 System.out.println (“No Solution.”);31 } 3233 // Three positive integers, finds quadratic root 34 private static boolean Root (int A, int B, int C)35 { 36 float D; 37 boolean Result;38 D = (float) Math.pow ((double)B, (double2-4.0)*A*C ); 39 if (D < 0.0) 40 { 41 Result = false; 42 return (Result);43 } 44 Root1 = (float) ((-B + Math.sqrt(D))/(2.0*A)); 45 Root2 = (float) ((-B – Math.sqrt(D))/(2.0*A)); 46 Result = true; 47 return (Result);48 } / /End method Root4950 } // End class Quadratic

Example – Quadratic

Introduction to Software Testing (Ch 2)

last-defs

first-uses

Pairs of locations: method name, variable name, statement (main (), X, 15) – (Root (), A, 38)!(main (), Y, 16) – (Root (), B, 38) !(main (), Z, 17) – (Root (), C, 38)!(main (), X, 21) – (Root (), A, 38)!(main (), Y, 22) – (Root (), B, 38)!(main (), Z, 23) – (Root (), C, 38)


Introduction to Software Testing (Ch 2) © Ammann & Offutt© Ammann & Offutt 11

25 ok = Root (X, Y, Z);26 if (ok) 27 System.out.println 28 (“Quadratic: ” + Root1 + Root2); 29 else30 System.out.println (“No Solution.”);31 } 3233 // Three positive integers, finds quadratic root 34 private static boolean Root (int A, int B, int C)35 { 36 float D; 37 boolean Result;38 D = (float) Math.pow ((double)B, (double2-4.0)*A*C ); 39 if (D < 0.0) 40 { 41 Result = false; 42 return (Result);43 } 44 Root1 = (float) ((-B + Math.sqrt(D))/(2.0*A)); 45 Root2 = (float) ((-B – Math.sqrt(D))/(2.0*A)); 46 Result = true; 47 return (Result);48 } / /End method Root4950 } // End class Quadratic

Example – Quadratic

Introduction to Software Testing (Ch 2)

last-defs

first-usePairs of locations: method name, variable name,

statement (Root (), Root1, 44) – (main (), Root1, 28)!(Root (), Root2, 45) – (main (), Root2, 28)!(Root (), Result, 41) – (main (), ok, 26 )!(Root (), Result, 46) – (main (), ok, 26 )


Introduction to Software Testing (Ch 2) © Ammann & Offutt© Ammann & OffuttIntroduction to Software Testing (Ch 2) 14

Quadratic – Coupling DU-pairs

Pairs of locations: method name, variable name, statement (main (), X, 15) – (Root (), A, 38)!(main (), Y, 16) – (Root (), B, 38) !(main (), Z, 17) – (Root (), C, 38)!(main (), X, 21) – (Root (), A, 38)!(main (), Y, 22) – (Root (), B, 38)!(main (), Z, 23) – (Root (), C, 38)!

(Root (), Root1, 44) – (main (), Root1, 28)!(Root (), Root2, 45) – (main (), Root2, 28)!(Root (), Result, 41) – (main (), ok, 26 )!(Root (), Result, 46) – (main (), ok, 26 )



Mutation Testing

• Mutation operators • Generate mutants • Strongly kill a mutant • Weakly kill a mutant • Mutation Coverage (MC)

!22


Courtesy of Dawson Engler

Condition Analysis

!23

Original Method int Min (int A, int B) { int minVal; minVal = A; if (B < A) { minVal = B; } return (minVal); } // end Min

Mutant int Min (int A, int B) { int minVal; ∆ 1 minVal = B; if (B < A) { minVal = B; } return (minVal); } // end Min

• Reachability:

• Infection:

• Propagation:

Unavoidable;

Need B ≠ A;

Need B > A;Wrong minVal needs to return to the caller; that is, we can't execute the body of the if statement.


Courtesy of Dawson Engler

Necessary and Sufficient Conditions

• Reachability: unavoidable;

• Infection: need B ≠ A;

• Propagation: need B > A.

Wrong minVal needs to return to the caller; that is, we can't execute the body of the if statement.

!• Condition for weakly killing mutant: B ≠ A

• Condition for strongly killing mutant: B > A

��24


Last Hints

• Make sure you know how to do the in-class exercises. !

• Be familiar with questions in 2012’s midterm. !

• If a topic was not mentioned here, it does not imply that it will not appear in the exam. !

• Good luck!20-MidtermReview - February 28, 2014

Documents

20-MidtermReview