-
Testing PracticeCS 130Lecture 5CS 130 Lecture 5*
CS 130 Lecture 5
-
Why Test?CS 130 Lecture 5*
CS 130 Lecture 5
-
Why Test?CS 130 Lecture 5*
CS 130 Lecture 5
-
Software is Buggy!On average, 1-5 errors per 1KLOCWindows 2000
35M LOC 63000 known bugs at the time of release 2 bugs per 1000
linesFor mass market 100% correct software is infeasible, butWe
must verify software as much as possibleCS 130 Lecture 5*
CS 130 Lecture 5
-
Verification and ValidationValidation: Are we building the right
product?
Verification: Are we building the product right?CS 130 Lecture
5*
CS 130 Lecture 5
-
Verification and ValidationValidation: Are we building the right
product? To which degree the software fulfills its (informal)
requirements? Usability, feedback from users, Verification: Are we
building the product right? To which degree the implementation is
consistent with its (formal or semi-formal) specification? Testing,
inspections, static analysis, CS 130 Lecture 5*
CS 130 Lecture 5
-
Approaches to VerificationTesting: exercising software to try
and generate failuresStatic verification: identify (specific)
problems by looking at source code, that is, considering all
execution paths staticallyInspection/review/walkthrough: systematic
group review of program text to detect faultsFormal proof: proving
that the program text implements the program specificationCS 130
Lecture 5*
CS 130 Lecture 5
-
ComparisonTesting Purpose: reveal failure Limits: small subset
of the domain (=> risk of inadequate test setStatic verification
Purpose: consider all program behaviors (and more) Limits: false
positives, may not terminateReview Purpose: systematic in detecting
defects Limits: informalProof Purpose: prove correctness Limits:
complexity/cost (requires a formal spec)CS 130 Lecture 5*
CS 130 Lecture 5
-
Today, QA is mostly testing50% of my company employees are
testers, and the rest spends 50% of their time testing!Bill Gates
1995CS 130 Lecture 5*
CS 130 Lecture 5
-
What is testing?Testing == To execute a program with a sample of
the input dataDynamic technique: program must be executedOptimistic
Approximation: The program under test is exercised with a (very
small) subset of all the possible input data We assume that the
behavior with any other input consistent with the behavior shown
for the selected subset of input dataCS 130 Lecture 5*
CS 130 Lecture 5
-
Role of TestingTesting is basic to every engineering discipline
Design a drug Manufacture an airplane Etc.Why? Because our ability
to predict how our creations will behave is imperfect We need to
check our work, because we will make mistakesCS 130 Lecture 5*
CS 130 Lecture 5
-
Testing Real Software is Hard (E.g. Word)CS 130 Lecture 5*
Many input forms mouse, keyboard, tablet
Many document formats .doc, .html
Outputs printer, screen
External libraries Fonts, Dictionaries, Languages
Interoperation VB, Excel, email Many features 100+ commands 34
toolbars
Constraints Huge user base
CS 130 Lecture 5
-
Independent TestingProgrammers have a hard time believing they
made a mistake Plus a vested interest in not finding mistakesDesign
and programming are constructive tasks Testers must seek to break
the softwareCS 130 Lecture 5*
CS 130 Lecture 5
-
Independent TestingWrong conclusions: The developer should not
be testing at allRecall test before you code Testers get only
involved once software is done Toss the software over the wall for
testingTesters and developers collaborate in developing the test
suite Testing team is responsible for assuring qualityQuality is
assured by a good software processCS 130 Lecture 5*
CS 130 Lecture 5
-
The Purpose of TestingTwo purposes:
Find bugs Find important bugs
Elucidate the specification When testing the prototype or
strawmanCS 130 Lecture 5*
CS 130 Lecture 5
-
CS 130 Lecture 5*
CS 130 Lecture 5
-
Granularity Levels of TestingUnit testing: verification of
single modulesIntegration testing: verification of the interactions
among the different modulesSystem testing: testing of the system as
a wholeAcceptance testing: validation of the software against the
user requirementsRegression testing: testing of new versions of the
softwareCS 130 Lecture 5*
CS 130 Lecture 5
-
CS 130 Lecture 5*
CS 130 Lecture 5
-
Unit TestingFocus on smallest unit of design A procedure, a
class, a componentTest what? Local data structures Basic algorithm
Boundary conditions Error handlingMay need drivers and stubsGood
idea to plan unit tests aheadCS 130 Lecture 5*
CS 130 Lecture 5
-
JUnitUnit testing framework for JavaTest a class by making a
subclass of TestCaseEach test is a method in the subclass Each unit
requires several testsSetup that is common to all tests is in the
setUp methodCS 130 Lecture 5*
CS 130 Lecture 5
-
JUnit testing Stack.javapublic class Stack {public Stack() {
}public boolean isEmpty() { }public int size() { }public Object
peek() {if (isEmpty()) throw new NoSuchElementException("Stack
underflow) }public clear() { }public void add(Object item) {
}public Object remove() {if (isEmpty()) throw new
NoSuchElementException("Stack underflow");}public boolean
equals(Stack s) { }}CS 130 Lecture 5*
CS 130 Lecture 5
-
JUnitimport org.junit.*;import static org.junit.Assert.*;import
java.util.NoSuchElementException;public class StackTest {private
Stack s;private String n1, n2, n3;@Before public void setUp(){s =
new Stack();n1 = "Able"; n2 = "Baker"; n3 = "Charlie";}@Test public
void clear(){s.add(n1); s.add(n2); s.add(n3);assertTrue("before
clear, size != 3", s.size() == 3);s.clear();assertTrue("after
clear, size != 0", s.size() == 0);}CS 130 Lecture 5*
CS 130 Lecture 5
-
JUnit@Test public void peek() {s.add(n1);assertEquals("first
peek failed",n1,s.peek());s.add(n2);assertEquals("second peek
failed",n2,s.peek());s.add(n3);assertEquals("third peek
failed",n3,s.peek());s.remove();assertEquals("first peek after
remove failed",n2,s.peek());s.remove();assertEquals("second peek
after remove failed",n1,s.peek());s.remove();try
{s.peek();fail("peek on empty stack failed to throw
NoSuchElementException");} catch (NoSuchElementException e) {}}CS
130 Lecture 5*
CS 130 Lecture 5
-
JUnit@Test public void isEmpty() {assertTrue("not isEmpty after
construction", s.isEmpty());s.add(n1);assertFalse("isEmpty after
add", s.isEmpty());s.remove();assertTrue("not isEmpty after
add/remove", s.isEmpty());}@Test public void equals()
{AbstractStack s2 = new Stack();s2.add(n1); s2.add(n2);
s2.add(n3);s.add(n1); s.add(n2); s.add(n3);assertEquals("3 element
stacks not equals", s, s2);assertEquals("3 element stacks not
equals (symmetric)", s2,s);s.remove();assertFalse("3/2 element
stacks equals", s.equals(s2));assertFalse("3/2 element stacks
equals (symetric)", s2.equals(s));}CS 130 Lecture 5*
CS 130 Lecture 5
-
JUnit@Test public void addRemove() {s.add(n1); s.add(n2);
s.add(n3);assertSame("on first String remove",
n3,s.remove());assertSame("on second String
remove",n2,s.remove());assertSame("on third String remove",
n1,s.remove());assertTrue("after 3 add/remove, size != 0", s.size()
== 0);try {s.remove();fail("remove on empty (String) stack failed
to throw NoSuchElementException");} catch (NoSuchElementException
e){ }for (int i=0; i=0; i--) assertEquals("on String remove "+i,new
Integer(i),s.remove());assertTrue("after 1000 add/remove, size !=
0", s.size() == 0);try {s.remove();fail("remove on empty (Integer)
stack failed to throw NoSuchElementException");} catch
(NoSuchElementException e){}}}CS 130 Lecture 5*
CS 130 Lecture 5
-
XUnithttp://www.xprogramming.com/software.htmHTMLUnit web
sitesNUnit any .NET systemAnd over a hundred other testing
frameworksCS 130 Lecture 5*
CS 130 Lecture 5
