CS 3773 Software Engineering Basic Concepts of Software Testing Unit Testing.

  • Published on
    23-Dec-2015

  • View
    213

  • Download
    1

Embed Size (px)

Transcript

<ul><li> Slide 1 </li> <li> CS 3773 Software Engineering Basic Concepts of Software Testing Unit Testing </li> <li> Slide 2 </li> <li> 2 Why testing? Examples Mars Climate Orbiter ($165M, 1998) Sent to Mars to relay signal from the Mars Lander Smashed to the planet: failing to convert between English measure to metric values Shooting down of Airbus A300 (290 death, 1988) US CG-49 shoot down a Airbus A300 Misleading output of the tracking software THERAC-25 Radiation Therapy (1985) 2 cancer patients at East Texas Cancer center received fatal overdoses Miss-handling of race condition of the software in the equipment </li> <li> Slide 3 </li> <li> 3 Why testing? Numbers On average, 1-5 bugs per KLOC (thousand lines of code) In mature software More than 10 bugs in prototypes Windows2000 35MLOC 63K known bugs at the time of release 2 bugs per KLOC $59.5B loss due to bugs in US 2002 (estimation by NIST) It is not feasible to remove all bugs But try to reduce critical bugs </li> <li> Slide 4 </li> <li> 4 Answer is testing, why? 50% of my employees are testers, and the rest spends 50% of their time testing ---- Bill Gates, 1995 More reliable than inspection, relatively cheap Actually in the old days, when testing is expensive, inspection was the major answer You get what you pay (linear rewards) Compared to other 3 approaches Inspection, static checking, formal proof </li> <li> Slide 5 </li> <li> 5 Testing: Concepts Test case Test oracle Test suite Test script Test driver Test result Test coverage </li> <li> Slide 6 </li> <li> 6 Testing: Concepts Test case An execution of the software with a given list of input values Include: Input values, sometimes fed in different steps Expected outputs Test oracle The expected outputs of software by feeding in a list of input values A part of test cases Hardest problem in auto-testing: test oracle problem </li> <li> Slide 7 </li> <li> 7 Testing: Concepts Test suite A collection of test cases Usually these test cases share similar pre-requisites and configuration Usually can be run together in sequence Different test suites for different purposes Smoke test, Certain platforms, Certain feature, performance, Test Script A script to run a sequence of test cases or a test suite automatically </li> <li> Slide 8 </li> <li> 8 Testing: Concepts Test Driver A software framework that can load a collection of test cases or a test suite It can usually handle the configuration and comparison between expected outputs and actual outputs Test Coverage A measurement to evaluate how well the testing is done The measure can be based on multiple elements Code Input combinations Specifications </li> <li> Slide 9 </li> <li> 9 Granularity of Testing: V-model </li> <li> Slide 10 </li> <li> 10 Granularity of testing Unit Testing Test of a single module Integration Testing Test the interaction between modules System Testing Test the system as a whole, by developers on test cases Acceptance Testing Validate the system against user requirements, by customers with no formal test cases Regression Testing Test a new version with old test cases </li> <li> Slide 11 </li> <li> 11 Unit testing Testing of an basic module of the software A function, a class, a component Typical problems revealed Local data structures Algorithms Boundary conditions Error handling </li> <li> Slide 12 </li> <li> 12 An example: w riting test cases without a test framework Class Definition Consider a class that handles and manages a Fibonacci array Initialized the class Fibonacci f = new Fibonacci (); Extend to a certain length f. extend (length) Provide the number at certain index f. get(index) Get Length f.getLength() Provide a range of numbers f. getRange(start, end) </li> <li> Slide 13 </li> <li> 13 An example: writing automatic test cases without a test framework public static void main(String args[]){ Fibonacci f = new Fibonacci(); if(f.getLenght != 0){ System.err.print(wrong results ); } f.extend(50); if(f.getLenght != 50){ System.err.print(wrong results ); } int num = f.get(50); if(num != 12586269025){ System.err.print(wrong results ); } int[] range = f.getRange(40, 50) if(!ValueEquals(range, )){ System.err.print(wrong results ); } } Not Enough! What happened if an exception is thrown </li> <li> Slide 14 </li> <li> 14 An example: writing automatic test cases without a test framework public static void main(String args[]){ Fibonacci f = new Fibonacci(); if(f.getLenght != 0){ System.err.print(wrong results ); } f.extend(50); if(f.getLenght != 50){ System.err.print(wrong results ); } int num = f.get(50); if(num != 12586269025){ System.err.print(wrong results ); } int[] range = f.getRange(40, 50) if(!ValueEquals(range, )){ System.err.print(wrong results ); } } Potential test interference! </li> <li> Slide 15 </li> <li> 15 Common Operations Comparison between actual results and expected results Report error messages Catch Exceptions Re-initialize the class to be tested </li> <li> Slide 16 </li> <li> 16 Unit test framework xUnit Created by Kent Beck in 1989 This is the same guy we mentioned in XP, design patterns The first one was sUnit (for smalltalk) Junit The most popular xUnit framework There are about 70 xUnit frameworks for corresponding languages Never in the annals of software engineering was so much owed by so many to so few lines of code -------Martin Fowler </li> <li> Slide 17 </li> <li> 17 State machine of JUnit Initialization Of Test Class setUp try{ testMethod } catch {} tearDown Why? Try and Catch? </li> <li> Slide 18 </li> <li> 18 Writing a Test Case public class FibonacciTest extends TestCase { protected Fibonacci fTest; protected static int groudtruth[] = {0, 1,...,12586269025}; public void setUp(){ fTest = new Fibonacci(); } @Test public testInitialize(){ assertEquals(Length of fTest after initialization, 0, fTest.getLength()); } @Test public testExtend(){ fTest.extend(50); assertEquals(, 50, fTest.getLength()); } @Test public testGet(){ fTest.extend(50); assertEquals(, groudtruth[49], fTest.get(50)); } } obj.m2, test obj.m2 first Why? You may not understand the error in m1 if yo"&gt; </li><li> 21 Tips of writing test cases Bottom Up If you know that obj.m1 -&gt; obj.m2, test obj.m2 first Why? You may not understand the error in m1 if you dont know whether m2 is working or not Externalize data Externalize expected results to files and load them at the test case constructor Why? Easier to maintain if you want to reuse them in several places </li> <li> Slide 22 </li> <li> 22 Tips of writing test cases Use development Database Do not use real database for testing Why? Obvious Do enough logging in the code during testing Why? Help debugging Testing for exceptions How? public testException(){ try{ do(invalid); fail(); }catch(xxxException e){} } </li> <li> Slide 23 </li> <li> 23 Assertions public void testCapacity() { // a test method . assertTrue(fFull.size() == 100+size); //assertion } If assertion fails: Assertion failed: myTest.java:150 (expected true but was false) Not so good! Try: assertEquals(100+size, fFull.size()); //expected value first Assertion failed: myTest.java:150 (expected 102 but was 103) Better! Try: assertEquals(list length, 100+size, fFull.size()); Assertion failed: myTest.java:150 (list length expected 102 but was 103) </li> <li> Slide 24 </li> <li> 24 Assertions Extend TestCase and write your own assertions AssertStringContains, AssertArrayEquals, Use fail(String) to fail a test with certain message Feel free to change the fixture Each test will re-setup public void testRemoveAll() { fFull.removeAllElements(); fEmpty.removeAllElements(); assertTrue(fFull.isEmpty()); assertTrue(fEmpty.isEmpty()); } </li> <li> Slide 25 </li> <li> Tear down Consider the following test code void setUp() { File f = open(foo); File b = open(bar); } void testAAA() { use f and b } void testBBB(){ use f and b } Problems? void setUp() { File f = open(foo); File b = open(bar); } void testAAA() { try { use f and b } finally { clean&amp;close f, b } void testBBB() { try { use f and b } finally { clean&amp;close f, b } Better? </li> <li> Slide 26 </li> <li> 26 Tear down Consider the following test code void setUp() { File f = open(foo); File b = open(bar); } void testAAA() { use f and b } void testBBB(){ use f and b } void tearDown{ clean&amp;close f, b } Problems? void setUp() { File f = open(foo); File b = open(bar); } void tearDown{ try{ clean&amp;close f }catch{ } the same for b } </li> <li> Slide 27 </li> <li> 27 Tear down Be careful about tear down If tear down is not complete, a test failure may affect the following test cases Recover the changes done to global data that are not well handled by the setup Database, files, network, global variables Clean resources Caution of exceptions in tear down itself </li> </ul>