Introduction to Software Testing. Types of Software Testing Unit Testing Strategies – Equivalence Class Testing – Boundary Value Testing – Output Testing.

  • Published on
    03-Jan-2016

  • View
    215

  • Download
    3

Embed Size (px)

Transcript

  • Introduction to Software Testing

  • Introduction to Software TestingTypes of Software TestingUnit Testing StrategiesEquivalence Class TestingBoundary Value TestingOutput TestingSystem TestingUsability Testing

    Outline*

  • What is Software Testing?*

  • software testing: Evaluating software by observing its executionPrimary goal of testing is to find bugs!During testing, should execute tests that find bugs.Testing can only prove the presence of bugs, not their absence. - Dijkstra When do you stop testing?

    What is Software Testing?*

  • test case: A set of inputs, execution conditions, and a pass/fail criterion.Example: height input is 5, width input is Hello, the test passes if the program produces an error message.

    test or test execution: The activity of executing test cases and evaluating their results.

    test suite: A set of test cases.

    Test Terminology*

  • defect: A mistake in the source code (static)Synonyms: fault, bug

    failure: An observable incorrect program behavior (output)

    debugging: The process of finding and correcting a defect in a program given a failure

    More Test Terminology*

  • Software Testing is Hard! Why?*

  • // Solve ax2 + bx + c = 0struct RootAnswer { float root_one; float root_two;}RootAnswer roots(float a, float b, float c); Which values for a, b, c should we test?

    Need to select test cases intelligently!

    Testing Inputs*

  • Due to time constraints, only a small subset of actual inputs can be used to test.Need to choose a test suite such thatEach functionality provided by a program is tested by at least one test case.Tests target common programming mistakes.Tests exercise areas of code that are likely to be buggy. Test Case Selection*

  • Code Inspection: Looking at the source code directly for bugs.

    Toolscompilerstatic analyzermemory leak detectorprofiler / performance monitors

    Other Techniques for Finding Bugs*

  • Introduction to Software TestingTypes of Software TestingUnit Testing StrategiesEquivalence Class TestingBoundary Value TestingOutput TestingSystem TestingUsability Testing

    Outline*

  • Functional testing: Testing to determine if the software is working properly based on the specification (has the proper function).Non-functional testing: Testing other desirable properties of the software.Examples: performance, usabilityTesting is vastly different.

    Functional vs. Non-Functional Testing*

  • Unit Testing: Testing of a unit in the program.Unit: smallest part of the program only assigned to one developer.Often a single function or class.Focuses on the functionality of that unit.Can be automated using unit testing toolsVisual StudioJunitGoogle Testing FrameworkUnit Testing*

  • Integration Testing: Testing the interactions of the different units.Assumes unit testing is complete or will catch errors within the unit.Focuses on the interactions between units.Can be tested manually or can be automated.Integration Testing*

  • System Testing: Testing the entire system.Requires the entire system to be present.Used to test common usage scenarios.Used to test non-functional requirements.Best tested manually.Testing approaches are vastly different for non-functional requirements.

    System Testing*

  • Introduction to Software TestingTypes of Software TestingUnit Testing StrategiesEquivalence Class TestingBoundary Value TestingOutput TestingSystem TestingUsability Testing

    Outline*

  • Derive tests from the specification of the unit.Need to choose testsIntelligentlySystematicallyThoroughlyConsider the unit under a test as a function with inputs and outputsAll unique and interesting inputs should be testedAll unique and interesting outputs should be producedFocus on common errors

    Unit Testing Strategies*

  • Introduction to Software TestingTypes of Software TestingUnit Testing StrategiesEquivalence Class TestingBoundary Value TestingOutput TestingSystem TestingUsability Testing

    Outline*

  • Equivalence Class Testing: Partition the input space into equivalence classes.The key is to divide the input space into equivalence classes such that all inputs within each equivalence class have the same functionality.Very likely that each input within a equivalence class will exercise the same code within the function. Need to have at least one test in each equivalence class.More tests are better.

    Equivalence Class Testing*

  • Equivalence Class TestingFailure (valuable test case)Functional testing is one way of drawing orange lines to isolate regions with likely failuresThe space of possible input values No failure*

  • The division into equivalence classes is based on the desired / specified functionality of the subject under test.Equivalence classes may consist of:one particular valuea range of valuesa subset (possibly discontinuous) set of valuesThe set of equivalence classes should include both valid and invalid inputs.

    Equivalence Class Testing*

  • Volume discount if you buy more than 1000 units.Function that returns the number of days in a month (1-12).If the area code is 800 or 888, there is no charge for the call.Shipping rates differ depending on what country you are in.

    Equivalence Class Testing Examples*

  • Identify equivalence classes to test for each for a function that returns a letter grade (A, B, C, D, or F) based on a numeric score from 0-100.

    Class Problem: Equivalence Class Testing*

  • Introduction to Software TestingTypes of Software TestingUnit Testing StrategiesEquivalence Class TestingBoundary Value TestingOutput TestingSystem TestingUsability Testing

    Outline*

  • Boundary Value Testing: Test the values near and at the boundary of an ordered input.

    For ordered inputs, history suggests that values near the boundary are most error prone.Programmers may forget to check for a boundary conditionProgrammers may code the boundary incorrectly (use < instead of

  • For each boundary n, want to have three cases:n 1, n, n +1For a range of values from x to y, have at least six cases:x 1, x, x + 1, y 1, y , y + 1Boundary value testing can be combined with equivalence class testing:Boundaries mark the endpoints of equivalence classes.Need to test the boundaries of each equivalence class.Still desirable to test one or more values in the middle of the range.

    Boundary Value Testing*

  • Some values are outside of the range:May also be part of another equivalence class (worry about redundancy later)May be invalid should check to make sure its invalidBoundary analysis can be applied to more situations than handling numeric data:string lengthsize of arrays (or other data structures)data type limitations (size of integer)

    Boundary Value Testing*

  • Class Problem: Boundary Value Testing*Identify boundary values to test for each for a function that returns a letter grade (A, B, C, D, or F) based on a numeric score from 0-100.

  • Introduction to Software TestingTypes of Software TestingUnit Testing StrategiesEquivalence Class TestingBoundary Value TestingOutput TestingSystem TestingUsability Testing

    Outline*

  • Equivalence class and boundary value testing can also be applied to the output of a function.Outputs can be divided into equivalence classes.At least one test should be executed for each output equivalence class.In many cases, but not all, tests created for output are redundant to equivalence class tests created for inputs. Boundaries of an ordered output should also be tested.Output Testing*

  • Introduction to Software TestingTypes of Software TestingUnit Testing StrategiesEquivalence Class TestingBoundary Value TestingOutput TestingSystem TestingUsability Testing

    Outline*

  • System Testing focuses on the whole system:Consists of not only software, but possibly hardware and other components. Necessary to understand the solution / product as a whole.Encompasses functional and non-functional testing:Functional: Does the software work?Non-functional: Performance, usability, security, etc.

    System Testing*

  • Scenario-Based Testing: Develop a scenario or a script that is important to execute.Scenarios tend to mimic realistic use of the system. How would a normal user use the software?Include invalid input scenarios:How would a user accidentally or intentionally misuse the system?Scenario-Based Testing*

  • Exploratory Testing: Test the program without constraint.Also called ad-hoc testing.No script or scenario.Allows the tester to creatively try tests as they explore the application. Exploratory Testing*

  • Introduction to Software TestingTypes of Software TestingUnit Testing StrategiesEquivalence Class TestingBoundary Value TestingOutput TestingSystem TestingUsability Testing

    Outline*

  • What makes a software product usable?Usability Testing*

  • Usability Testing: Tests the usability of a software product.Typically done with representative samples of real users to try out the software.Objective criteria:Time and number of operations to perform a taskFrequency of user errorblame user errors on the product!

    Usability Testing*

  • Ask questions with the user in mind:Does the user interface make sense?Are the prompts / messages clear?Is the response appropriate when an invalid input is entered?Is the program intuitive?May want to have someone outside the class try out your programs.Dont let them read the specifications. Basic Usability Features*

    *Ask the class on what they think software testing is.*When testing, you need to put on your testing hat and try to break the program.

    It can be difficult to test your own code because you often assume it is correct. You need to assume it is not correct and try to get break the program.

    Even after thorough testing, you cannot be absolutely certain that your program is free of errors.

    When do stop testing is a difficult question but depends on many factors in practice: time, money, how buggy the software is, how critical the software works correctly (software in an airplane vs. solitaire), etc. *Key points:Distinguish between a defect and a failure.Distinguish between testing and debugging.*Ask the class. Hope to hear:Too many input combinations to testHard to know what the correct output is in some cases

    The next three slides focus on specific testing challenges: too many inputs, test case selection*Note that to test all possible inputs for a, b, and c:- Would require (232)3 1028 legal inputs- At 1 trillion tests / sec, would require 2.5 billion years!

    Point out this is a simple function. Imagine testing Microsoft Word there are an infinite number of inputs.*The term functionality is rather vague here. Eventually this is an equivalence class in equivalence class testing.

    Example of a common programming mistake: incorrect or missing boundary

    Areas of code that are likely buggy: could be a function where many bugs have been discovered or a function that the developer is not confident about.*Point out the compiler can catch syntax mistakes. More sophisticated tools can catch other mistakes.

    Depending on when this unit is covered, the students may or may not be familiar with a memory leak.*This unit assumes there is a specification that properly describes what the software is supposed to do.*The tutorial uses the Google Testing Framework*I dont expect integration testing to be much of a focus in 152 but is mentioned here for completeness.*The focus of unit testing here is to derive test cases from the specification, not the code. While some testing strategies derive tests from the code, it is simply not the focus here.*A particular equivalence class should have either all valid or all invalid inputs. *Examples:Equivalence classes: 1000Equivalence classes: < 1 (invalid), 30-day months, 31-day months, February, > 12 (invalid).Equivalence classes: 800 and 808, everything elseEquivalence class for each country

    Notes:Could have additional invalid equivalence classes for 1, 3, and 4. For example, a negative number of units for example 1 would be invalid.Could argue to test each month in 2 and each country in 4 to make sure the element is in the proper set.*Solution: = 101 (invalid)*Solution:*For equivalence class = 101 (invalid) need: 100, 101, 102

    Putting all the tests together (removing redundant test cases) yields:-2, -1, 0, 1, 58, 59, 60, 61, 68, 69, 70 ,71, 78, 79, 80, 81, 88, 89, 90, 91, 99, 100, 101, 102

    *Could make an argument that you need tests at the most negative and most positive integers.

    *The intentional misuse scenarios are important for programs where security is important. *Typically some constraints are given in larger software projects to avoid redundancy.*Ask the class for possible answers.*I included this usability testing section to reinforce having intuitive user interfaces. *

Recommended

View more >