Automated Testing Nathan Weiss April 23, 2007. Overview History of Testing Advantages to Automated Testing Types of Automated Testing Automated Testing

  • Published on
    22-Dec-2015

  • View
    214

  • Download
    2

Embed Size (px)

Transcript

  • Slide 1
  • Automated Testing Nathan Weiss April 23, 2007
  • Slide 2
  • Overview History of Testing Advantages to Automated Testing Types of Automated Testing Automated Testing Tools Disadvantages to Automated Testing
  • Slide 3
  • History of Testing
  • Slide 4
  • Four main phases in testing methodology: Debugging (1950s) Exhaustive Testing/Prove it Works (1960s) Prove it Does Not Work (1970s) Defect Prevention & Testing Process (1980s)
  • Slide 5
  • Debugging No distinction between debugging and testing. Consisted of making code compile and changing code so that it would not crash.
  • Slide 6
  • Exhaustive Testing/ Prove it Works Exhaustive testing in terms of code paths and given input. Found to be impossible Write tests to prove the program works. Testing defined as what is done to demonstrate correctness of a program.
  • Slide 7
  • Prove it Does Not Work Testing not for proving correctness, but for finding defects. Started testing to see if a program did what it was not suppose to do, not just test to prove that it does what it was suppose to do.
  • Slide 8
  • Defect Prevention & Testing Process Testing traditionally referred to what was done to the system after working code was produced. Testing moved towards an activity done during a phase, not a phase by itself. Studies show that 50% of bugs are created in requirements and design stages.
  • Slide 9
  • Advantages to Automated Testing Test scenarios impossible/unfeasible using manual testing. Lowers testing time and costs in an iterative environment. Remove the human error factor. Introduces stability in regression tests. Increase in testers morale.
  • Slide 10
  • Types of Automated Testing Split into two categories Black box tests White box tests PC applications Embedded Systems
  • Slide 11
  • Black Box Automated Testing GUI testing Always been prime candidates for manual testing. Need to automate has created many commercial tools. Two ways to create the tests: Scripts Recordings
  • Slide 12
  • Black Box Tests - Scripts Less of a testing tool/environment and more of a testing library. Flexible. Easy to run batch scripts to run multiple tests.
  • Slide 13
  • Black Box Tests - Recordings Full testing tool/environment. Tester performs a test once while tools is recording his actions. Tester can add ending conditions to test the success of a test. Very useful for testing GUIs of stand alone applications.
  • Slide 14
  • White Box Automated Testing Module testing. More APIs then environments. Embedded systems introduce more complexity to white box testing.
  • Slide 15
  • White Box Tests PC Apps Tests usually written in the same language as the system being tested. Tools usually come in the form of an API containing asserts and other testing utility methods. Tests can be performed on any level of abstraction required by the project.
  • Slide 16
  • White Box Testing Embedded Systems Test code usually not stored/run on the same hardware as the system code. Do not have direct access to system code or memory. Requires an interface with an embedded system debugger.
  • Slide 17
  • Automated Testing Tools xUnit SilkTest Selenium Custom
  • Slide 18
  • xUnit Compilation of popular code driven testing frameworks. Frameworks based on the design by Kent Beck. Originally implemented for SmallTalk as SUnit.
  • Slide 19
  • xUnit Partial List of Frameworks NUnit (.net) JUnit (java) CUnit ( C ) CPPUnit (C++) SUnit (SmallTalk)
  • Slide 20
  • xUnit xUnit test execution Setup Test body Teardown
  • Slide 21
  • xUnit
  • Slide 22
  • SilkTest Created by Borland. Works with: Web Browsers(IE and Firefox) Java GUIs .Net GUIs Win32 GUIs Price : 4-6k
  • Slide 23
  • Selenium Testing tool for web applications. Comes in API and tool form. Supports: Windows: IE, Firefox, Seamonkey, Opera Mac OS X: Safari, Firefox, Camino, Seamonkey Linux: Firefox, Konqueror
  • Slide 24
  • Selenium Can create tests using the Selenium library in many languages (Java,.NET, Perl, Python and Ruby) Or you can create tests using their Selenium IDE which is implemented as a Firefox extension. Once created in Firefox, the tests can be run in any of the suported browsers.
  • Slide 25
  • Selenium
  • Slide 26
  • Custom Write your own custom testing environment or API. Used when you have very specific testing needs. Often used for embedded systems.
  • Slide 27
  • Disadvantages to Automated Testing Initial cost. Who tests the testers?
  • Slide 28
  • References Borland. (2007). Borland SilkTest: An Automated Regression and Functional Software Testing Tool. Retrieved April 2, 2007, from http://www.borland.com/us/products/silk/silktest/index.html http://www.borland.com/us/products/silk/silktest/index.html [2] Dustin, E., Rashka, J., Paul, J. (1999). Automated Software Testing: Introduction, Management, and Performance. Addison-Wesley. [3] Lewis, E., W., (2005). Software Testing and Continuous Quality Improvement. CRC Press LLC. [4] NUnit. (2006). Nunit. Retrieved April 2, 2007, from http://www.nunit.org http://www.nunit.org [5] OpenQA. (2006). OpenQA: Selenium. Retrieved April 2, 2007, from http://www.openqa.org/selenium/index.htmlhttp://www.openqa.org/selenium/index.html