Upload
softwarecentral
View
295
Download
1
Embed Size (px)
DESCRIPTION
Citation preview
11
Software Software TestingTesting
Does your software product perform as it is Does your software product perform as it is intended to perform?intended to perform?
Presented by:Presented by:
Andy BoutsAndy Bouts
John KinuthiaJohn Kinuthia
22
ContentContent
Introduction Introduction Background on software testingBackground on software testing Modern views of software testingModern views of software testing Key TerminologyKey Terminology Categories of software testingCategories of software testing OO software testing environmentOO software testing environment Lessons Learned; Object oriented testingLessons Learned; Object oriented testing ConclusionConclusion
33
What is Software Testing?What is Software Testing?
Process of examining a software product or part of Process of examining a software product or part of a software product to determine whether a a software product to determine whether a software performs as it is expected to performsoftware performs as it is expected to perform
Software testing is a part of quality assuranceSoftware testing is a part of quality assurance
Quality assurance: takes into account all business Quality assurance: takes into account all business processes and not just testingprocesses and not just testing
44
Testing outcomesTesting outcomes
Desired results are / are not achievedDesired results are / are not achieved
Non – desired results are / are not achievedNon – desired results are / are not achieved
Users accept / do not accept productUsers accept / do not accept product
55
History of Software TestingHistory of Software Testing
There are four distinct periods of time that mark the There are four distinct periods of time that mark the development of software testing during the twentieth development of software testing during the twentieth century.century.
During these periods, software testing has changed During these periods, software testing has changed from its earliest days of continuously debugging to the from its earliest days of continuously debugging to the more proactive approaches of today.more proactive approaches of today.
66
Four Periods of DevelopmentFour Periods of Development
1957-1978: Engineers distinguish between testing and 1957-1978: Engineers distinguish between testing and debugging software. This is the first time they were debugging software. This is the first time they were able to prove that a software system can meet able to prove that a software system can meet requirements.requirements.
1979-1982: Engineers began trying to break the 1979-1982: Engineers began trying to break the system to find errors, instead of just debugging the system to find errors, instead of just debugging the known errors.known errors.
77
Four Periods of DevelopmentFour Periods of Development
1983-1987: Engineers began offering ‘observational 1983-1987: Engineers began offering ‘observational periods’ on software, today these periods as called periods’ on software, today these periods as called beta programs.beta programs.
1988-Present: As software projects are more closely 1988-Present: As software projects are more closely managed in terms of financials, risk, and timing, the managed in terms of financials, risk, and timing, the focus has shifted toward detecting errors as soon as focus has shifted toward detecting errors as soon as possible. This is because early detection leads to possible. This is because early detection leads to better products which are often delivered on time and better products which are often delivered on time and with less risk to manage.with less risk to manage.
88
Software Testing: Modern ViewSoftware Testing: Modern View
Software should be tested during different stages of Software should be tested during different stages of system development life cycle.system development life cycle.
Almost all software is developed using a type of Almost all software is developed using a type of iterative beta program where programmers, testers, iterative beta program where programmers, testers, developers, and users are encouraged to break the developers, and users are encouraged to break the software or find software functionality that does not software or find software functionality that does not meet desired results.meet desired results.
It has been proven that it is cheaper to fix defects if It has been proven that it is cheaper to fix defects if detected early in development stages.detected early in development stages.
99
Key Terms: Black Box TestingKey Terms: Black Box Testing
Black, White, or Grey Box Testing refers to the point Black, White, or Grey Box Testing refers to the point of view taken while designing test cases and plans.of view taken while designing test cases and plans.
Black Box Testing (BBT) takes an external view of the Black Box Testing (BBT) takes an external view of the test object.test object.
BBT focuses on inputs and outputs of the system, BBT focuses on inputs and outputs of the system, user functionality of the system, and the higher level user functionality of the system, and the higher level workings of the system.workings of the system.
BBT is often automated and is usually one of the final BBT is often automated and is usually one of the final steps before software is released.steps before software is released.
1010
Key Terms: White Box TestingKey Terms: White Box Testing
White Box Testing (WBT) is focused on the internal White Box Testing (WBT) is focused on the internal aspects of the system.aspects of the system.
WBT test cases and test plans are usually developed WBT test cases and test plans are usually developed by programmers and engineers who understand and by programmers and engineers who understand and want to examine the inner workings of the system.want to examine the inner workings of the system.
WBT is usually among the earliest tests performed in WBT is usually among the earliest tests performed in software development.software development.
WBT can also be used to evaluate and fix system WBT can also be used to evaluate and fix system errors for software that is already in production.errors for software that is already in production.
1111
Key Terms: Grey Box TestingKey Terms: Grey Box Testing
Grey Box Testing (GBT) is a hybrid of the black and Grey Box Testing (GBT) is a hybrid of the black and white box testing.white box testing.
GBT is usually focused around databases and GBT is usually focused around databases and connectivity.connectivity.
GBT often requires the tester to populate a GBT often requires the tester to populate a development database with sample data to evaluate development database with sample data to evaluate the proper workings of the data store.the proper workings of the data store.
Testers for GBT are often proficient with SQL, XML, or Testers for GBT are often proficient with SQL, XML, or other data centric languages / technologies.other data centric languages / technologies.
1212
Three Types of TestingThree Types of Testing
““Component testing tests the minimal software item Component testing tests the minimal software item that can be tested.” that can be tested.” 11
““Component integration testing exposes defects in the Component integration testing exposes defects in the interfaces and interaction between integrated interfaces and interaction between integrated components.” components.” 22
““System testing tests an integrated system to verify System testing tests an integrated system to verify that it meets its requirements.” that it meets its requirements.” 33
A team’s failed integration test…A team’s failed integration test… 1,2,3: 1,2,3: http://en.wikipedia.org/wiki/Software_testinghttp://en.wikipedia.org/wiki/Software_testing
1313
Acceptance TestingAcceptance Testing
““Acceptance testing - allows the end-user or customer Acceptance testing - allows the end-user or customer to decide whether or not to accept the product.” to decide whether or not to accept the product.” 11
““Alpha testingAlpha testing is simulated or actual operational is simulated or actual operational testing by potential users/customers or an testing by potential users/customers or an independent test team at the developers’ site.” independent test team at the developers’ site.” 22
““Beta testingBeta testing comes after alpha testing. Versions of comes after alpha testing. Versions of the software, known as beta versions, are released the software, known as beta versions, are released to a limited audience outside of the company.” to a limited audience outside of the company.” 33
1,2,3: 1,2,3: http://en.wikipedia.org/wiki/Software_testinghttp://en.wikipedia.org/wiki/Software_testing
1414
Regression TestingRegression Testing
Regression testing is often used to re-test functionality Regression testing is often used to re-test functionality that has previously passed system or acceptance that has previously passed system or acceptance testing.testing.
It is being re-tested because of changes to the It is being re-tested because of changes to the software code to ensure that the system still works software code to ensure that the system still works properly and that the changes have not introduced properly and that the changes have not introduced unintentional regression of previously achieved results.unintentional regression of previously achieved results.
A vendor’s failure to thoroughly regression test…A vendor’s failure to thoroughly regression test…
1515
OO Testing Environment … Unique OO Testing Environment … Unique ChallengesChallenges
In an Object Oriented (OO) programming environment, In an Object Oriented (OO) programming environment, individual parts (objects) work together as a system.individual parts (objects) work together as a system.
An object has a set of attribute values that define a An object has a set of attribute values that define a state of the object and the actions the object can do. state of the object and the actions the object can do.
Because objects are the basic building blocks of a Because objects are the basic building blocks of a system, any change to their attribute values system, any change to their attribute values functionality, or accessibility may cause unintended functionality, or accessibility may cause unintended changes downstream in the environment.changes downstream in the environment.
1616
OO testing environment: So what’s OO testing environment: So what’s the problem?the problem?
In the paper, In the paper, Developing an Object-Oriented Software Developing an Object-Oriented Software Testing and Maintenance Environment, Testing and Maintenance Environment, the main the main problems seen during OO testing areproblems seen during OO testing are11:: The understanding problemThe understanding problem Complex interdependencyComplex interdependency Object state behavior testing problemObject state behavior testing problem The tool support problemThe tool support problem
1: 1: Article CitationArticle Citation
1717
The Understanding Problem The Understanding Problem 11
It becomes difficult to decipher to process / data flow It becomes difficult to decipher to process / data flow when encapsulation and information hiding are used.when encapsulation and information hiding are used.
Often multiple objects are used with multiple classes Often multiple objects are used with multiple classes creating a system that can use an exponentially large creating a system that can use an exponentially large number of object combinations. number of object combinations.
1,2: 1,2: Article CitationArticle Citation
1818
The Understanding Problem The Understanding Problem 11
Understanding the relationships between objects is Understanding the relationships between objects is important for a successful test.important for a successful test.
““Although each of the member functions is very small Although each of the member functions is very small (a total of 15 lines of code, or an average of 5 lines per (a total of 15 lines of code, or an average of 5 lines per member function), it took, on average, 0.95, 0.93, and member function), it took, on average, 0.95, 0.93, and 0.79 person-hours to prepare the test data, test driver, 0.79 person-hours to prepare the test data, test driver, and test stub, respectively.” and test stub, respectively.” 22
1,2: 1,2: Article CitationArticle Citation
1919
Complex Dependency Problem Complex Dependency Problem 11
Analysts and testers spend large amounts of time Analysts and testers spend large amounts of time trying to understand classes in large OO program and trying to understand classes in large OO program and determine if the classes depend on other classes.determine if the classes depend on other classes.
““Without sufficient insight, a tester may not know where Without sufficient insight, a tester may not know where to start testing an OO library.” to start testing an OO library.” 22
1,2: 1,2: Article CitationArticle Citation
2020
Complex Dependency Problem Complex Dependency Problem 11
““It is difficult to identify and test the effect of It is difficult to identify and test the effect of polymorphism and dynamic binding.” polymorphism and dynamic binding.” 22
““It is difficult to identify change impact in OO It is difficult to identify change impact in OO maintenance, since the impact may ripple throughout maintenance, since the impact may ripple throughout the OO program through the complex dependencies.” the OO program through the complex dependencies.” 33
1,2,3: 1,2,3: Article CitationArticle Citation
2121
The State Behavior The State Behavior Testing Problem Testing Problem 11
What an object returns is very dependant on the state What an object returns is very dependant on the state of the object when it is told to act.of the object when it is told to act.
It is erroneous to assume a product works fine by just It is erroneous to assume a product works fine by just testing individual operations without evaluating the testing individual operations without evaluating the object’s state.object’s state.
Integration testing doesn’t solve this problem; it will just Integration testing doesn’t solve this problem; it will just test that individual member functions are interfacing test that individual member functions are interfacing correctly.correctly. 1: 1: Article CitationArticle Citation
2222
The Tool Support Problem The Tool Support Problem 11
Conventional testing methods used by commercially Conventional testing methods used by commercially available software may not necessarily deal with available software may not necessarily deal with problems identified in object oriented testing.problems identified in object oriented testing.
It is better to have a formal specification language that It is better to have a formal specification language that will be used by a testing tool to derive test cases will be used by a testing tool to derive test cases customized to meet software specifications.customized to meet software specifications. 1: 1: Article CitationArticle Citation
2323
An OO Testing SituationAn OO Testing Situation
““Central to the environment is a mathematically defined Central to the environment is a mathematically defined test model, consisting of three types of diagrams:” test model, consisting of three types of diagrams:” 11
ORD: Object relation diagram - represents inheritance, ORD: Object relation diagram - represents inheritance, aggregation, association aggregation, association 22
BBD: block branch diagram - represents the control BBD: block branch diagram - represents the control structure of a member function and its interface to structure of a member function and its interface to other member functions other member functions 33
OSD: Object state diagram - represents the state OSD: Object state diagram - represents the state behavior of an object class behavior of an object class 44
1,2,3,4: Article Citation 1,2,3,4: Article Citation
2424
Advantages of OO Test ModelAdvantages of OO Test Model
It uses graphical models to explain the relationships It uses graphical models to explain the relationships between classes and objects. between classes and objects.
It accelerates preparations by helping the tester / It accelerates preparations by helping the tester / analyst work more efficient.analyst work more efficient.
It assists with defining and analyzing OO testing specs.It assists with defining and analyzing OO testing specs. ““It facilitates the development of the various algorithms It facilitates the development of the various algorithms
and capabilities for OOTM.” and capabilities for OOTM.” 11
1: Article Citation1: Article Citation
2525
Software Testing as an IndustrySoftware Testing as an Industry
Software testing has grown and is continuing to Software testing has grown and is continuing to develop as an industry.develop as an industry.
Larger companies have teams and / or departments Larger companies have teams and / or departments that are solely focused on testing.that are solely focused on testing.
Often firms utilize automated testing tools to Often firms utilize automated testing tools to supplement human testing.supplement human testing.
2626
Testing Industry: CompaniesTesting Industry: Companies
http://www-306.ibm.com/software/rational/offerings/testhttp://www-306.ibm.com/software/rational/offerings/testing.htmling.html
http://www-128.ibm.com/developerworks/rational/kits/tehttp://www-128.ibm.com/developerworks/rational/kits/tester/?S_TACT=105AGX23&S_CMP=SWQster/?S_TACT=105AGX23&S_CMP=SWQ
IBM will provide consulting or testing packages.IBM will provide consulting or testing packages.
2727
Automated Test SoftwareAutomated Test Software
http://www.surgient.com/solutions/devtest/?gclid=CL-Phttp://www.surgient.com/solutions/devtest/?gclid=CL-P2f73sIgCFQ5_UAodB1er3g2f73sIgCFQ5_UAodB1er3g
http://www.seapine.com/products.htmlhttp://www.seapine.com/products.html
Seapine offers a full suite of software development Seapine offers a full suite of software development tools supporting the entire development life cycle.tools supporting the entire development life cycle.
2828
ConclusionsConclusions
Test early and test oftenTest early and test often Integrate the application development and testing life cycle. Integrate the application development and testing life cycle.
You'll get better results and you won't have to mediate You'll get better results and you won't have to mediate between two armed camps in your IT shop.between two armed camps in your IT shop.
Formalize a Formalize a testingtesting methodology; you'll test everything the methodology; you'll test everything the Same way and you'll get uniform results. Same way and you'll get uniform results.
Develop a comprehensive test plan; it forms the basis for Develop a comprehensive test plan; it forms the basis for the vesting methodology. the vesting methodology.
Use both static and dynamic Use both static and dynamic testingtesting. .
2929
ConclusionsConclusions
Define your expected results. Define your expected results. Understand the business reason behind the Understand the business reason behind the
application. You'll write a better application and application. You'll write a better application and better better testingtesting scripts. scripts.
Review and inspect the work; it will lower costs. Review and inspect the work; it will lower costs. Use multiple levels and types of Use multiple levels and types of testingtesting
(regression, systems, integration, load). (regression, systems, integration, load). Don't let your programmers check their own work; Don't let your programmers check their own work;
they'll miss their own errorsthey'll miss their own errors
3030
QuestionsQuestions
Do you have any questions?Do you have any questions?
3131