INTRODUCTION TO

SOFTWARE TESTING INDUSTRY

February 2015

Purpose of the Presentation ?

To provide some context on the software testing industry (broader than our day-to-day duties).

What is Software Testing ?

Software Testing is …

• Software testing is a process of executing a program or

application with the intent of finding the software bugs (ISTQB)

• The principle objective of software testing is to give confidence in the software. (Unknown Author)

• Testing is an infinite process of comparing the invisible to the ambiguous in order to avoid the unthinkable happening to the anonymous (James Bach)

• Software testing is an investigation conducted to providestakeholders with information about the quality of the product or service under test. (Bertrand Meyer)

Money in Software Testing …

Money in Software Testing Industry

Pierre Audoin Consultants ( https://www.pac-online.com )

has forecasted the software testing spends to hit

Euro 100 billions by 2014.

IDC and Gartner forecast that the enterprise spending on independent testing services would cross the benchmark of USD 30 billions (Euro ~25 billions) by 2015.

(http://conference.eurostarsoftwaretesting.com/2011/testing-times-stay-factors-influencing-spending-testing-services/)

Money (Lithuanian GDP)

Estimate for LT GDP 2014 – USD 50 Billions.

Billions of EURO

0

20

40

60

80

100

120

LT GDP 2014 Independent Testing by2015

Overall Testing 2014

Money in SW Testing Industry vsLithuanian GDP

History of Software Testing…

http://www.softwaretestpro.com/Item/4537/History-of-Ideas-in-Software-Testing/Agile-Performance-Automation-Metrics-Development-Acceptance-Exploratory-Functional-Integration-Process-Software-Strategy-Testing-Teams-Unit-Six-Sigma-selenium-Web

http://www.testingreferences.com/testinghistory.php

History of Testing [before 1970]

1958 – First independent Test Team at IBM

1960s:

• Testing is mentioned in “Computer Programming Fundamentals” book by Jerry Weinberg

• Some of the Testing methods whitepapers are published (IBM, 1967)

• NATO report mentions Software Quality (1968)

History of Testing [1970s]

Cyclomatic complexity, Quality metrics, structure based testing, code inspections…

1979 - Glenford J. Myers in his “The Art of Software Testing” introduces separation of debugging from testing and summarizes the black box testing methods.

History of Testing [1980s]

Boris Beizer

“Software Testing Techniques”,

“Black-Box Testing”

IEEE 829

Exploratory Testing introduced by Cem Kaner (1988)

Black-box and White-box testing techniques are defined

History of Testing [1990s]

Boris Beizer “Bug Taxonomy”, “Pesticide Paradox”

Cem Caner “Testing Computer Software”

Mercury Interactive WinRunner started

First Star conference (1992, Las Vegas)First EuroStar conference (1993, London)

Tmap published (1995)ISEB certification created (1998)

James Bach – Context Driven Testing (1999)

History of Testing [2000s …]

James Bach – Rapid Software Testing, Session Based Testing

James Bach, Cem Kaner, Bret Pettichord“Lessons learned in Software Testing”

ISTQB founded (2002)

IBM Rational Tester (2002), Selenium (2004)

"Schools of Software Testing“, 2003 - Bret Pettichord

uTest – crowdsourcing, 2008

History of Testing [2000s]

Gojko Adzic “Specification by Example” (2011)

Acceptance Test Driven Development (ATDD)

Behavior Driven Testing (BDT)

(TDD -> ATDD -> BDT)

Gherkin (Cucumber specification)

Fitness….

Some More on Software Testing …

Schools of Software Testing

• Analytical School (structural testing)

• Factory School (requirement traceability)

• QA School (gatekeeping over testing)

• Context Driven School

http://www.testingeducation.org/conference/wtst_pettichord_FSofST2.pdf

Software Testing Contributors

Software Testing Industry

Some Software Testing Folklore

Some Software Testing-Related Terms

Triangle Problem (1960s)http://www.testingeducation.org/conference/wtst3_collard5.pdf

Pesticide Paradox (early 1990s)http://www.softwaretestingclub.com/profiles/blogs/defect-clustering-pesticide-paradox

Software Archeology (early 2000s)http://en.wikipedia.org/wiki/Software_archaeology

Expensive Bugs

• Mariner Bugs Out (1962), Cost: $18.5 million Disaster: The Mariner 1 rocket with a space probe headed for Venus diverted from its flight path shortly after launch.

• World War III… Almost (1983), Cost: Nearly all of humanity Disaster: The Soviet early warning system falsely indicated the United States had launched five ballistic missiles

• Patriot Fails Soldiers (1991), Cost: 28 soldiers dead, 100 injured. Disaster: During the first Gulf War, an American Patriot Missile system in Saudi Arabia failed to intercept an incoming Iraqi Scud missile.

• Cancer Treatment to Die For (2000), Cost: Eight people dead, 20 critically injured.

http://top-100s.blogspot.com/2009/05/10-most-expensive-software-blunders.html

A Software Testing Urban Legend

Claim: A flavor of ice cream being transported affects a car's performance: every time vanilla is the driver's choice, the car stalls (vapor lock).

http://www.snopes.com/autos/techno/icecream.asp

(Collected on the Internet, 1998)

THANK YOU