Advanced Software Engineering: Software
TestingCOMP 3702 (Lecture1)
Sada NarayanappaSeif Azgandhi
Anneliese Andrews Thomas ThelinCarina Andersson
A Andrews - Software Engineering: Software Testing'06
27 dm3, 31 kg
4 KB read/write magnetic core memory
64 KB read-only memory
A Andrews - Software Engineering: Software Testing'06
A Andrews - Software Engineering: Software Testing'06
Software on Wings4 M LOC
A Andrews - Software Engineering: Software Testing'06
Facts about testing
System development:1/3 planning1/6 coding1/4 component test1/4 system test[Brooks75]
A Andrews - Software Engineering: Software Testing'06
Implementation
Analysis
Design Function test
System Test
A Andrews - Software Engineering: Software Testing'06
Good enough quality
To claim that any given thing is good enough is to agree with all of the following propositions:
It has sufficient benefits
It has no critical problems
The benefits sufficiently outweigh the problems
In the present situation, and all things considered, further improvement would be more harmful than helpful
James Bach, IEEE Computer, 30(8):96-98, 1997.
A Andrews - Software Engineering: Software Testing'06
Quality attributes – ISO 9126
A Andrews - Software Engineering: Software Testing'06
Quality attributes – McCall
A Andrews - Software Engineering: Software Testing'06
Why use testing?
Risk mitigationFaults are found earlyFaults can be preventedReduce lead-timeDeliverables can be reused…
A Andrews - Software Engineering: Software Testing'06
Why do faults occur in software?Software is written by humans
Who know something, but not everything Who have skills, but aren’t perfect Who don’t usually use rigorous methods Who do make mistakes (errors)
Under increasing pressure to deliver to strict deadlines No time to check, assumptions may be wrong Systems may be incomplete
Software is complex, abstract and invisible Hard to understand Hard to see if it is complete or working correctly No one person can fully understand large systems Numerous external interfaces and dependencies
A Andrews - Software Engineering: Software Testing'06
Origins of defects
Defect sources
Lack of educationPoor communicationOversightTranscriptionImmature process
Impact of software artifactsErrors
Faults / Defects
FailuresImpact from user’s
viewPoor quality software
User dissatisfaction
Fault model
A Andrews - Software Engineering: Software Testing'06
Whoops, that’s my calculator
A Andrews - Software Engineering: Software Testing'06
Testing, Verification & Validation
Definition 1Verification
is the product right?
Validation is it the right product?
Definition 2Verification
satisfies the conditions at the start of the phase
Validation satisfies the requirements
TestingThe process of evaluating a program or a
system
A Andrews - Software Engineering: Software Testing'06
Definitions
Failure is an event, fault is a state of the software caused by an error
Error – human mistake Fault / Defect – anomaly in the software Failure – inability to perform its required functions Debugging / Fault localization – localizing, repairing, retesting.
A test case consists of: A set of inputs Execution conditions Expected outputs
A Andrews - Software Engineering: Software Testing'06
Scripted and non-scripted testing
In scripted testing test cases are pre-documented in detailed, step-by-step descriptions
Different levels of scripting possible
Scripts can be manual or automated
Non-scripted testing is usually manual testing without detailed test case descriptions
Can be disciplined, planned, and well documented exploratory testing
or ad-hoc testing
A Andrews - Software Engineering: Software Testing'06
Test oracle
An oracle is the principle or mechanism by which you recognize a problemTest oracle provides the expected result for a test, for example
Specification document Formula Computer program Person
In many cases it is very hard to find an oracle Even the customer and end user might not be able to
tell which is the correct behaviour
A Andrews - Software Engineering: Software Testing'06
Principle 1 – purpose of testing
Testing is the process of exercising a software component using a selected set of test cases, with the intent of
1. Revealing defects
2. Evaluating quality
A Andrews - Software Engineering: Software Testing'06
Principles
2: A good test case – When the test objective is to detect defects, then a good test case is one that has high probability of revealing a yet undetected defect(s)
3: Test result – The results should be inspected meticulously
4: Expected output – A test case must contain the expected output
A Andrews - Software Engineering: Software Testing'06
Principles
5: Input – Test cases should be developed for both valid and invalid input conditions
6: Fault content estimation – The probability of the existence of additional defects in a software component is proportional to the number of defects already detected in that component
7: Test organization – Testing should be carried out by a group that is independent of the development group
A Andrews - Software Engineering: Software Testing'06
Principles
8: Repeatable – Tests must be repeatable and reusable
9: Planned – Testing should be planned
10: Life cycle – Testing activities should be integrated into the software life cycle
11: Creative – Testing is a creative and challenging task
A Andrews - Software Engineering: Software Testing'06
Test
Mat
urity
Mod
el
A Andrews - Software Engineering: Software Testing'06
Goals of the course
Knowledge
Skills
Attitudes
A test specialist who is trained as an engineer should have knowledge of test-related principles, processes, measurements, standards, plans, tools, and methods, and should learn how to apply them to the testing tasks to be performed.
A Andrews - Software Engineering: Software Testing'06
www.swebok.org
A Andrews - Software Engineering: Software Testing'06
www.swebok.org
A Andrews - Software Engineering: Software Testing'06
Lectures
Theory + discussions Cover the basic parts of software testing
1. Introduction2. Black-box testing, Reliability, Usability3. Inspections, white-box testing4. Lifecycle, documentation5. Organization, tools6. Metrics, TMM7. Guest lecture,
Technical
Overview
Technical / ManagerialManagerial
Economic
A Andrews - Software Engineering: Software Testing'06
Lab sessions
Preparation, Execution, Report
1. Black-box testing
2. Usage-based testing and reliability
3. White-box testing
4. Inspection and estimation
5. Software process simulation
A Andrews - Software Engineering: Software Testing'06
Project: Option 1
Learn a specific area of software testing Collect and summarize research information Critical thinking beyond the written
information Present information in a structured way Peer review
A Andrews - Software Engineering: Software Testing'06
Examination
Written exam based on the book and lab sessionsLab sessions (approved)Project (F / G / VG)
A Andrews - Software Engineering: Software Testing'06
Schedule
Read Course program Projects in Software Testing
Check homepageNot decided
Extra Lab dates
A Andrews - Software Engineering: Software Testing'06
This week
Read course programLecture 2 tomorrow Project
Read Projects in Software Testing Exercise on Thursday Decide subject (deadline Thursday 17/3)
Lab Prepare lab 1
Read Burnstein 1-3 Prepare Burnstein 4,12
A Andrews - Software Engineering: Software Testing'06
Project: Option 1
Research: solve a research problem; survey the state-of-the-art and identify the research problems in some area; develop and justify an extension to an existing technique; etc.Evaluation: apply and evaluate a technique or evaluate a commercial testing or, analysis tool.Practical: Use an existing technique to test a system or design and implement a prototype for a system.
A Andrews - Software Engineering: Software Testing'06
Project: Option 1
Read Projects in Software Testing Divide in groups (2-3 persons)
Discuss with me
http://www.cs.du.edu/~andrews/andrews/COMP3702/FilesFromCD/Project/Project_SwTest.pdf
A Andrews - Software Engineering: Software Testing'06
Project: Option 2
Experiment participation.Compare Fault finding
Unit test code inspection
Two sessions 3-4 less totalFull credit