Introduction to
Software Testing
Definition of Software Testing Previous
Testing is a part of coding workMake sure that a system does what it is supposed to do
PresentTesting is the process of analyzing a software item to detect the differences between
existing and required conditions and to evaluate the features of the software items. [IEEE/ANSI, Std 829-1998]
Testing is a concurrent lifecycle process of engineering, using and maintaining testware in order to measure and improve the quality of the software being tested. [Rick D Craig, 2002, “Systematic Software Testing”]
Software Testing To find and prevent defect To have confidence that the system
performs adequately To provide information about
quality To provide the insight of product
risk and quality To ensure a program corresponds to
its specification Defect Detection Mechanism
Software Test Process Test Strategy Test Plan Test Design Setup Test Environment Test Execution Defect Tracking Testing Tools Evaluate Test Exit Criteria Reporting Improve Test Process
Some terminologies
Error
Defect, Fault, Bug
Incident, Issue
Failure
Risk
Related to human=Mistake
Related to productBug= slang / dialect
Related to eventAccident
Risk = likelihood of failure x impact
Some terminologiesTest
DesignTest Case : a set of input values and expected results, developed for a particular objective or test condition
Test Case 1 to be executedTest Case 2 to be executedTest Case 3 to be executed
Test Case n to be executed
Order, story
included
Test Suite
Test Procedure= Test script
Test Scenario
Test Objectives (Why)Primary
•To find defects•To gain confidence of software•To prevent defects•To see software matching its requirement specifications•To reduce the business risk•To provide well-informed advice (quantitative metric)
Others•To check development process•To detect issues•To check logical design•To confirm that the system and software work properly
Characteristic in Software Testing
Exhausting testing is impossible There would be infinity path in a program There would be infinity combination of all inputs values There would be infinity timing or moment of occurring the
GUI event Testing cannot prove that there are no
defects It is hard to find defects if you produced
the product Practical Goal of Testing
“To maximize possibility of finding defects at the high risk area within the given resource (time, people) -> to minimize risk”
Why Software need to be tested as early as possible?
General Testing Principles Testing Shows presence of defects Exhaustive testing is impossible
(why? – 2 on/off button ,tab option, calculator)
Early Testing Defect Clustering Pesticide paradox Testing s context dependent Absence-of-error fallacy
The V Model
Integration test
Requirements
DetailedDesign
Moduleimplementation
Unit test
System test
Acceptancetest
Run
test
sW
rite tests
CustomerDeveloper
Functional(BB)
Structural(WB)
ArchitecturalDesign
Req. analysis
Logical spec.
Physical spec.
Program source
V model matrixTest Levels Goal Perform
er Env Test Basis Difficulty levels
Acceptance Req. User Real SRS User
System Non-functional Prof. Tester
Testing SDD + SRS
Integration Functional Developer Dev. SDD
Unit Find Defect Developer-peer Dev. Functions Technical
Validation vs. Verification
Verification ValidationDefinition Am I building the product
right?The review of interim work steps and interim deliverables during a project to ensure they are acceptable.
Am I building the right product?Determining if the system complies with the requirements and performs functions for which it is intended and meets the orgainization’s goals and user needs. It is traditional and is performed at the end of the project.
Characteristic Human-Based Testing- To find defects, read documentations
Computer-Based Testing- To find defects, execute the software
Type Manual review•inspection/walkthrough/peer review•Static analysis
Low level testing•Unit testing / integration testing
High Level testing• usability testing, Function testing, System testing, acceptance testing.
Classification of Testing Static Testing
Technical Review
Inspection
Requirement Testing
Design Testing
Static Analysis
Dynamic Testing Black box testing Risk Based Testing Spec-based testing Exploratory based testing Experience based testing Structure based testing (white box) Statistical testing CTM (Classification Tree Method) Use Case Testing State Transition Testing Boundary Analysis & Equivalent Partitioning Control Flow Testing Basis Path Testing Pairwise Elementary Comparison Test Structure Testing Acceptance Testing System Testing Integration Testing Componen (Unit) Testing Structure Testing Regression / Confirm Testing Functional / Non-Functional Testing
Testing Approach
Testing Design
Testing Levels
Testing Types
Effective and Efficient Testing What is effective testing ?
To make test result which is decided or desired Effective tester decide what test result to be drawn from the
testing effort What is efficient testing?
Desired test result can be achieved as soon as possible (performance)
Efficient tester allocate time, cost and resource properly
Effective testing vs. efficient testing?
When to stop? Where to test more? The best way for effective and
efficient = Risk based testing
Effective and Efficient Testing Effective testing vs. efficient testing?
The best way for effective and efficient testing => Risk based testing
Efficient
Effective
Risk MatrixHi Chances of FailureLow Business Impact
Hi Chances of FailureHi Business Impact
Low Chances of FailureLow Business Impact
Low Chances of FailureHi Business Impact
Business Impact
Chances of Failure(likelihood)
Standards ISO/IEC29119 – Standard for Softwar
e Testing Process IEEE829 – Standard for Software Test
ing Document
Testing ArchitectureMaster Test Plan
Test Policy
Test Process Improvement
StakeholdersEnd User manageme
nt
Test Basis
Test organization
Test Strategy Test Tech. Test
ItemsTest
Environment
Test Level
Lifecycle
Plan/Control
AnalysisDesign
Imple.Exec.
Exit / Reporting
ClosurePlan/Control
AnalysisDesign
Imple.Exec.
Exit / Reporting
Closure
Test Plan Test Spec.Test Case
Status Report
Test Procedur
eTest ResultResult Report
ReleaseFinal
Report
Test Process
evaluation and advice
Test Process
Improvement Plan
Testware
Defects Reporting Test
Reporting
Plan/Control
AnalysisDesign
Imple.Exec.
Exit / Reporting
Closure
Organization
Test OrganizationSteering
CommitteeBusiness
Testing (permanent / project base)
InstallCodingAnalysis / Design
Project Staff & QA
Project Management
Support•Strategy, tech, training•Infra & Tools•Subject Matter
Testing A
Test Management
Testing B Testing C
1. Tested by Developer2. Independent Tester inside Project Team
3. Project Testing Group in organization
4. Business OrganizationUser CommunityIndependent IT org.
6. Outsourcing
5. Independent Test specialistUsability / Security/ CertificateTesting