<Insert Picture Here>

SOFTWARETESTING PROCESS

Byungwook Cho C. (bwcho75@gmail.com)Sr Consultant 2008.08.08

ICE BREAK

AGENDA

• Test model• Test process• Static testing

Test model

Test oriented development process model

• V-Model• Enhanced traditional

waterfall model

• Verification & Validation

Verification Validation

To verify the artifacts that has been produced in each development cycle.

Valid & estimate the system

Review & Inspectionwith artifacts from each development step

Testing with system

Static testing Dynamic testing

< V-Model >

< Verification & Validation >

Test LevelLevel Description Testing Type Responsibility

Unit Test Verify software component White box test Lead by Dev Team Use Code coverage

Integration Test Verify integration between component. Verify software flow ,interface & interaction

White box test Lead by Dev Team Use Mock Object (Test Stub ,Driver)

System Test Test system over production environment to verify system itself and production environmentInclude Functional & Non Functional Test (Availibility, Stablity, Extendability, Performance etc)

Black box test Lead by QA Team(Specialized for system test)

Acceptance Test Verify customer’s requirement (End User)Verify customer’s Legal issue (Legal)Verify customer’s maintanance issue (SM)

Black box test Lead by QA Team or Customer

Alpha TestBeta Test

※ Integration strategiesTop down Bottom up Big-bang Backbone

Desc Integrate from top(use test stub)

Integrate from bottom(use test driver)

Integrate all in time Most important & high risk component first

Adv easily find architectural defect

Can test implementation logic

Short time

Ex. BPM SOA svc Small project

Test Cycle

• Each test is consists of test cycle• Test cycle

• Pre test - check up testability• Main test • Conformation test - check up the defect found in

previous “Main test”• Regression test - check up the impact from change

Pre Test Main Test

ConformationTest

RegressionTest

TEST CYCLE

Test Process

Master Test Plan

Organization

Policy

Strategy

Mgmt sub tests

Test ProcessTest plan

Define objective of Test and scope

Define Test strategy

Test estimation

Set up test organizationTest analysis and design

Evaluation test object & basis

Designing and prioritizing test case

Identify test

Identify & prepare test data

Designing test env & identify required infra & tools

Make test procedure

Test Implementation & Execution

Implement TC

Implement test script

Set up test environment

Pre Test

Run Test

Logging Test Result

Test Closure activity

Store test-ware into repository

Analyzing lesson learned for future test maturity

Evaluating exit criteria and reporting

Write a test summary report for stakeholders.

Assessing if more test are needed

Test control

Test mgmt & control

Test monitoring & reporting

Test plan

Define objective of Test and scope

Define Test strategy

Test estimation

Set up test organizationTest analysis and design

Evaluation test object & basis

Designing and prioritizing test case

Identify test

Identify & prepare test data

Designing test env & identify required infra & tools

Make test procedure

Test Implementation & Execution

Implement TC

Implement test script

Set up test environment

Pre Test

Run Test

Logging Test Result

Test Closure activity

Store test-ware into repository

Analyzing lesson learned for future test maturity

Evaluating exit criteria and reporting

Write a test summary report for stakeholders.

Assessing if more test are needed

Test control

Test mgmt & control

Test monitoring & reporting

Test plan

Define objective of Test and scope

Define Test strategy

Test estimation

Set up test organizationTest analysis and design

Evaluation test object & basis

Designing and prioritizing test case

Identify test

Identify & prepare test data

Designing test env & identify required infra & tools

Make test procedure

Test Implementation & Execution

Implement TC

Implement test script

Set up test environment

Pre Test

Run Test

Logging Test Result

Test Closure activity

Store test-ware into repository

Analyzing lesson learned for future test maturity

Evaluating exit criteria and reporting

Write a test summary report for stakeholders.

Assessing if more test are needed

Test control

Test mgmt & control

Test monitoring & reporting

Test plan

Define objective of Test and scope

Define Test strategy

Test estimation

Set up test organizationTest analysis and design

Evaluation test object & basis

Designing and prioritizing test case

Identify test

Identify & prepare test data

Designing test env & identify required infra & tools

Make test procedure

Test Implementation & Execution

Implement TC

Implement test script

Set up test environment

Pre Test

Run Test

Logging Test Result

Test Closure activity

Store test-ware into repository

Analyzing lesson learned for future test maturity

Evaluating exit criteria and reporting

Write a test summary report for stakeholders.

Assessing if more test are needed

Test control

Test mgmt & control

Test monitoring & reporting

Acceptance Test Plan

System Test Plan

Integration Test Plan

Unit Test Plan

Test Plan

• Test planning & control• Define activity of each testing level• Define milestone ,resource ,schedule.• Make a plan based on analyzed risk factor• Define test strategy based on risk• Define approach & techniques for testing

(testing techniques, coverage, test item, test ware)

• Must include time & resource for preparing testware

• Define completion condition

Define test scope

objective

Risk anal

Set up strategy

Estimateresource & time

Planning

PRE TEST

TEST

Monitoring

Report

Test Plan

• Risk analysis• Risk = Likelihood * impact• Risk identification

• Recommend < 36 identification.• Likelihood : Complexity, Implementation technical Level,

Size,Developer skill• Impact : Biz impact

Likelihood Impact

factor factor factor factor

Risk item

Risk item

Likelihood

Impact

STA

(Severe Test Area)

SSTA

(Strong Test Area)

FTA(Fundamental

Test Area)

ITA

(Intensive Test

Area)

Erik van neneendaal, Risk Based Testing, STAREAST,2006

Test Plan

• Risk based test strategy

• Other strategies• Analytical approaches• Model based approaches• Methodical approaches• Dynamic and heuristic approaches

Likelihood

Impact

STA

(Severe Test Area)

SSTA

(Strong Test Area)

FTA(Fundamental

Test Area)

ITA

(Intensive Test

Area)

Likelihood

Impact

STA

(Severe Test Area)

SSTA

(Strong Test Area)

FTA(Fundamental

Test Area)

ITA

(Intensive Test

Area)

( Low Level Test) ( High Level Test)

- Unit Test

- Integration Test

- System Test

- Acceptance Test

Test Analysis and Design

• Test analysis• Review - test basis, testability• Identify test requirement & test data• Identify test infra & tools

• Test design• Identify test condition• Test case specification• Define test procedure

Test Analysis and Design

• Design technique• Specification based

• Equivalence partitioning, Boundary value analysis• Pairwise, Decision table testing• State transition testing, Usecase testing

• Structure based• Control Flow Test• Basic Path Test – Based on Cyclomatic complexity (Edge –

Node + 2P) P = number of connected component• Elementary comparision test – Based on MC/DC Coverage* Coverage : Statement, Condition, Decision, Condition Decision

Coverage, Modified Condition/Descision Coverage(MC/DC), Multiple Condition Coverage

• Experience based• Exploratory Testing – Adhoc testing, based on test charter,

define timezone• Classification Tree method• CheckList

Test Execution

• Execution• Define serverity & priority• Develop TC,Test data,Test stub &

driver,Test script,Test suite

• Reporting• Logging expected result, real result• Current status• Time & resource usage

• Defect tracking• Use Bug(Issue) tracking system

( Mantis,Bugzilla,JIRA,Trac)• Define tracking workflow

TESTDRIVER

TESTSTUB

TARGET SYSTEM

TEST CASE

TESTINGTOOL

TEST DATA MODULE

MODULE MODULE MODULE

TEST SCRIPT

TEST SUITE1

TEST SUITE2

TEST

SIMULATOR

Test Execution

Test Report Defects

Issue Tracking System

Progress Report

Test logs

Evaluating exit criteria and reporting

• Test exit criteria• Coverage, Duration, number of unresolved defects

• Test result report• Progress report• Release advice• Final report per Level

• Matrix• Defects per KLOC, Days test effort per requirement,

Unresolved defect impact analysis, Defects / Hour ,Cumulative defects / Day ,Test coverage / Day , Risk vs number of TC

Test closure activities

• Test closure activities• Evaluating

• Evaluate test target• Evaluate test process for maturing next text

• Final reporting• Store test-ware

Static Testing

Static testing techniques

• Performed in Verification phase• Test with artifacts not a runtime software• Test by human

Manual Review

• Inspection• Objective : To find defects• Inspect artifacts• Lead by moderator

• Walkthrough• Objective : Knowledge sharing• Knowledge transfer by author

• Code review• Objective : Find defects and share knowledge• Lead by Programming Leader

• Pair programming• XP • Lead by mentor or coworker• Share just “ONE” keyboard

Formal action

Informal action

Static Analysis

• Static Analysis• Test code or diagram with “static analysis tool”• Analysis

• Can find syntax error or well known bugs• Validate naming rules• Can find dead code or missing parameter• Can analysis code complexity & dependency

• Quality?• Not so smart• It need a rule and need a maturing

• Easy to appeal your manager• Tools

• Open source – PMD, Find Bugs• Commercial – Klockworks, Jtest etc.