單元 5 ：軟體測試與驗證

Software Engineering for Digital Home

單元 5 ：軟體測試與驗證


Outline

• Introduction• Strategies• Exercise

2


Testing

• To demonstrate to the developer and the customer that software meets its requirement ( 是否合乎需求 )

• Testing is a fault detection technique that tries to create failures or erroneous states in a planned way.(Testing 就是試著弄壞系統找出 Bug)

3


Test Stubs & Test Driver• Test Stubs: If the function A you are testing calls another

function B, then use a simplified version of function B, called a stub.( 虛構的程式方法 )

• Test driver – allows you to call a function and display its return values.( 可以啟動你的程式，並驗證回傳結果 )

4

int YourFunction(){

// 呼叫其他程式…

…}

YourFunction

YourOtherFunction

int YourOtherFunction(){

return 1;}

Test Stubint main() {

// 呼叫被測試程式…

// 驗證結果…

}

Test Driver

YourFunction


Software Lift Cycle

5

Software Life Cycle

Development cycle

Design Implementation Testing

Maintenance

RequirementsEngineering

System Design

Object Design

RequirementsElicitation

Analysis

Testing Plan (Test case) Testing Report

Unit TestingIntegration

Testing System Testing


Bug

• Fault – 又稱作 bug 或 defect ( 缺陷 ) ，是 design 或 coding錯誤導致反常的結果• Erroneous state – 有 fault 在系統執行中發生• Failure – 系統實際的行為與預計不同，通常由 1 個以上的

erroneous state 導致

6

Software

Design/Implement Time

Fault(Bug, Defect)

Software

Runtime Time

Erroneous state

System view User View

Software

Failure


Testing

Software developer ( 程式開發者 )

Testing Strategies• Testing Plan – 包含測試相關資訊、測試種類、 test case 與測試進度 (IEEE 829)• Testing Report – 測試報告

7

Testing Plan( 測試計劃 )

Testing Report( 測試結果報告 )

Bug Report( 問題報告 )

Test Case Result( 試案案例的結果 )Test Case

( 試案案例 )

Unit Testing( 單元測試 )

Integration Testing( 整合測試 )

Independent testing team( 獨立測試小組 )

System Testing( 系統測試 )


Bug Report

8


Test Case

• 設計一個情況 (Condition) ，然而軟體程式在這種情況下，必須能夠正常運作並且達到程式所設計的執行結果 (Expect Result) 。

• 如果程式在這種情況下並不能正常運作，而且這種問題是能夠被重複產生的 (Reproducible) ，那表示軟體程式人員已經測出軟體有瑕疵 (Bug) ，這時候就必須將這個問題標示出來，並且輸入至問題追蹤系統 (Bug Tracking System) 內，通知軟體開發人員。

9


Test Case Design

• 原則– Involves designing the test cases (inputs and outputs) used to

test the system.(設計測試案例要有輸入輸出 )

– Design inputs that cause buffers to overflow.(設計會產生 overflow的資料 )

– Force computation results to be too large or too small.(選擇很大與很小的資料去測試 )

• 方針– Equivalence Testing– Path Testing

10


Requirement

11

Use case name 建立作業Participating actors

Initiated by 老師Communicates with 學生

Flow of events 1. 老師要建立作業2. 老師輸入作業的名稱 (50 字元 ), 作業的要求 (2000 字

元 ), 截止日期3. 系統檢查輸入的作業名稱 , 作業的要求 , 截止日期是否正確4. 如果正確 , 系統就建立止作業

Exceptions 3.1 如果所輸入的資料不正確 , 系統要求使用者重新輸入Entry condition 老師己登入教學網站Exit conditions 老師在教學網站上建立了一個新的作業


Test Case

12

Title 建立作業Test Purpose 測試建立作業功能是否正確

Test Steps1. 輸入作業名稱 (50 字元 ) 、作業說明 (2000 字元 ) 、截止日期後按下新增按鈕2. 顯示已新增作業的作業清單

Interface

Input Interface Output Interface


Test Case – I/O Form

13

Items Input Output 是否正確作業名稱長度 1個字 A

作業名稱長度 25個字 25 個 A

作業名稱長度 50個字 50 個 A

作業說明長度 1個字 A

作業說明長度 1000個字 1000 個 A

作業說明長度 2000個字 2000 個 A

作業名稱未輸入作業名稱長度 51個字 51 個 B

作業說明未輸入作業說明長度 2001個字 2001 個 B


Equivalence Testing• The program behaves in an equivalent way for equivalence

partition.( 同區塊的資料效力視為等價 )

• Test cases should be chosen from each partition.( 所有部份需要被選取 )

• Requirement ：臨時帳號密碼的長度限定為 2-6 碼

141 2 4 6 7

帳號長度 < 2 帳號長度 >= 2 AND 帳號長度 <= 6 帳號長度 > 6

Boundary Boundary


Path Testing

• Execute each possible path • Not practical with many nested conditionals• Impossible for most loops

15


Path Testing

• 測試活動圖每一個可能路徑• 修改學生資料正確路徑

16

學生點選修改資料

顯示修改完成訊息

更新學生資料寄發確認電子郵件

顯示修改學生資料表單輸入修改資料後送出

驗證學生資料

正確錯誤


Path Testing

• 測試活動圖每一個可能路徑• 修改學生資料例外路徑

17

學生點選修改資料

顯示修改完成訊息

更新學生資料寄發確認電子郵件

顯示修改學生資料表單輸入修改資料後送出

驗證學生資料

正確錯誤


Unit Testing (Component Testing)

• Testing individual subsystems (collection of classes)

• Goal: Confirm that subsystem is correctly coded and carries out the intended functionality

18

SubsystemCode

System Design Document

Unit Test


Unit Testing

19

1. 寫好一個物件的功能

2. 決定決定測試資料3. 開始測試 Test Case Validate

userID Return 測試結果確認False False ○

C True True ○Bill True True ○

Office True False ×William False False ○

/* 帳號的長度限定為 1 到 6 個字元 */class system{

bool ValidateUserID(const char* userID){int num = strlen(userID);

return (num >= 1 )&& (num <= 5 );}

}

測試結果第四項目不符合表示程式有錯


Integration Testing• Testing groups of subsystems and eventually the entire system

• Goal: Test interfaces between subsystems

20

SubsystemCode

System Design Document

Integration Test

SubsystemCode

SubsystemCode


Integration Testing• Integration testing detects faults that have not been detected during unit

testing by focusing on small groups of components.(integration testing 測試的單位為一群 components)

• Two or more components are integrated and tested, and when no new faults are revealed, additional components are added to the group.( 先從兩個以上有關聯的 component 當作一個群組開始測試，該群組沒發現新bug 後才增加其他有關聯的 component 進入該群組 )

• Strategies– Bottom-Up Strategy– Top-Down Strategy– Sandwich Strategy

21


Bottom-Up Strategy

• Strategy1. Start with subsystems in lowest layer of call hierarchy2. Test subsystems that call the previously tested subsystems3. Repeat until all subsystems are included

• Advantage– Interface faults can be more easily found.

( 開發者可控制一個 component ，掌握該 component 與相關底層component 的 inputs 、 outputs ，故容易從 component 的 interface找出 faults)

• Disadvantage– Faults found in the top layer may often lead to changes in the

subsystem interfaces of lower layers.( 越往頂層發現的錯誤牽連越廣，要修正的地方越多 ) 22


Bottom-Up Strategy

23

MSSQLDatabase Server

ODBC/ADO

IISWeb Server

ASP UploadServer-UFTP Service

DMLApplication

DMLInterface


ODBC/ADO

Test

ASP Upload

IISWeb Server

Test

Server-UFTP Service Test

DMLApplication

DMLInterface

Test

Test


Top-Down Strategy

• Strategy1. Start with subsystems in top layer of call hierarchy2. Include subsystems that are called by the previously tested subsystems3. Repeat until all subsystems are included Advantage

• Advantage– It starts with user interface components. Supports test cases for the

functionality of the system.( 由使用者觀點進行測試 )

• Disadvantage– The development of test stubs is time-consuming and prone to error.

( 製作一堆 test stub ，費時又容易發生錯誤 )

24


Top-Down Strategy

25


ODBC/ADO

IISWeb Server


DMLApplication

DMLInterface


ODBC/ADO

Test

ASP Upload

IISWeb Server

Server-UFTP Service

DMLApplication

DMLInterface

Test

Test

Test


Sandwich Strategy

• Combines top-down with bottom-up strategy

• Advantage– Many testing activities can be performed in parallel.

(Top-down 與 Bottom-up 可同時進行 )– It lead to a significantly shorter overall testing time than top-down or

bottom-up testing.(Sandwich strategy花費的測試時間比 Top-down或是 Bottom-up都短 )

• Disadvantage– It is the need for additional test stubs.

(依然會有製作 test stub 所產生的缺點 )26


Sandwich strategy

27


ODBC/ADO

IISWeb Server


DMLApplication

DMLInterface


ODBC/ADO

Test

ASP Upload

IISWeb Server

Server-UFTP Service

DMLApplication

DMLInterface Test

Test

Test

Test


System Testing• Testing the entire system

• System testing is a black-box technique.(黑箱測試的 test case 是根據 use case 而來 )

• Goal: Determine if the system meets the requirements (functional and non-functional)

28

RequirementsSpecification

SystemTest

EntireSystem


Functional Testing• The goal of the tester is to select valid and invalid inputs that are relevant

to the user and have a high probability of uncovering a failure.( 測試人員要以使用者角度選用各種可能發現 failure 的 input 進行測試 )• Test cases are derived from the use cases. (test case依據 use case 而來 )• We do not consider implementation details. ( 不需了解實做細節，只管

inputs 與 results)• Impossible to generate all possible inputs

29

Input data

Expected

Invalid

Output data

Expected

Invalid

System1 12 2

334

4


Non-functional Testing• Product testing

– Usability Testing– Performance Testing– Space Testing– Reliability Testing– Availability testing– Portability Testing

• Organisational testing– Delivery Testing– Implementation Testing– Standards Testing

• External Testing– Interoperability Testing– Ethical Testing– Privacy Testing– Safely Testing

• Configuration Testing

30


Performance Testing

• Timing Testing(時間測試 )Response times and time to perform a function

• Stress Testing(壓力測試 )Stress limits of system (maximum number of users, peak demands)

• Recovery Testing(回復測試 )System’s response to presence of errors or loss of data

31


Tools for performance testing• WAPT 4.0 http://www.loadtestingtool.com/• TestView http://www.radview.com/products/testview.asp• AdventNet QEngine Web Performance Test tool

http://www.adventnet.com/products/qengine/web-performance-test.html

32


Exercises

• 試完成第 20張投影片 (Bottom-up)及第 22張投影片 (top-down)全部測試的順序• 請依照教學網站作業功能製作 I/O Form

• 至教學網站下載 testing plan 進行 system testing ，需使用 bugzilla 進行 bug report

33

Documents

單元 5 ：軟體測試與驗證