Requirement & Release Management in context of IFDK reference product

Requirement & Release Management in context of IFDK reference product

About this course material- This material if for general training for

requirement and release management- Material is more supportive in class

room - Material will be updated during courses- FreeNest Portable Project Platform is

used to demonstrate things only in practice. This is not limiting usage for material for other training environments (I hope )

About material

Why Requirements?Why Requirements?

IDEALIDEAL

Requirements help

Design

Requirements help

Design

REQ-XREQ-X

REQ-YREQ-Y

REQ-ZREQ-Z

REQ-OREQ-O

REQ

REQ

Yläotsikko

Requirement ManagementRequirement Management

Between 40% and 60% of software failures and defects are the result of poor software management and requirements definition. In plain English, this means that about half of the problems encountered could have been avoided by making it clear, from the very beginning, what the customer expected from the respective project. This is to say that the programming was fine and the developers did their job well – only they did a different job from what they were supposed to.

http://www.softwareprojects.org

Read also http://www.projectshrink.com/why-requirements-change-270.html

Yläotsikko

http://www.softwareprojects.org/

Example of IFDK product concept

IFDK = Internal Flame Drum Kit

Requirement Management

Requirements and ImplementationRequirements and Implementation

Business and customer

Development Team

Business Requirements

Business Requirements

TechnicalRequirements

TechnicalRequirementsNEEDNEED SOLUTIONSOLUTION

Design & ArchitectureImplentation

Design & ArchitectureImplentation

AgreementAgreement

“One of sides should not dominate in design process”“One of sides should not dominate in design process”

Yläotsikko

ImplentationImplentation

Example of product architecture

Yläotsikko

Yläotsikko

Seller

Customer

Developer

Different aspects to product

Hacker

Nature of a good requirement?- One sentence which can be tested

- Sentence has to be understandable, measurable and testable

Yläotsikko

• All team board should have own color palette • Board can be "soft locked" for further changes by user• Board can be also seen as locked mode without edit• Selected board's can be set in full screen slide show

mode • User will be able to upload own board as a background• All backgrounds are available for all team members• There can be max 20 different backgrounds

• Tractor has four wheels• Tractor has one exhaust pipe• Engine of tractor is capable of use flexi fuel• The tractor has a hook for trailer• The tractor shall have a enhanced driving system

• Tractor has four wheels• Tractor has one exhaust pipe• Engine of tractor is capable of use flexi fuel• The tractor has a hook for trailer• The tractor shall have a enhanced driving system

• Simply ask questions!• You can gather requirements from several sources: Users,

Stakeholders, Business and Development team and many others• Requirements are trying to define nature of

feature/system/solution more specific than common written document does. This information is helping development team to design a solution for a need

• There is several common methods to define and gather requirements.

• Traditional Requirement modeling (http://en.wikipedia.org/wiki/Requirements_management)

• RUP/UML based Use Case modeling (http://en.wikipedia.org/wiki/IBM_Rational_Unified_Process)

• Agile XP oriented User Story’s (http://en.wikipedia.org/wiki/Agile_software_development)

• Your task is to figure out a small difference between them

Read more http://en.wikipedia.org/wiki/Requirements_management

How to find requirements?

FunctionalRequirements


Non-Functional

Requirements

Non-Functional

Requirements

FunctionalityFunctionality

StabilityStability

SecuritySecurity

PerformancePerformance

UsabilityUsability

User Interface Design?User Interface Design?

Feature XFeature X

Traditional Requirement Modelling and FeaturesTraditional Requirement Modelling and Features

..............

Yläotsikko

ScalabilityScalability

StabilityStability


SecuritySecurity


StressStress

UsabiltyUsabilty

Different points of view on systemRequirement Category‘s

Requirements from product development point of viewRequirements from product development point of view

„scalability“„scalability“

„Stability“

„Functionality“

Yläotsikko

• Different requirement levels

Customer/Business/Stake Holder Requirements


System RequirementsSystem Requirements

Design RequirementsDesign Requirements

Component RequirementsComponent Requirements

ImplementationImplementation

ScalabilityScalability

StabilityStability


SecuritySecurity


StressStress

UsabiltyUsabilty

System TestingSystem Testing

Integration TestingIntegration Testing

Unit TestingUnit Testing



System RequirementsSystem Requirements

Design RequirementsDesign Requirements

Component RequirementsComponent

Requirements

Acceptance TestingAcceptance Testing

Why we need requirements from testing point of view?Why we need requirements from testing point of view?


„Developer's Area“

„Test Engineers Area“

”Traditional Testing Levels”

Functional and non-functional requirementsFunctional and non-functional requirements



Non-FunctionalRequirements


Functional Requirement

"User can select application from ui

by using wheel button”

”Tractor can be driven both directions”

Functional Requirement


by using wheel button”

”Tractor can be driven both directions”

Non-Functional Requirement

"Performance Requirement"

”Tractor Startup should take minimum 10 seconds”

”Usability Requirement”

”User interface should be able to control using simple wheel quide”

”The hook can last max 20Kkg trailer load”



”Tractor Startup should take minimum 10 seconds”

”Usability Requirement”

”User interface should be able to control using simple wheel quide”

”The hook can last max 20Kkg trailer load”

How it works? How fast it is?How stable it is?

Do some googling!!

Create a wiki page!!

AboutUserStory

Do some googling!!


AboutUserStory

Yläotsikko

Practice: Define traditional requirements for you feature

Practice: Define traditional requirements for you feature

Functional RequirementsFunctional Requirements Non-Functional RequirementsNon-Functional Requirements

Yläotsikko

Use CaseUse Case

USE CASE

Written scenario for action. Also execeptions included

Use Case: Open IFDK Main Application

Actor: IFDK User

Step1: Gadget User touches home button

Step2: UI wakeup initiated (if standby)

Step3: Home screen is activated

Setp4: User browses applications specific icons using wheel button

Step5. Icons are moving on screen left and right

Step6: User selects application by pushing wheel button

Step7: Application starts up <4 seconds

Execptions:

1. If application cannot start there will be note on screen about problem

USE CASE


Use Case: Open IFDK Main Application

Actor: IFDK User








Execptions:

1. If application cannot start there will be note on screen about problem

Use Case

A use case in software engineering and systems engineering, is a description of steps or actions between a user (or "actor") and a software system which leads the user towards something useful.[1]

Wikipedia

Yläotsikko

http://en.wikipedia.org/wiki/Software_engineering

http://en.wikipedia.org/wiki/Systems_engineering

http://en.wikipedia.org/wiki/Actor_(UML)

http://en.wikipedia.org/wiki/Use_case

Why Use Cases are used?Why Use Cases are used?

Use cases are used for define functional requirements!

Yläotsikko

Brief History of Use CasesBrief History of Use Cases

• Defined by Ivar Jacobson http://en.wikipedia.org/wiki/Use_case

• Used with UML (Unified Modeling Language) http://en.wikipedia.org/wiki/Unified_Modeling_Language

• RUP (Rational Unified Process) • http://fi.wikipedia.org/wiki/RUP

Yläotsikko

http://en.wikipedia.org/wiki/Unified_Modeling_Language

User StoryUser Story

USER STORY Examples

Simple phrase describes a need. This can lead to several other storys!

"As user I would like to open application easily”

As a user I would like to select application from icon list on screen

As a user I would like to configure amount of application icons on screen

"As a user I would like to use wheel for speeding up selection process"

"As a user I would like to initate application fast enough"

"As a tractor driver I would like to have enhanced driving system”

USER STORY Examples


"As user I would like to open application easily”

As a user I would like to select application from icon list on screen

As a user I would like to configure amount of application icons on screen

"As a user I would like to use wheel for speeding up selection process"

"As a user I would like to initate application fast enough"

"As a tractor driver I would like to have enhanced driving system”


Yläotsikko

RequirementMeasurable

Testable

RequirementMeasurable

Testable

Use Case, User Story, Requirement

USE CASE


Use Case: Open Application

Actor: Gadget User








Execptions:1. If application cannot start there will be note on screen about problem

USE CASE


Use Case: Open Application

Actor: Gadget User








Execptions:1. If application cannot start there will be note on screen about problem

USER STORY


"As user I would like to open main menu using left

button"

"As a user I would like to use mouse wheel for

zoom"

"As a user I would like to initate application fast

enough"

USER STORY


"As user I would like to open main menu using left

button"

"As a user I would like to use mouse wheel for

zoom"

"As a user I would like to initate application fast

enough"

Non Functional Requirement


"Application Startup should take minimum 4 seconds"

Non Functional Requirement


"Application Startup should take minimum 4 seconds"

Functional Requirements


by using wheel button"

Functional Requirements


by using wheel button"

Yläotsikko

Practice 1: Drafting some requirements

Yläotsikko

Define 10 requirements in a team for a selected product

• Card Game

Functional Requirement Non-Functional

Requirement

LOAD SECURITY MAINTENANCE

IMPLEMENTATION




Non-Functional RequirementAgile

Epic / Theme

SCALING

FUNCTION

SAFETY


RECOVERY?

NON FUNCTIONAL

NON FUNCTIONAL

PERFORMANCE?

Main {….} ?

Stake Holders? Customer / User ? Business Request ?

Use Case NON FUNCTIONAL

PERFORMANCE?

SCALINGUSABILITY

UC

AcceptanceCriteria? RECOVERY?

NON FUNCTIONAL

User Story ?

User Story Addition +

SCENARIO ?

Traditional Feature

?

Yläotsikko

Who?When?How?Cost?

Who?When?How?Cost?

Product Idea! $$$

• Cat Safehouse with web cam support• Cat’s owner can “communicate” with cat using

“skype” alike technology.

CAT HOUSEService

30€/day Food, drinks and internet connection

includedLive Camfeed

Web portal login

BringCat to safe house

GetCat back home

Heading- Sed posuere interdum sem.- Quisque ligula eros ullamcorper quis,

lacinia quis facilisis sed sapien.- Mauris varius diam vitae arcu. Sed arcu

lectus auctor vitae, consectetuer et venenatis eget velit.

- Sed augue orci, lacinia eu tincidunt et eleifend nec lacus.

Yläotsikko

• Product Management

Idea#1Idea#1

Defining a Feature? Defining a Feature?

Customer

I would like to haveInternal Flame Drum Kit

Could you deliver it to us?

I would like to haveInternal Flame Drum Kit

Could you deliver it to us?

Actually We have severalWow features for it here

Actually We have severalWow features for it here

Ok!What's a plan

Ok!What's a plan

Nice looking feature propoals.We have to do some evaluationNice looking feature propoals.

We have to do some evaluation

Idea#2Idea#2

Idea#3Idea#3 Idea#4Idea#4

Idea#1Idea#1-Technology?-Knowledge-Resource-Solution?-Priority?

FeatureFeature

FeatureFeature

Feature is functionality of product/software which can be seen as one module of whole product

Internal Flame kit has WLAN support Internal Flame Kit has touch screen user interface

Feature is functionality of product/software which can be seen as one module of whole product

Internal Flame kit has WLAN support Internal Flame Kit has touch screen user interface

Do some googling!!


AboutUserStory

Do some googling!!


AboutUserStory

Yläotsikko

Is product a combination of features?Is product a combination of features?

Calory CounterCalory Counter

Drum MetronomeDrum Metronome

Table Drum ModeTable Drum Mode

Standby ModeStandby Mode

MIDI SupportMIDI Support

Touch Screen with single tapTouch Screen with single tap

Yläotsikko

Core Software/Platform

Core Software/Platform

Is product a combination of features?Is product a combination of features?


Drum MetronomeDrum Metronome

Table Drum ModeTable Drum Mode Standby ModeStandby Mode



Yläotsikko

Customer/Business Requirements?Customer/Business Requirements?

Calory CounterCalory CounterDrum MetronomeDrum Metronome


Simple Training ModeSimple Training Mode



Customer Type 1 Customer Type 2 Customer Type 3 Customer Type 4

Who are our target customers?

Customer StrategyCustomer Strategy

Drum MetronomeDrum Metronome Table Drum ModeTable Drum Mode

Simple Training ModeSimple Training Mode



Customer Type 1 Customer Type 2 Customer Type 3 Customer Type 4

What is our key customer?

Primary Target


Secondary Target

Requirement

Requirement

Requirement USE CASE #2

USE CASE #1

USE CASE #3

Requirement

Requirement

Requirement

USE CASE #1 User Story #1

User Story #2

User Story #3

RequirementRequirement

RequirementRequirement USE CASE #2USE CASE #2

USE CASE #1

USE CASE #3USE CASE #3

RequirementRequirement USE CASE #1USE CASE #1 User Story #1User Story #1

Requirement

Requirement


USE CASE #1

USE CASE #3

Requirement

Requirement

Requirement

USE CASE #1 User Story #1

User Story #2

User Story #3




USE CASE #1





USE CASE #1USE CASE #1 User Story #1User Story #1

User Story #2User Story #2

User Story #3User Story #3

Requirement

Requirement


USE CASE #1

USE CASE #3

Requirement

Requirement

Requirement

USE CASE #1RequirementRequirement


USE CASE #1RequirementRequirement



Features and release planningFeatures and release planning

Release 0.1

Release 1.1 Release 1.2

Feature: Simple Training Mode

Feature: Table Drum mode

Feature Touch Screen with single tap

Release 1.0

TIME TO MARKET!! For Target Group 3TIME TO MARKET!! For Target Group 3

CORE/Platform Software Development



Practice: Defining FeaturesPractice: Defining Features

Play problem domain card game with team to

search for features?

Yläotsikko

Yläotsikko

Feature X * nFeature X * n

Feature example 1 (Invented on course 2009-2010)Feature example 1 (Invented on course 2009-2010)

Calory Counter:

Player can measure calories during training session. This can be seen as exercise result in web service eg. Facebook application

Energy usage

Yläotsikko

Feature example 2 (Invented on course 2009-2010)Feature example 2 (Invented on course 2009-2010)

Table Drummer:

Player drums table board instead of drum can. IFDK kit is able to use DSP algorithm to detect correct drum sound from environment.

In training mode IFDK is trained to detect drum sounds for environment.


DSPAlgorithm

DSPAlgorithm

Yläotsikko

DefinesDefines






Use CasesUse Cases

Vision of product

Simple Requirement Management ProcessSimple Requirement Management Process

Problem Domain Solution Domain

Solution Proposal

Customer/Marketing/business

User StorysUser Storys

FEATUREVISION/NEED/

PROPOSAL

FEATUREVISION/NEED/

PROPOSAL

YOU!YOU!

Design documents & implementation

Test Case

Yläotsikko

Non-Functional Requirement category examplesNon-Functional Requirement category examples

Scalability

How our implementation is scaling in situation X?

Scalability

How our implementation is scaling in situation X?

Stability

Is our implementation stable on situation like

zzzZZZ?

Stability

Is our implementation stable on situation like

zzzZZZ?

Functionality

Implementation should work like this way

Functionality

Implementation should work like this way

Security

Is our implementation secure enough against

attack type xxx?

Security

Is our implementation secure enough against

attack type xxx?

PerformanceHow good performance our implementation provides against competitor?

PerformanceHow good performance our implementation provides against competitor?

StressHow much we can stress our implentation without

a problems?

StressHow much we can stress our implentation without

a problems?

UsabiltyIs implementation usable for target

customer?

UsabiltyIs implementation usable for target

customer?

MaintenanceIs implementation easy to maintain?

MaintenanceIs implementation easy to maintain?

Yläotsikko

Practice: Create Use CasesPractice: Create Use Cases

ACTOR

Use Case

SYSTEM Actor:

System: IFDK kit

Use Cacenario: Standby mode after boot

1. User turn’s on IFDK by pressing red button on front panel

2. Screen wil flash and show welcome text ”Hello my friend!”

3. User interface opens after seconds

4. Screen will show three selection buttons

5. After 30 seconds user inteface goes to standby mode

Exeption:

1. If user activates screenby tapping it standby counter will be reseted

Actor:

System: IFDK kit

Use Cacenario: Standby mode after boot

1. User turn’s on IFDK by pressing red button on front panel

2. Screen wil flash and show welcome text ”Hello my friend!”

3. User interface opens after seconds

4. Screen will show three selection buttons

5. After 30 seconds user inteface goes to standby mode

Exeption:

1. If user activates screenby tapping it standby counter will be reseted

Yläotsikko

USE CASE:_____________________________________________

Actor:__________________________________

Scenario:

Execeptions:

USE CASE:_____________________________________________

Actor:__________________________________

Scenario:

Execeptions:

Yläotsikko


• http://en.wikipedia.org/wiki/User_story

Yläotsikko

http://en.wikipedia.org/wiki/User_story

Brief History of User StorysBrief History of User Storys

• Used in agile development process for requirement definition and gathering

• Backlog Item• Simple way to describe needed functionality

Yläotsikko

DefinesDefines






Use CasesUse Cases

Vision of product

Simple Requirement Management ProcessSimple Requirement Management Process

Problem Domain Solution Domain

Solution Proposal

Customer User StorysUser Storys

FEATUREVISION/NEED/

PROPOSAL

FEATUREVISION/NEED/

PROPOSAL

YOU!YOU!

Design documents & implementation

Test Case

Yläotsikko

Practice: Create User StorysPractice: Create User Storys

USER STORY:

As a bad behavin person I cannot access IFDK using wlan without encryption

How to test?Acceptance Criteria?

USER STORY:

As a bad behavin person I cannot access IFDK using wlan without encryption


USER STORY:

As a member of audience I can hear effect sound that player is having electrical shocks


USER STORY:

As a member of audience I can hear effect sound that player is having electrical shocks


Yläotsikko

Agile Requirement Management Agile Requirement Management

Epic Story

User StoryUserStory0001

RequirementId0002

RequirementId0003

EpicStory0001

As a userI would like to use

product Which is fast to power

on

As a userI would like to use

product Which is fast to power

on

As a Customer I wouldlike to have topquality product

As a Customer I wouldlike to have topquality product

NOTE:Gadget should

have >30fpsUI performace

NOTE:Gadget should

have >30fpsUI performace

NOTE:Gadget should

Startup <5seconds

NOTE:Gadget should

Startup <5seconds

AcceptanceCriterias?

Yläotsikko

EPIC STORY: As a user I would like to use product which is fast to startup

Story Points:

Notes:

• Gadget should startup <5seconds

How to show it is tested? (eg. Acceptance Criteria)

Use timer to verify startup time. Measure time from power on to main screen activated.

• Time

EPIC STORY: As a user I would like to use product which is fast to startup

Story Points:

Notes:




• Time

USER STORY: As user I would like to see startup message on screen, so I could be sure a gadget is started up

Story Points:

Notes:




• Time

USER STORY: As user I would like to see startup message on screen, so I could be sure a gadget is started up

Story Points:

Notes:




• Time

Yläotsikko

USER STORY:

Story Points:

Notes:


USER STORY:

Story Points:

Notes:


USER STORY:

Story Points:

Notes:


USER STORY:

Story Points:

Notes:


Yläotsikko

http://www.sysml.org/http://www.sysml.org/

UML

OMT

OMT

System Engineering

Google: SysML, UML, Systems engineering, Software Design,

Code Code

Software Engineering - ArchitectureSysML

UML

OMT

.NET, JAVA, C++

Software Engineering - Design

Requirements

http://www.sysml.org/


http://www.opfro.org/

http://www.google.fi/url?sa=t&rct=j&q=requirement%20specification%20example&source=web&cd=1&ved=0CBgQFjAA&url=http%3A%2F%2Fwww.opfro.org%2FComponents%2FWorkProducts%2FRequirementsSet%2FSystemRequirementsSpecification%2FSystemRequirementsSpecificationExample.doc&ei=aiWETqjnG-P04QTMzNivDw&usg=AFQjCNFd4r5LLJgQj3foixq_jxZHjk78pQ&sig2=ILMCdBta4j8JVTCbQfgZgQ&cad=rja

Interesting Links ?Interesting Links ?

Yläotsikko


You have vision of productWhat means testing in brief?

You have vision of productWhat means testing in brief?

FeaturesFeatures Test CaseTest Case

Use CasesUse Cases


Product Design & Implementation

RequirementsRequirements

Test CaseTest Case

Test CaseTest Case

Test CaseTest Case

Ready to testReady to test

Testing & Quality Assurance

Can we deliverProduct

Ready to Deliver

Ready to Deliver

? CustomerNot ready

todeliver

Yläotsikko

SW Development Process (Waterfall)SW Development Process (Waterfall)

RequirementGathering/Evaluation

RequirementGathering/Evaluation

DesignDesign


Verification &Validation

Verification &Validation

MaintenanceMaintenanceError ManagmentProcessError ManagmentProcess

Task1Task1Task1Task1 Task1Task1 Task1Task1

Mile Stone 1Mile Stone 1 Mile Stone 2Mile Stone 2 Mile Stone 3Mile Stone 3

Task1Task1 Task1Task1

SW Development Process (Agile)SW Development Process (Agile)

User Story YUser Story Y

SprintSprint

Task1Task1

Task2Task2

Task3Task3

SprintSprint SprintSprint SprintSprint

Task4Task4

Task5Task5

Task6Task6

User Story XUser Story X

DesignImplementation

Verification


Verification

Product Backlog


Verification


Verification


Verification


Verification


Verification


Verification

User Story ZUser Story Z

Task7Task7

Task8Task8

Task9Task9

User Story ZUser Story Z

Yläotsikko

Customer/BusinessRequirements


Sub SystemRequirementsSub System

Requirements

ComponentRequirementsComponent

RequirementsComponent / Unit

TestingComponent / Unit

Testing

IntegrationTesting

IntegrationTesting

SystemTesting

SystemTesting

AcceptanceTesting

AcceptanceTesting

SystemRequirements

SystemRequirements

IFDK SystemVerification and Validation

IFDK SystemVerification and Validation

Use CasesUse Cases


FeaturesFeatures

RequirementsRequirements

Validation = Are we building the right product?Verification = Are we building the product right?Validation = Are we building the right product?

Verification = Are we building the product right?

Architecture&Design&

Implementation


Implementation

ProductProductVALIDATIONVALIDATION

VERIFICATIONVERIFICATION

IFDK Product IdeasIFDK Product Ideas

Yläotsikko

V-Model in testingV-Model in testing

Customer Requirements

Sub System Requirements

Component Requirements Component/Unit Testing

Integration Testing

System Testing

Acceptance Testing

System Requirements

Verification and Validation

Verification = Are we building the product right?Validation = Are we building the right product?

Yläotsikko

Component Design

ClassClass

AttributesAttributes

ClassClass

MethodsMethods

ClassClass


ClassClass

MethodsMethods

Yläotsikko

Testing according ISEB standard

Functional Testing

Testing based on an analysis of the specification of the functionality of a component or system. See also black box testing.)

Non-Functional Testing

Testing the attributes of a component or system that do not relate to functionality, e.g. scalabilty, stability, reliability, efficiency, usability, maintainability and portability.

http://www.bcs.org/upload/pdf/glossary-current.pdf

Yläotsikko

Black Box vs White Box Testing

Unit Testing is White Box testingUnit Testing is White Box testingSystem Testing is Black BoxTestingSystem Testing is Black BoxTesting

????

Yläotsikko

We need to capsule all projects as sub projects, which have more independence

• Project Teams has to have own test engineer –> Integration Test Engineer• this lead’s to better defect prevention• Integration Test Engineer is part of project team, part of team at start of project• Test Engineer provides valuable information for design and enables better testability for project product

Define new roles for validation engineers

• Check next page

Lower level for test automation

• Enables regression test (method to block new implementation defects)• Support for executing more complex test scenarios• Continuous test execution (stability, performance gain etc)

Notes

Yläotsikko

Test Level1 – Component/Unit testing

• Component/Class level unit testing (eg. xUnit framework)

Test Level2 - Integration Testing (Feature, Application)

• Integration test for feature/application with “stub” interface components• Functionality SW Testing• Non-Functional SW Testing according needs (eg. feature/application specific performance, stability)

Test Level3 – Regression Testing (Target HW+Android platform, tool, terminal etc.)

• Functional SW testing for all possible features• Non-Functional SW testing according needs• Non-Functional HW Testing according needs

Test Level4 – System Integration Testing (Target Platform, Tool, Terminal etc)

• Functional SW testing for all new features• Non-Functional SW Performance, Stability, Scalability

Test Level5 – Acceptance Testing

• Functional/Non-Functional test according customer requirements

HWT1 – Hardware Testing• HW Integration test with limited environment. • Conducted performance verified

Testing Level descriptions

Yläotsikko

Testing Levels (with TAF = test automation framework)IFDK Release v1.0

Feature Project 2

Regression Testing (ISEB)System Integration

TAF

Component/Unit Testing (ISEB)

IntegrationTesting (ISEB)

System Testing (ISEB)

AcceptanceTesting (ISEB)

Feature Project 1

TAF

TAF Project

TAF enables

TAF

Suppo

rts

Verification and Validation

Yläotsikko

Refrerences & LinksRefrerences & Links

http://www.rbcs-us.com/images/documents/The-ISTQB-Advanced-Syllabus.pdfhttp://www.rbcs-us.com/images/documents/The-ISTQB-Advanced-Syllabus.pdf

Yläotsikko

VERIFICATION & VALIDATIONVERIFICATION & VALIDATION

Validation = Are we building the right product?

Verification = Are we building the product right?

Yläotsikko

Create a Test Case!

Functional?

Your SourcesFor Test Case

Functional Test Case:

●Verify functionality of XXXX

Non-Functional Test Cases

●Verify Stability of XXXX●Verify Performance of XXX●Verify Security of XXXX●Verify Usability of XXXX●Verify Scalability of XXXX●etc...

Non-Functional?

Which Type?

●Requirement●Use Case●Feature●User Story

●Customer's Idea●Brainstorm●Intitution●Exploratory

Check also.....

●Correct functionality path●Miss-usage of functionality●Boundary Check

Check also.....●Check Possiblity to automated testing?

How to create Test Case???

Regression Test Case??

Write a Case

Write a Case

Yläotsikko

Verify what?Using configuration?

With tools?

What is verdict?

●Verify drum track player pause mode functionality.●Do this with IFDK software release X and playing song ”Show must go on by Freddy Mercury”●Test should be done using android emulator environment and using your hands, ears and eyes”

Add Information about case

●Pre State:●Android emulator is running●Release X is installed on emulator

●Test Case Steps:●1. Open drum kit player application●2. Select song ”Show must go on”●3. Start to play●4. Press Pause and check song is paused●5. Check memory usage from system application●6. Press Play●7. jump to 4 several time (<10)●8. Listen song to the end●9. Exit player using ”exit button”

●End State:●IFDK Kit in main screen mode

●Test Case Id●Test Case owner/writer●Date●comments

●If Pause is working result is PASS. If Pause mode failed result is FAIL

Define pre-stateDefine Steps

Define end-state

What Information Test Case should contain?

Yläotsikko

Component /Unit TestingComponent /Unit Testing

ClassClass


ClassClass

MethodsMethods

ClassClass


ClassClass

MethodsMethods

ClassClass


ClassClass

MethodsMethods

ClassClass


TestClassTestClass

MethodsMethods

Unit Test Frame WorkUnit Test Frame WorkResultResult

Yläotsikko

Integration testing using emulatorIntegration testing using emulator

Yläotsikko

Yläotsikko

Test Case

SUT/DUTIFDK

What is generated as results from test case execution

LOG FILE

EVENTS

NOTIFICATIONS

SUT = System Under TestDUT = Device Under Test

ENVIRONMENTANDROID EMULATOR

TOOLSTOOLSTOOLSscripts/grep

TEST CASE

Yläotsikko

●TEST CASE ID XXXXX

●Step1●Step2●Step3.●Step4.

INCIDENT(Huomio)

Bug/Defect(Vika)

Defect Database

IFDKSystem

EXECUTE TEST !

Test Engineer

Notes

Reports

What means error reporting?

Yläotsikko

Component TestingComponent Testing



Sub SystemRequirementsSub System

Requirements

ComponentRequirementsComponent

RequirementsComponent / Unit

TestingComponent / Unit

Testing

IntegrationTestingIntegrationTesting

SystemTestingSystemTesting

AcceptanceTestingAcceptanceTesting

SystemRequirements

SystemRequirements


Implementation


Implementation

ProductProduct

VALIDATIONVALIDATION

VERIFICATIONVERIFICATION

Yläotsikko

How to verify component implementation

How to verify component implementation

-Unit Testing-Code Coverage-Branch Coverage-Complexity Analyse

-Unit Testing-Code Coverage-Branch Coverage-Complexity Analyse

Yläotsikko

Code ComplexityCode Complexity

Example tool CCCC

Yläotsikko

Branch coverageBranch coverage

The percentage of branches that have been

exercised by a test suite. 100% branch coverage

implies both 100% decision coverage and 100%

statement coverage.

Yläotsikko

Code CoverageCode Coverage

An analysis method that determines which parts of

the software have been executed (covered) by the

test suite and which parts have not been executed,

e.g. statement coverage, decision coverage or

condition coverage.

http://en.wikipedia.org/wiki/Code_coverage

http://www.atlassian.com/software/clover/

Yläotsikko

http://en.wikipedia.org/wiki/Code_coverage

Release/Configuration Management &Integration Testing

Day 6Day 6

Yläotsikko

Putting all tools together!Putting all tools together!

Continous Integration

http://hudson-ci.org/

Yläotsikko

Design verification -Unit Testing

Design verification -Unit Testing

Yläotsikko

Unit Test example Re-run with MeegoUnit Test example Re-run with Meego

Example of xUnit in QT environment

Yläotsikko

Integration TestingIntegration Testing


Sub SystemRequirements

ComponentRequirements

Component / UnitTesting

IntegrationTesting

SystemTesting

AcceptanceTesting

SystemRequirements


Implementation

Product

VALIDATION

VERIFICATION

Yläotsikko

Why Integrate first? Avoid Big Bang!Why Integrate first? Avoid Big Bang!

HW ComponentHW Component

Data BaseData BaseComponent/Application 10% testedComponent/Application 10% tested

Web ServiceWeb Service

Tested Component/ApplicationTested Component/Application

Yläotsikko

Integration Test with stubsIntegration Test with stubs

Tested Component/ApplicationTested Component/Application

LogLog

STUB/MOCK ComponentSTUB/MOCK Component

Scripted STUB InterfaceScripted STUB Interface

ControlConfigureControlConfigure

SimulatedInterfaceSimulatedInterface

Messages/EventsMessages/Events


Control InterfaceControl Interface

Yläotsikko

in practice #1 IFDK android setupin practice #1 IFDK android setup

Tested ComponentApplication


Activate/ControlActivate/Control



ControlConfigureControlConfigure



WEB SERVER simulatingFacebook interfaceWEB SERVER simulatingFacebook interface


Trace/LogTrace/Log

Yläotsikko

in practice #2 server component testingin practice #2 server component testing



Trace/LogTrace/Log

Activate/ControlActivate/Control

Mock Server/DaemonMock Server/Daemon


AutomatedTest Interface

AutomatedTest Interface



WEB SERVERWEB SERVER


Operating SystemOperating System

NeededFake Application

NeededFake Application

Junit Scripted InterfaceJunit Scripted Interface

Yläotsikko

Release Management and Integration Testing

Release Management and Integration Testing

IFDK ApplicationIFDK Application

Facebook Web ServiceFacebook Web Service

HW ComponentCalore Meter

Enabled Drum Stick

HW ComponentCalore Meter

Enabled Drum Stick

Calore Meter SW Component 10% testedCalore Meter SW Component 10% tested

Data Base Schema DesignData Base Schema Design

Stubsneeded

Stubsneeded

Stubsneeded

Stubsneeded

Stubsneeded

Stubsneeded

Stubsneeded

Stubsneeded

First System TestWith all componentsFirst System TestWith all components

Yläotsikko

Trace/Log as feedbackTrace/Log as feedback

Log contains important information. Log

Simple tool for log analysing in Linux ”grep” command ,

TAIL -F /var/log/messages | grep error

-Specific Inhouse log capturing and analyse tools

Yläotsikko

AgendaAgenda

Brief guidance for NEST Project Platform 1.3

Background story for reference project IFDK

Some Definitions

SW Development Process

Release + Configuration Management

Testing Levels and Error Management

Hands On: Test Link + Bugzilla

Error Reporting, Metrics and daily usage

Closed Software Project vs Open Source Project

DiscussionYläotsikko

Error/Bug/Defect ReportError/Bug/Defect Report

●Defect/Burg/Error ID●Reporter●Time●Founded where●Which way?●Test Case●Test Setup/Configuration●Describe scenario?●Attachements? Picture/Log/etc..

●Defect/Burg/Error ID●Reporter●Time●Founded where●Which way?●Test Case●Test Setup/Configuration●Describe scenario?●Attachements? Picture/Log/etc..

Yläotsikko

About Error ManagementAbout Error Management

RequirementManagementRequirementManagement

ImplementationProcess

ImplementationProcess

Verification&

Validation

Verification&

Validation

FailureReportFailureReport

Yläotsikko

DefinitionsDefinitions

Failure -

Fault, Defect, Bug -

Incident, Failure, Error

Example forum thread: http://www.allinterview.com/showanswers/36257.html

ISTQB syllabus

Yläotsikko

http://www.allinterview.com/showanswers/36257.html

Release & Configuration Management

Release & Configuration Management

Yläotsikko

Release ManagementRelease Management

Version 0.1

Version 0.1 Version 0.2Version 0.2 Version 0.3Version 0.3

Version 0.2.1Version 0.2.1

Version 0.2.2.1Version 0.2.2.1


Version 0.4Version 0.4TrunkTrunk

Customer 1Customer 1

Customer 1Customer 1 Version 0.2.3Version 0.2.3



Yläotsikko

Release & Configuration Managmement

Version 0.1

Version 0.1 Version 0.2Version 0.2 Version 0.3Version 0.3




Version 0.4Version 0.4TrunkTrunk






FeaturesFeatures

Release 1.0Release 1.0



FeaturesFeatures

FeaturesFeatures

Yläotsikko

Release PlanningRelease Planning

Yläotsikko

Test ManagementTest Management

Yläotsikko

Validaton& Verificaton (Testing) Management

Version 0.4Version 0.4


Version 0.2.2

Version 0.2.2

Test PlanTest CasesFor

Features

Test PlanTest CasesFor

Features

Tested Release/configurationTested Release/configuration

Error/DefectReport

Error/DefectReport

Error/DefectReport

Error/DefectReport

Error/DefectReport

Error/DefectReport

Yläotsikko

Feature PackFeature Pack

Project ManagerProject Manager

Designer/CoderDesigner/Coder

IntegrationTest EngineerIntegrationTest Engineer

Test ManagerTest Manager

System TestingSystem Testing

IFDK System TestingIFDK System Testing

Feature Unit/Integration TestingFeature Unit/Integration Testing

IFDK System Acceptance TestingIFDK System Acceptance Testing

SystemTest EngineerSystemTest Engineer

Test AutomationEngineerTest AutomationEngineer

AcceptanceTest EngineerAcceptanceTest Engineer

ValidationValidation

VerificationVerification

IFDK Verification/Validation (Testing Organization)IFDK Verification/Validation (Testing Organization)

Regression Testing

Regression Testing

Integration Testing

Integration Testing

System Testing

System Testing

Acceptance Testing

Acceptance Testing

Test AutomationTest Automation

Unit Testing

Unit Testing

Product ReleaseProduct Release

Yläotsikko

Error and change managementError and change management

Yläotsikko

Example Sources for error reportExample Sources for error report

CRM CRM

Field TestingField Testing

Testing ProcessTesting Process

CustomerFeedback /Customer

Feedback /

Error ReportError Report Change Request?Change

Request?

N x IncidentsN x Incidents

Yläotsikko

Change ManagementChange Management

Sometimes founded defect can lead to change

Bug?Bug?

Change Request?Change Request?Not ClearRequirementsNot Clear

Requirements

Feature ?Feature ?

Yläotsikko

Error Reporting and Process

Yläotsikko

Hands On: Bugzilla Error DatabaseHands On: Bugzilla Error Database

http://www.bugzilla.org/http://www.bugzilla.org/installation-list/

http://www.bugzilla.org/http://www.bugzilla.org/installation-list/

What is Bugzilla?

Bugzilla is a "Defect Tracking System" or "Bug-Tracking System". Defect Tracking Systems allow individual or groups of developers to keep track of outstanding bugs in their product effectively. Most commercial defect-tracking software vendors charge enormous licensing fees. Despite being "free", Bugzilla has many features its expensive counterparts lack. Consequently, Bugzilla has quickly become a favorite of thousands of organizations across the globe.

What is Bugzilla?

Bugzilla is a "Defect Tracking System" or "Bug-Tracking System". Defect Tracking Systems allow individual or groups of developers to keep track of outstanding bugs in their product effectively. Most commercial defect-tracking software vendors charge enormous licensing fees. Despite being "free", Bugzilla has many features its expensive counterparts lack. Consequently, Bugzilla has quickly become a favorite of thousands of organizations across the globe.

Yläotsikko

http://www.bugzilla.org/

http://www.bugzilla.org/

Other Defect Database Solutions

JIRA – Commercial

Requisite Pro – Commercial

Rational Synergy - Commercial

Mantis – Open Source

Yläotsikko

Yläotsikko

BugzillaBugzilla

Yläotsikko

Reporting, Metrics and daily usageReporting, Metrics and daily usage

Yläotsikko

CMMI Process framework

CMMI – covers ”error management” in several process areas

SP3.2 Analyze Verification results

Typical Work Products: Trouble reports

- Analyze the verificationd ata on defects

- Record all results of the analysis in a report

- Provide infromation on how defects can be resolved (including verfiocation methods, criteria, and verification environment) and initiate corrective action

Project Monitoring and Control

SG2 Manage Corrective Action to Closure

SP2.1 Analyze Issues

SP2.2 Take corrective Action

SP2.3 Manage corrective Action

SG2 Validate Product or Product Components

SP2.2 Analyze Validation Results. Change request management & Configuration Management process

SG2 Track and Control Changes

SP2.1 Track Change Requests

SP2.2 Control Configuration Items

Yläotsikko

Traditional SW Project vs Open Source ProjectTraditional SW Project vs Open Source Project

Open Source – Crowd Sourcing

SW Relase tested without coordination by group of volunteers

Release tested by customer

Field Testing

Test Group

Yläotsikko

Documents

Requirement & Release Management in context of IFDK reference product