SIMULTANEOUS USEOF PROCESS-ORIENTED SOFTWARE QUALITY APPROACHESZádor Dániel Kelemen, Budapest University of Technology and Economics

Supervisors:Katalin Balla, Budapest University of Technology and

Economics

Jos Trienekens, Rob Kusters, Eindhoven University of Technology

Contents Terminology Multi-model process improvement Background (available solutions) Research focus, questions and objectives Structure of process-oriented quality approaches Mapping quality approach elements Example: tailoring a process from multiple process- oriented quality approaches

2

“If you wish to converse with me, define your

terms.” Voltaire

Terminology

Terms used Approach

a standard, model or method is an approach (gives a view of a solution) to one or more problem

Quality Approach an approach connected to quality

Software Quality Approach a quality approach used in software industry

4

Terms used Process-Oriented (Software) Quality

Approach a (software) quality approach which focuses on

processes (POSQA)

Simultaneous Use of Process-Oriented Software Quality Approaches ~ subset of multi-model process improvement process improvement based on multiple

software quality approaches

5

Situation, benefits, problems

Multi-model process improvement

Multi-model environment Several SPI

methods, standards and models are available

Companies use these approaches simultaneously

Some examples:CMMI, ISO/IEC/IEEE

12207:2008, ISO/IEC 15939:2002, ISO 9001:2008, ISO 9004:2000,ISO

90003:2000, IEEE 1028:2008, Agile

methods, ITIL, SPICE

7

Focusing on the same process from different perspectives

8

CMMI

ISO 12207

IEEE 1028

Peer Review

Peer Revie

w

Peer Review

Organizational Peer Review

Process

Organizational peer review needs

Benefits of Using Multiple Approaches

Quality approaches are often focusing on the same problems from different perspectives

A multi-model solution therefore coulde be more complete uses the best practices supports combined appraisals has higher ROI …

9

Problems of Using Multiple Approaches

Approaches have different structure terminology level of detail point of view on

quality

Difficult to tailor organizational processes from multiple quality

approaches

10

mappings, standardization, combined models, formal methods, further solutions

Backgorund,

Background (available solutions)

Mappings of Quality Approaches Wide literature on comparing CMMI to other

approaches ISO 9001 (Mutafelija) TSP SPICE (T. Route et al.) Agile methods SixSigma PMBOK etc…

12

Combined models Available combined models:

CITIL Developed by Wibas CMMI-ITIL

Appraisal Assistant Developed at Griffith University CMMI – SPICE Combined appraisal of CMMI and SPICE

13

Standardization International Organization for Standardization

(ISO) harmonizes its most widely used standards Eg. ISO 9001, 90003 and 14001 has been developed

with the same structure and terminology IEEE 1028:2008 and IEEE 12207:2008 are also

harmonized performing 1028 satisfies requirements of 12207

14

Formal methods Other researchers create more formal

solutions using natural language processing know-

how for processing standards and models or building ontologies.

Eg. Malzahn briefly presents an ontology based solution for multi-model process improvement

15

Further multi-model solutions QMIM (Quality through

Managed Improvement and Measurement) shows how the quality

standards and models are connected to quality objects

PrIME (Process Improvement in Multimodel Environments) A SEI project for maximizing

the ROI

Metric

Quality attribute

Definition

Quality specification dimension:

Objects dimension:Project

ManagementTechnical Process

Product

GQM ISO 9126(Boehm, McCall) ...

ISO 9001:2000CMM

SPICECMMI

ISO 15504TSP, PSP

PM methodologiesPeople CMM,

Weinberg’s theory ...

16

Characteristics of available solutions 1Solutio

n type

Solution

exampleMain strengths Main weaknesses

Standa

rdizati

on

ISO 9001 –

90003 – 14401

Common terminology,

structure,

Provides guidance

Must follow the structure

and terminology of iso

9001

IEEE 1028 –

12207Common terminology

1028 can be used only to

satisfy 2 requirements of

12207

Mappin

gISO9001-CMMI

Concrete (maps each

requirement, terminology,

gives guideline)

Different terminology,

structure, content

17

Characteristics of available solutions 2

Solutio

n type

Solution

ExampleMain Strengths Main Weaknesses

Com-

bined

model

CITIL

Combined appraisal,

terminology, structure,

content

Applicable only for two

approaches (CMMI and ITIL)

SPICE-CMMI,

Appraisal

Assistant

combined appraisal,

terminology,

Structure, content

Could be applicable only for

CMMI and SPICE structured

approaches

Formal

metho

d

Natural language

processing,precise, formal,

easy to process the model

with computers

still not available for wider

use, Often needs special

mathematical knowledgeontologies

fuzzy logic

18

Characteristics of available solutions 3

Solutio

n type

Solution

ExampleMain Strengths Main Weaknesses

Multi-

model

QMIMgives a starting point for multi-

model-oriented companiesTBD

PrIME increase ROI TBD

process-based

approach (with

possibility to

represent it

graphically)

A continuation of QMIM,

identifies main elements and

level of QAS

TBD

19

Summary of available solutions Significant effort has been spent on multi-model problem

Mappings are useful in using two different models or standards Combined appraisal models reduce the cost of the appraisals Formal methods could serve good solutions in the future Process modelling facilitate the better understanding and

maintenance of processes QMIM framework helps companies to start their process improvement

Multi-model process improvement is still an issue when companies want to use more than two or still not mapped approaches

20

Research focus, questions and objectives

Our solution

Research focus Our goal: to create a process based approach for supporting the use of multiple process-oriented (software) quality approaches

Excluding approaches in which the main focus is not on the process but on other quality objects, such as the product or resource. Using this approach, software companies can tailor their own quality system from different process-oriented software quality approaches

22

Research Questions Is a process model based approach (PBA) usable to support the

use of multiple process-oriented software quality approaches?A. Does the outcome of the PBA provide sufficient information to

unify selected parts of different process-oriented software quality approaches?

Objective: To determine the completeness of the material developed using the process based approach.

B. Is the outcome of PBA traceable? To identify the main element types to be extracted from quality approaches To determine the level of traceability of derived elements back to the selected

quality approaches

23

Research QuestionsC. Is the outcome of PBA adaptable and expandable?

To identify possible reactions to the changes in and number of quality approaches.

D.  Is the outcome of PBA usable in practice? To use the process based approach and its outcomes in practice.

E. Does the outcome of the PBA will use a harmonized terminology and content?

To provide a unified terminology and content for the selected parts of the used process oriented quality approaches.

F. Does the outcome of the PBA will support combined appraisals?

24

Searching for convertible quality approach elements…

The structure of quality approaches

The structure of quality approaches

In order to quickly create process models, would be useful to know, what kind of elements could be converted directly from textual information to graphical representation.

Aspects considered important: to analyse the structure of widespread approaches, to describe the structure of approaches analysed in a

very simple and understandable format. -> UML class seems suitable for describing structures

26

Selected approaches/based on surveys made at Hungarian companies/

The most used and “mandatory” quality approach: ISO 9001:2008 Quality management systems – requirements

Besides ISO 9001:2000, most used approaches are: CMMI & SPICE

Further, well known approaches at Hungarian companies: ISO 9004:2000 Quality management systems – Guidelines for performance

improvements ISO/IEC 90003:2000 Software Engineering – Guidelines for the application of

ISO9001:2000 to computer software ISO/IEC 15939-2002 – “Information technology - Software measurement process” IEEE/ISO/IEC 12207-2008 – “Information technology - Software life cycle process”

A peer review specific approach: IEEE 1028:2008

27

The structure of ISO 9001:2008

Elements:• Chapter• Subchapter• Sub-subchapter• Requirement

in (simple) sentences

ISO 9004 and 90003 have similar structure

28

The structure of CMMI-DEV v1.2

Elements: Process area, Specific Goal, Specific Practice, Typical Work Product, Subpractice, Generic Goal, Generic Practice, Generic Practice Elaboration

29

The structure of ISO-IEC 12207-2008Information Technology – Software Lifecycle Processes

Elements: Chapter, Process category, Process Description, Activity, Task, Purpose, Outcome, …

30

The structure of ISO-IEC 15939-2002Software engineering. Software measurement process

Elements:• Chapter / process

activity• Subchapter /sub-activity• Sub-sub-activity• Activity description

31

The structure of IEEE 1028:2008

32

mapping to process elements and process model elements

Mapping quality approach elements

Some elements ofprocess-oriented quality approaches

Chapter Subchapter Sub-subchapter Requirement (in /simple/ sentences) Guideline Process Process Description Activity Task Option Condition Documentation requirement Responsibility Chapter / process activity Subchapter /sub-activity Sub-sub-activity Activity description

• Process area• Specific Goal• Specific Practice• Typical Work Product• Sub Practice• Generic Goal• Generic Practice• Generic Practice Elaboration• Process• Mission Statement• Goal• Activity• Benefit• Critical Success Factor• Key Performance Indicator• Further Specific Informative Components• Method• Practice• Value• Feature• Process

34

Similar elements are marked with same color

Process vs. Quality Approach Elements

Process Elements:• Purpose• Inputs• Entry criteria• Activities• Roles• Measures• Verification steps• Outputs• Exit criteria

QAP Elements:

ChapterSubchapterSub-subchapterRequirement (in /simple/ sentences)GuidelineProcessProcess DescriptionActivityTaskOptionConditionDocumentation requirementResponsibilityChapter / process activitySubchapter /sub-activitySub-sub-activityActivity descriptionProcess areaSpecific GoalSpecific PracticeTypical Work ProductSub PracticeGeneric GoalGeneric PracticeGeneric Practice ElaborationProcessMission StatementGoalActivityBenefitCritical Success FactorKey Performance IndicatorFurther Specific Informative ComponentsMethodPracticeValueFeatureProcess

35

Mapping quality approaches to processes

Knowing the structure of quality approaches, their elements can be mapped to

Process elements Process modeling language elements

BPMN elements Elements of UML Activity diagrams EPC elements … other process modeling language elements

36

Eg: mapping CMMI elements 1

CMMI elements REQ. level Process elements

BPMN elements

Process area Calculated Process Process

Specific Goal REQUIRED (Calculated)

- -

Specific Practice

EXPECTED (Sub)process (Sub)process

Typical Work Product

INFORMATIVE Input & Output Artefact/Data Object

37

Eg: mapping CMMI elements 2

CMMI elements REQ. level Process elements

BPMN elements

Sub Practice INFORMATIVE Task Task

Generic Goal REQUIRED (Calculated)

- -

Generic Practice

EXPECTED (Sub)process (Sub)process

Generic Practice Elaboration

INFORMATIVE Task Task

38

Observations 7 quality approaches (QAP) were analysed Similar elements are present in the QAPs and processes

• Process, process description, activity, process activity, activity description and task are proportional to the activity element of processes.

• Documentation requirements and typical work products are proportional are similar to inputs and outputs.

• ... Element which have no similarities to process elements are also

present.• Such elements are eg. benefits, critical success factors, features or

key performance indicators in ITIL. • Difficult to convert ISO chapters, sentences...

39

An example of tailoring a process from

multiple process-oriented quality approaches

The peer review process

Quality approaches used CMMI, SPICE IEEE 1028:2008 ISO 12207 Fagan’s Inspections Volere Requirements Review Checklist Templates of Karl E. Wiegers Templates of the team of R. Kusters Personal Software Process template for code review Articles connected to reviews/inspections(mainly from Wiegers) Software Testing Books

41

General stepsof building the graphical notation

1st case: 2nd case:

Textual POQAs

Textual General Process

Textual POQAs

Graphical General Process

Tailored Organizational Textual and Graphical Process

TextualGraphic

al

Tailored Organizational Textual and Graphical Process

Textual POQAs

Textual General Process

Graphical General Process

Tailored Organizational Textual and Graphical Process

1b case:

Developed and practically performed

process

42

Peer Review specific steps of building the graphical notation (1st case)

Organizational level description (CMMI, SPICE, ISO12207)

Process, procedure level description(IEEE 1028, …)

Practical level description

(Temlates of Karl E. Wiegers, Kusters, …)

Textual description

Graphical description

Tailored Textual description for Org.

needs

Tailored Graphical description for Org.

needs

43

Quality elements connected topeer reviews on a high process level

One instance of a peer review (session) CMMI GGs and GPs are also included, represented Complete CMMI and/or ISO/IEC/IEEE 12207 compatibility can be achieved!

44

Mapped elementson a lower process level…

45

Summary Situation of multi-model process improvement A review of available solutions Research Questions Structure of process-oriented quality approaches

emphasis on convertible elements An example of a tailored process from multiple process-oriented quality approaches

the peer review process

46

Acknowledgement This project has been supported by

SQI Hungarian Software Quality Consulting Institute Ltd. BME (IT)2 (BME Innovation and Knowledge Centre of

Information Technology) within the frame of Peter Pázmány Programme, National Office for Research and Technology, Hungary.

BUTE project “Szoftverminőség-biztosítási szolgáltatás-csomag kidolgozása a nyílt dokumentumformátum alkalmazásaira”

47

Thanks for your attention! We are pleased to collaborate with you

from the industry to carry out a case study on using multiple quality approaches

from the academic field to cooperate with similar research projects

Send mail to: Zádor Dániel Kelemen

kelemend@iit.bme.huProject webpage:

http://sqi.hu/tools/multi-model-spi/

48