A Study of Process Improvement Best Practices

Aedah Abd Rahman

Software Engineering Section, MIIT Universiti Kuala Lumpur Kuala Lumpur, Malaysia

Shamsul Sahibuddin and Suhaimi Ibrahim

Faculty of Computer Science & Information Systems Universiti Teknologi Malaysia

Johor, Malaysia

Abstract—Software project success depends on various reasons including project control, software standards and procedures. Software development organizations realize the importance of using best practices to improve software development practices. An increasing number of literature have described about process improvement best practices and standards. Formal process improvement frameworks have emerged widely to promote the use of systematic processes for software engineering. These approaches identify best practices for managing software engineering quality. They provide methods for assessing an organization’s process maturity level and capability. In this article, recent process improvement best practices and standards are presented. Its objective is to analyze the existing approaches towards software process improvement initiatives. Another objective is to determine the issues related to adoption of process improvement and standards. The research outcome is to obtain the significant process improvement issues and formulate a classification of generic steps for process improvements.

Keywords-process improvement best practices; process improvement; process improvement framework; software engineering; standards

I. INTRODUCTION

Software project success depends on various reasons including project control, software standards and procedures. Due to this, software development organizations adopt best practices to improve their software development practices. This practice have been reported and cited mostly in literatures based on industrial adoption of process improvement best practices and standards. The software productivity and quality improvement requires high commitment from the senior management of the organization. Formal process improvement frameworks have emerged widely to promote the use of systematic processes for software engineering. These approaches identify best practices for managing software engineering quality for the products and services. Organization’s process maturity level and capability can be assessed by using these best practices. This article reviews recent process improvement best practices and standards. Moore (1999) mentioned that there are varieties of quality initiatives and frameworks emerged [13]. Best practices improve the software development processes [2-5,8,15,16] and services [6,7].

This paper focus on analyzing the existing approaches towards process improvement best practices in Section II. Section III discusses about the research method. Section IV describes the research results. Finally, we present our

conclusions including further research recommendation. The paper concluded with identification of issues in process improvements best practices and classification of generic steps for process improvements. The issues related to adoption of process improvement and standards will be presented.

II. RELATED RESEARCH

Recent research efforts about software quality and improvement demonstrated a concentrated effort to improve software processes in software development organizations. There are arising needs to solve the software quality problems and the complexity to manage the software development process. In order to solve these issues most software development organizations adopt process improvement best practices to ensure effective management of their processes. Each best practice have their own focus and direction towards quality. A selection of literature is widely available that reflects the existence of these process improvement best practices.

A. The Concept of Process Improvement Best Practices

Process improvement is not specific to software activities. The cyclic approach can be applied to any discipline. The cyclic approach introduced by Deming (1950) [24] in Plan-Do-Check-Act (PDCA) is the evidence of a successful process improvement approach. Despite the actual objective for PDCA is the application towards the manufacturing industries, the approach have been redefined by Basili and Rombach (1988) for the software engineering domain. The process improvement best practices are frameworks that focus on processimprovement aspect. The focus for each framework may differ. It may be focusing towards improving the software products, services, projects or organization. In general various best practices emerge today to assist organizations to evaluate and improve their software development processes and capabilities. The effort on process improvement was introduced byHumphrey (1989). He defines the initial step towards improving the capabilities of a software organization is to understand the current status of software development practices [20]. Research on process improvement related to software engineering has been conducted by Paulk, Weber, Curtis and Chrissis (1993) through the CMM model [16]. Glass (1998) in his work also determined that the area of software quality and its improvement has been rapidly vital since 1990s.

B. The Process Improvement Best Practices and Frameworks

The best practices presented in this section comprises of PDCA, Ishikawa, QIP, IDEAL model, SW-CMM, CMMI,

Proceedings of the 5th International Conference on IT & Multimedia at UNITEN (ICIMU 2011) Malaysia

14 – 16 November 2011

978-1-4577-0989-0/11/$26.00 ©2011 IEEE

BOOTSTRAP, ISO/IEC 15504 (formerly known as SPICE),ISO 9000 series, ISO 9126-1 and ISO/IEC 20000. The models to represent an approach for managing improvement have been initiated by several quality experts. (Shewart, 1939) was the first model that emphasize on continuity aspect of improvement actions [23]. Shewart started his work as early as 1931. This model later evolves to become Deming or PDCA cycle (Deming, 1986) in his model Plan-Do-Check-Act(PDCA) [24,25]. Ishikawa (1985) improvise Deming’s cycle name it Control Circle in Total Quality Control (TQC) [12]. Deming’s cycle and TQC which have been originally developed based on manufacturing industry has been redefined by Basili and Rombach (1994) in his work Quality Improvement Paradigm (QIP). He applied the concepts to software engineering and improvement field. QIP is based on the principles that software discipline is evolutionary and experimental [26]. QIP model include the QIP cycle which is broken into two iterative cycles which are the organizational (larger) cycle and the project (smaller) cycle. QIP classified improvement activities into project and organizational levels.

After the use of SW-CMM, there was no guidance on continuing the quality effort. IDEAL model was developed to overcome the weakness of The Software Capability Maturity Model (SW-CMM). There are two dimensions of IDEAL model [2]: the strategic level and tactical level. The strategic level comprises of Initiating, Leveraging and Managing SPI program phases. The tactical level consists of Diagnosing, Establishing and Acting phases.

ISO 15504 Part 7: “Guide for use in process improvement” has been developed and added to existing process reference model (IS0 15504-2 1998) and assessment model (ISO 15504-5 1998). ISO 15504 Part 7 is complementing the IDEAL improvement model and SW-CMM.

Software Capability Maturity Model (SW-CMM) describes the practices for software process maturity and assessment. It was developed for software organizations to improve the maturity of their software processes. SW-CMM provides evolutionary path from adhoc, chaotic processes to a more mature, disciplined software processes. SW-CMMstructure consists of five categories: maturity level (ML), key process area (KPA), goals, common features, key practices. The KPA are organised into five maturity levels: initial, repeatable, defined, managed and optimizing. These maturity levels are the foundation for continuous process improvement. Each KPA consists of a group of related activities. These activities are performed together to achieve a set of goals for enhancing process capability. In order to achieve these goals SW-CMM requires common features [15] which are defined as commitment to perform, ability to perform, activities performed, measurement and analysis; and verifying implementation. SW-CMM were sunset and efforts have been directed towards the establishment of CMMI. The CMMI project was developed to address the need of the integration of software engineering, system engineering and other models such as IPD-CMM. The models combined were: SoftwareCapability Maturity Model (SW-CMM), the Systems Engineering Capability Model (SECM) and the Integrated Product Development Capability Maturity Model (IPD-CMM)

CMMI framework describes discrete levels of processimprovement [3-5]. CMMI is a capability map which gives description of specific goals and practices that should be attained by an organisation in order to achieve a level of capability and maturity. CMMI has progresses from CMMI v1.1 until CMMI v1.3. CMMI v1.1 has already sunset in August 2007. CMMI v1.2 consists of three constellations of CMMI v1.2 which are development (CMMI-DEV v1.2), acquisition (CMMI-ACQ v1.2) and services (CMMI-SVC)[5]. This CMMI v1.2 combines the CMMI various variants into one common framework that covers CMM v2, EIA731 Systems and CMM (IPPD). CMMI related works also have been investigated by Kasse (2004), Kulpa and Johnson (2003), Jones and Soule (2002) [21,17,14]. CMMI has made some modification to the assessment approach based on BOOTSTRAP (Kuvaja, Simila, Krzanik, Saukkonen and Kochi (1994) and ISO 15504 ((ISO 15504-5 1998) [18,11].

Staged representation applies sets of predefined process areas to define improvement path [3]. It consists of five maturity levels: initial, managed, defined, quantitatively managed and optimizing. The evolution levels are described as in Table 1.

TABLE I. CMMI – STAGED APPROACH

MATURITY LEVELS DESCRIPTION OF LEVELS

Level 1 Initial Process is normally unstable and changeable Informal process control

Level 2 Managed Process starts with basic project management

Level 3 Defined Organizations already have their own set of standard processes Process standardisation throughout the projects

Level 4 Quantitatively Managed

Measurement and control mechanism

Level 1 Optimizing Continuous process improvement

The continuous representation allows an organization to perform improvement work for a selected process [4]. There are six levels of process capability known as incomplete, performed, managed, defined, quantitatively and optimizing. The continuous representation is grouped based on four categories: Process Management (five process areas), Project Management (six process areas), Engineering (six process areas) and Support (five process areas).

BOOTSTRAP is a complement to SW-CMM. BOOTSTRAP widened the assessment activities scope to the software development organizations and their processes, unlike BOOTSTRAP addresses the processes and practices forevaluating and improving the quality of the organization, project and software development processes. The process areas are organization, methodology and technology [18].

ISO/IEC 15504 is a process improvement and assessment standard originated from a special project called SPICE (Software Process Improvement and Capability dEtermination) [11]. ISO/IEC 15504 consists of fiveprocesses: organizational process (ORG), management

process (MAN), customer-supplier process (CUS), engineering process (ENG) and support process (SUP) [11]. Software process standards consist of a standard definition of software development practices documentation. International standards are used widely as reference process models particularly for improving their processes. ISO 9001 (2000) provides a model for quality assurance in design, development, production, installing and servicing [8]. ISO 9000-3 (1997) provides guidelines for the application of ISO 9001 in software development [10]. These models are not too complex and practical to be used in defining software development practices. However, the drawback is the lacking of guidance on improvement actions. ISO 9126-1 (2001) is a standard related to software product quality consisting of six characteristics such as functionality, reliability, usability efficiency, maintainability and portability [9].

The improvement method for software services is becoming increasingly vital apart from the improvement methods for software product and processes. The issues on services management particularly on the benefit, cost and success factors of service oriented IT management has been highlighted by Hochstein, Tamm and Brenner (2005) [1]. They presented that there are potential aspects for improvement in service oriented IT management context. ISO/IEC 20000 is the first international standard for IT Service Management and it contains two parts: ISO/IEC 20000-1 (International Standard Organisation, 2005a) and ISO/IEC 2000-2 (International Standard Organisation, 2005b) [6,7]. ISO/IEC 20000 framework can be divided into several processes as described in Table II.

TABLE II. ISO/IEC 20000

Processes Sub-Processes

Service Delivery Processes

Capacity Management, Availability and Continuity Management, Service Level Management, Service Reporting, Information Security Management, Budgeting and Accounting for IT Services

Resolution Processes Incident Management, Problem Management

Release Processes Release Management

Control Processes Configuration Management, Change Management

Relationship Processes Business Relationship Management, Supplier Management

III. RESEARCH METHOD

The research method focuses on the theoretical aspect and comparison approach. The research is divided into two research questions, the leading and decomposition research questions. The leading research question is “What are the recent best practices available that focus on process improvement. Decomposing this question we investigate the following problem: a) What are the issues arise in the adoption of process improvement best practices? and b) What are the suggested solutions for the issues? The first step of the research method is using the theoretical literature on a critical review based on the available literatures which has been discussed in previous Section II. We focus the literature on report types and

standards that present scientific researches, for example in conference, journal articles and standards. The second step focuses on identifying two categories of best practices which are assessment and non-assessment based. Upon decomposing the best practices their category, the comparison is performed among the best practices by investigating the levels, steps, activities or description. The result of comparison will be used as input foundation for finding the issues in process improvement best practices and generating the generic steps for process improvement (PI).

The tables III and IV below present the comparison among variety of process improvement best practices for assessment based and non-assessment based process improvement. We present the two tables III and IV to show the description between the two categories. In Table III, assessment based approach can be determined as best practices that focus on software process assessment. It emphasizes on the importance of software practices to be organised according to the reference process model. This approach assesses existing software development processes and compares them with the reference process models and their assessment methods. The result of assessment gives the indication whether current processes satisfy the methods requirement.

TABLE III. COMPARISON OF PROCESS IMPROVEMENT BEST

PRACTICES – ASSESSMENT BASED APPROACH

PI BEST PRACTICE LEVELS/STEPS/ACTIVITIES/ DESCRIPTION

SW-CMM

Maturity levels: 1.Initial 2.Repeatable 3.Defined 4.Managed 5.Optimizing

CMMI

Staged approach maturity levels: 1.Initial 2.Repeatable 3.Defined 4.Managed 5.Optimizing Continuous approach: 1. Process 2. Project 3. Engineering 4. Support

BOOTSTRAP

Maturity levels: 1.Initial 2.Repeatable 3.Defined 4.Managed 5.Optimizing

ISO/IEC 15504

Process Groups: Organizational process Management process Customer-Supplier process Engineering process Support process

Table IV indicates the non-assessment based approach which focuses on quality standards for software or services (first five rows) and managing the software processimprovement (last four rows).

TABLE IV. COMPARISON OF PROCESS IMPROVEMENT (PI) BEST

PRACTICES – NON-ASSESSMENT BASED APPROACH

PI BEST PRACTICE LEVELS/STEPS/ACTIVITIES/ DESCRIPTION

PDCA

1. Plan 2. Do 3. Check 4. Act

Ishikawa

1. Determine Goals and Targets 2. Determine the Methods of Reaching

Goals 3. Engage in Education and Training 4. Implement Work 5. Check the Effects of Implementation 6. Take Appropriate Action

QIP

1. Characterize and Understand 2. Set Goals 3. Choose Processes, Methods, Techniques and Tools 4. Execute the Processes (run the project

cycle) 5. Analyze Results 6. Package and Store Experience

IDEAL Model

1.Initiating 2.Diagnosing 3.Establishing 4.Acting 5.Leveraging

ISO 15504 Part 7

1. Examine Organizations Needs 2. Initiate Process Improvement 3. Prepare and Conduct Process

Assessment 4. Analyze Results and Derive Action Plan 5. Implement Improvements 6. Confirm Improvements 7. Sustain Improvement Gains 8. Monitor Performance

ISO 9000-3 Provides guidelines for the application of ISO 9001 in software development

ISO 9001 Provides a model for quality assurance in design, development, production, installation and servicing.

ISO/IEC 20000

An IT service management best practices which covers the following processes. 1.Service Delivery Processes 2.Resolution Processes 3.Release Processes 4.Control Processes 5.Relationship Processes

ISO 9126-1

It is a standard related to software product quality. It consists of six characteristics such as functionality, reliability, usability efficiency, maintainability and portability. It promotes a three-stage evaluation process for software product quality assessment. It does not provide method for measurement, rating or assessment.

Hence, we summarize that non-assessment based approach (PDCA, Ishikawa, QIP, IDEAL Model and ISO 15504 Part 7) focuses on SPI management. Another category non-assessment based approach covers ISO 9000-3, ISO 9001, ISO/IEC 20000 and ISO 9126 focuses on standard. There is a need to determine a solution to describe the continuous improvement of a particular process. Thus, we found that the management of software process improvement is not covered by the best practices in table III and covered partially in table IV. Hall, Rainer and Baddoo [19] described the importance of SPI

implementation by focusing and understanding the critical success factors. The success of SPI depends on the SPI management, improvement action and critical success factors.

IV. RESULTS AND DISCUSSION

This paper reviewed recent studies in process improvement best practices. The results from the comparison made are used to determine the issues in process improvements adoption and producing the generic steps for process improvement.

A. Issues in Process Improvement Best Practices

The comparison between assessment based and non-assessment based approaches indicates the lacking mechanism of software process improvement management. This is an important issue which arises when software development organizations adopt the best practices. The first issue that needs to be considered is the type of approach for the process improvement best practices. An organization that decides to pursue on best practice initiatives need to determine whether to use assessment based process approach or non-assessment based. There are two types of approached determined from comparison performed on the twelve best practices. The first four best practices SW-CMM, CMMI, BOOTSTRAP and ISO/IEC 15504 from table III are assessment based approach. In this approach, existing software development processes are selected and compared to specific reference process model assessment method. The assessment results are indication of how good the existing processes satisfy the methodsrequirements and criteria. The other nine best practices: PDCA, Ishikawa, QIP, IDEAL model and ISO 15504 Part 7, ISO 9000-3, ISO 9001, ISO/IEC 20000 and ISO 9126 are classified as non-assessment based approach. The second issue is determining the focus and direction of the best practices. Decision whether to improve product, project, organization or services affects the selection of best practice. Examples of best practices and their direction are given. SW-CMM and CMMI improve on software development practices, ISO/IEC 20000 improves services and BOOTSTRAP improves on all four aspects.

B. Generic Steps for Process Improvement Best Practices

Referring to the comparison made on the process improvement based practices. We based our findings based on the non-assessment based approach for process improvement. These best practices are chosen due to the life cycle components and improvement cycle that exist. They emphasize on management of SPI. We have looked from the view of best practices fulfilling the focus as standards and process assessment. We need to define a mechanism for guiding the improvement action continuously. Analyzing closer into the non-assessment based approach, we group some of the best practices as providing the platform for improvement action and cycle. The best practices that are extracted for comparison include PDCA, Ishikawa, QIP, IDEAL model and ISO 15504 Part 7. We derived the three generic steps for an improvement process as: Plan, Improve and Control.

The first step (plan) includes the process of examining the SPI requirements and defining the process improvement goals.

The second step (improve) includes determining the measures for establishing the baseline performance. The improvement opportunity or problem is quantified at this stage. Then, the process of executing the best suggested improvement solution is performed. The final step (control), include confirmation or checking that the problem and the root causes have been reduced or eliminated.

V. CONCLUSION AND RECOMMENDATION

This paper has reviewed recent studies in process improvement best practices. We have conducted comparison among twelve best practices and derived several issues related to process improvement. Two types of approaches have been identified which are the assessment based approach and non-assessment based approaches. Generic steps for process improvement best practices are derived consisting of plan, improve and control.

We recommend that the three phases to be used in our future works in improving processes. We expect to extend our work in working with software development organizations or software project in applying the framework to processes for further evaluation and improvement.

ACKNOWLEDGMENT

Authors wish to thank Universiti Kuala Lumpur and Universiti Teknologi Malaysia for the research support.

REFERENCES

[1] A. Hochstein, G. Tamm, and W. Brenner, “Service oriented IT management: Benefit, cost and success factors”, European Conference on Information Systems (ECIS 2005), Proceedings 13, Regensburg, Germany, 2005. [2] B. McFeeley, “IDEALSM: A user’s guide for software process improvement”, CMU/SEI-96-HB-001, Pittsburg, Pennsylvania [3] CMMI Product Team, “Capability Maturity Model Integration, Version

1.1, CMMI-SW/SE/IPPD/SS, Staged Representation”, Technical Note, CMU/SEI-2002-TR-011, 2002a.

[4] CMMI Product Team, “Capability Maturity Model Integration, Version 1.1, CMMI-SW/SE/IPPD/SS, Continuous Representation”, Technical Note, CMU/SEI-2002-TR-012, 2002b.

[5] CMMI Product Team, “CMMI for Development (CMMI-DEV 1.2)”, CMU/SEI-2006 TR-008, Software Engineering Institute, Carnegie Mellon University, 2006. [6] International Standard Organisation, “ISO/IEC 20000-1. Information

Technology – Service Management – Part 1: Specification”, International Standard Organisation, Geneva, Switzerland, 2005a.

[7] International Standard Organisation, “ISO/IEC 20000-2. Information Technology – Service Management – Part 2: Code of Practice”, International Standard Organisation, Geneva, Switzerland, 2005b.

[8] ISO 9000, “Quality management system – Fundamentals and vocabulary”, 2000. [9] ISO/IEC 9126-1, “Software Engineering – Product Quality – Part 1 : Quality Model”, 2001. [10] ISO 9000-3, “Guidelines for the application of ISO 9001:1994 to the development, supply, installation and maintenance of computer software”, 1997. [11] ISO/IEC 15504-5, “Information Technology – Software Process Assessment – Part 5, An assessment model and indicator guidance”, Technical Report Type 2, 1998. [12] K. Ishikawa, What is Total Quality Control? The Japanese Way.

Prentice-Hall, Eaglewood Cliffs, N.J. [13] J. Moore, “An integrated collection of software engineering standards”, IEEE Software, 16(6):51-57, 1999. [14] L.G. Jones and A.L. Soule, “Software process improvement and product line practice: CMMI and the framework for software product line practice”, Technical Note, CMU/SEI-2002-TN-012, 2002. [15] M.C. Paulk, C. Weber, B. Curtis, and M.B. Chrissis, “Capability Maturity Model for Software, Version 1.1”, Technical Report, Software Engineering Institute, Carnegie Mellon University, CMU/SEI-93-TR-024 ESC-TR-93-177, 1993. [16] M.C. Paulk, C. Weber, B. Curtis, and M.B. Chrissis, “The Capability Maturity Model: Guidelines for improving the software process, Version 1.1”, Boston: Addison Wesley, 1995. [17] M.K. Kulpa, and K.A. Johnson, Interpreting the CMMI: A Process Improvement Approach, Auerbach Publications, 2003.[18] P. Kuvaja, J. Simila, A. Krzanik, S. Saukkonen, and G. Kochi, Software

Process Assessment and Improvement – The BOOTSTRAP Approach, Oxford, UK: Blackwell Publisher, 1994.

[19] T. Hall, A. Rainer, and N. Baddoo, “Implementing software procecess improvement: An empirical study”, Software Process: Improvement and Practice, 7(1), pp. 3-15, 2002.

[20] W.S. Humphrey, Managing the Software Process. Reading, Massachusetts: Addison-Wesley, 1989.

[21] T. Kasse, Practical Insight into CMMI, Norwood, MA: Artech House, 2004. [22] T.P. Rout and A. Tuffley, “A harmonizing ISO/IEC 15504 and CMMI”, Software Process Improvement and Practice, 12(4): 361-371, 2007. [23] W.A, Shewart, “Statistical method from the viewpoint of quality control”, The Graduate School, George Washington University, Washington D.C., 1939. [24] W.E. Deming, Elementary principles of the statistical control of quality.

JUSE, 1950.

[25] W.E. Deming, “Out of crisis: Quality, productivity and competitive position”, MIT Center for Advanced Engineering Study, Cambridge, MA, 1986.

[26] V.R. Basili and H.D Rombach, “The TAME project: Towards improvement oriented software environments”, IEEE Transactions on Software Engineering. SE-14(6):758773, 1988.