“The Importance of Software Process Management …€œThe Importance of Software Process Management for Software Quality, Testing and Industry Development” Presented at the China’s

�The Importance of Software Process Management for Software Quality, Testing and

Industry Development�Presented at the

China�s Software Quality, Testing and Industry Development Strategy Seminar

China's Quality Seminar 1

Development Strategy SeminarBeijing October 23, 2004

Angel G. JordanUniversity Professor Emeritus

Provost EmeritusCarnegie Mellon University

Founder, Software Engineering Institute

Issues at the Seminar

nSoftware Quality

nScope of Testing


Scope of Testing

nRequirement Analysis

What I will cover in this presentation

n The Significance of Software Qualityn The Importance of Software Testingn Quality in Delivering Offshore IT Servicesn State of Practice Versus SEI’s Vision in Software Engineeringn Software Process Management


n Software Process Managementq CMMIq Capability Levelsq Maturity Levelsq How CMMI can help an organization

n Conclusions

Software Quality Assurance

n Defining Software Quality Assuranceq Monitoring the software development processq Complying with standards and accepted methods


q Complying with standards and accepted methodsq Anticipating errors --- prevention

Software Quality Assurance�

nComputer system failures are caused by software defects

nDefects caused not only by Programming errors, but byqNot clear application requirements or changing requirements

nSoftware complexity may also induce errors


nSoftware complexity may also induce errorsnOther causes of errors may be:

qTime pressure because of unrealistic schedulesqNot well documented codeqDefective and/or poorly documented development tools

Introducing Software QA processes in an organization

n In large organizations involving high risk projects formalized QA processes are mandatory.

n For lower risks, QA processes are to be balanced with productivity.

n For small groups or projects, ad-hoc processes may


n For small groups or projects, ad-hoc processes may be appropriate, depending on customers and projects.

n Dependence on:q Team leaders or managersq Feedback to developersq Communication between customers, managers, developers, and testers

Significant payoff from investment in QA comes from:

n Processes in management of requirements aiming atq Testable requirement specifications as part of requirements or design documentation


q Or in agile-type environments coordination with end users

q Inspections of the design and codeq Evaluation of past projects

Management can play a key role in:

n Recognizing:q The importance of Verification and Validationq The use of Walkthrough and Inspectionq Different testing methods to be considered


q Different testing methods to be consideredq An understanding of the typical problems arising in the software development process and n An appreciation of potential solutions

q An understanding of the true meaning of Software Quality and what it entails

Software Quality entails:n Recognition that Quality Software is:

q Reasonably bug-free, q delivered on time and within budget, q meets requirements and/or expectations, q and can be maintained

n Quality software incorporates


Quality software incorporatesq Good codeq Good design

n The vision of the SEI regarding Quality emphasizes:q Software Process Management, in order to: q Minimize Testing

n We will talk about this later

The Software Life Cyclen Starts when application is conceived and ends when it is no longer in use. It includes:q Initial conceptq Requirements analysisq Functional designq Internal designq Documentation planningq Test planning


q Test planningq Codingq Document preparationq Integrationq Testingq Maintenanceq Updatesq Retestingq Phase-out

Software Testing

n Defining Software Testingq Testing involves operation of a system or application under control conditions and evaluating the results


q Controlled conditions should be both normal and abnormal

q Testing should attempt to make things go wrongq The emphasis is in detection

Software Testing Engineeringn The role of the Software Test engineern The role of the Software QA engineern The role of the QA or Test managern The role of Documentation in QAn The importance of Requirementsn Developing and running software tests


n Developing and running software testsq The test planq The test case

n Actions needed after defects are foundn Configuration management

n Actions when software has many defects

Considerations in Testing

n The end of the testing processn Time allotted for comprehensive testingn The size of the project and extensive testingn Coping with changing requirements When the application has functionality that was


n When the application has functionality that was not in the requirements

n Software QA processes vis-a-vis productivityn Coping with fast growth and QA processes

Other considerations in Testing

n Client/Server Environments and Testing

n Testing World Wide Web sites

n Testing and object-oriented designs


n Testing and object-oriented designs

n Extreme Programming and Testing

Reference for QA and Testing: http://www.softwareqatest.com/

Software QA and Testing Resource Center

n http://www.softwareqatest.com/


n http://www.softwareqatest.com/

Software Quality Engineering

n http://www.sqe.com/index.aspn Collocated with STARWEST 2004.n Sponsors useful courses

Software Testing Foundations course


q Software Testing Foundations courseq Certification training courseq Writing Testable Requirementsq Essential Software Requirements

n Balancing Plan-Driven and Agile Approaches

Quality in Delivering Offshore IT Services

n The Delivery of Offshore Services n Quality Framework and its Business Benefits

q Organizational Effectivenessq Service delivery


q Service deliveryq Quality Certifications:

n CMM, CMMI, Six Sigma

n Reference: Stephanie M. Torto Ruthbea Yesner, ‘The Role and Benefits of Quality in Delivering Offsore IT Services’, December 2003, White Paper, Sponsored by Wipro Technologies.

Quality influences Business Results

n Organizational Effectivenessq Commitment to qualityq Highly skilled employeesq Responsiveness and flexibilityStrong work ethic


q Strong work ethicq Lower turnover of employeesq Robust project management processesq Strong communication flow between client and services firm

Outcomes-Benefits to the customer

n Lower maintenance costn Lower schedule overrun costn Lower development cost

n IDC Recommendations * q Separate the forest from the trees


q Separate the forest from the treesq Deepen understanding of organizational effectivenessq Quantify the benefits to the businessq Use a balance of onsite and offshore resources

* IDC is a leading provider of data driven research analysis http://www.idcindia.com/ http://www.idc.com.cn/ http://www.idc.com.cn/eindex.php

The Importance of Quality Certifications

n The Capability Maturity Model (CMM)n The People Capability Maturity Model (PCMM) n The Capability Maturity Model Integrated


(CMMI)q We will talk about these later

n Six Sigma ISO, IEEE, ANSI Standards

Definitions of the quality certifications mentioned in this document follow: ISO 9000 specifies requirements for a quality management system for any organization that needs to demonstrate its ability to consistently provide a product that meets customer and applicable regulatory requirements and aims to enhance customer satisfaction. The Capability Maturity Model (CMM) for software describes the principles and practices underlying software process maturity and is intended to help software organizations improve the maturity of their software processes in terms of an evolutionary path from ad hoc, chaotic processes to mature, disciplined software processes. The People Capability Maturity Model (PCMM) is a maturity framework that describes the key elements of managing and developing the workforce of an organization. It describes an evolutionary improvement path from an ad hoc approach to managing the workforce to a mature, disciplined development of the knowledge, skills, and motivation of the people that fuel enhanced business performance. The PCMM helps organizations to characterize the maturity of their human resource practices, set priorities for improving the competence of the workforce, integrate competence growth with process improvement, and establish a culture of workforce excellence. The Capability Maturity Model Integrated (CMMI) product suites process improvement models available for product and service development and maintenance and build on and extend the best practices of the CMM for Software (SW-CMM), the Systems Engineering Capability Model (SECM), and the Integrated Product Development Capability Maturity Model (IPD-CMM). CMMI best practices enable organizations to more explicitly link management and engineering activities to business objectives, expand the scope of and visibility into the product life cycle and engineering activities to ensure that the product or service meets customer expectations, incorporate lessons learned from additional areas of best practice (e.g., measurement, risk management, and supplier management), implement more robust high-maturity practices, address additional organizational functions critical to products and services, and comply more fully with relevant ISO standards. Six Sigma is a rigorous and disciplined methodology that uses data and statistical analysis to measure and improve a company's operational performance by identifying and eliminating "defects" in manufacturing and service-related processes.

SEI Strategic ThemesnPredictably better, faster, and cheaper by -

nMoving to the leftnEmbrace a systems engineering approach and make better decisions before coding to predictably improve quality, cost, and schedule.


nReusing everythingnReuse code, but also the architecture and knowledge from building similar systems.

nNever making the same mistake twicenLeverage lessons learned.

State of Practice Versus SEI’s Vision

Development Integration and System Test

Software state of practice (“test in” quality)

60 - 80 % of effort and cost


* move to the left !

* reuse everything

* never make the samemistake twice*Ref: Standish Group, www.standishgroup.com, 1999

Development Integration and System Test

World-class developers “design in” quality

SEI Technical Program

Capability MaturityModel

Integration

Integration Software Intensive Systems

Team Software Process

Survivable Systems

Product

The right software delivered defect free, on cost, on time, every time

High confidence, evolvable,product lines

with predictable and improvedcost, schedule, and quality


Management PracticeInitiatives

Integration

Acquisition Support Systems

Systems

Performance Critical Systems

Software Architecture Technology

Technical PracticeInitiatives

Software Engineering Measurement & Analysis

Product Line Practice

Predictable Assembly with

Certifiable Components

The program of work described in the FY01-02 Program Plan addresses development and transition activities focused on improving software technology in areas of critical importance to the DoD. In general, SEI activities are focused on leveraging best commercial practices for the benefit of DoD systems. The SEI’s program of work consists of nine initiatives grouped into two principal areas: software engineering technical practices (emphasis on technical aspects of engineering a software product) and software engineering management practices (emphasis on management practices contributing to improved product quality, timeliness, and reduced total operational cost). The latter includes transition-support practices (emphasis on methods for measuring and improving the widespread use of new technology). Collectively, this work is focused on advancing the state of practice of software engineering—transforming the practice so software-intensive systems are built and acquired with predictable and improved cost, schedule, and quality. A major emphasis is on techniques for assembling these systems from components and product lines to form systems that are evolvable over their life cycles.

General Definition of Process

nA process is a set of practices performed to achieve a given purpose; it may include tools, methods, materials, and/or people.


tools, methods, materials, and/or people.

Quality Leverage Points

nWhile process is often described as a leg of the process-people-technology triad, it may also be considered the “glue” that unifies the other aspects.

Everyone realizes the importance of having a motivated, quality work force but even our finest

PEOPLE


force but even our finest people can’t perform at their best when the process is not understood or operating “at its best.”

n Major determinants of product cost, schedule, and quality

PROCESSTECHNOLOGY

See notes on slide 5

Why Focus on Process?nProcess provides a constructive, high-leverage focus...

nas opposed to a focus on people n The experience and training of your work force is not always enough.


n Working harder is not the answer.n A well defined process can provide the means to work smarter.

n Shifts the “blame” for problems from people to the process

q as opposed to a focus on technologyn Technology, by itself, will most likely not be used effectively.

n Technology, in the context of an appropriate process roadmap, can provide the most benefit.

The Process Management Premise

nThe quality of a system is highly influenced by the quality of the process used to acquire, develop, and maintain it.

nThis premise implies a focus on processes as


nThis premise implies a focus on processes as well as on products.q This is a long-established premise in manufacturing.

q Belief in this premise is visible worldwide in quality movements in manufacturing and service industries (e.g., ISO standards).

Process Improvement Involves Dealing with Multiple Dimensions at One Time

OrganizationStructure

Technical

Strategic??


CMMI based improvement is Much More than simply askingpeople to use a different template.

Managerial Social/Cultural

Strategic

ISOISO

CMMICMMIQAQA

??

Simple Improvement Processes

nDetermine where you want to be.nMake a plan.nExecute the plan.nLearn lessons and do it again.


nAnother improvement process is Plan Do Check Act.

nIDEALSM

nDetermine where you are.

SetContext

BuildSponsorship

CharterInfrastructure

Pilot/Test

RefineSolution

ImplementSolution

AnalyzeandValidate

ProposeFutureActions

Stimulus for Change Acting

Learning

The IDEALSM Model


CharacterizeCurrent &Desired States

DevelopRecommendations

SetPriorities Develop

Approach

PlanActions

CreateSolution

Pilot/TestSolution

Initiating

Diagnosing

EstablishingSM IDEAL is a service mark of Carnegie Mellon University.

Documented Results of CMM-based PI

q Process improvement benefits fall into one or more of these general categories:n improved schedule and budget predictabilityn reduction of reworkn improved cycle timen increased productivity


n improved quality (as measured by defects)n increased customer satisfactionn improved employee moralen increased return on investmentn decreased cost of quality

nAll this is in addition to meeting your customer’s requirement....

See notes on page 11

CMMI Transition Status- 234 Introduction to CMMI instructors trained- 402 SCAMPI Lead Appraisers trained- 830 students trained in Intermediate Concepts of CMMI

- 179 Introduction to CMMI instructors authorized- 16,161 students trained in Introduction in CMMI


- 16,161 students trained in Introduction in CMMI- 293 SCAMPI Lead Appraisers authorized- 80 active transition partners that offer Introduction to CMMI training

- 147 active transition partners that offer SCAMPI appraisal services

Number of Appraisals Conducted by YearReported as of 2 March 2004

500

600

700

800


0

100

200

300

400

198719881989199019911992199319941995199619971998199920002001200220032004

SPA SEI CBA IPI SCAMPI vX

What is CMMI?

nCMMI q is not a process; does not tell you How to do your work

q is a model of best practices in systems and


q is a model of best practices in systems and software development and in product development

qdoes tell you What to doq is based on the process management premise

CMMI Model Representations

An organization may choose an approach to process improvement from either of the following:process area capability


qprocess area capabilityqorganizational maturity

CMMI models support each approach with a representation.qprocess area capability - continuous representationqorganizational maturity - staged representation

Continuous Representation

•Some benefits of choosing the continuous representation are

• It allows you to select the order of improvement that best meets your organization’s business objectives and mitigates your organization’s


objectives and mitigates your organization’s areas of risk.

• It enables comparisons across and among organizations on a process area by process area basis.

• It provides an easy migration from EIA/IS-731 to CMMI.

PA PA

Process Area Capability

PA

ProjectManagement

Process AreasCategory

Project PlanningProject Monitoring and ControlSupplier Agreement ManagementIntegrated Project Management for IPPDRisk ManagementIntegrated TeamingIntegrated Supplier Management

Organizational Process FocusOrganizational Process DefinitionOrganizational TrainingOrganizational Process PerformanceOrganizational Innovation and Deployment

ProcessManagement


Requirements ManagementRequirements DevelopmentTechnical SolutionProduct IntegrationVerificationValidation

Engineering

Configuration ManagementProcess and Product Quality AssuranceMeasurement and AnalysisDecision Analysis and Resolution Organizational Environment for Integration Causal Analysis and Resolution

Support

Integrated Supplier ManagementQuantitative Project Management

Capability Levels

nCapability levels, used in the continuous representation, describe a sequential order for approaching process improvement within each process area.

nCapability levels build on each other, providing a


nCapability levels build on each other, providing a recommended order for approaching process improvement within a selected process area.

nCapability levels enable you to track, evaluate, and demonstrate your organization’s progress as you improve processes associated with a process area.

Staged Representation

•Some benefits of choosing the staged representation are

• It provides a proven sequence of improvements, each serving as a foundation for the next.

• It permits comparisons across and among organizations by the use of maturity levels.


the use of maturity levels.

• It provides an easy migration from the SW-CMM to CMMI.

• It provides a single rating that summarizes appraisal results and allows comparisons among organizations.

Organizational Innovation and DeploymentCausal Analysis and Resolution5 Optimizing

4 Quantitatively Managed

3 Defined

ContinuousProcess Improvement

QuantitativeManagement

ProcessStandardization

Organizational Process PerformanceQuantitative Project Management

Requirements DevelopmentTechnical SolutionProduct IntegrationVerificationValidationOrganizational Process FocusOrganizational Process Definition

Higher Quality ProductivityLower Risk

Process Areas for SE/SW/IPPD/SSLevel Focus


3 Defined

2 Managed

Standardization

BasicProjectManagement

Organizational Process DefinitionOrganizational Training Integrated Project Management for IPPDRisk ManagementIntegrated TeamingIntegrated Supplier ManagementDecision Analysis and ResolutionOrganizational Environment for Integration

Requirements Management Project PlanningProject Monitoring and ControlSupplier Agreement ManagementMeasurement and AnalysisProcess and Product Quality AssuranceConfiguration Management

Higher RiskRework1 Initial

Maturity Levels-1

nMaturity levels, used in the staged representation,organize selected process areas into five evolutionary plateaus to support and guide process improvement across the organization.


nMaturity levels are defined evolutionary plateaus of process improvement, in which each level is supported by the characteristics of the process areas implemented within the lower levels.

nMaturity levels represent a process improvement evolution for the entire organization.

Maturity Levels-2

Process measuredand statistically controlled

Focus on continuousimprovement

Optimizing

QuantitativelyManaged

Optimizing

4

5


Process unpredictable, poorly controlled and reactive

Process characterized for projects and is often reactive

Process characterized for the organization and is proactive

Defined

Initial

Managed

Defined3

2

1

Staged

ML3

ML4

ML5

Continuous. . .for a single process areaor a set of process areas



ML 1

ML2

ML3

. . .for a specified set ofprocess areas across anorganization

PA PA


PA

Comparing the Different Representations

n Both representations provide ways of implementing process improvement to achieve business goals.


n Both representations provide essentially the same content but are organized in different ways.

n There is nothing that requires that you use one or the other representation – you can use them bothconcurrently if that suits your business needs.

Representation Synergyn The continuous representation provides additional de factogranularity in support of a staged-focus initiative:• as a guide for detailed tactical planning• to demonstrate intermediate progress short of process area or maturity level

• to allay concerns and build support among stakeholders


n The staged representation can provide structure for a continuous-focus initiative:• as a guide for big picture “strategic” planning• to “chunk” higher-granularity activities for senior management

• as a means for representing high-level success in “industry standard” terms key stakeholders will more readily understand

CMMI Can Benefit Youn CMMI provides

q Efficient, effective assessment and improvement across multiple process disciplines in an organization

q Improvements to best practices incorporated from


q Improvements to best practices incorporated from the Software CMM

q A common, integrated vision of improvement for all elements of an organization

q A means of representing new discipline-specific information in a standard, proven process-improvement context

Conclusions

n Developing Quality Software is essential in any software enterprise

n Software Testing is a close companion of Software Quality

n Quality is essential in Delivering Offshore IT Servicesn Quality Impacts Business Results


n Quality Impacts Business Resultsn Quality Certifications are highly recommendedn The SEI vision is to develop software reducing the testing effort

n The importance of “Process” in Software Developmentn CMMI can benefit any Software enterprise

Questions?Comments?


Documents

“The Importance of Software Process Management …€œThe Importance of Software Process Management for Software Quality, Testing and Industry Development” Presented at the China’s