Debunking 3 Common CMMI Peer Review Myths

8/6/2019 Debunking 3 Common CMMI Peer Review Myths

1/12

A SmartBear White Paper

This white paper provides straight talk and advice on what the CMMI really requires with respect to peer

reviews, how to get the most value rom your peer reviews, and what to look or in peer review sotware to

improve the perormance o your organization while achieving CMMI compliance. You will learn techniques to

eliminate wasted eort while tuning your teams perormance or optimal competitive advantage.

Debunking 3 Common CMMI Peer

Review Myths

Contents

Executive Summary ......................................................................1CMMI & Peer Review Myths ......................................................2

Myth 1................................................................................................2

Case Study 1....................................................................................4

Myth 2................................................................................................5

Case Study 2....................................................................................6

Myth 3................................................................................................7

Case Study 3....................................................................................8

Summary...........................................................................................9

CMMI Peer Review Sotware and Best Practice

Checklist ......................................................................................... 10

Author Bio...................................................................................... 11

About SmartBear........................................................................ 12

Ensuring Sotware SuccessSM

www.smartbear.com/codecollaborator

Executive Summary

Today there are a lot o misunderstandings when it comesto the Capability Maturity Model Integration

(CMMI)[1] and peer reviews. But why should you care?

First, peer reviews are an expected practice and required

to achieve CMMI compliance. Second, while peer reviews

are commonly held today, the way these reviews are oten

conducted is costing many organizations wasted time and

eort, and ailing to provide the promised higher peror-

mance payback on their investments. In some cases poor

peer review practices may result in delaying, or not achiev-

ing, CMMI compliance. Recent case study data [2] reveals a

major cause o these ailures can be traced to the act that

many organizations dont realize the options and exibility

they have to use peer reviews to gain higher organizational

perormance.

By Paul E. McMahon
http://smartbear.com/products/software-development/code-reviewhttp://smartbear.com/products/software-development/code-review


2/12

More specifcally you will learn:

Three commonly held myths associated with the CMMI and peer reviews

CMMI key acts related to peer review requirements

Common peer review best practices used by CMMI mature organizations

Three case studies explaining:

What many mistakenly believe the CMMI requires with respect to the way peer reviews must be conducted

How many organizations over-interpret the CMMI requirements with respect to data collection and product

reviews leading to wasted eort

How the right sotware peer review tool set up appropriately can result in higher quality products and more

satisfed customers

Key checklist items that can help you implement a peer review process optimized to meet the needs o yourorganization and customers while maintaining CMMI compliance

CMMI & Peer Review Myths

Historically there has been a tendency or people to read things into the CMMI model that arent really there

creating unnecessary non-value-added work. Understanding three commonly held myths related to the CMMI

and peer reviews along with the real acts o what the CMMI requires can help an organization improve its per-

ormance and achieve CMMI compliance aster and more eectively.

Myth 1: The CMMI specifes how peer reviews must be perormed including who must participate in reviews and

when peer reviews must take place

There is a common myth that peer reviews must be run in a ormal way, at a specifed time, and participants

must be sotware practitioner peers inside your own organization. Organizations that succumb to this myth

expend unnecessary eort, and oten ail to gain the greatest potential benefts rom their peer reviews early

critical deect removal and products validated early by satisfed customers.

2

CMMI Stages and associated Peer Review Activities
http://www.smartbear.com/


3/12

What the CMMI really requires

One o the goals o the CMMI Verifcation Process Area is to perorm peer reviews. However, the CMMI doesnt

dictate how peer reviews must be conducted, nor does it dictate the degree o ormality. There is also noth-

ing in the CMMI that specifes who the reviewers must be, or when the reviews must take place. A common

best practice in many CMMI mature organizations is to provide guidelines on how-to options to conduct peer

reviews, including recommended stakeholders to involve in the dierent types o peer reviews that make sense

or your organization.

Summary CMMI Benefts [3]

33% decrease in the average cost to fx a deect -Boeing, Australia

30% increase in sotware productivity - Lockheed Martin M&DS

15% improvement in internal on-time delivery - Bosch Gasoline Systems

20 +/- 5 deects per KLOC - Northrop Grumman

$2 million in savings rom early detection and removal o deects - Sanchez Computer Associates, Inc.

Increased ocus on quality by developers - Northrop Grumman

Peer Review Best Practices o Successul CMMI Companies

Customize their peer review process based on company and customer needs

Careully document alternative review processes & options

Provide guidelines on stakeholders to involve, including outsiders such as customers

Detail the types o peer reviews specifc to each business

3


4/124

Case Study 1 - I thought the CMMI required that...?

When your goal is to achieve CMMI compliance, start by talking to the practitioners in your organi-

zation. As they tell you how they do their job eventually they will get to the products they produce.

Then ask: Does anyone else look at these products? Anticipate a common response heard today in

many organizations just starting a new CMMI eort: We dont do ormal peer reviews.

A CMMI lead appraiser who I worked with many years ago taught me to--in these cases-- always ask

the intent question. From a tip in the CMMI guidelines we learn the intent o peer reviews is to iden-

tiy deects or removal and to recommend changes that are needed. This leads to a new question

that should be asked: How do you identiy deects or removal?

When I asked this question at BOND, one o my client organizations seeking to become CMMI Level

3, I heard: We demonstrate our products to the customer early and oten, and they give us eedback.

And We meet daily with our team and discuss our products, and We check our code into a library

daily and encourage our teammates to review and give us eedbackand they do.

What we realized was when those practitioners at BOND said, We dont do ormal peer reviews,they meant there wasnt a set time everyone went into a room to provide a list o deects ound in

their work products. But we also realized they were achieving the intent o peer reviews through a

dierent how-to option.

In todays collaborative environments customers oten operate as teammates providing eedback

early through demonstrations as we saw at BOND. Customer eedback can be a great way to fnd

deects early and help validate that your products will meet your customers needs. BOND had a his-

tory o producing high quality low deect products. When BOND explained this to their CMMI LeadAppraiser she had no difculty accepting this alternative peer review process that was achieving

the goal-- or intent-- o the CMMI peer review expected practice. We documented the process and

trained it just like BOND was carrying it out, and it helped the BOND organization achieve their CMMI

Level 3 goal.


5/125

In the next myth and case study we examine more closely how-to options, and give the reader some guidance

in deciding which ones are right or their organization.

Myth 2: The CMMI specifes what data must be collected in peer reviews and what products must be

peer reviewed

There is a common myth that every deect reported in peer reviews must include inormation related to thetype and cause o the deect. There is also a common myth that certain types o products must always be peer

reviewed on every project. Organizations that succumb to this myth oten fnd themselves expending signifcant

cost and eort without the accompanying perormance payback.


The CMMI expects you to analyze data rom your peer reviews, but

it doesnt tell you what data you need to collect and analyze, nor

does it tell you what products you need to review.

So how does an organization determine what data should beanalyzed, and what products should be reviewed? To answer these

questions, frst ask yoursel: What are our measurement objec-

tives? Establishing your measurement objectives is an expected practice in the CMMI model. Then ask yoursel:

What data should we collect rom our peer reviews that supports our objectives?

With respect to which products to peer review, ask yoursel, What products does our customer require us to

produce? Then ask, What products do we know we need to produce to ensure we have satisfed customers?

The right data to be analyzed rom peer reviews, and the right products to be peer reviewed or an organization

depends on that organizations specifc business domain, their specifc measurement objectives, and their cus-

tomers needs. A best practice in many CMMI mature organizations is to provide documented criteria that helpproject owners make these decisions given their specifc project situation.


Establish organizational measurement objectives based on your business needs

Collect and analyze data based on your companys established objectives-- not the CMMI

Provide criteria based on your company and customer needs to help your people make better decisions on

what products to peer review

Simpliy the deect reporting process as much as possible to maximize participation

The CMMI leaves the decision up to

each organization to decide what work

products to peer review... create criteria to

help a project leader make more dynamic

real time decisions with respect to peer

reviews [4]


6/126

Case Study 2 - Dont we need that data for the CMMI appraisal?

LACM is a CMMI level 3 organization that undertook a process improvement eort to streamline its

processes. The peer review process at LACM required a great deal o data to be collected about each

deect identifed and it required certain documents to be produced and reviewed on every project.The process also required periodic analysis o the collected data. When we looked close we ound that

practitioners were entering all the data about each deect because the company peer review sotware

tool required it, but we also ound no one was going back and analyzing this data.

We also discovered that some o the documents that were being produced and peer reviewed on

every project the customer didnt require and no one in the LACM organization was using them. On

urther investigation we ound the reason the data and those documents were required was because

someone inside LACM wrongly thought the CMMI required them. Practitioners also told us that

because the peer review sotware tool was so difcult to use it actually had discouraged practitioners

rom entering real deects ound.

Ater streamlining the process, peer review sotware, and related training the company ound the

CMMI implementation was strengthened because the people were now ollowing the peer review

process with greater discipline. This improvement in process adoption occurred because the peer

review sotware and process were easier to use and the practitioners in the organization

could now see the beneft o their eort.[5]


7/127

In the next section we examine how sotware tools can simpliy peer reviews and help with CMMI appraisals.

Myth 3: The CMMI doesnt speciy that sotware should be used or peer reviews, so a sotware tool

cant be helpul

It is true that the CMMI doesnt speciy that sotware should be used or peer reviews. But organizations that

succumb to the myth that peer review sotware cant be helpul oten end up making the eort o producing aquality product more difcult or their practitioners and fnd themselves trailing the competition.


While the CMMI doesnt speciy anything about tools, the model does expect each organization to analyze

data rom peer reviews. A common best practice ound in many CMMI mature organizations is to establish a

standard approach to the data that is collected and analyzed. This can help project practitioners in carrying out

their responsibilities and help organizations determine where they need improvement to stay ahead o their

competition.


Create a standard approach to data collection and analysis using peer review sotware

Customize peer review sotware to the process not the other way around

Increase adoption by using sotware that makes the actual peer review easy and unobtrusive

Assign and train personnel responsible or data collection and analysis

Centralize reporting across projects to identiy continuous improvement opportunities and simpliy

CMMI appraisals

Peer review software being used for review of software code.


8/128

Case Study 3 - How can peer review software help?

When we talked to sotware practitioners at one o our clients about peer reviews we heard:

Sure, we do peer reviews. But we werent getting a warm-eeling this was true because we

could fnd no objective evidence to back it up and their customers had been complaining aboutlatent deects. At the time they used no peer review sotware, nor required any related peer

review documentation.

We put a simple peer review tool in place requiring minimal data collection, and periodic

analysis. The tool helped us maintain records that were utilized as evidence during the CMMI

ormal appraisal. During the CMMI appraisal our interviews which included multiple sotware

practitioners revealed that beore the tool was installed and used the practitioners oten

cut corners and skipped peer reviews when pressed or time. They told us this was easy or

them to do since no records were kept.

They also told us when the peer review sotware started to be required they knew there would

be a record, and it helped discipline them to ollow the process even when they were pressed

or time. Once people started using the tool regularly, they realized that their products had

ewer latent deects, and customer satisaction increased noticeably. Using peer review sot-

ware helped this company more rapidly prepare or the CMMI appraisal process and ultimately

achieve Level 3 compliance.


9/129

Summary

The CMMI doesnt dictate how-to conduct peer reviews, nor what products you need to peer review, nor what

data you need to collect, or who needs to participate in these reviews. However, it does expect each organization

to make all o these decisions based on its own business needs, document its decisions, and train its people in

their established peer review process.

The CMMI doesnt dictate the use o peer review sotware, but the right peer review sotware set up with the

right data specifc to business needs has proven to help many organizations achieve their CMMI goals o im-

proving the quality o their products and code, increasing their sotware productivity, and saving money by fnd-

ing and fxing deects earlier in the process. This allows CMMI compliant organizations to continuously optimize

their processes, be high perorming and ultimately keeping them ahead o the competition.

Author Bio

Paul E. McMahon, Principal, PEM Systems (www.pemsystems.com), helps large and small organizations as they

improve their technical and management processes and move toward increased agility and process maturity.

He has taught Sotware Engineering at Binghamton University, State University o New York; conducted work-

shops on Engineering Process and Management, and published more than 35 articles on sotware and systems

development and management. His experience, insights, and recommendations he shares with his clients reect

24 years o engineering and management experience working or such companies as Link Simulation and Lock-

heed Martin, and 14 years o independent consulting. Paul is the author o two books, Integrating CMMI and

Agile Development: Case Studies and Proven Techniques or Faster Perormance Improvement (August, 2010)

and a book on collaborative development, Virtual Project Management: Sotware Solutions or Today and the

Future (CRC Press, 2000).

Endnotes[1] CMMI stands or Capability Maturity Model Integration CMMI is a process improvement reerence model

developed by the Sotware Engineering Institute (SEI) and is required or contractors to bid on many U.S.

Department o Deense contracts.

[2] McMahon, Paul E, Integrating CMMI and Agile Development: Case Studies and Proven Techniques or Faster

Perormance Improvement, Addison-Wesley, August 2010

[3] Demonstrating the Impact and Benefts o CMMI: An Update and Preliminary Results

(http://www.sei.cmu.edu/reports/03sr009.pd)

[4] McMahon, Paul E, Integrating CMMI and Agile Development: Case Studies and Proven Techniques or Faster

Perormance Improvement, Addison-Wesley, August 2010

[5] For more detailed inormation on the BOND and LACM Case Studies and Peer Reviews reer to [2].
http://www.pemsystems.com/http://www.sei.cmu.edu/reports/03sr009.pdfhttp://www.sei.cmu.edu/reports/03sr009.pdfhttp://www.pemsystems.com/http://www.smartbear.com/


10/12

CMMI Peer Review Sotware & Best Practices Checklist

Use the ollowing checklist to help determine the right peer review process, data, and sotware or your

organization.

10

Checklist Item Dont forget...!Yes No ?

People and Processes

Have your practitioners been trained

in your peer review process and sotware?The CMMI does expect you to train your

personnel.

Does your peer review process identiy

criteria or how your projects and person-nel decide what work products must

undergo a peer review?

The CMMI expects you to peer review work

products, but it does not dictate what theproducts need to be. You decide.

Does your peer review process provide

guidance as how to select thestakeholders or a peer review?

The CMMI expects you to involve stakehold-

ers in peer reviews.

Does your process identiy the minimum

data items that must be entered related

to a peer review?

The CMMI doesnt dictate what data to

collect. Each organization should make this

decision based on business needs.

Does each required peer review data itemsupport a dened measurement objec-

tive in your organization?

All required data collected in the peer reviewprocess/tool should support your dened

measurement objectives.

Has the person or role who is responsible

or data analysis been identied andassigned?

The CMMI does expect you to assign

resources to analyze data.

Peer Review Software

Is your peer review sotware customiz-able to align your data requirements with

your organization/s peer review process?

Sotware tools should be fexible tosupporting an organizations process.

Is your sotware fexible enough to

support peer review comments thatcome rom varied sources, such as,

collaborating teammates & customers?

Peer reviews can be implemented in

dierent ways. The CMMI doesnt dictatethe how to.

Does your peer review sotware & processprovide helpul hints to aid your people

making better peer review decisions?

Practitioners need reminders to help makethe best decision with respect to peer

reviewing products.

Does your peer review sotware & process

help your practitioners collaborate andcommunicate deects in a non-intrusive

way?

The primary intent o peer reviews is to iden-

tiy deects or removal and to recommendchanges that are needed.

Does your peer review sotware & processsupport peer review o appropriate

artiacts such as images, requirements

documents, schematics, and code.

Peer reviews can be applied to multipletypes o artiacts. The CMMI does not

dictate which artiacts need to be reviewed.

Is the peer review data centralized andeasily available or analysis and

reporting?

The CMMI does expect you to analyze peerreview data.


11/1211

About CodeCollaborator

SmartBears peer review sotware, CodeCollaborator, helps companies successully use peer reviews to reach their

CMMI Level 3 and above goals. CodeCollaborator is an enterprise-level sotware solution that helps standardize

and expedite peer reviews with team members no matter where they are located, capturing the review metrics

required or CMMI audits, and helping to improve overall product quality.

Reach your teams CMMI goals more quickly using CodeCollaborator to:

Quickly implement the standardized and documented peer review processes required or CMMI

Accelerate and simpliy the CMMI audit review process with centralized reporting

Enable continuous process improvements to get the most rom CMMI and peer review

To learn more about how CodeCollaborator can support your teams peer review needs and CMMI goals visit

http://www.smartbear.com and sign up or a ree trial o CodeCollaborator

Try CodeCollaborator Free or 30 Days

Scan and download your ree trial to see why

users choose SmartBear or peer code review.
http://smartbear.com/products/software-development/code-reviewhttp://www.smartbear.com/http://smartbear.com/products/software-development/code-review/free-codecollaborator-trialhttp://www.smartbear.com/http://smartbear.com/products/software-development/code-review/free-codecollaborator-trialhttp://www.smartbear.com/http://smartbear.com/products/software-development/code-review


12/12

About SmartBear

SmartBear Sotware provides tools or over one

million sotware proessionals to build, test, and

monitor some o the best sotware applications and

websites anywhere on the desktop, mobile and in

the cloud. Our users can be ound worldwide, in small

businesses, Fortune 100 companies, and government

agencies. Learn more about the SmartBear QualityAnywhere Platorm, our award-winning tools, or join

our active user community at www.smartbear.com,

on Facebook, or ollow us on Twitter @smartbear.

SmartBear Sotware, Inc. 100 Cummings Center, Suite 234N Beverly, MA 01915+1 978.236.7900 www.smartbear.com 2012 by SmartBear Sotware Inc.Specifcations subject to change. CC-WP-3CMMI-042012-WE
http://www.smartbear.com/http://www.facebook.com/smartbearhttps://twitter.com/#!/smartbearhttps://twitter.com/#!/smartbearhttp://www.facebook.com/smartbearhttp://www.smartbear.com/

Documents

Debunking 3 Common CMMI Peer Review Myths