2 1 Determine Measure Vsw Edits(Rev)

© 2007 IBM Corporation For internal use only. Subject to change after publication date. All contents strictly IBM Confidential.

Lean Six Sigma Black Belt

6SigmaTek LLC Proprietary Material

Measure Performance – 2.1

Presented by: Stephen Griffin, CEO

Determine What to Measure

Global Business Services Lean Six Sigma Black Belt DMAIC

© 2007 IBM Corporation For internal use only. Subject to change

after publication date. All contents strictly IBM Confidential.



2.1 Determine What to Measure

Objective:

To identify different types of measures and an understanding of how the measures relate to critical customer requirements.

Key Topics:

Performance Measurement. Input, Process, and Output Indicators. Indicator Relationships.

2.1Determine

What to Measure

2.2 Manage

Measurement

2.3Understand

Variation

2.5Managing the Measurement

System

2.4Determine

Sigma Performance






Performance Measures: Achieving Customer Value

Important decisions based on linking

customer expectations to

process performance

SuppliersProcess Inputs

BusinessProcesses

ProcessOutputs

CriticalCustomer

RequirementsInput Measures

Process Measures

Output Performance Measures

Customer Value

Good process decisions require comprehensive process data.






Suppliers: Inputs:

Start Boundary ____________

Outputs: Customers:

End Boundary ____________

Process

Process Elements and Indicator Relationships






Inputindicators

Processindicators

Output(performance)

indicators

Efficiency Measures

Cost per transaction Time per activity Amount of rework Turnaround time Variability of an activity

Effectiveness Measures

Percent defective Number of errors Total response time Invoice/billing accuracy Revenue

Input Process Output Indicators






I nput Indicators Process Indicators Output Indicators

Measures that evaluate the degree to which the inputs to a process – provided by suppliers – are consistent with what the process needs to efficiently and effectively convert into customer-satisfying outputs.

Examples: Number of customer

inquiries. Type of customer inquiries. Number of orders. Number of open positions. Type of position open. Accuracy of credit analysis. Timeliness of contract

submitted for review.

Measures that evaluate the effectiveness, efficiency, and quality of the transformation processes – the steps and activities used to convert inputs into customer-satisfying outputs.

Examples: Service Personnel availability. Required time for credit

review. Percentage of non-standard

approvals required. Number of qualified

applicants. Total cost of service delivery. Total overtime hours.

Measures that evaluate dimensions of the output – may focus on the performance of the business as well as that associated with the delivery of services and products to customers.

Examples: Number of calls/hour taken

by each service rep. Second year customer

retention figures. Total number of meals

delivered. Percentage of customer

complaints.

Input Process Output Measurements






Process Output Indicators: CCRs and CBRs

Output Indicators

VOB

________

________

CBRs________

________

BusinessIssues

________

________

CCRs________

________

VOB - Voice of the BusinessCBR - Critical Business Requirements

VOC - Voice of the CustomerCCR - Critical Customer Requirements

CCR’s________

________

________

_________

________

________

________

_________

VOC________

________

________

_________

CustomerIssues






EXAMPLES: CCR and CBR

CCRs

Output Indicators

CBRs

________

________

________

________

CCRs

Price/Unit

Delivery Time

Dimensions

Purity

Reliability

Color

Service Level

CBRs

Cost/Unit

Productivity

Compliance withRegulations

Changeover Time

Safety

Training Hours

Critical to: The BusinessThe RegulatorThe Employees

Critical to: The Customer

The Market






x3,1 x3,2 x3,3 x3,4 x3,N

Example: Medical Diagnostic Tube Life

y1

Oil Dielectric Quality GTD-3

y2

Focal Spot ControlGTD-7

y3

Generators SpitsGTD-17

y4

Rotor Failures GD-14

yn

x1,1 x1,2 x1,3 x1,4 x1,N

x2,1 x2,2 x2,3 x2,4 x2,N

Top level Y is big enough to be seen at OBU level – an operational business objective.

Frequently the parent project does at least a verification of this top level Y with the customer.

x4,1 x4,2 x4,3 x4,4 x4,N

After eight (8) projectsAfter eight (8) projects From January ’05 to May ’06 the average tube life doubled.From January ’05 to May ’06 the average tube life doubled.

Success Derived from Project Focus






Selecting the Right Output Indicator Good output indicators relate directly to CCR.

The correlation of the output indicator to the CCR is the most important consideration…otherwise the indicator is meaningless.

A balance must be established between the following:

– Ease of obtaining data.– Time available to the project team.– Cost associated with obtaining the data for each indicator.






Clarify and Refine CCRs Before Selecting Indicators

Key Customer I ssue

The real customer concerns, values, or expectations regarding a product or service. Void of emotion or bias, the statement describes the primary issue a customer may have with the use of product or service. Description of or experience surrounding the attributes of the product or service expected or desired by the customer.

Critical Customer Requirements

The specific, precise, and measurable expectations which a customer has regarding a product or service.

Voice of the Customer

Actual customer statements and comments which reflect their perception of an: Attribute of a product or

service. Experience with a

product or service delivery.

Encounter or experience with a business process or representative.






Associated CCR Output IndicatorsOutput Indicator

Product delivery cycle time from the completion of the customer order to the delivery of the product.

Number and type of vehicle specifications delivered correctly.

Actual delivery time vs. promised.

Delivery time for each vehicle. Number of times vehicles were

delivered to location other than what is specified on agreement.

CCR Product is delivered within

three hours of order taken.

Right vehicle is delivered at the right time to the right location.

Output Indicator Number and type of vehicle

specifications delivered correctly.

Actual delivery time vs. promised delivery time for each vehicle.

Number of times vehicles were delivered to location other than what is specified on agreement.

CCR

The vehicle delivered meets the vehicle specifications as described in the contract.

The vehicle is delivered within the time specified in the contract.

The vehicle is delivered at the location specified in the contract.

An example of oneCCR with four

output indicators.

An example of oneCCR with four

output indicators.

Three CCRs withone correspondingoutput indicator.

Three CCRs withone correspondingoutput indicator.

OR






Output Indicators Guidelines

Designate output indicators for each CCR.

Decide on an appropriate measurement frequency.

Ensure measurement instructions are clear and understandable.

Measure defects of the process.

Use actual results for analysis.

Think ahead and anticipate measures to enhance the defect analysis.

Limit the number of indicators to three (3).






Objective:– Identify a list of potential output indicators to evaluate the extent which the process meets CCRs.

Instructions:

1. List a critical customer requirement in the right column of the table below.

2. Determine if the CCR describes a single or multiple customer expectations associated with an output.

3. Identify ways to measure whether or not the process is meeting the critical customer requirement. What are the units of measure? When and where would the data be collected?

4. Select one or more measures that may serve as good potential output indicators and list in the table below.

5. Repeat the process for the remaining CCRs.

6. Be prepared to discuss your conclusions with the class. CCRs

Output Indicator Critical Customer Requirement

Course Exercise: Identify Potential Output Indicators






Course Exercise: Link to Customer

Objective:

– Identify the customer.

Instructions:

For each output indicator on the previous page:

Identify the customer. What should be measured (operational definition)? How would the data be collected?






Measurement Considerations…

One Rule: use as few indicators as possible.

There is a cost/benefit and relevance relationship to measuring.

– Too many measures are costly, unproductive, and may divert attention from critical areas.

– Not enough or incorrect process indicators will not allow you to identify defects before they occur.

The appropriate level of measurement will enable the organization to focus on the correct areas and provide a means for early detection of defects and out-of-control activities necessary to prevent customer dissatisfaction.






Clarifying CCRs

As a class, consider each of the CCRs translated from Voice of the Customer (VOC).

Confirm each CCR meets the criteria as described in the table below. If it does not, write the draft CCR in the VOC column and identify the key customer

issues associated with the statement. Identify and list the final CCR in the appropriate space. Does this CCR need to be validated? If so, what validation data may be available?

Key Customer I ssue

The real customer concerns, values or expectations regarding a product or service. Void of emotion or bias, the statement describes the primary issue a customer may have with the use of product or service. Description of or experience surrounding the attributes of the product or service expected or desired by the customer.

Critical Customer Requirements

The specific, precise and measurable expectations which a customer has regarding a product or service.

Voice of the Customer

Actual customer statements and comments which reflect their perception of an: Attribute of a product or

service. Experience with a

product or service delivery.

Encounter or experience with a business process or representative.






CCRs and Multiple Output Indicators

In the previous section, teams translated a variety of VOC data into critical customer requirements. Teams were careful to recognize some customer feedback and statements need to be clarified, and a process for specifying CCRs involves considering key issues customers may have with a product or service. From these issues, the team was able to specify the critical customer requirements of the process output.

Some CCRs may be measured in terms of one specific expectation a customer has. Others may require several output indicators. The table shows how one CCR can have one or several associated output indicators.

This table lists the second CCR from the above table, but shows it as three (3) different critical customer requirements associated with the delivery of a vehicle.

Regardless of how the CCR is stated, teams must be sure to identify all CCRs associated with a process production service and make sure CCRs are measured by the output indicator.

Output Indicator• Product delivery cycle time

from the completion of the customer order to the delivery of the product.

• Number and type of vehicle specifications delivered correctly.

• Actual delivery time vs. promised.

• Delivery time for each vehicle.

• Number of times vehicles were delivered to location other than what is specified on agreement.

Product is delivered within three (3) hours of order taken.

Right vehicle is delivered at the right time to the right location.

Output Indicator• Number and type of vehicle

specifications delivered correctly.

• Actual delivery time vs. promised delivery time for each vehicle.

• Number of times vehicles were delivered to location other than what is specified on agreement.

The vehicle delivered meets the vehicle specifications as described in the contract.

The vehicle is delivered within the time specified in the contract.

The vehicle is delivered at the location specified in the contract.

Critical Customer Requirement

Critical Customer Requirement






Next Step: Develop Process Indicators

Organizations must have information about process performance before products or services are delivered.

– It allows a business to identify abnormal variation or special cause before a defect occurs.

– Early defect detection prevents dissatisfied customers and retains customers.

– As CCRs change and opportunities for improvement are identified, the business will know where in the process improvement resources need to be assigned.

– If the output is defective (e.g., not meeting CCRs) we will have data to begin looking for undesired variation and identify root cause.






Effective Process Indicators

Effective process indicators must:

– Be based on facts and data.– Predict, or lead, at least one output indicator.– Provide valid and quantifiable data.– Enable the evaluation of cause-and-effect relationships occurring

inside the process.– Lead the team to root cause analysis, if CCRs are not being met.

This step allows the team to begin capturing data about the process to ultimately have an impact on the customer.






Select Where and What to Measure

While reviewing the process map, choose logical control points to take measurements:

– Activities or decisions that if performed incorrectly or inadequately will result in high probability of a poor output indicator result.

– Decision blocks – How many rejects or special cases are there which are non-value-adding?

– Activities with a high volume of work built up.– Hand-offs to the next functional area.– Time-consuming activities.

When the control point is identified, determine the best measure for that process indicator.






Course Exercise: Identify Additional 0utput Indicators

Objective:

– Identify indicators necessary to measure the teams process performance, in terms of CCRs and business expectations.

Instructions:

– As a team, review the customer value equation:

Quality + Service = Customer Value Cost + Time

– Discuss and respond to the questions on the following page.






Course Exercise: Identify Additional 0utput Indicators (cont.)

Discuss and respond to these questions:

1. Do the CCRs used in the previous exercise account for what we want this process to accomplish? If not, what is missing?

2. Do the CCRs from the previous exercise address the time it takes to perform the process? If not, is it important to make improvements to the process cycle time? If so, are any additional indicators needed?

3. Is it important to reduce the cost of the process? If so, what additional indicator(s) may be needed?

4. Do the output indicators measure the quality and service expectation of the customers and the business? If not, what additional indicator(s) may be needed?

5. If necessary, work with leadership to establish critical requirements that may not be considered critical to the customer, but are very important to the business.

6. Identify additional output indicators needed to achieve the business objectives, strategies, and goals.

7. Review the output indicators selected with the criteria and questions addressed in Selecting the Right Output Indicators.






Selecting the Right Process IndicatorsIn addition to making sure the indicators provide the team with valid and quantifiable data, teams must be sure what they are measuring actually enables them to evaluate the cause and effect relationships occurring inside the process. Below is a list of questions each team should review after identifying output, process, and input indicators:

– Are each of the process indicators true predictors, or leading indicators, of at least one output indicator?

– Do the process indicators evaluate areas of the process known to adversely affect the quality of the process output?

– Has the team identified process indicators for the process variables that most influence the ability of the process in meeting critical customer requirements and, therefore, the output indicators?

– If a critical customer requirement is not met, does the team know why (root cause)? What additional process indicators may be needed to answer this question?






Course Exercise: Identify Potential Process IndicatorsObjective:

– Determine potential process indicators that serve as good predictors of the ability to meet the CCRs.

Instructions:

1. Carefully review functional deployment maps created for the process.2. List the output indicators in the appropriate spaces of the table below.3. Working from the beginning of the process, identify the activities or decisions likely to

have a significant influence on the output indicators.4. Of the activities or decisions identified, which ones are known, or are assumed to, have

been a problem historically?5. Select the activities that serve as the best predictors of process output success.6. Of the measures identified, evaluate the process indicators selected by applying the

characteristics of good measures, and selecting the right process indicators.7. Identify any activities or decisions currently under review or already selected as

improvement opportunities? Do any of the steps or activities have readily accessible data available?






Selecting the Right Input Indicators

Input indicators allow measurement of the consistency of the inputs to the process.

– Do the input indicators measure the critical requirements we have of our suppliers’ products or services?

– Do the indicators measure elements of the input known to affect the ability of our process to meet critical customer requirements?

– Are the input indicators true predictors, or leading indicators, of at least one process indicator?

– Do the indicators measure aspects of the input that would, within a specified tolerance, eliminate significant inspection, scrap, rework, or excessive cycle time?






Course Exercise: Identify Input Indicators

Objective:

– Determine the input indicators that serve as the best “predictors” of the ability to meet critical process requirements.

Instructions:

1. Carefully review the list of inputs provided to the process.2. Of the inputs listed, which ones have the potential for having a drastic impact

on process indicators, outcome indicators, and/or CCRs?3. Select the process inputs that, if not perfect (accurate, complete, timely,

defect-free, etc.) will prevent the process from meeting CCRs or add significant cost and additional effort to the process.

4. List the process requirement for your supplier's input.5. Determine the output indicators needed to help ensure process activities and

decisions meet desired performance.6. Of the measures identified, select one or two input indicators by applying the

characteristics of good measures and selecting the right indicators, shown earlier.

7. Repeat the previous steps for the remaining process inputs.






Indicator Relationships

Link output performance to process and input indicators.

First, look to establish output indicators because they indicate how effective your process is at meeting CCRs. Once you understand the key output performance measures, determine what key input and process indicators you need in order to meet the desired outcomes and, therefore, satisfy customer requirements.

You can use a relationship matrix to help show the relationship between the output performance measures and key input and process measures. The relationship matrix will help prioritize the most important measures.

Note: The strength of the relationship is based on how likely changes in the input or process indicator will cause changes in the output indicator.






Link Output Performance to Process & Input Indicators

Relationship of process & input measures

Strong Relationship

Medium Relationship

Weak Relationship

No RelationshipBlank

Output Performance Indicators

Process & Input Indicators

Call Abandon Rate

Customer Satisfaction

AnswerSpeed

EmployeeExperience

First TimeResolution

The strength of the relationship is based on how likely changes in the input/process measure will cause changes in the output performance measure.






Course Exercise: Indicator Relationships

Objective:

– To determine the strengths of the relationships between the team's Input, Process, and Output indicators.

Instructions:

1. List the draft output indicators on the far left column of the Relationship Matrix below.2. List the draft input and process indicators along the top row of the Relationship Matrix.3. As a team, discuss the following questions for each of the pairs of input or process indicators

and the associated output indicators: – To what extent do the input or process indicators predict the output?– If the input or process indicator were to suddenly change, would there be a corresponding

change in the output indicator? When would it occur?– What other activities taking place in the process may have a more dramatic effect on the

output? How may we measure that?4. Using the scale at the bottom of the page, assign a value of strong, medium, weak, or none to

describe the relationship between each possible pair of indicators.5. Add any additional indicators the team may have identified and assign a corresponding

relationship strength value.6. Be prepared to present and discuss the team conclusions with other course participants.






Measurement Plan: Uphold Principles

The measure must be important to the customer.

The measure must be easy to understand.

The measure is sensitive to the right things.

The measure must promote appropriate analysis and action.

The data needed must be easy to obtain.

The measure needs to be driven by business objectives.

The measure needs to lead to process improvement.






Choose Measurement Indicators Wisely

Choose as few indicators as possible.

Too many measures are costly to collect and analyze.

Too few measures will limit your ability to identify defects.

Characteristics of Good Measures:

– Linked to CCRs;– Simple to read and understand; – Help drive appropriate corrective

actions; – Provide insight to cause-and-effect

relationships;– Representative and relevant to the

process;– Reveal trends and cycles;– Easy to collect;– Clearly defined;– Statistically valid; and– Repeatable.






Review and Transition

In section 2.1 Determine What to Measure, we learned:– The role data plays in process improvement.– The cause-and-effect relationships that occur inside the team's process.– How to determine the indicators needed to evaluate current process

performance.

In section 2.2 Manage Measurement, we will learn:– The different types of data and how each type can provide the team with

different insights and knowledge of a process.– How to develop operational definitions and data collection plans to build

validity and consistency in the data which the team gathers.

2.1Determine

What to Measure

2.2 Manage

Measurement

2.3Understand

Variation

2.5Managing the Measurement

System

2.4Determine

Sigma Performance