View
248
Download
0
Category
Tags:
Preview:
Citation preview
LeanSigma® Facilitator Training
Module 12 – Measuring Success
Topics
Measuring Success
• Process performance measures……………………………………… 5 - 17
• Process capability ………………………………………………………… 18 - 23
• Program performance measures ………………………………… 24 - 34
2
Lean Sigma has six steps to optimize processes.
3
Introduction
• We’ll look at several types of measures:− Process performance: cycle time, KPIs, DPMO and yield
− Process capability: capability indices
− Program performance: pace, cost savings, implementation progress
• Selected process measures should follow from CTQs
• We want quantitative measures of a process because they’re much more objective than qualitative or ad hoc data.
• Performance data can help us to set a baseline, so we know when to intervene with more process improvement work.
4
Cycle time is our most common measure of process improvement.
• We create an estimate of future state cycle time in kaizens based on the improvements we plan to make.
• After the future state is implemented, you’ll measure the actual cycle time improvement.
• This validates how much time we’ve redeployed to other work. You’ll capture this measure in the Lean Sigma PMO
5
Annualized Labor Hours Saved
IMS FTE Reduc-tion
Outsourced FTE Reduction
Capacity redeployed
You will receive a demo of the PMO within the next few weeks.
Other common process improvement measures
• Reduction in lead time− May support a reduction in DAP; but often process improvements
must first be implemented in other, related processes
• Process improvement measures are often not measured on an on-going basis, but rather before and after process changes (2x)
6
VA Tas
ks
Non-V
A Ta
sks
Delay
s
Loop
back
s
Hand-
Offs
Decisio
n Po
ints
Lead
Tim
e
Cycle
Tim
e
0%10%20%30%40%50%60%70%80%90%
100%
Percent Improvement
Typical kaizen event measures
Key performance indicators (KPIs) are tracked to monitor the health of the process over time.
• Identify the “vital few” measures that tell the story of process performance
Define Collect Analyze Report Take Action
On-time delivery, query responsiveness, defect rate, throughput, etc.
Clear metrics definitions are important!
• Each KPI must have an Operational Definition:− An exact definition of the measure
− How it should be measured
− Measurement frequency
− Who should measure
− The threshold or expected service level
• Reporting: KPIs are evaluated in 30 / 60 / 90 day post-kaizen reviews.
• KPIs are monitored on an on-going basis by Process Owners, who are accountable for taking the appropriate action when results fall below corresponding thresholds
8
Defects per unit and per million opportunities are common process measures.
Term Definition
Defective Any transaction that contains at least 1 defect (error)
Defect Any part of a single transaction that has an error
Opportunity Any part of a single transaction that can cause a defect• A transaction could have any number of opportunities contained within it, a measure of
complexity• Every chance to do something either “right” or “wrong”
DPU Defects Per Unit – Calculates the percentage of the time in decimal form how many defects occurred out of the total transactions evaluated
DPMO Defects Per Million Opportunities – Calculates the number of times out of a million a defect would occur based upon the total opportunities for error
Evaluated # Total
Defects of # TotalDPU
000,000,1*iesOpportunit Total
Defects of # TotalDPMO
9
Calculating DPMO
• A project team is working on improving invoice accuracy
• The number of defect opportunities per invoice is 10. The team evaluated 100 invoices and discovered 35 defects
DPMO = (# of defects x 1,000,000) (# of Defect Opportunities/Unit) x # of Units
DPMO = 35 x 1,000,000 10 x 100
• “Six Sigma quality” means DPMO ≤ 3.4
10
Calculating DPMO
• A delivery process of a courier company has 3 defect opportunities (timeliness, accuracy of delivery, accuracy of invoice)
• The company has delivered 5000 mail pieces, 8 of them were too late, 3 of them delivered to the wrong address, and 2 had a wrong amount on the invoice
• The DPMO is
866000,000,1*5000*3
238
DPMO
11
Process Yield Concepts
• LeanSigma Yield concepts are:
−Throughput Yield – one process step: ratio of defects vs. defect opportunities
−Rolled Throughput Yield (RTY) – multiple process steps: multiplication of Throughput Yields
−Normalized Yield – multiple process steps: root of the multiplied Throughput Yields (“average process yield”)
• “Traditional” Yield concepts:
−First Time Yield – ratio of defectives vs. units processed
−Final Yield – ratio of defectives (after rework) vs. units processed
12
First Time Yield
• FTY – the percentage of units that pass through an operation without any defects
• If you describe the yield as 80%, the true performance of the process is not measured
• FTY = 20%, therefore the percentage defective = 80%
13
FTY + p(d) x 100% = 100%
Percentage Defective
“Traditional” Yield Metrics
• First Time Yield (FTY)
• Final Yield (FY)
Focus on Defectives and units
No. of units: 1057
No. of defective: 58
FTY: 94.5%
Step 1 Step 2 Step 3 Step 4 Step 5 Step 6
No. of units: 1057
No. of defective: 45
FY: 95.7%
Rework
Step 1 Step 2 Step 3 Step 4 Step 5 Step 6
14
FTY Example
• 1056 invoices were processed in one day. 924 passed and 132 failed for various reasons
• # of bad units + # of good units = Total # of units
• # of bad units can be expressed as a probability of a defective, p(d): or a percentage defective: p(d) x 100% = percent defective
15
%1001056
132
1056
924
%100%5.12%5.87
FTY + percent defective = 100%
FTY = 87.5%, percent defective = 12.5%
LeanSigma Yield Metrics
• Throughput Yield (TY)
− Focus on Defects and defect opportunities
No. of defect opportunities per unit: 4
No. of units processed: 268
No. of total defect opportunities (4x268): 1072
No. of defects: 34
Throughput Yield: 96.8%
Step 1
16
Rolled Throughput Yield
• What is the probability of producing this product error free?
17
Invoice Generation
Invoice Processing
Billing Output
TY = 97%
Op1 x
TY = 98%
Op2 x
TY = 91%
Op3= 86.5%
86.5% probability an invoice will pass through defect-free
How can we establish thresholds for selected process measures?
• Thresholds or “specifications” should be established for all KPIs to flag the need for adjustment to the process
• Specifications are usually based on:− Historical results
− Compliance/regulatory requirements
− Benchmark data
− Management judgment
• In most cases, IMS specs are one-sided limits defined by internal customers
− Coding Accuracy: Error (defect) rate < 1%
− Panel fulfillment > 90%
• Specifications are often documented in Service Level Agreements (SLAs)
18
How can we determine if our process measures can routinely comply with established specs?
• Process capability is the inherent ability of a process to meet the specifications− The performance of the process when it is stable and predictable,
or operating in a state of statistical control, i.e., the mean and the variation in the process are not shifting
− Assumes no “special cause” variation, e.g., a “market event”, or a hurricane and resulting production delays. Verify this!
• Process capability measures provide insight as to whether the process has:− A centering issue relative to the specifications− A variation issue− A combination of centering and variation issues− Inappropriate specifications
• Process capability measures provide a baseline assessment
19
When specifications cannot be changed, actions must be taken to improve process capability
• LSL = Lower Spec Limit• USL = Upper Spec Limit• Target is usually the
midpoint between the specs
20
Standard deviation describes how far results in a process vary from the mean result.
• Processes with higher standard deviations have more variation.
• They are less predictable and less stable than healthy processes.
• Processes with less variation will have a higher “sigma level” and fewer defects.
21
Variation(Dispersion)
1
)( 2
n
XXs
NormalDistribution(“Bell”)Curve
StandardDeviation =
We assess process capability using process capability indices.
• Cp = =
− What is possible if your process is perfectly centered
− The process potential
• Cpk = minimum ( , )
− The reality of your process performance
− How the process is actually running relative to the specifications
− The “k” is the penalty for being off-center
specification width
process width
(USL – LSL)
6s
(USL – X)
3s
(X - LSL)
3s
22
The “sigma level” of the process provides another measure of process capability.
• σlevel = minimum ( , )
− Number of standard deviations between the center of the process and the nearest specification
• “Six-sigma quality” means:− Cp ≥ 2.0
− Cpk ≥ 1.5
− σlevel ≥ 4.5
(USL – X)
s
(X - LSL)
s
Process capability should be periodically reassessed, especially following the implementation of process improvements.
23
Program metrics help us monitor the health and benefits of the LeanSigma program.
• Are we achieving our mission to deliver more value to our customers and stakeholders by identifying and implementing measureable process improvements?
• Regular Lean governance meetings help us: − Align on priorities and targets− Validate results meet expectations− Ensure support− Mitigate risks
24
Costs:• Lean facilitator & SME time• Travel $• Implementation costs
Benefits:• Enable IMS to deliver on
strategic priorities• Transform the culture
Lean program reporting summarizes results from all Lean events.
• Cadence
− Pace of events by month, department & geography
• Benefits
− Net EBIDTA savings
− Annualized hours saved (capacity redeployed, FTE savings)
− Percent improvement cycle / lead time, delays, etc.
− Highlights of exceptional events, case studies
• Implementation
− Percent of identified savings removed from budget
− Percent of kaizen action items completed
• Focus areas
− Prioritization of processes based on exec sponsor discussions
− Pipeline of Lean events
• Cultural entrenchment
− Number of employees trained / certified
− Number of employees participating in kaizens
− Communications
25
95% of reporting is enabled by web queries against the Lean PMO
26
Sample DashboardU.S. Events by Month
n= 110completed
events
Process Focus to Date
Annualized Labor Hours Saved= 31,086
IMS FTE Re-duction 8,040
hrs; 26%
Outsourced FTE Reduction
7,236 hrs; 23%
Capacity re-de-
ployed15,810 hrs; 51%
2012 U.S. EBIDTA Contribution: $817K*100% of savings realized
Production & Data Coll.
26%
Del ivery & Support
29%
PD / Stat Svcs3%
US BU6%
HR26%
Finance6%
GPS2%
Legal1%
Feb Aor Ju
nAug Oct
Dec Feb
Apr Jun
Aug Oct0123456789
pla
nn
ed
2011 2012
Training grants23%
Delivery & support
46%
Producton & Data Coll
31%
80.23% of all U.S. kaizen action items completed
Cost Savings Detail Example
27
Qualitative Reporting Sample
28
Case Study: U.S. Market Research Client Support
29
February 2011Sponsor: Manny TriggianoProcess Owner: Brenda CapobiancoLean Process Lead: Carissa WaltenbaughTeam: Kevin Downing, Chris Davies, Cathy Pileggi-Jefferson, Katrina Kulp, Kyle Burke, Michele Agnoli, Joy Cross, Gerard Avillo, Lisa Cooney, Jeff Northern
Key Kaizen Findings
Issues Noted Process Changes
Some CSAs less able to handle complex questions.
CSAs sometimes call multiple times to refine the client question.
CSAs sometimes do unnecessary research.
Triage client questions and assign CSA resources to a Level 1 pool (simple) or Level 2 pool (complex).
Create a checklist to guide CSAs when refining details of a client’s question.
Create a decision tree to identify what research is required for a particular question. Improve how info is organized on ARC.
90 Days After the Kaizen
Clients are assigned a CSA even if they call only a few times a year.
Pool CSA resources so questions are answered by the first available CSA.
A pilot was completed to test the central support pool for smaller clients.
The centralized model went live June 13, 2011.
QMS was enforced as a central information system.
Tier 1 (simple) and 2 (complex) resource pools were created.
Checklists and decision trees were created to clarify info needs.
Case Study: U.S. Market Research SupportFindings 15 months after implementation
30
Results:• The central service center remains in place successfully.• Clients talk to CSAs directly more often.• Very few clients have questioned the change.• Three positions were eliminated.• In addition, the team took on additional work without adding
headcount. (SMART roll-out). • Cycle time savings were estimated at 65% initially. The savings
observed are about 50%.
Critical factors made the change possible: Pilot to test client feedback Excellent communication to the team and clients about changes Extreme organization and focus
Improve process workflows for custom deliverables
Improve quality & consistency of RFP responses
Reduce time spent screening & validating data
Reduce admin burden for GTM resources
Reduce time to implement
Reduce admin burden
Improve the process for IC implement-ations
APR 16
Focus areas – Completed Events
MAR 6-8
Resu
lts
Obje
ctiv
es
MMS Screening & Validation
MAS RFP Response mini
kaizen
Billing Initiation Process
MAS Custom Project Process
Performance Analytics
Deployment Standardization
Opportunity Management /
CRM Improvement
SAS Incentive Compensation
Implementation Process
MAY 1-3 MAY 15-17 MAY 15-17 JUN 11-12 JULY 17 -19
31% reduction in cycle time
Process improve-ments imple-mented
Identified 3 improve-ment work streams
Results to be rolled out at July 10 report out
Delivery time reduced by 25%
Eliminated time spent entering detail into early stage opps
31
Process Priorities – Sample from Operations
32
Gather / enter customer-specific order requirements
Collect dataPerform
statistical data transformation
Manage offering
production
Plan / QC the order
Perform delivery supportStandardize / cleanse data
Schedule / run customer-specific
production
Create output / deliver order
Track and update order
Define & communicate order
management procedures
Perform customer-specific
QC on requirements
Perform quality control on final
output
Build / Deliver the order
Test & certify supplier
Receive, cleanse & bridge/code
transaction inputs
Create / maintain statistical
methodology
Define & communicate
product creation procedures /
methodologies
Add new supplier to database
Perform transaction quality
assurance
Define / perform global quality management
Schedule production
Establish ongoing data collection
process
Build transaction database
Project / impute transaction data
Execute production jobs
Maintain reference database
Validate projected data
Manage rerun / rework
Perform change management
Maintain product related data
Maintain historical data
Perform in-process
production quality control
Provide customer training
Define & communicate
customer service & support procedures
Close loop to Marketing & Sales
Receive / escalate customer inquiry Service Center
Receive / escalate customer inquiry
Call Center
Analyze & resolve inquiries / complaints
Provide field services and
support
Manage credits & invoices
Process changes & maintain customer order information
Maintain customer support systems
and data
Measure & report customer delivery
performance
Manage customer delivery vendors /
subcontractors
Manage customer delivery information
systems
Data Collection Production Delivery
High Priority Process
Evidence of Cultural Entrenchment
• Lean continues to be a high priority for Seyed and teamLeadership
• U.S. has held more Lean events than any other country (110 events since Feb 2011)
• 30 additional events are in the pipelineLean Events
• More than half of all certified Lean facilitators are in the US
• 19 certified and 48 in progressSkill Development
• Over 25% of Lean’s global savings come from the U.S.
• Implementation is strong (>80% kaizen action items complete)
Results
33
Lean Measurement & Reporting Tools
• Lean Event Executive Summary Template:− http://arc.imshealth.com/gm/document-1.9.1639940
• Guidelines for 30 / 60 / 90 Day Reviews− http://arc.imshealth.com/gm/document-1.9.2276470
• Lean PMO− http://arc.imshealth.com/gm/leansigmaroadmap
• PMO Users Guide:− http://arc.imshealth.com/gm/folder-1.11.1540558
• Project Statistics (PMO Web Query):− http://arc.imshealth.com/gm/folder-1.11.1517854
• Guidance on standard metrics and metrics-monitoring tools− Under development. Contact Tamra Kirkpatrick.
34
US & Canada: ISO Compliance
• In the US and Canada, the Kaizen management process is maintained in compliance with ISO standards
• Facilitators are required to comply with the Kaizen Management procedure− http://arc.imshealth.com/gm/document-1.9.2241809
− The procedure is written to provide a great deal of flexibility in selecting, planning and conducting kaizen events
• Process effectiveness is assessed:
• Results are reviewed in quarterly internal management reviews, and periodically, by external ISO auditors
35
Number of kaizen events per quarter, based on a rolling
four quarters15≥
Recommended