Upload
adhitya-setyo-pamungkas
View
217
Download
0
Embed Size (px)
Citation preview
7/27/2019 Measuring Team Productivity
1/27
Nashville
Measure what is measurable and make measurable what is not so
Galileo Galilei(1564-1642)
O P M
Measuring Team
ProductivityOregon
ProductivityMatrix
using
specifically for Support teams
An experience paper on improving PPQA
team productivity by
Srinath RamaseshWipro Quality Consulting Group
7/27/2019 Measuring Team Productivity
2/27
2
O P M
Fundamentals| The Oregon Productivity Matrix (OPM in short) takes multiple
metrics, uses a eight step process* and subjective weightingsystem to rank/prioritize the metrics. The metrics are thenadded up to give a cumulative composite score
| This single index indicates how well thecompany/organization/team is doing against the performancemetrics
*as we implemented it.
7/27/2019 Measuring Team Productivity
3/27
3
O P M
Theme| Preamble - on Productivity
| Productivity in Support Teams -Really?
| Background The team which implemented OPM
| OPM as a solution
| OPM History - and advantages| The OPM Grid - What is OPM?
| The Eight steps to Implementation The core details
| Avenues to improving OPM
| Lessons Learned
| Q&A
7/27/2019 Measuring Team Productivity
4/274
O P M
Preamble| Productivity is a major concern in an IT industry, especially so for
support teams (or non-coding jobs)z PPQAs (Process and product Quality Assurance)
z Project Management Office
z Administration
| Productivity by definitionz "Goods and services produced per unit of labor and expense
orz The relationship between the inputs required to produce a product or
service, and the value of the output produced
z In a software development activity, this is relatively easy to calculate. We
could measure code productivity as Function pointsor lines of codedeveloped per unit effort or cost
Goods + Services
Labor + Expenses
7/27/2019 Measuring Team Productivity
5/275
O P M
Productivity of Support
Teams| Inputs are fairly self-evident: staff, equipment, costs, etc
z but what of the outputs?
| What exactly do support teams produce?z Multiple deliverables,
z Non periodic deliverables,
z Multiple tasks,
| Measured by multiple parameters
| Example:z An accounts department, for instance, will produce a variety of reports, some
regular and some adhoc They work on several tasks simultaneously, withconstant interruptions, and its impossible to apportion the total input to each
Can Productivity for a support team be clearly established?
Is it possible to measure the value driven by all these activities and deliverables?Is it possible to measure improvement collectively?
7/27/2019 Measuring Team Productivity
6/276
O P M
Its Possible!!!| The productivity of support teams can be measured!!!,
z Changes up and down can be monitored allowing managerialaction to be taken to ensure that improvement takes place
The Hypothesis:Anything that increases the quantity and/or quality of output, without a
corresponding increase in the utilization of resources, creates a gain inproductivity *
Oregon
ProductivityMatrix
Using
Goods + Services
Labor + Expenses
Volume + Value
Efficient Resource Consumption
This presentation covers an experience of implementing a research basedmethodology called Oregon Productivity Matrix with a Process and Product QualityAssurance (PPQA) team in a leading automobile company in North America
=
*Productivity by the Objectives Matrix Oregon Productivity Center
7/27/2019 Measuring Team Productivity
7/277
O P M
Background| The PPQA group at an IT division of a leading Automobile company
assisted projects in process verification and validation activities
| The teams key activities includedz Process validation,
z Issue of phase approvals,
z
Conducting configuration management audits,z Reviewing work products,
z Mentoring projects,
z Facilitating root cause analysis meetings with Project teams
| The Challenge:z With multiple type of activities and multiple measurement parameters, the
teams challenge lay ahead in identifying a single indicator for measuringteams productivity, showing continual improvement, and establishing aperformance measurement system
QA
7/27/2019 Measuring Team Productivity
8/278
O P M
The Solution| The team agreed that the solution was to establish a concrete method
for measuring the PPQA teams productivity, particularly as it relates tothe team's continuous improvement indicators
| Based on a thorough literature search we evaluated that OregonProductivity Matrix (OPM) best fits this scenario
| In addition, we had experiences in the team of successfullyimplementing OPM in an operations management area to indicate asingle performance indicator of all operations
7/27/2019 Measuring Team Productivity
9/279
O P M
OPM History| Developed by the Oregon Productivity Center at Oregon State University in 1986,
(Productivity By the Objectives Matrix (OMX) Handbook-1986)z the OPM was first introduced at Boeing with a pilot team in 1990(Defense and Space
facility, Corinth)
| Similar to Balanced score card* with an exception that the OPM can beimplemented at a shop floor level without getting into strategic perspectives of theorganization
| Has been applied in various industries. Is immensely versatile and adaptable to
various applications in any setting like:z Hardware Network productivity ITz Office productivity (administration)- Servicez Sales productivity - Fabricationz Production control- Food Processingz Purchase-Fabricationz Accounting -Electronics
z Human Resources Bottlingz Planning and Development- Governmentz Police Governmentz Team productivity- Aircraftz Foreclosures- Mortgage Company
*Balanced Scorecard is method developed by Kaplan and Norton that enables organizations to clarifytheir vision and strategy and translate them into action
7/27/2019 Measuring Team Productivity
10/2710
O P M
Advantages of OPM| Flexible
z Goals can be agreed to be adjusted based on continuous improvementz Allows for multiple measurement categories where baseline and current
performance levels are depicted
| Standardizedz A valuable method for standardizing team performance by the use of an index
that can be uniformly interpreted throughout the organization
| Versatilez Can be used in a variety of work environments to establish a performance
measurement system
| Prioritizesz Weights are established for each of the performance areas, thus focusing the
team on the most important areas to improve
| Team orientedz Goals are set by the teams
z Teams are the users and take ownership| Rolled-Up
z May be rolled up to an organizational level with multiple OPMs
| Visualz OPM graphically displays team performance against goals
7/27/2019 Measuring Team Productivity
11/2711
O P M
The OPM GridCategories Point Score Goal
10.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00
Goal
Limits Point Weight Total
Performance
Category 1 100.0 99 98 97 96 95 93 91 87 83 82 94% 25
Performance
Category 2 100.0 98 96 94 92 90 88 86 84 82 80 100% 15
Performance
Category 3 100.0 99.5 99 98 97 96 94 92 90 88 85 98% 10
Performance
Category 4 100.0 99 98 98 97 96 95 94 93 92 90 97% 30
PerformanceCategory 5 100.0 99.5 99.0 98.5 98 97 96 95 94 92 90 98% 20
Point ScoreTable
4
Weights3
Goal Limits2
PerformanceCategory
1OPM may be seen as a grid in which the differentPerformance categories(which are monitored using quantitative metrics data) are depicted vertically
along the grid. Goal Limits are identified for each category. Weights each ofthe Performance Category are established to identify importance. A PointScore table is used to earn pointsof performance.
7/27/2019 Measuring Team Productivity
12/27
12
O P M
When Rubber Hits the
Road
(Step 8)Analysis
(Step 1)
Identify Team
Goals
(Step 2)
Define &Validate
Performancecategories
(Step 3)
Set Performance
Goal Limits
(Step 6)
Identify Scores for
Actual Performance
(Step 7)
Arrive at OPM
Index
OPM 2
3
4
5
6
7
8
1
(Step 5)
Establish Performance
indicators(Step 4)
Priori tize Categories
by assigning weights
8
7/27/2019 Measuring Team Productivity
13/27
13
O P M
OPM 2
3
4
5
6
7 1
Identify Team Goals| Identification of the team goals is the first activity towards establishing
any performance measurement systemz Understand expectations from managementz Align the goals with Business goals and objectivesz Brainstorm with team
| What are we trying to achieve?
G1
Achieve consistency in PPQA team performance
G2 Change perception of PPQA from inspectors to mentors
G3 Improve process compliance performance in projects
8
7/27/2019 Measuring Team Productivity
14/27
14
O P M
OPM 2
3
4
5
6
7 1
Define and Validate
Performance Categories| Use the Team goals to arrive at
performance categories for
monitoring| The PPQA team used the Goal
Question Metric (GQM) paradigm * asa method to identify the measurementcategories or performancecategories
| Identify Performance categories thatindicate what the team does
A family of measures thus created can reflect the work of the team in a way no single measure can.
*GQM method was originated by V.Basili and D.Weis
G1 G2 G3
Q1 Q2 Q3 Q4 Q5 Q6 Q7
PerformanceCategory1(Metric1)
PerformanceCategory 2(Metric 2)
PerformanceCategory 3(Metric 3)
PerformanceCategory 4(Metric 4)
PerformanceCategory 5(Metric 5)
PerformanceCategory 6(Metric 6)
Team Goals
Questions derived from goals thatmust be answered in order todetermine if the goals are achieved
Measurement Categories that provide
the most appropriate information foranswering the questions
8
7/27/2019 Measuring Team Productivity
15/27
15
O P M
OPM 2
3
4
5
6
7 1
Define and Validate
Performance Categories1 Performance
Category
Performance
Categories*
Metric
Phase approvals On-Time
Number of approvals issued ontime/Total number ofapprovals issued *100
CM Audits On-Time Number of CM Audits conductedon-time/Total number of CMAudits Planned *100
QA Report Quality % Compliance score based onDefects found from review ofQA Reports in a month
ImprovementEffectiveness
Overall Score Improvement
Index: Total compliancequestions from all
projects/Total applicablequestions in month for allprojects expressed in %age
Meeting Compliance Number of Meetings conducted inthe month /Scheduledmeetings*100
| Validate that these categoriesencompass the range of inputs
used: labor, materials, capital andequipment, and the output factors ofquantity, quality, timeliness and cost
| Validate that each performance
category is positive driven and ismeasured in percentage
z i.e. each metric is measured forcomplianceand not defects or noncompliance
*Note: these are samples and do not cover all categories for a PPQA function
8
7/27/2019 Measuring Team Productivity
16/27
16
O P M
OPM 2
3
4
5
6
7 1
Set Goal Limits1 Performance
Category
2 Goal Limits
Performance Categories
Goal
Limits
Phase approvals On-Time 94%
CM Audits On-Time 100%
PPQA verification Run Quality 98%
Improvement Effectiveness 97%
Meeting Compliance 100%
| Assess Current level of Performance
z Historical data to generate the initial
OPMz Conduct pilot
| Review and analyze with team
| Establish the goal limits for eachperformance category with team
consensus
This step is useful in itself, as people and work groups should have targets for quality of service so that they know the
standards of work that they are expected to achieve.
7
8
1
7/27/2019 Measuring Team Productivity
17/27
17
O P M
OPM 2
3
4
5
6
7 1
Prioritize the Categories| Assigning importance weights (iw)
for each Performance Category based
on the value of the output summing upto 100
| How to assign weights for aperformance Category?
z Take expert judgment based on
experiences and alignment to businessobjective
z Take an simple average based on asurvey
z May be jointly established with theteam and management representativesto ensure alignment to the Businessvalue.
Performance Categories Weight
Phase approvals On-Time 25
CM Audits On-Time 15
PPQA verification Run Quality10
Improvement Effectiveness30
Meeting Compliance 20
100
(Analysis of the following for each category providesthe key to decide the weight )
o Who uses the services or deliverable ?o What is value of this delivery?
o How is performance of each of the services/deliverable affected?
o What is impact of wrong service or deliverable?
1 PerformanceCategory
2 Goal Limits
3 Weights
Weighing by ExpertiseWeighing by CompromiseWeighing by Consensus
Aligning the teams service more closely to business goals and objectives will promote significant productivity gains by
eliminating unnecessary work.
7
8
1
7/27/2019 Measuring Team Productivity
18/27
18
O P M
OPM 2
3
4
5
6
7 1
Establish Performance
IndicatorsPoint Score
10.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00
100.0 99 98 97 96 95 93 91 87 83 82
100.0 98 96 94 92 90 88 86 84 82 80
100.0 99.5 99 98 97 96 94 92 90 88 85
100.0 99 98 98 97 96 95 94 93 92 90
100.0 99.5 99.0 98.5 98 97 96 95 94 92 90
Point ScoreTable
4
Weights3
Goal Limits2
PerformanceCategory
1
| May be achieved by establishing a point score table
| Point score table as seen in the figure is a range of values fragmented horizontallyin varying percentage fragments between two limits and each percentage indicatinga certain pointearned in a scale of 0 and 10
| The lower limit of this target range is the minimum level of service that can bepermitted or expected called as the minimum expected performance score(meps). The maximum range or the upper limit is fixed at 100% (Optionally, themaximum scale can be the goal score also)
| Intermediate values are determined for scores between these ranges based on past
experiences and adjusted based on pilots
The targets goals levels may be set for each performance area either in terms of in terms of percentage or point
score values.
7
8
1
7/27/2019 Measuring Team Productivity
19/27
19
O P M
Establish Performance
Indicators
OPM 2
3
4
5
6
7 1
Categories
10.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00
Goal
Limits Point Weight Total
Phase approvals On Time 100.0 99 98 97 96 94 93 91 87 83 82 94% 5 5 25
CM Audits On-Time 100.0 98 96 94 92 90 88 86 84 82 80 100% 10 5 50
PPQA Verification r un Quality 100.0 99.5 99 98 97 96 94 92 90 88 85 98% 7 10 70
Improvement Effectiveness 100.0 99 97 96 95 93 91 89 86 83 80 97% 6 35 210
Meeting Compliance 100.0 99.5 99.0 98.5 98 97 96 95 94 92 90 98% 6 20 120
Performance Category 100.0 98 97 96 95 94 93 91 89 87 85 95% 6 1 6
Performance Category 100.0 98 97 96 95 94 93 91 89 87 85 95% 6 2 12
Performance Category 100.0 98 97 96 95 94 93 91 89 87 85 95% 6 2 12
Performance Category 100.0 99.5 99 98 97 96 95 94 93 92 90 97% 6 20 120
meps 100 625
(iw)
Point Score Goal
GOAL
INDEX
| Establish a grid with performance categories, point score table goal limits and assigned weights
| The goal limits of each performance category corresponds to a point score. The product of point scoreand weight gives the total goal scores.
E.g. :z in the first row: 94% goal limit corresponds to 5pointsz Point score (5) X assigned weight (5) = 25 ( Total goal limit score for category Phase approvals On time)
| The sum of point scores based on the individual goal limits gives a single number that is an aggregate,weighted index called Goal Index
7
8
1
7
8
1
7/27/2019 Measuring Team Productivity
20/27
20
O P M
OPM 26
3
4
5
7 1
OPM 26
3
4
5
7 1
Record Actual Scores
| The actual measurement scores for each of the performance categories are recorded andtranslated as pointsearned from the point score table
z This was done using a specific reporting and consolidation template in a spreadsheet designed to do thenumber crunching of all the measurement attributes and arrive this table.
| The Performance score for each performance category is then a weighted score of the point
| The sum of all performance category point scores gives a single number that is an aggregate,weighted index called OPM index
E.g. :z
in the first row: 83% Actual score of Phase approvals On-Time corresponds to 1 point earned from the
OPMINDEX
Ver 4
Categories
10.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00 GoalLimits Point Weight Total Actual Poin ts Weight
Perf
Score
Max
Possible
Phase approvals On Time 100.0 99 98 97 96 94 93 91 87 83 82 94% 5 5 25 83% 1 5 5 50
CM Audits On-Time 100.0 98 96 94 92 90 88 86 84 82 80 100% 10 5 50 98% 9 5 45 50
PPQA Verification run Quality 100.0 99.5 99 98 97 96 94 92 90 88 85 98% 7 10 70 97% 6 10 60 100
Improvement Effectiveness 100.0 99 97 96 95 93 91 89 86 83 80 97% 6 35 210 95% 4 35 140 350
Meeting Compliance 100.0 99.5 99.0 98.5 98 97 96 95 94 92 90 98% 6 20 120 91% 1 20 20 200
Performance Category 100.0 98 97 96 95 94 93 91 89 87 85 95% 6 1 6 97% 8 1 8 10
Performance Category 100.0 98 97 96 95 94 93 91 89 87 85 95% 6 2 12 98% 9 2 18 20
Performance Category 100.0 98 97 96 95 94 93 91 89 87 85 95% 6 2 12 95% 6 2 12 20
Performance Category 100.0 99.5 99 98 97 96 95 94 93 92 90 97% 6 20 120 88% 0 20 0 200
meps 100 625 100 308 1000(iw)
Actual
OPM Measurement ReportMonth1
Point Score Goal
Performance ScoreActual Scores
tablez Point score (1) X assigned weight (5) = 5 (Total performance score for the category Phase approvals On
time)z Sum of all performance scores =308 = OPM Index for the Month 1
7
8
1
7/27/2019 Measuring Team Productivity
21/27
21
O P M
OPM 2
3
4
5
6
The Moment of Truth
PPQA OPM Dashboard
395 410
485
415
515 515
635 645 650685 685
318285
308
0
100
200
300
400500
600
700
800
9001000
Oct-04 Nov-
04
Dec-
04
Jan-05 Feb-05 Mar-
05
Ap r-05 May-
05
Jun-05 Jul-05 Aug-
05
Sep-
05
Oct-05 Nov-
05
Dec-
05
GOAL LINE
STRETCH GOAL LINE
Calibrate(initial)
Calibrate(yearly)
(Optional)
| The OPM index trend indicates the Productivity line and Goal index gives theGoal Line.
| The OPM indices (productivity line) may be compared to the Goal line tomeasure performance and demonstrate improvement in productivity
7
8
1
7/27/2019 Measuring Team Productivity
22/27
22
O P M
OPM 2
3
4
5
6
Analyze OPM
| Initial ( Pilot Analysis)z Calibrate OPM based on pilot
z Adjust goal limits, point scores or weights
| Monthlyz Team analyses to measure productivity improvements
z Causal Analysis and discussion on improvement areas
z Elimination of assignable causes
| Yearlyz Root cause analysis to identify process improvements
z Calibration and Program corrections
Questions/Symptoms Actions Recommended
Change in scope of activities? Perform GQM
Do we have the right measures? Review performance categories
Performance goal is consistently not achieved Review Goal limits
Consistent 100-percent-of-goal performance on aperformance category
(or becomes the current level of performance)
Review Goal Limits, Point Score, Weights
Re evaluate performance category
7/27/2019 Measuring Team Productivity
23/27
23
O P M Avenues to Improving OPM
| OPM is a process completely driven by the team from selecting the performancemeasurement areas to actually improving them. Implementation successes may
be replicated at other teams in the organization
| The point score table may be embellished with more objective sub- criteria ofperformance (0-10 levels)
| It is possible to combine all the team OPMs into a single OPM for the entireorganization
| Once piloted and proven, build it as a part of your quality system
7/27/2019 Measuring Team Productivity
24/27
24
O P M Lessons Learned
| Increases team ownership for improvement (Process, Product, Productivity)
| Leave it to the practicing team to analyze and improve
| Initially coming to consensus with goals and getting acceptance from all teammembers may be slow, but its worth the time. The process accelerates upon teamagreement and involvement
| Use it as a process to improve the performance areas and not as a end in itself
| Be open to continuous improvement based on root cause analysis
| Productivity is better measured by overall value delivered per unit effort/cost thanby quantity delivered per unit effort/cost
7/27/2019 Measuring Team Productivity
25/27
25
O P M CAUTION !
| DO NOT use OPM to tie the results to any form of reward
system (or worse penalizesystem)z This does not yield good results in long term
z The team may not necessarily focus on the right performancecategories for improvement
| DO NOT use OPM to foster competition amongst teams
z The objective is to improvenot to prove
|
DO NOT use the tool to identify whois the problemz Let the team decide where is the problem and improve it
7/27/2019 Measuring Team Productivity
26/27
26
O P M References
| OSU Archives - Oregon Productivity Center Records (RG 219)
z OMX Handbook
| Productivity by Objectives : By J ames L. Riggs, Glenn H. Felix
| The Use of Performance Metrics as a Strategic Management Tool -Terry R. Collins, Manuel D. Rossetti and J ulie A. Watson paper,University of Arkansas
| Rose, K. (1995) A Performance Measurement Model.. Quality Progress,V28, N2, 63-6
| R.Solingen, E.Berghout: "The Goal/Question/Metric Method" McGraw-Hill Publishing Company, 1999
7/27/2019 Measuring Team Productivity
27/27
O P M
The primary objective of any process improvement activity is to improveand not just prove
Thank you
586-354-4813
mailto:[email protected]:[email protected]