Enterprise Services Planning: Defining Key Performance Indicators

[email protected] @leankanbanu Copyright Lean Kanban Inc.

Learn to care about what the customer

cares about

KPIs should shape improvements to service delivery

Enterprise Services Planning

Defining Key Performance Indicators

Presenter

David J. Anderson

Swift Kanban

Webinar

29 April 2015


Lean Kanban North America 2015 conference

• “Back to our roots”

• Implementing Kanban

• Looking to the future…

• Enterprise Services Planning


• 2 Days of Learning Sessions

• Choose Your ½ Day Workshops

• Risk Profiling – David J Anderson

• Cost of Delay – Don Reinertsen

• Project Management with Kanban

• Cynefin 101 for Portfolio Kanban

• Simple Probabilistic Forecasting

…and more!

• Becoming Data-Driven

• Objective Retrospectives

• Forecasting

• Enterprise Kanban & Lean Startup

• Scrumban

• Kanban Coaching

• Blockers for Improvement

• RBS Project Sizing

• Kanban Academic Research …


Kanban experience reports

Ultimate

Software

• BazaarVoice

• web recommendations

app and mobile development

Including…




Fitness for Purpose


Which system is fitter?

We don’t know!

System B is faster but without understanding customer expectations, both may be fit enough

0

2

4

6

8

10

12

14

Lead Time (Days)

System A

Frequency

0

5

10

15

20

25

30

5 10 15 20 25 30 More

Lead Time in Days

System B

Frequency

Mean 17 days Mean 12 days


Measuring delivery against expectation

0

2

4

6

8

10

12

14

Lead Time (Days)

System A

Frequency

0

2

4

6

8

10

12

Lead Time Expectation Spread (Days)

System A

Frequency

0

5

10

15

20

25

30

5 10 15 20 25 30 More

Lead Time in Days

System B

Frequency

0

5

10

15

20

25

30

35

40

45

-15 -10 -5 0 5 10 15 20 More

Lead Time Expectation Spread (Days)

System B

Frequency

Mean 17 days Mean 12 days

System B is clearly fitter!

System B delivers 5/7 within expectations

System A only delivers 3/7 within expectations


What makes a pizza delivery service“fit for purpose” ?

Fitness criteria are metrics that measure things customers value when selecting a service again & again

Delivery time

Quality

Predictability

Safety (or conformance to regulatory requirements)


Meet Neeta - a project manager

Neeta’s team are working late (again)

Neeta needs to feed them with pizza

What attributes do her team care about in a pizza delivery service?

• Delivery time =approximately 1 hour

• Non-functional quality =tasty & hot

• Functional quality (order accuracy) =doesn’t matter if small mistakes are made, geeks will eat any flavor of pizza

• Predictability =+/- 30 minutes is acceptable

• Safety =so long as health & safety in food preparation is good, it’s fine


Neeta is also a working mom!

Neeta gets home late. Her kids are really hungry and even though she shouldn’t she decides to order pizza for them

What makes a pizza delivery service acceptable to her kids age 4, 6, 9 & 11 years?

• Delivery time =20 minutes

• Non-functional quality =doesn’t matter too much, it’s pizza!!!

• Functional quality (order accuracy) =it must be cheese pizza! No other flavor is acceptable! (even if you take the pepperoni off)

• Predictability =+/- 5 minutes maximum!!!

• Safety =only mommy worries about that stuff!


To be “fit for purpose” there is a product component & a service

delivery component

We need to offer a selection of different recipes which are tasty & popular. However, we must

also deliver with speed & predictability

Lesson


Modern creative & knowledge worker businesses often obsess

with product definition & strategyOperational excellence and service

delivery excellence are often overlooked or treated as inferior

management skills

Lesson


Neeta has 2 identities –Mother and Project ManagerEach of Neeta’s identities

represents a different market segment for the pizza delivery

service

Traditional demographic & income group segmentation does

not accurately capture the context to understand

“fit for purpose”

Nor, for that matter, do personas. As Neeta represents

two segments not just one persona

Lesson


Exercise – Understanding Fitness for Purpose

Pick a service with which you are familiar

Consider what makes it “fit for purpose?”

Which attributes make you select the service, again and again?

What are your expectations for each attribute? Why?

What (business) risks drive your expectations?


Fitness Criteria DriveEvolutionary Change


Fitness criteria are metrics that measure observable external outcomes

Fitness criteria are metrics that measure things customers or other external stakeholders value Delivery time

Quality

Predictability

Safety (conformance to regulatory requirements)

or metrics that qualitatively assess actual outcomes such as customer satisfaction

employee satisfaction


Evolutionary change has no defined end point

EvolvingProcess

Rollforward

Rollback

InitialProcess

Future process is emergent

EvaluateFitness

EvaluateFitness

EvaluateFitness

EvaluateFitness

EvaluateFitness

We don’t know the end-point but we do know our emergent

process is fitter!


Business Risks, Fitness Criteria & Classes of Service should all align

Business risks are things which are uncertain that affect the performance of our business such as nature of demand, consistency of supply, delivery predictability, seasonal windows of opportunity, time value of money

Classes of service offered should align with business risks

Metrics used to evaluate service delivery capability should be fitness criteria that are derived from specific business risks

For example, opportunity cost of delay requires us to measure lead time and understand sensitivity to schedule uncertainty


Select Key Performance Indicators Carefully!

KPIs should be fitness criteria metrics with threshold values that represent “good enough” – the level where the service delivery is “fit for purpose”

KPIs should assess service delivery capability and indicate fitness for purpose. In doing so, a KPI indicates your likelihood of success – of surviving and thriving - by adequately satisfying your customers?

KPIs should be recognizable by your customers as something meaningful!

If your customer doesn’t recognize the metric it isn’t a “key” “performance” indicator, it is some other kind of metric


Other Useful Metrics

Some other metrics are useful

Those which guide improvements

Those which indicate general health

Is your metric evaluating and guiding a specific change to improve fitness of your business such as an initiative to improve vendor response times?

Or, is it a general business health indicator such as liquidity?

If neither of these, then it is a metric that you almost certainly don’t need! It should be removed!


Defining Fitness Criteria


Market Adoption Lifecycle Segmentation

Enthusiasts EarlyAdopters

EarlyMajority

LateMajority

Laggards

Rate

Of

Mark

etAdop

tion

time

Moore’sChasm

LittleChasm

HipCool

BuggyCommunitydevelopment

NicheMarket

Features

Good func qualityAdequate non-func

quality

PermissionGiving

Early adopter

Exceptional func and non-func quality

CostEffectiveBroad Features

Exceptional func and non-func quality

Low CostEasy Access

Forced adoptionViewed as

taxation

Fit for purpose Fit for purposeChanges over time


Customer Storytelling & Clustering

Tell stories about real customers, their motivation, what they buy and why. Cluster similar stories

Give each cluster a “nickname”e.g.

• “All ins”• “Aspirationals”• “Bet hedgers”• “Boy scouts”


You can’t just ask!

Neeta, how fast would you like your pizza delivered? How predictable do you need us to be with our delivery estimate?

Customers will tend to tell you they need better service and more features than they

really need!

Would you pay more for the things you say you

need and want?

No, probably not!

Believe what customers actually do, do not believe what they say they’ll do!

Actually behavior will vary from declared intent!


Who knows your customers?

Front line staff

Those who take and those who deliver orders

Those who provide “customer care”

Often the lowest paid staff in a business

Often the highest turnover, shortest tenured positions

And yet, they have the vital information that enables the business to survive, thrive and compete


Pizza boy knows Neeta’s Story!

Staff who meet customers can be trained to learn what matters to them and why

Create ways to capture customer stories or directly involve customer facing staff

when defining customer segments fitness criteria


GT car manufacturer story

A well known manufacturer of GT cars determined customers were prepared to wait 21 months to take delivery

They learned this by letting delivery time slip to 27 months and receiving cancellations and customers switching to a rival

manufacturer

Determining fitness criteria thresholds by

reducing service levels until customer complaints rise to dangerous levels isn’t a “safe to fail” approach!

Damaging your brand, your reputation and your profitability is a strange way to discover how to

be…

“fit for purpose!”


Retarding customer service until customers complain vehemently or take their business elsewhere could be damaging

• Undermines brand• Damages reputation• Loss of market share• Loss of revenue

Probing for threshold values by reducing service quality isn’t “safe to fail”

Is it “safe to fail”?

We need general guidance that allows us to probe for fitness criteria threshold values that is “safe to fail”

If we can’t ask, and we can’t allow service to decline until

complaints make the threshold evident, what can

we do?


Probing for Threshold Levels


Look for clusters or patterns of demand, or

patterns of similar expectations, or new

sources of demand that may represent an emerging segment

Probe with classes of service

Create a class of service to respond to the believed new segment

• Set service levels at or close to anticipated threshold levels


Probe with classes of service

Observe take up of class of service

• Is it over-used? (or abused?) If so tighten qualification criteria

• Is it under-used? Consider removing it

• Is it used but you fail to deliver to expectations? Do people complain? If no then consider removing it. You are over promising

Fixed delivery date class of service emerged this way.

Initially abused by marketing, eligibility

criteria were tightened up.


Telecom Equipment Example

A platform maintenance department at a telecom equipment manufacturer

receives demand only from internal application

departments…

… Each request is tagged with the originating telco operator for whom the request is being implemented. Each operator is given a lane on the kanban board


Telecom Equipment Example

Imagine 3 American telco operators with different strategic positions…

• Verizon value quality most• Sprint value time-to-market• Voicestream/T-Mobile USA values low

cost

Now design and offer 3 classes of service…

• High quality, tight “done” criteria for each step

• Short lead time – pull priority, looser “done” criteria

• Low cost – junior staff, lowest priority compared to other work


Different lanes, different risks

Done

F

E

I

Engin-eeringReady

Deploy-mentReady

G

D

GY

PB MN

10 ∞

P1AB

Ongoing

Development Testing

Done Verification Acceptance10 10

Verizon 10

10

Sprint

T-Mobile

10

DE

DA

Each lane represents a different source of demand but also different fitness criteria and threshold values

Different classes of service and different pull criteria policies are defined for each lane providing service levels tuned to the “fitness for purpose” expectations of each customer


To have confidence you are offering a service that is “fit for

purpose”, you must offer different classes of service

To serve more than one market segment adequately, you must offer a selection of classes of

service

Lesson


Classes of service should align to market segments and fitness

criteria (or stakeholders needs)

Lesson

KPIs cannot be general! They need to be tied to customer expectations. Different segments have different

expectations. Hence, different threshold levels of the KPI


Classifying Metrics


85% at10 days

Mean5 days

98% at25 days

Change

Req

ues

ts

Pro

duct

ion D

efec

ts

85% at60 days

Mean 50 days

98% at150 days

Median 45 days

Lead Time Distribution is a KPI


Blocker Cluster Data Guides Improvements

Harvested blocker tickets over a 1 month period

Cluster blockers based on the stories behind the delay

Each cluster represents a risk

Identify Likelihood, Total Impact & Average Impact

Identify whether occurs in the tail of the lead time distribution

Define risk reduction & mitigation actions

http://www.klausleopold.com/2013/09/blocker-clusters-problems-are-not.html


Avg. Lead Time

Avg. Delivery RateWIP

Poolof

Ideas

ReadyTo

Deliver

Cumulative Flow is a General Health Indicator


TestReady

Flow Efficiency is a General Health Indicator

F

E

I

GD

GYPBDE MN

P1

AB

Customer Lead Time

Waiting Waiting WaitingWorking

IdeasDev

Ready

5Ongoing

Development Testing

Done

3 35

UATReleaseReady

∞ ∞

Flow efficiency % = Work Time x 100%

Lead Time

Working WaitingWorking


Liquidity is a General Health Indicator

The volume of pull transactions in a kanban system defines its liquidity


Volatility is a General Health Indicator

The derivative of liquidity shows us kanban system volatility


Wrap Up


ESP Training Modules

Module 1 Portfolio Management (Day 1 & 2) Strategy, fitness for purpose, KPIs, Cost of Delay, Scheduling,

Sequencing, Portfolio risk, risk hedging, risk profiling, aligning strategy & capability, strategy review

Module 2 Enterprise Services (Day 3) Understanding kanban systems, real options, upstream Kanban,

commitment & replenishment, lead time, chance vs assignable cause variation

Module 3 Project & Demand Management (Day 4) Demand analysis, demand shaping, capacity planning, project

forecasting, risk review, labor pool liquidity, workflow liquidity

Module 4 Real Options, Portfolios, Programs, Dependencies & Scaling(Day 5) Scaling Kanban, dependencies, visualizing dependencies, portfolio

Kanban, stand ups, service delivery reviews, ops reviews


ESP Training

ESP training is delivered on-premises with clients around the world

To order an ESP training class, contact Wes Harris, Commercial Director of David J. Anderson & Associates, [email protected]


Thank you!


About

David Anderson is an innovator in management for 21st Century professional services businesses. He leads a training, consulting, publishing and event planning company dedicated to developing, promoting and implementing new management thinking & methods…

David has 30+ years experience in the high technology industry starting with computer games in the early 1980’s. He has led software organizations delivering superior productivity and quality using innovative methods at large companies such as Sprint and Motorola.

David defined Enterprise Services Planning and originated the Kanban Method an adaptive approach to improved service delivery. His latest book, published in June 2012, is, Lessons in Agile Management – On the Road to Kanban.

David is also Chairman of Lean Kanban Inc., a business operating globally, dedicated to providing quality training & events that bring Kanban and Enterprise Services Planning to a broad audience of professionals around the world.


Leadership & Management

Enterprise Services Planning: Defining Key Performance Indicators