© 2015 Cogentus Consulting Ltd

The human element in prioritizing R&D projects

2

People vs. Process

3

People vs. Process “Soft” vs. “Hard”

4

People vs. Process “Soft” vs. “Hard” Managing contradictions

5

People vs. Process “Soft” vs. “Hard” Managing contradictions •  A bit about me •  The Case Study

•  US Department of Energy (DOE) •  Prioritization •  Process •  Issues •  Lessons Leaned

•  Summary •  Questions

Agenda

6  

Ian Seed  

7  

Ian Seed, CEO Cogentus  

8  

Chemical Engineer  

Ian Seed, CEO Cogentus.  

9  

Chemical Engineer MBA  

Ian Seed, CEO Cogentus.  

10  

Chemical Engineer MBA Technical, financial and business positions  

Ian Seed, CEO Cogentus.  

11  

Chemical Engineer MBA Technical, financial and business positions Heavily involved in OR for most of my career  

Ian Seed, CEO Cogentus.  

12  

13

Case Study

14

140 R&D projects

n  US DOE, R&D to support nuclear clean up n  $120M a year budget, 4 departments n  Multi-discipline, complex wide

n  Problem: More R&D projects than funding

n  Solution: Resource Allocation

About the case study

15

n  Each R&D project will cost money and will deliver some benefits to DOE.

n  We just need to work out the optimum combination of projects that will give DOE the best benefit for their budget.

n  Used MCDA – specifically resource allocation n  Criteria n  Options n  Weights n  Scores

Methodology

16

n  Criteria are the factors against which options will be judged

n  We develop criteria from the hierarchy of objectives n  Line of Sight / Strategic alignment

n  DOE has mission & vision so it should be easy to identify criteria

n  No it isn’t!

Step 1 - Criteria

17

n  Some not aware of what has been published

n  What do the words really mean?

n  How do you measure? n  Differences amongst

stakeholders n  Inside team n  Outside team n  Outside DOE

Issue 1 – Leadership Team Knowledge

18

n  Do not assume that previously published and approved information has been read

n  Expect to revisit objectives and success criteria n  Strategy development session with Lead Team. n  Useful and necessary but not really “prioritization”

n  Expect that different stakeholders will have very different views on objectives and success criteria n  Use multiple sessions or role play to elicit those

views

Criteria – Lessons Learned

19

n  Options are the things you want to compare. n  In this case, options are the R&D projects that

people want to do. n  All they need to do is to explain their proposed

project so it should be easy to identify options. n  No it isn’t.

Step 2 - Options

20

n  Wide variation in “quality”. n  Might have a meaningful title if we’re lucky

n  Variable descriptions n  Technospeak n  Vague

n  Different stages of maturity of thinking

Issue 2 - Options

21

n  Standardise quality n  Each option (project) should be at the same level of

detail n  Template n  Technical Support n  Peer reviews

n  In department n  Across departments

n  Maturity index

Options – Lessons Learned

22

n  Yet to meet anyone who knows what weighting actually means.

n  Most believe it is “importance” n  But all criteria, if aligned to the strategy, are equally

important. n  It’s all about equalizing scale lengths.

n  Lessons Learned n  Expect to have to re-educate everyone what

weighting means!

Step 3 – Weighting

23

n  Scoring is where the performance of each option is established on each criterion

n  Most participants expect a “scoring workshop” n  Consensus decision making n  Expert knowledge n  A good thing right?

n  Not particularly!

Step 4 – Scoring

24

n  Experts aren’t expert on things outside of their expertise n  That’s all criteria except one (if we’re lucky)

n  Scores are a matter of opinion, not fact n  There’s no evidence provided

n  Workshops are subject to Groupthink n  Groupthink, gaming, influencing

n  Impossible to validate scores n  Different groups on different days will provide

different scores

Issue 4 – Scoring

25

n  Collect the evidence beforehand n  Datasheets for each option n  Performance data for each criterion n  Criteria stage will have defined relevant measures

n  Peer Review and validate data n  Use workshop for experts to investigate the

data on each option, but not to “score” n  Datasheets provide the evidence-base

necessary for any future reviews n  This gives robustness

Scoring – Lessons Learned

26

Step 4 – Results

27

Compare portfolios. Can we develop ones that have greater benefit to the Organization for the same, or less budget?

Step 4 – Results

28

Balancing. Cash Flow – is the annual budget within required limits?

n  Departmental Heads prioritise on benefits only n  Cost is secondary n  Huge disparity between opinion and “fact”

n  People cannot process multiple datasets n  Trading off benefits and costs n  Actual performance of options against criteria n  Lifetime cost vs. annual costs n  Balancing across departments n  Short terms vs. long term, big vs. small

n  That’s why we do multi-criteria!

Issue 5 – Results

29

n  Need substantial time to analyze n  People need to unravel their own perceptions n  Why are their favorite projects not more of a priority? n  What about projects already started?

n  Lewin’s is a good model to use n  Unfreeze, Change, Refreeze

Results – Lessons Learned

30

31  

Who is the DM?

n  The “process” is straightforward. n  People make it more difficult.

n  More people generally means more difficult. n  Every stage has its issues. n  Lessons learned to address most of them n  Still left with: Who is the DM?

n  Challenge is to present and communicate. n  Not models, networking and negotiation. n  No such thing as “optimal”. n  Good enough. Needs to be better than the others!

Summary

32

33

Who is the DM?

Erroll Southers n  Counter Terrorism Expert. n  FBI Special Agent. n  Deputy Director for California

Office of Homeland Security, appointed by Governor Schwarzenegger.

n  Nominated by President Obama to lead the Transportation Security Administration.

n  "In my 30+ years of experience, Cogentus facilitated the most effective brain-storming session I have attended. It was challenging and produced viable future considerations for a major interdisciplinary initiative. Excellent use of time and expertise!”

People can be very nice!

34

35  

Ian  Seed    

iseed@cogentus.co.uk  0787  282  5027  

www.cogentus.co.uk  

Ques=ons?