Embed Size (px)
Citation preview
1
EVIDENCE-BASED HR
WHAT IS IT AND IS IT REALLY HAPPENING YET?
Rob B Briner
CEBMa center forEvidence-Based Management
2
Most stuff available www.cebma.org
2
3
Objectivesu What is evidence-based practice in HR?u Why all practitioners need evidence-based
practiceu How evidence-based is HR?u Mini case of how evidence-based approaches
can lead to more effective decisions
3
4
What is evidence-based practice for?u To help with this big problem: Practitioners
in all fields use evidence (or information) in their decision-making
u BUT the evidence used is often limited– Not enough of it– Not from/of multiple and diverse sources/types– Not critically appraised for its quality or relevance– Not used in a systematic way (e.g., not focusing
on the best available evidence, not weighting, not summarized)
4
5
The logic of evidence-based practiceu Using limited quantities, sources, and types
of evidence in an uncritical and unsystematic way will increase the chances of poor decision processes and unintended outcomes
u Using greater quantities, sources and types of evidence in a critical and systematic way will increase the chances of better decision processes and intended outcomes
5
6
What is it EBMgt?Evidence-based practice is about making decisions through the conscientious, explicit and judicious use of the best available
evidence from multiple sources by
1. Asking: translating a practical issue or problem into an answerable question2. Acquiring: systematically searching for and retrieving the evidence3. Appraising: critically judging the trustworthiness and relevance of the evidence4. Aggregating: weighing and pulling together the evidence5. Applying: incorporating the evidence in the decision-making process6. Assessing: evaluating the outcome of the decision taken
to increase the likelihood of a favourable outcome.6
7
What is EBMgt?u Making decisions through theu conscientious (you try hard, you are
systematic)u explicit (you write down, show and share
your evidence) andu judicious (you judge the quality of the
evidence) use of theu best available evidence fromu multiple sources (not just one or two) 7
88
9
Example: Evidence-Based Performance Managementu Suppose a senior manager believes that
there is a problem with low employee performance in the organization
u They believe the employees could perform better
9
1010
Example: Evidence-Based Performance Management
Element 1: Professional expertiseu Why do I and my experienced colleagues think there is
low employee performance?u Have I seen this before? What happened?u What are my personal beliefs about the causes of low
performance?u What’s worked to improve performance in the past and
why?u What are my hunches?u What do I think are the causes and possible solutions?u Is this situation occurring elsewhere?u How relevant and applicable is my experience?
1111
Example: Evidence-Based Performance Management
Element 2: Organizational datau What actually is the performance level?u What type of poor performance and where?u What are local explanations for poor performance?u What performance management is currently in place and
is it working?u What do managers think is happening?u What are the possible costs and benefits of
interventions? Is it worth intervening here?u What is happening or what is going to happen that
might be affecting performance?
1212
Example: Evidence-Based Performance Management
Element 3: Scientific research findingsu What are the average levels of employee performance in
my sector and location – is the performance level really ‘low’?
u What does systematically reviewed research evidence suggest to be the major causes of poor performance?
u How relevant and applicable is that evidence here?u What does research evidence from systematic reviews
suggest as effective interventions to increase employee performance?
u How well might the interventions the research describes work here?
1313
Example: Evidence-Based Performance Management
Element 4: Stakeholders’ values and concerns
u How do employees feel about their work performance?u What do employees feel about the proposed
interventions?u Do they see downsides or unintended negative
consequences?u How do managers feel about the performance
interventions?u How practical do those responsible for implementing the
interventions feel they are?u What alternative explanations and proposed solutions for
poor performance do others have?
14
Why is it a particular problem in HR?u It may not be but management and HR has
come very late to evidence-based practiceu The concept is not well-known or understood by
organizations, managers or management schools
u It seems that much HR practice…– is dominated by fads and fashions– not into basing decisions on good quality evidence
from multiple sources– applies ‘solutions’ without identifying problems
u HR is not alone in this…
14
1515
1616
1717
1818
1919
2020
2121
2222
23
24
Why do we need it? Because our decisions strongly shaped by…u Strong beliefs which don’t seem to be trueu General cognitive biases that limit decision-
making and judgementu Management fads, fashions and quick fixesu Power and politics that determine seniorityu Consultants/advisors/fad vendors
24
25
The dangers of holding strong beliefs if they aren’t quite right
u It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so. (Mark Twain)
u The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge(Stephen Hawking)
u Ignorance is fine – false beliefs not so
25
26
Some examples…u What’s happened to average job tenure in
the past 10 years? Up? Down? Stayed same?
u Job satisfaction in past 15 years? Up? Down? Stayed same?
u World moving faster – more VUCA (volatility, uncertainty, complexity and ambiguity)? Faster? Slower? Same?
u Generational differences in work attitudes? Large? Small? None? 26
27
Job tenure UK
27
2828
2929
30
More VUCA?
30
31
Economist: Creed of Speed December 2015u The idea that time is speeding up is clearly
popular. It is also plausible. There is just one problem. It is very hard to prove that it is actually happening.
u Hard evidence of a great acceleration is hard to come by. The Economist has considered a variety of measures by which the speed of business in America can be quantified. A few do show some acceleration. But a lot do not.
31
3232
33
Generational differences in work attitudes?
33
34
Employee engagement: What do we know for sure that just ain't so?1. Engagement is clearly defined2. There are valid and reliable measures of
engagement3. Engagement is different from previous concepts
(e.g., satisfaction, commitment)4. There plenty of good quality evidence that
demonstrates value of engagement• Increases in engagement > increases in performance• Engagement interventions > increases in engagement >
increases in performance
5. Claims made about the importance and role of engagement are reasonable and accurate
34
3535
3636
37
Cognitive biases
A bat and ball cost one pound and ten pence. The bat costs a pound more than the ball. How much does the ball cost?
37
38
Error and biases in problem-solving and decision-making – some examplesu Confirmation bias: Tendency to interpret and search for
information consistent with one’s prior beliefsu Hindsight bias: Tendency to see past events as being more
predictable than they were before the event occurredu Loss aversion: Tendency to prefer avoiding losses than to
acquiring gainsu Anchoring effect: Tendency to rely too heavily or over-
emphasize one piece of information (e.g., restaurant wine lists, large reductions in price in sales)
u Framing effect: Drawing different conclusions from exactly the same information presented in different ways (e.g., would you prefer a ready meal that’s “85% fat free” or “15% fat”?)
u Meta-cognitive bias: The belief we are immune from such biases38
39
Evidence-based practice can help because it’s about the conscientious, explicit, and judicious use of different sources of information
39
40
Example fads and fashionsu Business process re-engineeringu Total quality managementu Quality circlesu Talent managementu Leanu Outsourcing
40
4141
4242
4343
4444
4545
46
Fad and fashion article titles: Miller et al (2004)u Stage 1 - Ascendancy: Total Quality: Wave of the
Future, Reengineering: It’s Totally Radical, Welcome to the Revolution, The Promise of Reengineering, How to Work Wonders, Completely.
u Stage 2 – Maturity: Reengineering: The Hot New Managing Tool, The Reengineering Rage, Warning: This Good Idea May Become a Fad, Reengineering: Beyond the Buzzword.
u Stage 3 – Decline: Ten Reasons Why TQM Doesn't Work, TQM: The Mystique, the Mistakes, The Hocus-Pocus of Reengineering, Why TQM Fails and What to Do About It.
46
4747
48
Related concept of the quick fixu Focus on style and presentation not content or
processu Not be evaluatedu Not be as quick as had been hopedu Not be effective so followed by another quick fixu Become subject to organizational amnesia*u Can be career-enhancing for managers (e.g.,
issue selling, kick-ass CEOs)
*Kitchen equipment analogy
48
4949
50
So why are fads and fashions and quick fixes followed?u Promise to deliver a lot and fastu Appear simpleu New and shinyu Will make everything alrightu Help contain anxieties around intractable problemsu Help user feel effective and cutting edgeu Seems very ‘human’ to want to find quick, easy answers
that other people are adopting too
Evidence-based management not really much of a fad, fashion or quick fix using these criteria
50
51
Evidence-based practice can help because it’s about the conscientious, explicit, and judicious use of different sources of information
51
52
Power and politics as barriers to evidence-based managementu How do people really get ahead and into
positions of power?u Is it by being experts in and advocates of
evidence-based management?
52
53
Espoused and implicit goals of managers that deter them from EBPESPOUSED GOALSu To do what works (but few
evaluations)u To help organization fulfil its
missionu To identify and solve important
problemsu To do what mattersu Treating everyone equallyu Speak truth to power(?)
IMPLICIT GOALSu To get things doneu To further careeru To avoid troubleu To fix political or presenting
problemsu To meet targetsu To do what gets measuredu Favour those who help
advance personal goalsu Say what higher-ups want to
hear
53
54
Evidence-based practice can (maybe) help because it’s about the conscientious, explicit, and judicious use of different sources of information
54
55
What are the incentives for fad-selling consultancies to be evidence-based?u Get the work, get more work, and keep getting
work (so depends almost entirely on what clients want)– Persuade (may not take much) clients they need some
new thing, idea, technique, approach– Sell them the relevant product or service or intervention
based on that idea, technique or approach– Saturate the market until everyone’s bought it– Invent or borrow new ideas, techniques and
approaches clients do not yet use– Sell them the relevant product or service or intervention– Repeat
55
56
So why do managers and HR need evidence-based management?u Because their decisions are shaped by
– Strong beliefs which may not be true– General cognitive biases that limit decision-
making and judgement– Management fads, fashions and quick fixes– Power and politics that determine seniority– Consultants/advisors/fad vendors
56
57
Is there such a thing as evidence-based is HR?
57
5858
5959
6060
6161
62
How evidence-based is HR?u Evidence is always used but…u Too much reliance on
– Professional expertise– Benchmarking/copying– Fads and fashions– ‘Analytics’ alone
u HR professional training does not support evidence-based practice
u Do HR teams have the capability?62
63
Mini-case: Risk culture in global banku Managers believed serious errors caused by
poor ‘risk culture’u Regulator required banks to do something
around risk cultureu What evidence was collected?u What other evidence could have been used?
63
6464
problem solution
Practitionersprofessional expertise
Organizationinternaldata
Stakeholdersvaluesandconcerns
Scientificliteratureempiricalstudies
AskAcquireAppraiseAggregateApplyAssess
65
What evidence was collected?u Some stakeholder evidence (managers and
regulators)u No scientific evidenceu Limited professional expertiseu Little analysis of organizational data (though
much available)
65
66
What was done?u Interviews/working groups around the world
(about 80)u Did not ask whether there was a problem or
what that problem might beu Asked participants what they thought a ‘sound’
risk culture looked like (assumption)u From this developed a survey and framework –
not psychometrically sound not properly validated
u So what would a more evidence-based approach be?
66
67
What organizational data could have been collected?u A thorough analysis of the types and
frequency of errors being madeu A judgement of the quality of the available
organizational datau An analysis of other data that might help
explain the causes of errors (e.g., location, managers, shift, employee training, age, etc)
67
68
What scientific evidence could have been collected?u What is known, generally, about why errors
are made? What is number one cause?u Concept of risk culture not really known –
general view that ‘culture’ is not something that can be measured (no validated surveys) and little agreement on meaning
u However, concept of safety climate well known: Free to use well-validated questionnaires – links to errors across industries (e.g., aviation, medicine, oil, etc)
68
69
Questions already raised in industry about risk culture conceptu McKinsey (2008) analyzed the 'risk culture' at
Goldman Sachs: “A model for all firms … were the firm to stumble it would not be for the lack of a powerful risk-informed culture that combines intelligence, discipline and common sense".
u 2010, Goldman Sachs > the largest penalty ever for a Wall Street firm at the time (500 mil), in a settlement with the SEC concerning misleading "investors in a subprime mortgage product just as the U.S. housing market was starting to collapse" (Securities Exchange Commission 2010).
69
70
What stakeholder evidence could have been collected?u A closer examination of what the regulators
actually wantedu A clearer analysis of what senior managers
really felt the problem to beu A critical appraisal of the quality of this
evidence and what could be taken from it
70
71
How could an evidence-based approach helpu More evidence used to provide a stronger
analysis of the problemu More evidence from a range of sources about
a range of possible solutionsu A critical analysis of that informationu Greater learning for the organizationu A solution that’s more likely to work
71
72
Disadvantages of EBMgtu Challenges beliefs, prejudices and pet projectsu Requires more effortful critical thinkingu Requires collecting and using more data (of
different types from different sources)u Makes it hard to follow fadsu Challenge to authority and power based on
hierarchy or “because I say so” or JFDI or HiPPOu Reduces action-orientation of management
where doing stuff more important than doing stuff that works
72
73
Advantages of EBMgtu All the disadvantages also advantages
– Challenges beliefs, prejudices and pet projects– Requires more effortful critical thinking– Requires collecting and using more data (of different types from
different sources)– Makes it hard to follow fads– Challenge to authority and power based on hierarchy or
“because I say so” or JFDI or HiPPO– Reduces action-orientation of management where doing stuff
more important than doing stuff that worksu Develops deeper knowledge and understanding (of
organizational and other data)u Develops essential skillsu More likely to do stuff that will lead the outcomes you
want73
74
It’s a sort of choiceu How do you personally want to practice as a
professional?u How would you like your skills, knowledge
and effectiveness to develop over the course of career?
u How would you like your team to develop its capabilities and function to help the organization meet its objectives?
74
75
How to become a little more evidence-basedu Check our CEBMa websiteu Accept our ignorance and limits of all forms
of knowledge and evidence and experienceu Allow yourself to think and say the three
hardest words in management…
75
76
“I don’t know”
But I know how to find out…
Evidence-Based Management
76
