View
364
Download
0
Embed Size (px)
DESCRIPTION
Presentation Evidence-Based Heathcare Logistics Erasmus University Oct 13, 2011 Eric Barends
Citation preview
Congres Evidence-Based Zorglogistiek, 13 oktober 2011
Evidence-Based Management
What is it?
Why do we need it?
How does it look like in practice?
1. Evidence based management:
What is it?
Postgraduate Course
Postgraduate Course
Definition
Evidence-based management means making decisions
about the management of employees, teams or
organizations through the conscientious, explicit and
judicious use of four sources of information:
1. The best available scientific evidence
2. Organizational facts, metrics and characteristics
3. Stakeholders’ values and concerns
4. Practitioner expertise and judgment
Postgraduate Course
Four sources
Evidence is not the same as ‘proof’ or ‘hard facts’
Evidence can be
- so strong that no one doubts its correctness, or
- so weak that it is hardly convincing at all
What is evidence?Postgraduate Course
McMaster University Medical School, Canada
Medicine: Founding fathers
David Sackett Gordon Guyatt
Postgraduate Course
Management: Founding MotherPostgraduate Course
Jeffrey Pfeffer Robert Sutton
Management: Founding FathersPostgraduate Course
2. Evidence-based management:
Why do we need it?
Postgraduate Course
EBMgt: some basic assumptionsPostgraduate Course
Research produced by management scholars could be useful to
organizations
Drawing on available evidence (including research produced by
academics) is likely to improve decisions
Organizations do not appear to be strongly aware of nor use
research findings
EBMgt is a potentially useful way of thinking about how we can
incorporate research evidence into decision-making
Reason 1:
Errors and Biases of Human Judgment
Postgraduate Course
Seeing order in randomness
Mental corner cutting
Misinterpretation of incomplete data
Halo effect
False consensus effect
Reinterpreting evidence
Group think
Postgraduate Course
Errors and Biases of Human Judgment
Confirmation bias
Authority bias
In-group bias
Recall bias
Anchoring bias
Inaccurate covariation detection
Distortions due to plausibility
Seeing order in randomness Mental corner cutting
Misinterpretation of incomplete data
Halo effect
False consensus effect
Reinterpreting evidence
Group think
Postgraduate Course
Errors and Biases of Human Judgment
Confirmation bias
Authority bias
In-group bias
Recall bias
Anchoring bias
Inaccurate covariation detection
Distortions due to plausibility
We are predisposed to see order, pattern and causal
relations in the world.
Patternicity: The tendency to find meaningful patterns in
both meaningful and meaningless noise.
Postgraduate Course
Seeing order in randomness
We are patern seeking primates: association learning
Postgraduate Course
Seeing order in randomness
Postgraduate Course
Points of impact of V-1 bombs in London
Postgraduate Course
Points of impact of V-1 bombs in London
A Type I error or a false positive, is
believing a pattern is real when it is not
(finding a non existent pattern)
A Type II error or a false negative, is
not believing a pattern is real when it is
(not recognizing a real pattern)
Postgraduate Course
Errors and Biases of Human Judgment
Dr. Michael Shermer
(Director of the Skeptics Society)
A Type I error or a false positive: believe that the
rustle in the grass is a dangerous predator when it is
just the wind (low cost)
Postgraduate Course
Errors and Biases of Human Judgment
A Type II error or a false negative: believe that the
rustle in the grass is just the wind when it is a
dangerous predator (high cost)
Postgraduate Course
Errors and Biases of Human Judgment
Pattern detection problem
Assessing the difference between a Type I and
Type II error is highly problematic (especially in
split second ‘life and death’ situations), so the
default position is to assume
all patterns are real.
Postgraduate Course
Errors and Biases of Human Judgment
Postgraduate Course
Errors and Biases of Human Judgment
Jennifer Whitson, University of Texas Austin, corporate environments
Postgraduate Course
Errors and Biases of Human Judgment
Erroneous beliefs plaque both experienced
professionals and less informed laypeople alike.
stress peptic ulcer
Peptic ulcer – an infectious disease!
This year's Nobel Prize in Physiology or Medicine goes to Barry Marshall and Robin
Warren, who with tenacity and a prepared mind challenged prevailing dogmas. By
using technologies generally available (fibre endoscopy, silver staining of
histological sections and culture techniques for microaerophilic bacteria), they
made an irrefutable case that the bacterium Helicobacter pylori is causing disease.
By culturing the bacteria they made them amenable to scientific study.
In 1982, when this bacterium was discovered by Marshall and Warren, stress and
lifestyle were considered the major causes of peptic ulcer disease. It is now
firmly established that Helicobacter pylori
causes more then 90% of duodenal ulcers.
The link between Helicobacter pylori
infection and peptic ulcer disease has been
established through studies of human
volunteers, antibiotic treatment studies and
epidemiological studies.
Oct 2005
Postgraduate Course
Errors and Biases of Human Judgment
Doctors, teachers, lawyers and managers hold many
erroneous beliefs, not because they are ignorant or
stupid, but because they seem to be the most sensible
conclusion consistent with the available evidence.
They hold such beliefs because they seem to be the
irresistible products of their own professional experience.
They are the products, not of irrationality, but of flawed
rationality
Seeing order in randomness
Mental corner cutting
Misinterpretation of incomplete data
Halo effect
False consensus effect
Reinterpreting evidence
Group think
Postgraduate Course
Errors and Biases of Human Judgment
Confirmation bias
Authority bias
In-group bias
Recall bias
Anchoring bias
Inaccurate covariation detection
Distortions due to plausibility
Postgraduate Course
Group think: management fads
The nearly-forgotten fads:
Scientific Management/Taylorism
Business Process Reengineering
Management by results
Excellence
Total Quality Management
Learning Organizations
Knowledge Management
Postgraduate Course
Group think: management fads
The fads that haven’t been forgotten
(yet):
Talent management
Management development
Executive coaching
Emotional intelligence
Employee engagement
Knowledge management
Myers Briggs Type Indicator
Belbin Team Roles
Postgraduate Course
Group think: management fads
“And there we see the power of any big
managerial idea (or fad). It may be smart, like
quality, or stupid, like conglomeration. Either
way, if everybody's doing it, the pressure to do it
too is immense. If it turns out to be smart, great.
If it turns out to be stupid, well, you were in good
company and most likely ended up no worse off
than your competitors. Your company's board
consists mostly of CEOs who were probably
doing it at their companies. How mad can they
get?
Postgraduate Course
Group think: management fads
The true value of conventional management
wisdom is not that it's wise or dumb, but that it's
conventional. It makes one of the hardest jobs in
the world, managing an organization, a little
easier. By following it, managers everywhere see
a way to drag their sorry behinds through
another quarter without getting fired. And isn't
that, really, what it's all about?”
(Colvin, 2004, Fortune)
Postgraduate Course
So?
Managers seem to be extremely good at generating
ideas, theories, and explanations that have the ring of
plausibility. They may be relatively deficient, however,
in evaluating and testing those ideas once they are
formed.
This requires that we think critically about experience,
question our assumptions, and challenge what we
think we know
(Show me the evidence!)
Postgraduate Course
Errors and Biases of Human Judgment
Reason 2:
De ‘buitenwereld’ wordt steeds kritischer
Postgraduate Course
Postgraduate Course
Probleem 2: kritische geluiden
“Managers maken Nederland ziek ... Steeds
meer vakmensen (zoals docenten, verpleegkundigen,
artsen) hebben het gevoel dat ze worden aangestuurd
door managers die van het vak geen verstand hebben
maar wel de dienst uitmaken.”
Ad Verbrugge
Postgraduate Course
“Of het nu gaat om een ziekenhuis of een dropfabriek,
te veel managers hebben de pretentie dat ze alles
kunnen managen zonder ook maar te letten op de
inhoud van het werk. Het zijn figuren die als een vlo
van de ene "uitdaging" naar de andere springen, een
spoor van verbittering en vernieling
achter zich latend.”
Geert Mak
Probleem 2: kritische geluiden
Postgraduate Course
“Nog meer managers, nog meer reorganisaties, nog
meer power point-presentaties, nog meer holle
retoriek over topprestaties en topkwaliteit. De
groeiende korst van nepfuncties die onze bedrijven,
scholen en andere organisaties nutteloos belasten
wordt almaar dikker .”
Dorien Pessers
Probleem 2: kritische geluiden
Postgraduate Course
Brede maatschappelijke ontwikkeling
Postgraduate Course
“Waar de overheid, de dokter, de pedagoog en
de manager vroeger een eenvoudig beroep op
hun autoriteit konden doen, zullen zij nu met
getallen en statistiek hun gelijk moeten
aantonen.”
Brede maatschappelijke ontwikkeling
Postgraduate Course
Evidence based medicine
Evidence based education
Evidence based criminology
Evidence based social welware
Evidence based management?
Brede maatschappelijke ontwikkeling
Evidence based management:
How does it look like in practice?
Postgraduate Course
Postgraduate Course
Four sources
Postgraduate Course
JAMA, 1992
Postgraduate Course
Push vs Pull
Push: teaching (management) principles
based upon a convergent body of
research and telling students what to do.
Pull: teaching (managers) how to find,
appraise and apply the outcome of
research (evidence) by themselves
Postgraduate Course
The 5 steps of ‘pull’ EBP
1. Formulate an answerable question
2. Search for the best available evidence
3. Critically appraise the evidence
4. Integrate the evidence with your managerial
expertise and organisational concerns and apply
5. Monitor the outcome
Postgraduate Course
The 5 steps of ‘pull’ EBP
1. Formulate an answerable question
2. Search for the best available evidence
3. Critically appraise the evidence
4. Integrate the evidence with your managerial
expertise and organisational concerns and apply
5. Monitor the outcome
Answerable question
Postgraduate Course
I am a consultant, my client a large health-care
organization. The board of directors has plans for a
merger with a smaller healthcare organization. However,
it’s been said that the organizational culture differs widely
between the two organizations. The board want’s to
know if this can impede a successful outcome.
Postgraduate Course
P = Population or problem
I = Intervention or successfactor
C = Comparison
O = Outcome
C = Context
Answerable question: PICO(C)
Answerable question: PICOC
Postgraduate Course
P: What kind of Population are we talking about? Middle managers,
back-office employees, medical staff, clerical staff?
O: What kind of Outcome are we aiming for? Employee productivity,
return on investment, profit margin, competitive position, innovation
power, market share, customer satisfaction?
P/C: And how is the assumed cultural difference assessed? Is it the
personal view of some managers or is it measured by a validated
instrument?
Postgraduate Course
The 5 steps of ‘pull’ EBP
1. Formulate an answerable question
2. Search for the best available evidence
3. Critically appraise the evidence
4. Integrate the evidence with your managerial
expertise and organisational concerns and apply
5. Monitor the outcome
Where do we search?Postgraduate Course
How do we search?Postgraduate Course
Postgraduate Course
The 5 steps of ‘pull’ EBP
1. Formulate an answerable question
2. Search for the best available evidence
3. Critically appraise the evidence
4. Integrate the evidence with your managerial
expertise and organisational concerns and apply
5. Monitor the outcome
Critical appraisal
How to read a research article?
Postgraduate Course
Critical appraisalPostgraduate Course
1. Study designs
2. Levels of evidence
3. Bias / confounding
4. Effect sizes
5. External validity
Postgraduate Course
Which study for which question?
Research designs
The “best” evidence depends on the question type !
Postgraduate Course
Levels of evidence
Postgraduate Course
The 5 steps of ‘pull’ EBP
1. Formulate an answerable question
2. Search for the best available evidence
3. Critically appraise the evidence
4. Integrate the evidence with your
managerial expertise and organisational
concerns and apply
5. Monitor the outcome
Postgraduate Course
1. Is your organization / division / population so different from those in the study that its results cannot apply?
2. How relevant is the study to what you are seeking to understand or decide?
3. What are your organization’s potential benefits and harms from the intervention?
4. Is the intervention feasible in your setting?
Organization concerns
Always ask yourself to what extent the evidence is applicable in your situation:
Postgraduate Course
The 5 steps of ‘pull’ EBP
1. Formulate an answerable question
2. Search for the best available evidence
3. Critically appraise the evidence
4. Integrate the evidence with your managerial
expertise and organisational concerns and apply
5. Monitor the outcome
Postgraduate Course
Monitor the outcome
Uitkomst gemeten?
Voormeting?
Controlegroep?
Postgraduate Course
Do a trial!
Postgraduate Course
Monitor the outcome
Business Process Redesign?
Six Sigma?
Lean management?
Lean Six Sigma?
TOC/ Theory of Constraints?
Performance Management?
Of …..
Postgraduate Course
Vragen?