Upload
stasia
View
28
Download
0
Tags:
Embed Size (px)
DESCRIPTION
PDW, Annual AOM 2012, Boston. Evidence-Based Management: Three New Approaches to Teaching the Practice of Management. Eric Barends. Blake Jelley. Wendy Carroll. Denise Rousseau. PDW, Annual AOM 2012, Boston. Denise Rousseau: Introduction Wendy Carroll & Blake Jelley: Push Approach - PowerPoint PPT Presentation
Citation preview
Postgraduate CoursePostgraduate Course
Evidence-Based Management: Evidence-Based Management:
Three New Approaches to Teaching the Three New Approaches to Teaching the
Practice of ManagementPractice of Management
PDW, Annual AOM 2012, BostonPDW, Annual AOM 2012, Boston
Eric Eric BarendsBarends
Blake Blake JelleyJelley
Denise Denise RousseauRousseau
Wendy Wendy CarrollCarroll
Postgraduate CoursePostgraduate Course
1.1. Denise Rousseau: Denise Rousseau: IntroductionIntroduction
2.2. Wendy Carroll & Blake Jelley: Wendy Carroll & Blake Jelley: Push ApproachPush Approach
SubgroupsSubgroups
3.3. Eric BarendsEric Barends: : Pull approachPull approach
4.4. Video (9 min)Video (9 min)
SubgroupsSubgroups
5.5. Denise Rousseau: Denise Rousseau: Process ApproachProcess Approach
PDW, Annual AOM 2012, BostonPDW, Annual AOM 2012, Boston
Postgraduate CoursePostgraduate Course
Definition
Evidence-based management means making decisions Evidence-based management means making decisions
about the management of employees, teams or about the management of employees, teams or
organizations through the conscientious, explicit and organizations through the conscientious, explicit and
judicious use of four sources of information:judicious use of four sources of information:
1. The best available scientific evidence1. The best available scientific evidence
2. Organizational facts, metrics and characteristics2. Organizational facts, metrics and characteristics
3. Stakeholders’ values and concerns3. Stakeholders’ values and concerns
4. Practitioner expertise and judgment4. Practitioner expertise and judgment
Postgraduate CoursePostgraduate Course
Four sources
Teaching EBMgt: The “Push” Approach
R. Blake Jelley & Wendy R. Carroll
Barends, Rousseau, Carroll, & Jelley2012 Academy of Management PDWEvidence-Based Management: Three New Approaches to Teaching the Practice of Management
Overview
• Our Perspectives and Context
• Importance of the “Push” Approach
• Principles and Resources
Our Perspectives and Context• Our backgrounds
– Education– Applied experiences
• Teaching in the UPEI School of Business– Undergraduate– EMBA (launched 2008)
• Oxford Handbook of EBMgt chapter– Jelley, Carroll, & Rousseau (2012). Reflections on teaching evidence-
based management.• Less about the “push” approach
Importance of the “Push” Approach• Bounded Rationality, Heuristics, Biases
• Kahneman (2011). Thinking, Fast and Slow– “System 1” (fast; automatic)– “System 2” (slow; deliberate)
• See also Kahneman & Klein (2009). Conditions for Intuitive Expertise: A Failure to Disagree(2009, American Psychologist)
Importance of the “Push” Approach• A path toward the development of expertise in management?
• Developing expert skill and intuition (see Kahneman & Klein, 2009; Kahneman, 2011)
– A sufficiently regular, predictable environment– Opportunities to learn regularities through prolonged
practice and feedback• The management domain is not highly favourable to skilled
intuition– Intuition is an important consideration, not the final word– Managers need to avoid overconfidence in intuition
Importance of the “Push” Approach
• System 1 will engage!
• Expert intuition is not magic…– “You can feel [Herbert] Simon’s impatience with the
mythologizing of expert intuition when he writes: ‘The situation has provided a cue; this cue has given the expert access to information stored in memory, and the information provides the answer. Intuition is nothing more and nothing less than recognition’” (Kahneman, 2011, p. 11).
Importance of the “Push” Approach• Making intuition more friendly to EBMgt.
– “You do as much homework as possible beforehand so that the intuition is as informed as it can be” (Kahneman, In Kahneman & Klein, 2010, McKinsey Quarterly).
– “It is easier to make good decisions quickly if managers are educated and evidence savvy” (John Zanardelli, 2012, p. 196; President & CEO, Ashbury Heights).
– Program System 1 with evidence-based principles.
• Think fast, well, and set triggers for System 2.
• Bolster, not replace, more deliberate processing.
Importance of the “Push” Approach• Practitioners are not well-informed about management-
related knowledge– E.g., Senior SHRM members = 57% correct (Rynes et al., 2002)
• Are educators much better?
• Various ways to “push” EB knowledge.– Management education as a key.
• Also, ME can integrate push, pull, and process approaches
Principles and Resources• Use of Diagnostic Quizzes
– Examples…• HRM (Rynes et al., 2002)• “100 things… & 50 more things you need to know” books• Advertising (Armstrong & Green’s adprin.com)
– Discussion of dissemination vs. exposing students as uninformed
– Links to critical thinking and the “pull” approach
Principles and Resources• Concerns about what and how we teach… and who does the
teaching
– Our body of knowledge• Benefits of systematic research• Volume of research• Focus on novelty over integration, etc.• Pluralism
– Textbooks– Instructors– Teaching methods
Principles and Resources
Other References:• Existing research syntheses;• Individual synthesis and translation articles
– E.g., Allen, Bryant, & Vardaman (2010). Retaining Talent: Replacing Misconceptions with Evidence-based Strategies. AOM Perspectives [Best Paper]
• SHRM Effective Practice Guidelines; • SHRM-SIOP’s new collaborative series.
Principles and Resources• Identify and teach the “core” management body of
knowledge; less content, more practice
• Focus on topics, theories, and principles that:– (a) Have a solid evidence-base– (b) Are practical to apply
• Are role-relevant• Have implications for practice; address important practice issues• Involve procedural as well as declarative knowledge
– (c) Are durable• Over time• Applicable in various situations
(Miner, 2003; Rousseau & McCarthy, 2007)
Since we can’t teach everything, what are the most important evidence-based things we need to program into our students?
ReferencesAllen, D. G., Bryant, P. C., & Vardaman, J. M. (2010). Retaining talent: Replacing misconceptions with evidence-based strategies.
Academy of Management Perspectives, 24(2), 48-64.Armstrong, J. S., & Green, K. C. (2012). Advertising principles: Evidence-based knowledge on persuasion through advertising.
Retrieved from http://advertisingprinciples.com/ [see http://advertisingprinciples.com/en/try/test-your-advertising-iq]Charlier, S. D., Brown, K. G., & Rynes, S. L. (2011). Teaching evidence-based management in MBA programs: What evidence is
there? Academy of Management Learning & Education, 10(2), 222-236.Eichinger, R. W., Lombardo, M. M., & Ulrich, D. (2004). 100 things you need to know: Best people practices for managers & HR
(Vol. 1). Minneapolis, MN: Lominger.Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Strauss, Giroux.Kahneman, D. & Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 64(6), 515-526.Kahneman, D. & Klein, G. (2010). When can you trust your gut? McKinsey Quarterly, Issue 2, 58-67. Jelley, R. B., Carroll, W. R., & Rousseau, D. M. (2012). Reflections on teaching evidence-based management. In D. M. Rousseau
(Ed.), The Oxford Handbook of Evidence-Based Management (pp. 337-355). New York: Oxford University Press.Latham, G. P. (2009). Becoming the evidence-based manager: Making the science of management work for you. Boston: Davies-
Black.Locke, E. A. (2009). Handbook of principles of organizational behavior: Indispensable knowledge for evidence-based management
(2nd ed.). Chichester, UK: Wiley.Miner, J. B. (2003). The rated importance, scientific validity, and practical usefulness of organizational behavior theories: A
quantitative review. Academy of Management Learning & Education, 2(3), 250-268.Pearce, J. L. (2009). Organizational behavior: Real research for real managers. Irvine, CA: Melvin & Leigh.
ReferencesPearce, J. L. (2012). Creating evidence-based management textbooks. In D. M. Rousseau (Ed.), The Oxford Handbook of Evidence-
Based Management (pp. 377-386). New York: Oxford University Press.Pfeffer, J., & Sutton, R. I. (2006). Hard facts, dangerous half-truths, and total nonsense. Boston, MA: Harvard Business School
Press.Rousseau, D. M. (Ed.) (2012). The Oxford Handbook of Evidence-Based Management. New York: Oxford University Press. Rousseau, D. M. (2012). Designing a better business school: Channelling Herbert Simon, addressing the critics, and developing
actionable knowledge for professionalizing managers. Journal of Management Studies, 49(3), 600-618.Rousseau, D. M., & McCarthy, S. (2007). Educating managers from an evidence-based perspective. Academy of Management
Learning & Education, 6, 84–101.Rynes, S. L., Colbert, A. E., & Brown, K. G. (2002). HR professionals’ beliefs about effective human resource practices:
Correspondence between research and practice. Human Resource Management, 41(2), 149–174.Society for Human Resource Management Foundation (2012). Effective practice guidelines series.
http://www.shrm.org/about/foundation/products/pages/default.aspxSociety for Industrial and Organizational Psychology & Society for Human Resource Management (2012). Publication and
dissemination of science to practice: A research collaboration between the Society for Human Resource Management (SHRM) and the Society for Industrial and Organizational Psychology (SIOP). http://www.siop.org/SIOP-SHRM%5Cdefault.aspx
Ulrich, D., Eichinger, R., Kulas, J., & De Meuse, K. (2007). 50 more things you need to know: The science behind best people practices for managers & HR professionals (Vol. 2). Minneapolis, MN: Lominger.
Zanardelli, J. (2012). At the intersection of the academy and practice at Ashbury Heights. In D. M. Rousseau (Ed.), The Oxford Handbook of Evidence-Based Management (pp. 191-197). New York: Oxford University Press.
Postgraduate CoursePostgraduate Course
Eric BarendsEric Barends
PDW, Annual AOM 2012, BostonPDW, Annual AOM 2012, Boston
Part 2: Part 2:
The 5-step pull approachThe 5-step pull approach
Evidence-Based Management: Evidence-Based Management: Three New Approaches to Teaching the Practice of ManagementThree New Approaches to Teaching the Practice of Management
Postgraduate CoursePostgraduate Course
Trust me, I’m a manager.Trust me, I’m a manager.
Postgraduate CoursePostgraduate Course
Push vs PullPush vs Pull
Push: Push: teaching management principles teaching management principles
based upon a convergent body of based upon a convergent body of
research and telling students what to do.research and telling students what to do.
Pull: Pull: teaching students how to find, teaching students how to find,
appraise and apply the outcome of appraise and apply the outcome of
research (evidence) by themselvesresearch (evidence) by themselves
Postgraduate CoursePostgraduate Course
Why do we need a Why do we need a
pull approach?pull approach?
Postgraduate CoursePostgraduate Course
Problem I: too much ‘evidence’Problem I: too much ‘evidence’
HRM: 1,400 articles in 2011 (ABI/INFORM). For an HR HRM: 1,400 articles in 2011 (ABI/INFORM). For an HR
manager to keep up this means reading 3 to 4 articles manager to keep up this means reading 3 to 4 articles
every day (for a ‘general’ manager more than 50!)every day (for a ‘general’ manager more than 50!)
Postgraduate CoursePostgraduate Course
Half of what you learn will be shown to be either dead Half of what you learn will be shown to be either dead
wrong or out-of-date within 7 years of your graduation; wrong or out-of-date within 7 years of your graduation;
the trouble is that nobody can tell you which halfthe trouble is that nobody can tell you which half
Problem II: false informationProblem II: false information
Postgraduate CoursePostgraduate Course
5 years? 5 years? 7 years? 7 years? 10 years?10 years?
Problem III: half time valueProblem III: half time value
Postgraduate CoursePostgraduate Course
PullPull
Pull: Pull: teaching students how to find, appraise teaching students how to find, appraise
and apply evidence by themselvesand apply evidence by themselves
Postgraduate CoursePostgraduate Course
Starting pointStarting point
Problem basedProblem based
Real live caseReal live case
Just in timeJust in time
Start with a practical question,Start with a practical question,
(not with an academic answer)(not with an academic answer)
Postgraduate CoursePostgraduate Course
The 5 steps of ‘pull’ EBPThe 5 steps of ‘pull’ EBP
1.1. Formulate a focused question (Formulate a focused question (AskAsk))
2.2. Search for the best available evidence (Search for the best available evidence (AcquireAcquire))
3.3. Critically appraise the evidence (Critically appraise the evidence (AppraiseAppraise))
4.4. Integrate the evidence with your managerial Integrate the evidence with your managerial
expertise and organisational concerns and apply expertise and organisational concerns and apply
((ApplyApply))
5.5. Monitor the outcome (Monitor the outcome (AssessAssess))
Postgraduate CoursePostgraduate Course
1. Formulate a focused question1. Formulate a focused question
Asking the right question?Asking the right question?
Does team-building work?Does team-building work?
Does the introduction of self-steering teams work?Does the introduction of self-steering teams work?
Does management development improve the Does management development improve the
performance of managers?performance of managers?
Does employee participation prevent resistance to Does employee participation prevent resistance to
change?change?
Is 360 degree feedback effective?Is 360 degree feedback effective?
Postgraduate CoursePostgraduate Course
Postgraduate CoursePostgraduate Course
What is a ‘team’?What is a ‘team’?
What kind of team?What kind of team?
In what contexts/ settings?In what contexts/ settings?
What counts as ‘team-building’?What counts as ‘team-building’?
What does ‘work’ mean?What does ‘work’ mean?
What outcomes are relevant?What outcomes are relevant?
Over what time periods?Over what time periods?
Focused question?Focused question?
Does team-building work?Does team-building work?
Postgraduate CoursePostgraduate Course
PP == PopulationPopulation
I I = = Intervention or success factorIntervention or success factor
CC = Comparison = Comparison
OO = Outcome= Outcome
CC = Context = Context
Answerable question: PICOCAnswerable question: PICOC
Postgraduate CoursePostgraduate Course
PP == PopulationPopulation
I I = = Intervention or successfactorIntervention or successfactor
CC = Comparison = Comparison
OO = Outcome= Outcome
CC = Context = Context
Focused question: PICOCFocused question: PICOC
Employee productivity?
Job satisfaction?
Return on investment?
Market share?
Organizational commitment?
Postgraduate CoursePostgraduate Course
2. Finding the best available evidence2. Finding the best available evidence
Searching evidenceSearching evidence
Where Where do we search?do we search?
Postgraduate CoursePostgraduate Course
DatabasesDatabasesPostgraduate CoursePostgraduate Course
ABI/INFORMABI/INFORM
Business Source EliteBusiness Source Elite
PsycINFOPsycINFO
Web of KnowledgeWeb of Knowledge
ERICERIC
Google ScholarGoogle Scholar
Searching evidenceSearching evidence
How How do we search?do we search?
Search StrategySearch Strategy
Postgraduate CoursePostgraduate Course
Two types of search strategiesTwo types of search strategies
Search strategySearch strategy
Building blocks methodBuilding blocks methodSnowball methodSnowball method
Postgraduate CoursePostgraduate Course
Hands on instructionHands on instructionPostgraduate CoursePostgraduate Course
Postgraduate CoursePostgraduate Course
3. Critical appraisal of studies3. Critical appraisal of studies
Making sense of evidenceMaking sense of evidence
Postgraduate CoursePostgraduate Course
The best available evidence = The best available evidence =
Studies with the highest Studies with the highest internal validityinternal validity(does it work?)(does it work?)
Studies with the highest Studies with the highest external validityexternal validity(does it work for my employees / my organization?)(does it work for my employees / my organization?)
Postgraduate CoursePostgraduate Course
Which study for which question?Which study for which question?
Research designsResearch designs
The “best” evidence depends on the question type !The “best” evidence depends on the question type !
Postgraduate CoursePostgraduate Course
Which design for which Which design for which question?question?
Explanation
Postgraduate CoursePostgraduate Course
Best research design?Best research design?
Postgraduate CoursePostgraduate Course
Critical appraisalCritical appraisal
Critical appraisalCritical appraisalPostgraduate CoursePostgraduate Course
1.1. Is the study design appropriate to the stated aims?Is the study design appropriate to the stated aims?
2.2. Was a control group used?Was a control group used?
3.3. Was a pretest used?Was a pretest used?
4.4. Are the measurements likely to be valid and reliable?Are the measurements likely to be valid and reliable?
5.5. Could bias or confounding have occurred?Could bias or confounding have occurred?
6.6. How large was the effect size?How large was the effect size?
Postgraduate CoursePostgraduate Course
Step 4: Turning evidence into Step 4: Turning evidence into practicepractice
Postgraduate CoursePostgraduate Course
1.1. Is your organization / population so different from those in the Is your organization / population so different from those in the study that its results are difficult to apply?study that its results are difficult to apply?
2.2. How relevant is the study (or outcome) to what you are How relevant is the study (or outcome) to what you are seeking to understand or decide? seeking to understand or decide?
3.3. What are your organization’s potential benefits and harms What are your organization’s potential benefits and harms from the intervention?from the intervention?
4.4. Is the intervention feasible in your setting?Is the intervention feasible in your setting?
Organization concernsOrganization concerns
Always ask yourself to what extent the evidence Always ask yourself to what extent the evidence is applicable in your situation:is applicable in your situation:
Postgraduate CoursePostgraduate Course
Four sources
Postgraduate CoursePostgraduate Course
Feasible?
organizational facts and characteristicsorganizational facts and characteristics
cultural aspectscultural aspects
stakeholders’ values and concernsstakeholders’ values and concerns
political aspectspolitical aspects
financial aspects /cost-effectiveness / ROIfinancial aspects /cost-effectiveness / ROI
prioritiespriorities
change readiness / resistance to change change readiness / resistance to change
implementation capacity implementation capacity
timingtiming
Postgraduate CoursePostgraduate Course
Popular management book / guided field trip Popular management book / guided field trip
Surfacing assumptionsSurfacing assumptions
Needle & haystack assignmentNeedle & haystack assignment
Myth busters, snake oil symposiumMyth busters, snake oil symposium
Find the FlawsFind the Flaws
Persuasive paper / presentationPersuasive paper / presentation
CATCAT
Exercises / AssignmentsExercises / Assignments
Postgraduate CoursePostgraduate Course
CAT: Critically Appraised TopicCAT: Critically Appraised Topic
Postgraduate CoursePostgraduate Course
CAT: Critically Appraised TopicCAT: Critically Appraised Topic
A critically appraised topic (CAT) is a A critically appraised topic (CAT) is a
structured, short (2 pages max) summary structured, short (2 pages max) summary
of evidence on a topic of interest, usually of evidence on a topic of interest, usually
focused around a practical problem or focused around a practical problem or
question..question..
Postgraduate CoursePostgraduate Course
CAT: structureCAT: structure
1)1) Background / contextBackground / context
2)2) Question (PICOC)Question (PICOC)
3)3) Search strategySearch strategy
4)4) Results / evidence summaryResults / evidence summary
5)5) FindingsFindings
6)6) LimitationsLimitations
7)7) RecommendationRecommendation
Postgraduate CoursePostgraduate Course
CAT-walkCAT-walk
Postgraduate CoursePostgraduate Course
CAT: exampleCAT: example
See: www.cebma.org/presentations
(CAT Organizational Trust and Job Satisfaction)
Postgraduate CoursePostgraduate Course
What are the skills that are manifest in What are the skills that are manifest in
this video that are relevant to what you this video that are relevant to what you
are trying to teach at your classes?are trying to teach at your classes?
Denise M. Rousseau
Barends, Rousseau, Carroll, & Jelley2012 Academy of Management PDWEvidence-Based Management: Three New Approaches to Teaching the Practice of Management
The “Human” Problem
Bounded Rationality
The Small Numbers Problem of Individual Experience
Prone to See Patterns Even in Random Data
Evidence-Based Practice
Critical Thinking
Decision Supports
Research
• Large Ns > individual experience
• Controls reduce bias
The “Human” Problem
Bounded Rationality
The Small Numbers Problem of Individual Experience
Prone to See Patterns Even in Random Data
Evidence-Based Practice
Critical Thinking
Decision Supports
Research
• Large Ns > individual experience
• Controls reduce bias
Get critical “evidence” in advance• Prime your KSAs
Make the right decision as fast as needed• Not necessarily “as fast as possible”
Learn best (evidence-based) strategies for different decision types
• Identify the type of decision you face• Then, engage the right decision strategy
Routine decisions for which there is a “best evidence-based way” to do things• Hiring call center workers, management trainees• Giving periodic performance feedback• Running a geographically distributed meeting
Acquire science-based evidence and org facts to identify effective practices
Develop standard operating procedures with users Gather org-evidence to evaluate SOP effects Modify as needed Put in user-friendly form (Checklist, Diagram)
Non-routine Decisions (# Stakeholders & Goals)• New facility start up• Solving space problem in existing building• Developing a company-wide performance management system
Evidence-Based Pull Approach
Yates’s Cardinal Rules
Note: What is non-routine to one organization may be routine in another (e.g., new store start ups are routine in McDonald’s )
Beneficiary Interests/Values
DecisionOther
Contributors
Preliminaries1—Need 2—Mode
3—Investment
Core4—Options
5—Possibilities6—Judgment
7—Value8—Tradeoffs
Aftermath9—Acceptability
10—Implementation
Decision Processes: Cardinal Issue Resolution
Resolution Contributors: E.g., Resources, Tools, Biology, Habits, “Natural” Experiences, Training, History, Culture
A Big Picture
Hypercomplex Decision with High Risk and Many Unknowns (i.e. Black Swans)
Use Sensemaking
Weick and Sutcliffe’s Resilience Process• Gather information and check assumptions• Run experiments (in parallel if several alternatives are identified)• Multiple trials to learn by doing• Build on small wins• Continue to question assumptions
Routine Decisions• Atul Gawande
Novel Decisions (due # Stakeholders & Goals)• Frank Yates
Hypercomplex with Many Unknowns/Risky Decisions• Karl Weick & Kathleen Sutcliffe
What type of decision situation do you face?
DIAGNOSIS: Appropriate decision strategy?
Product of this is critical thinking that overtime helps you become more aware of assumptions and gaps in logic
Heath, C., Larrick, R. P., & Klayman, J. (1998) Cognitive repairs: How organizational practices can compensate for individual shortcomings. Review of Organizational Behavior, 20, 1–38.
Gawande, A. (2009). Checklist manifesto: How to get things right. New York: Henry Holt.
Larrick, R.K. (2009/) Broaden the decision frame to make effective decisions. In E.A. Locke (ed.), Handbook of principles of organizational behavior: Indispensable knowledge for evidence-based management. New York: Wiley (pp. 461-515).
Taleb, N. N. (2010). The black swan: The impact of the highly improbable. (2nd ed.) New York:: Penguin.
Weick, K.E, & Sutcliffe, K. (2007). Managing the Unexpected: Resilient Performance in an Age of Uncertainty. New York: Wiley.
Yates, J. F. (2003). Decision management. San Francisco: Jossey-Bass.
Yates, J.F. & Potwoworski, G. (2012). Evidence-based decision management. In D,M, Rousseau (ed): Handbook of Evidence-Based Management: New York: Oxford University Press, this volume.