Upload
john-windmueller
View
259
Download
1
Embed Size (px)
DESCRIPTION
Connecting Curiosity with Evaluation Tools to Learn from and Assess ADR Programs and Practices
Citation preview
Learning From our PracticeConnecting Curiosity with Evaluation Tools to Learn
from and Assess ADR Programs and Practices
1
Thank you for joining this conversation.Thanks to the folks who organized this...
Cindy MazurPamela Pontillo Stephanie Fell
2
UB Program in Negotiations & Conflict Management
John Windmueller, Ph.D.
3
Scholarship
• PhD from ICAR in 2006• MA and BA in International
Relations• Teaching background at UB,
ICAR, CAR, & NCC• Research focus on civic
engagement and civil society; dialogue; conflict resolution evaluation and assessment methodologies; and emerging practices of narrative and frame-based conflict interventions
• Began practice work as a court and community mediator in 1993
• Practice work has included conflict resolution, facilitation, and consulting work in the contexts of large-scale community conflicts, organizational conflicts, and international development
Practice
4
The vast bulk of what we do as conflict resolution practitioners is not backed or guided by research.
We should and can change that.
5
6
7
http://www.xkcd.com8
I have some ideas about how conflict works and how to constructively intervene in conflict.
Those ideas might be wrong.
I should test them.
9
Community Sensemaking & Healing in the Wake of Political Violence
10
1991 Mount Pleasant Riot
11
9/11
12
13
14
Community Resilience Project
15
Dialogue
16
By dialogue I mean...
A deliberate process of conversation where emphasis is put on building understanding and empathy among the participants.
17
DebateArgument
Consensus BuildingAction Planning
Everyday Conversation
18
Theoretical parents of “modern” dialogue:
David BohmBohm, D. (1996). On dialogue. New York: Routledge. [originally a
series of papers done between 1985-1991]
Martin BuberBuber, M. (1970). I and thou (W. Kaufmann, Trans.). New York:
Simon and Schuster. [original in 1923]
19
• Quantum Physicist
• None of us see the whole picture.
• We need to improve the process and quality of our communication.
• Emphasis is on growing understanding of complex problems.
David BohmBohm, D. (1996). On dialogue. New York: Routledge.[originally a series of papers done between 1985-1991]
20
• Jewish, philosopher
• Distinguishes between I/it versus I/thou (Ich-Du and Ich-Es) ways of interacting and relating, and hence way of being, in the world
• Stresses the importance of empathy and relationship
Martin BuberBuber, M. (1970). I and thou (W. Kaufmann, Trans.). New York:
Simon and Schuster. [original in 1923]
21
What do we know about dialogue?
22
0
23
What we know about dialogue:
• Those who facilitate dialogue describe it in positive terms, anecdotally reporting that it raises empathy and understanding.
• There are many (again, anecdotal) reports of participants in dialogue reporting that they found the experience worthwhile.
• There are many (again, anecdotal) reports of dialogues that produced later collective actions by the dialogue participants.
24
Methodology
Grounded TheoryNarrative Analysis
25
Shifts Happen!
• Relational ShiftsMove from talking at to talking with over time
• Frame / Narrative / Discourse ShiftsThere is a substantive negotiation taking place over how to jointly make sense and meaning in the wake of political violence.
26
We need to get over ourselves!
Facilitators had the lowest ability in the room to influence the quality and direction of dialogue
27
Participants Matter!• Substantive diversity plays a significantly
positive role in dialogue shifts
• Participant skills have a greater influence than facilitators’ in shaping dialogue quality Open and revealing about their personal experiencesDiscussed what others saidMade personal connectionsReached out to silent membersRecognized both common ground & differencesForegrounded tacit frame negotiations
28
The Clever Models?
Failed
29
Research and evaluation can help prove the value and results of what we do.
Research and evaluation can challenge our assumptions and reshape our practice.
30
Evaluation & Assessment is something you do
every day
31
really
32
You probably even did it on your way here.
33
34
35
36
37
38
39
40
41
42
What would your ideal mediation dashboard look like?
43
44
Evaluation & Assessment 101
45
(that thing you already informally do every day)
46
Don’t Panic
It’s About Curiosity
It’s interesting, challenging, and fun.
47
48
49
50
51
Guiding Evaluation Principles
• Systemic inquiry
• Evaluator competence
• Integrity and honesty
• Respect for the security & dignity of evaluation stakeholders
• Consideration of general and public welfare
52
good evaluation...
• Can surprise you and disprove your assumptions
• Produces data you are curious about
• Produces data that can influence action
• Is designed with stakeholders
• Evolves
53
examples of data collection
54
55
Conflict Climate Assessment
56
57
Where can you get data?
58
Measurement can Matter“The score turned an intangible and impressionistic clinical concept—the condition of new babies—into numbers that people could collect and compare. Using it required more careful observation and documentation of the true condition of every baby. Moreover, even if only because doctors are competitive, it drove them to want to produce better scores—and therefore better outcomes—for the newborns they delivered.”
Virginia Apgar
Gawande, A. (2007). Better: A surgeon's notes on performance. New York: Metropolitan Books.
59
Why not anecdote?
60
what color is this slide?
61
Anecdote -> small N
62
The naked eye was inadequate for learning what you need to know to evaluate players. Think about it. One absolutely cannot tell, by watching, the difference between a .300 hitter and a .275 hitter. The difference is one hit every two weeks...If you see both 15 games a year, there is a 40 percent chance that the .275 hitter will have more hits than the .300 hitter...The difference between a good hitter and average hitter is simply not visible--it is a matter of record.
-- Bill James
Anecdote -> small N
63
There is no controversy in social science which shows such a large body of qualitatively diverse studies coming out so uniformly in the same direction as this one. When you are pushing over 100 investigations, predicting everything from the outcome of football games to the diagnosis of liver disease, and when you can hardly come up with half a dozen studies showing even a weak tendency in favor of the clinician, it is time to draw a practical conclusion.
- Paul Meehl (cited in Ayres)
Expert judgement isn’t what it’s cracked up to be
64
Get data from many sources
65
Tools for Data Analysis
• Excel
• SPSS
• SAS
• JMP
• R
• StatCrunch
• Nud*ist
• NVivo
• AtlasTI
• HyperResearch
• TAMSAnalyzer
66
67
68
69
What does research have me curious about and questioning?
The role of mindset on learning/trainingOur (in)ability to predict satisfactionQuirky human decision making & heuristics The neurobiology of conflict resolution
70
Mindset1. Your intelligence is something very basic about you
that you can't change very much.
2. You can learn new things, but you can't really
change how intelligent you are.
3. No matter how much intelligence you have, you can
always change it quite a bit.
4. You can always substantially change how how
intelligent you are.
FIXED
GROWTH
Dweck, C. (2006). Mindset. New York: Ballantine.
Negotiation ability, like plaster, is pretty
stable over timeFIXED
GROWTHNegotiation ability is changeable and can
be developed
71
Mindset
Participants w/Growth mindset were more likely (88% vs 53%) to tackle challenging negotiation scenarios
Participants w/Growth mindset were typically twice as successful in negotiation scenarios
Participants w/Growth mindset earned higher course grades
Kray, L., & Haselhuhn, M. (2007). Implicit theories of negotiating ability and performance: Longitudinal and experimental evidence. Journal of Personality and Social Psychology, 93, 49-64.
Negotiation ability, like plaster, is pretty
stable over timeFIXED
GROWTHNegotiation ability is changeable and can
be developed
72
We are terrible at predicting the happiness that will result from different outcomes
73
Anchoring & Framing
Risk aversion
Status quo bias
Optimism and over-confidence
Those Quirky Decision Makers
Schwartz, B. (2004). The paradox of choice. New York: Harper Collins.
Thaler, R., & Sustein, C. (2008). Nudge. New Haven: Yale Universiy Press.
Kahneman, D., Slovic, P., & Tversky, A. (Eds.). (1982). Judgment under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press.
74
How many African countries are members of the UN ?>< 10 -- average answer 20>< 60 -- average answer 45
Anchoring & Framing
75
Imagine that you have decided to see a concert where admission is $20 a ticket. As you enter the concert hall, you discover that you have lost a $20 bill. Would you still pay $20 for a ticket to the concert?
Imagine that you have decided to see a concert and already purchased a $20 ticket. As you enter the concert hall, you discover that you have lost your ticket. The seat was not marked and the ticket cannot be recovered. Would you pay $20 for the additional ticket?
76
Get $100
Flip a coin. If it’s heads, you get $200
Would you rather...
(Probability of success x Gain) - (Probability of failure x Loss)(.5x200)-(.5x0) = 100
Average amount it typically takes to induce the risk: $240
Risk Aversion
77
Risk Aversion: We are more sensitive to loss than gain
78
To keep your mug
To buy a mug
How much would you pay...
People bidding to keep mugs value them at almost twice the level of those bidding to get mugs
Status Quo Bias
79
<5 % of MBA students predict that they will be in the lower 50% of their class.
94% of professors report that they are better than the average professor.
Optimism & Overconfidence
80
The neurobiology of conflict resolution
?
81
So what next?
82
vs.
83
On the tail end of things...
Redelmeier, D., & Kahneman, D. (1996). Patients' memories of painful medical treatments: Real-time and retrospective evaluations of two minimally invasive procedures. Pain(116), 3-8.
84
Discomfort Stage 1
Discomfort Stage 2
Discomfort Stage 3
Discomfort Stage 4
8 5 8
8 5 8 3
A
B
85
All’s well that ends well...
86