Michael Quinn Patton DE Book Launch

DevelopmentalEvaluation:

Applying Complexity Conceptsto Enhance Innovation and Use

Michael Quinn Patton SIG October 4,2010

Michael Quinn Patton

1

In the beginning…

Michael Quinn Patton SIG October 4,2010 2


Innovation and Evaluation

Systems thinking & complexity science

as frameworks for conceptualizing interventions:

Developmental Evaluation

3

5 types of DE

1. Ongoing Development

Developmentvs.

Improvement


Blandin Community Leadership Program


Beyond Formative and Summative to

Developmental Evaluation

as an option in the repertoire of approaches


Mrs. McCave and her 23 Daves

7Michael Quinn Patton SIG October 4,2010


Challenge:Matching the evaluation process and design to the nature of the situation:

Contingency-basedEvaluation

8


Conditions that challenge traditional model-testing evaluation

• High innovation• Development• High uncertainty• Dynamic• Emergent• Systems Change

AdaptiveManagement

andDevelopmental

Evaluation

9


First DE Type

Ongoing development in adapting a project, program, strategy, policy, or other innovative initiative to new conditions in complex dynamic systems.

10


Mintzberg on Strategy

Unrealized StrategyIntendedStrategy

Deliberate Strategy

Realized Strategy

Emergent Strategy

11


12

Uncertainty and Emergence“No battle plan ever survives contact with the

enemy.” Field Marshall Helmuth Carl Bernard von Moltke

“Everyone has a plan…until he gets hit.”Former World Heavyweight boxing champion, Mike Tyson

5 Types of DE2. Pre-formative development of a potentially

scalable innovation:• Developing an innovation to the point

where it is ready for traditional formative and summative evaluation

• Pre-formative developmental evaluation works with emerging ideas and visionary hopes in a period of exploration to shape them into a potential model that is a more fully conceptualized, potentially scalable intervention.



PANARCHY MODEL

14


Stored

HARVESTING LESSONS

DEVEOPMENTAL EVALUATION

FORMATIVE

SUMMATIVE

Phases of Technological &

Social Innovation 15

Polio Example Bill & Melinda Gates Foundation: $1 billion on polio

eradication: Vertical, focused strategy "There's no way to sugarcoat the last 12 months,"

Bruce Aylward, a WHO official, told Mr. Gates in June -- the virus was rippling through countries believed to have stopped the disease.Mr. Gates asked: "So, what do we do next?“

New strategy: Disease-specific wars can succeed only if they also strengthen the overall health system in poor countries.

http://online.wsj.com/article/SB10001424052702303348504575184093239615022.html?mod=WSJ_hps_InDepthCarousel




5 Types of DE

3. Adapting effective general principles to a new context as ideas and innovations are taken from elsewhere and developed within a new setting, the work of developmental evaluation in the dynamic middle between top-down and bottom-up forces of change.



Fundamental Issue:How the World Is Changed

Top-down dissemination of “proven models”

versusBottoms-up adaptive management

18


Prescriptive Models vs. Adaptive Principles

Identifying effective principles for adaptive management

(bottoms-up approach)versus

Identifying and disseminating “proven” models

(top down approach)19


Top-down dissemination of “proven models”

Developmental Evaluation: Navigating the murky,

complex, dynamic MIDDLE

Bottom-up adaptive management


Personal Factor

Damiano example, Jean Gornick

21

5 Types of DE4. Major systems change and cross-scale

developmental evaluation• Providing feedback about how major systems

change is unfolding• Evidence of emergent tipping points• System inter-relationships as the “unit of

analysis”• Evaluating strategy



Systems

• Parts are interdependent such a change in one part changes all parts

• The whole is greater than the sum of the parts

• Focus on interconnected relationships• Systems are made up of sub-systems and

function within larger systems

23



Understanding the Elephant

from a Systems Perspective

25





http://en.wikipedia.org/wiki/File:Malaelephants.jpg

http://www.lifeinthefastlane.ca/wp-content/uploads/2008/11/elephant_9sfw.jpg



The relationship between what goes in and what comes

out What conceptual framework informs evaluation?

31


Teen Pregnancy Program Example

32


Logic Model for Pregnant Teens Program1. Program reaches out to pregnant teens

2. Pregnant teens enter and attend the program (participation)

3. Teens learn prenatal nutrition and self-care (increased knowledge)

4. Teens develop commitment to take care of themselvesand their babies (attitude change)

5. Teens adopt healthy behaviors: no smoking, no drinking,attend prenatal clinic, eat properly (behavior change)

6. Teens have healthy babies (desired outcome)

33


Systems web showing possible influence linkages to a pregnant teenager

Teachers/ other adults

Youngpregnantwoman's

attitudes &behaviors

Herparents &

other familymembers

Child'sfather &

peers

Prenatal program

staff

Her peer group

34


Program systems web showing possible institutional influences affecting pregnant teenagers:

SCHOOL SYSTEM

Youngpregnantwomen's

attitudes &behaviors

PrenatalClinic andHospitalOutreach

Church

Prenatal program

Other community-based youth programs

Other Systems-- welfare-- legal -- nutrition programs-- transportation-- child protection-- media messagesContext factors-- politics-- economic incentives-- social norms-- culture-- music

YouthCulture

35

5 Types of DE

5. Developing a rapid response in the face of a sudden major change or a crisis, like a natural disaster or financial melt-down:

• Exploring real time solutions and generating innovative and helpful interventions for those in need.

• Dealing with high uncertainty, turbulence, turmoil, high stakes, and often conflict.


Refugee Camps



About this book

Preview this book

Shake Hands with the Devil By Roméo Dallaire

This is a preview. The total pages displayed will

be limited. L

Loading...

Loading...


resultSearch Booksresultoe9S6SgfGeneral RPP1PP1ACfU3U1E


Photo by Lynsey Gornick


Seeing Through A Complexity Lens

“You don't see something until you have the right metaphor to let you perceive it”. Thomas Kuhn

41

Getting to Maybe: How the World Is

Changed? Frances Westley,Brenda Zimmerman, Michael Q. PattonRandom House

Canada,2006



Complex Nonlinear Dynamics• Nonlinear: Small actions can have large

reactions. “The Butterfly Wings Metaphor”• Emergent: Self-organizing, Attractors• Dynamical: Interactions within, between, and

among subsystems and parts within systems can volatile, changing

• Getting to Maybe: Uncertain, unpredictable, uncontrollable

• Co-evolution: Integrating innovation and evaluation

43

Three Cups of TeaBaltistani proverb:

First cup you share, you are a stranger.

Second cup, you are an honored guest.

Third cup, you are in relationship.



“A Leader's Framework for Decision Making” by David J. Snowden and Mary E. Boone, Harvard Business Review,November, 2007:

Wise executives tailor their approach to fit the complexity of the circumstances they face.

45

Wise evaluators tailor their approach to fit the complexity of the circumstances they face.



5 Types of DE

1. Ongoing development and adaptation2. Preformative evaluation to support

exploration and innovation3. Supporting local adaptation of general

principles to navigate top-down and bottom-up forces for change

4. Evaluating major systems change5. Evaluating in turbulent, disaster situations

47


Challenge:Matching the evaluation process and design to the nature of the situation:

Contingency-basedEvaluation

48



ReferencesDevelopmental Evaluation: Applying Complexity

Concepts to Enhance Innovation and Use. Guilford Press, June 2010.

Getting to Maybe: How the World Is Changed? Frances Westley, Brenda Zimmerman, Michael Q. Patton, Random House Canada, 2006

Utilization-Focused Evaluation, 4th ed., Michael Quinn Patton, Sage, 2008.

50

Documents

Michael Quinn Patton DE Book Launch