UFE Curriculum 2 UFE Primer - UFE & Research … · 11/2/2009 · DECI-UFE-Curriculum-2009 Premises of UFE No evaluation should go forward unless and until there are primary intended

DECI-UFE-Curriculum-2009

UFE Curriculum

UFE Primer

In this module we will:

1. Define UFE

2. List the premises behind UFE

3. Distinguish the main steps of UFE

2


Program evaluation is…

Systematic collection of information about the

activities, characteristics, and results of programs to

make judgments about the project / network, improve

or further develop project / network effectiveness,

inform decisions about future programming, and / or

increase understanding.


Utilization focused evaluation is…

Evaluation done for and with specific, intended primary

USERS for specific, intended USES.





Premise: evaluations should be JUDGED by their utility

and ACTUAL USE.




Premise: evaluations should be JUDGED by their utility

and ACTUAL USE.

USE concerns how real people in the real world

APPLY evaluation findings and experience the

evaluation PROCESS. (Patton, 2008: 37 - Ch.2)




UFE is…

A decision-making framework for

enhancing the utility and actual

use of evaluations. (Patton, 2008a: slide 9)



Premises of UFE

No evaluation should go forward unless and until there are

primary intended users who will use the information that

can be produced.

Primary intended users are involved in the process.


Premises of UFE



can be produced.


Evaluation is part of initial program design. The primary

intended users want information to help answer a question

or questions.


Premises of UFE



can be produced.




or questions.

Evaluator’s role is to help intended users clarify their

purpose and objectives.


Premises of UFE



can be produced.




or questions.

Evaluator’s role is to help intended users clarify their

purpose and objectives.

Make implications for use a part of every decision

throughout the evaluation – it is the driving force of the

evaluation process. (Patton, 2008a: slide 20)



Think back about your best evaluation experience…

To what extent was your best evaluation

experience compatible with UFE?


Think back about your best evaluation experience…

To what extent was your best evaluation experience compatible

with UFE?

Review it in terms of:

• The USERS: were they identified?

• The specific USES: were they clear?

• User ENGAGEMENT: how and why?


1. Project / network readiness assessment.

2. Evaluator readiness and capability assessment.

3. Identification of primary intended users.

4. Situational analysis.

5. Identification of primary intended uses.

6. Focusing on evaluation.

7. Evaluation design.

8. Simulation of use.

9. Data collection.

10. Data analysis.

11. Facilitate use.

12. Meta-evaluation.

UFE in 12 steps


Primary tasks for Step 1: Steps in DECI

1. Assess research partners’

commitment to doing useful

evaluation based on an

explanation of UFE

2. Assess if projects/networks

are ready to commit time and

resources

3. Determine willingness by

research partners to assess

various stakeholder

constituencies to select

primary intended users.

4. Assess what needs to be

done to enhance readiness.

1.

2.

3.

4.


Primary tasks for Step 2: Steps in DECI

• Assess the match between

the evaluators’ knowledge

and what will be needed in the

evaluation.

1. Assess the match between

the evaluators’ commitment

and the likely changes of the

situation.

2. Assess the match between

the evaluators’ skills and what

will be needed in the

evaluation.

3. Ensure evaluators are

prepared to be judged by the

use of the evaluation by the

primary intended users.

•

1.

2.

3.


Traditional (conventional) Vs Developmental Evaluation

Render definitive judgment of success or failure

Measure success against predetermined goals

Position the evaluator outside the assure

independence and objectivity

Design the evaluation based on linear cause-

and-effect logic models

Aim to produce generalizable findings across

time and space

Accountability focused on and directed to

external authorities, stakeholders and

funders

Accountability to control and locate responsibility

Evaluator determines the design based on the

evaluator’s perspective about what is

important. The evaluator controls the

evaluation

Evaluation results in opinion of success or

failure, which creates anxiety in those

evaluated


Traditional (conventional) Vs Developmental Evaluation

Render definitive judgment of success or failure

Measure success against predetermined goals

Position the evaluator outside the assure

independence and objectivity

Design the evaluation based on linear cause-

and-effect logic models

Aim to produce generalizable findings across

time and space

Accountability focused on and directed to

external authorities, stakeholders and

funders

Accountability to control and locate responsibility

Evaluator determines the design based on the

evaluator’s perspective about what is

important. The evaluator controls the

evaluation

Evaluation results in opinion of success or

failure, which creates anxiety in those

evaluated

Provide feedback, generate learnings, support

changes in direction

Develop new measures and monitoring mechanisms

as goals emerge and evolve

Position evaluation as an internal, team functions

integrated into action and ongoing

interpretive processes

Design evaluation to capture systems dynamics,

interdependencies, models and emergent

interconnections

Aim to produce context-specific under-standings to

inform ongoing innovations

Accountability centred on the innovators’ deep

sense of fundamental values and

commitment

Learning to respond to lack fo control and stay in

touch with what’s unfolding and thereby

respond strategically

Evaluator collaborates with those engaged in the

change effort to design an evaluation

process that matches philosophically with

an organization’s principles and objectives

Evaluation support ongoing learning


Sources

McConnell Foundation. (2006) Sustaining social innovation: Developmental

evaluation. www.mcconnellfoundation.ca/default.aspx?page=139 11/Feb/09

Patton, M.Q. (2008) Utilization focused evaluation, 4th Edition. Sage.

Patton, M.Q. (2008a) Utilization focused evaluation. Presentation to the AEA.

Documents

UFE Curriculum 2 UFE Primer - UFE & Research … · 11/2/2009 · DECI-UFE-Curriculum-2009 Premises of UFE No evaluation should go forward unless and until there are primary intended