A plain English Description of the components of evaluation

Craig Love, Ph.D.Westat

An evaluation is a systematic way of determining whether the project activities are consistent with what is intended and is achieving the end results that was determined was needed to be accomplished.

Just share common methods and values Research intended to create generalizations,

to create new knowledge Evaluation is intended to provide

practitioners, funders, community members and other stakeholders information as to whether the intent of the program was met.

Evaluation is amenable to community participation- ergo fits most Native American cultures.

Formative evaluation is a systematic collection of data that, in turn, is used to assess progress toward goals and to verify that the project is implemented as intended

Used to adjust the program and is ongoing

Summative evaluation is an assessment of how well the program was implemented and how well it achieved its intended outcomes

Usually completed at or near the end of the project

For example, it can be used to determine whether the project is worthy of continuing after funding

Documentation of the initiating and implementation of the project.

Included in both formative and summative evaluations

Important to explanation of what is associated with the outcomes

Assessment of how well the outcome was achieved.

Addresses questions- Did you meet your goals? Did you get what you wanted out of the project?

Most effective when associated with process

An important aspect of the program planning and evaluation process

Usually based on a survey of consumers, practitioners and other stakeholders

Provides information on deficits- barriers to the desired level of performance.

Is a good way to establish how to design the program to meet the identified needs in the community to be served.

We have determined that there is a need

We have the following resourcesWe are aware of the following barriers to be overcome

We have a program designed to overcome barriers and/or create outputs that will help produce the outcomes needed

The program design begins with the needs/goals of the project

The interventions (e.g. rewrite math curriculum) are intended to produce outputs (e.g. more interest on the part of students) that create the desired outcomes (e.g. improved math skills) and more Native American students pursue science and math careers (long-term outcome).

Identified problem- students in our college do not do well in math achievement tests (Need identified in achievement tests)

Want to improve achievement scores by 20% within the next two years (goal- intended outcome)

Barrier, the students are using a curriculum designed for big cities and white culture (Needs assessment by survey of students, review of text materials)

Intervention- modify curriculum so that the examples, issues etc. are appropriate for our population

Expect to gain better response to students, help make the curriculum more relevant to students (output)

Higher math achievement scores will be achieved (outcome) the students are better able to relate to the math text.

Logic model- Need to improve math scores- the curriculum found to be irrelevant to students- adjust curriculum to make it more relevant to the students- increase student attention and interest in math curriculum- increase scores on math achievement test

Formative evaluation- review of revised text to determine wither the new text is indeed more relevant. Also determine whether students are more attentive and teachers approve of the new curriculum, identify unintended barriers

Process evaluation- identify whether the new curriculum is implemented, identify issues in implementation, creation of the modified curriculum, accepted by teachers, students seem more interested? Note attendance records, teacher comments, observe classes.

Outcome evaluation- assess student perspective on new math curriculum, monitor math grades, assess standard scores before and after implementation of new curriculum

Questions or comments?