MULTICULTURAL HEALTH · health-related programs and the like. This shortcoming is unfortunate considering the increasing cultural diversity of communities within the United States

MULTICULTURAL

HEALTHEVALUATION

Overview of Multicultural and CulturallyCompetent Program Evaluation

Issues, Challenges and Opportunities

OVERVIEW OF MULTICULTURAL AND CULTURALLY

COMPETENT PROGRAM EVALUATION

Issues, Challenges and Opportunities

Fall 2003

Written by Rodney Hopson, Ph.D.Duquesne University

In partnership with:

Social Policy Research Associates1330 Broadway, Suite 1426Oakland, CA 94612Phone: (510) 763-1499

This paper was made possible through the support of The California Endowment’s

Diversity in Health Evaluation Project. For more information on the Diversity in Health

Evaluation Project, please contact Dr. Astrid Hendricks-Smith at (818) 932-3528 or

Social Policy Research Associates at (510) 763-1499.

�

ISSUES AND CHALLENGES IN ADVANCING THE MULTICULTURAL

AND CULTURALLY COMPETENT EVALUATION OF PROGRAMS

Until very recently, multicultural and culturally competent approaches to evaluation

have not been given the same kind of attention in the field as they have been given in

many other disciplines. Whereas recent discussions, conference meetings and requests

for proposals have drawn attention to multicultural and culturally competent

evaluation, there is a dearth of knowledge about how multiculturalism and cultural

competence can be applied in the evaluation of social programs, community organizations,

health-related programs and the like. This shortcoming is unfortunate considering the

increasing cultural diversity of communities within the United States.

The failure of university departments, training programs and professional development

programs to teach evaluation approaches and methods that address issues of culture and

diversity contributes to a scattered focus on the topic and results in little formal

treatment of multicultural and culturally competent evaluation. Some might argue that

this failure stems from the simple lack of agreement on the meaning of basic concepts

such as cultural context, cross-cultural, cultural responsiveness, cultural competence

and cultural sensitivity.1 An additional problem is that journals for the evaluation

practitioner and academic are either skeptical of cultural competence in evaluation or

silent about it. As a result of these compounding factors, the potential benefits of using

cultural competence and multicultural awareness in programs involving differences in

race, culture and power in this country remain largely unfulfilled.

Those who attempt to study and advance knowledge of multicultural and culturally competent

evaluation inevitably find that the central issue is to move beyond narrow culture-bound

assumptions toward diverse sociocultural perspectives and experiences. As Nick Smith and

ISSUES AND CHALLENGES

OVERVIEW OF MULTICULTURAL PROGRAM EVALUATION | Issues, Challenges and Opportunities

page 1

1 This paper highlights the peculiar distinctions between these terms according to evaluation scholars; rather,the concept of multicultural/culturally competent evaluation will be used interchangeably with these terms asa way to show their interrelated nature.

Soojung Jang (2002) suggest in their analysis of cultural sensitivity in a South Korean

context, cultural differences must be understood at the level of social structure and

intercultural or cross-cultural communication. That is, cultural differences are not

merely surface variations in style, preference and behavior, but fundamental differences in

how people experience social life, evaluate information, decide what is true, attribute causes to

social phenomena and understand their place in the world.

Thus, developing multicultural and culturally competent evaluation strategies and

approaches implies a paradigm shift in the field. Under the new paradigm, certain skills

and attributes are likely to be of critical importance for evaluators as they seek to

improve evaluation practice across and within racially and ethnically diverse

populations. Moreover, competence in a multicultural context will involve

recognizing the “epistemological ethnocentricism” (Reagan, 1996) that privileges

the dominant worldview and values of the White middle class. If evaluators are to

challenge the “tendency to make one’s own community the center of the universe

and the conceptual frame that constrains all thought” (Gordon, et. al, 1990:15)

— and thereby question the hidden assumptions and norms used in evaluation —

multiculturalism and cultural competence will need to become the new lenses

through which we reexamine what we do. The challenge for evaluators is to

understand how awareness and knowledge of cultural differences in evaluation

work can contribute to different kinds of understandings about what evaluation is

and what it can be.

Paper Purpose

This paper intends to help fill a conceptual void in the evaluation field by reflecting on the

history and significance of the “movement” to incorporate multiculturalism and cultural

competence into the field. Within it, efforts are made to highlight how the new focus on

culture in evaluation integrates and crosscuts multiple disciplines, relying on similar

ways of understanding the importance of cultural understanding and competence.

ISSUES AND CHALLENGES


page 2

Another purpose of the historical background and overview is to set the stage for identifying

the important tenets of culturally competent evaluation, and for suggesting important

considerations for designing appropriate programs, standards, interventions and measures

that incorporate the uniqueness of cultural groups. Overall, the paper aims to document the

timeliness of understanding how the study of culture is relevant in the evaluation context.

A HISTORICAL OVERVIEW OF MULTICULTURAL/CULTURALLY

COMPETENT EVALUATION: INTERNATIONAL AND NATIONAL STRANDS

A historical overview of multicultural/culturally competent evaluation reveals that the

movement has two strands. One has historically focused on evaluator skills, capacities

and frameworks in an international, cross-cultural context; that is, how North American

(and sometimes western European) evaluators understand how culture matters when

they do evaluation in countries other than their own. The other has focused on

evaluator skills, capacities and frameworks, as well as a host of accompanying issues, in

the context of the multicultural diversity of the United States. Rarely do scholars who

focus on the international cross-cultural aspects of evaluation merge ideas with those who focus

on the more domestic multicultural aspect of evaluation and vice versa. However, as this

overview intends to show, there are obvious similarities between the two.

The International, Cross-Cultural Strand

Arguably, the work that has most influenced the consideration of culture in the

evaluation field from the international, cross-cultural perspective is Michael

Patton’s (1985) edited volume New Directions on Program Evaluation. Patton

wondered “What happens when we export the ideas, concepts, models, methods, and

values of evaluation to other countries and cultures?” and used the volume to answer

this question, thus provoking evaluators from generally Western perspectives to

consider the problems and potentials of doing cross-cultural international

evaluations. While Patton’s reference to the exportation of ideas, concepts, models,

HISTORICAL OVERVIEW


page 3

methods and values implies a one-way transfer of ideas (from the West to the rest

of the world instead of or in addition to the other way around), his work filled a

gap in the consideration of how culture matters in evaluation.

The authors in the Patton volume suggest how to take into account cultural issues,

factors and context in Canadian, Caribbean, Dutch, Egyptian and Israeli settings,

suggesting how variations in the perception of the purposes, methods, and

priorities of evaluation matter in an international, cross-cultural context. Cultural

differences in the conception of time and their implications for the expectations

and goals of an evaluation is a common theme. For example, Merry Merryfield

(1985) contends that “[d]iffering cultures’ perceptions of time can create not only

logistical problems in development projects, but also affect conceptualization of

the project and its evaluation.” Similarly, Marlene Cuthbert (1985) writes:

“Perhaps the cultural pattern to which North Americans need most to be

sensitized is the sense of time. Although urban and rural areas as well as

individuals vary somewhat in their observance of time, time is generally regarded

not as clock time but as time for human interactions.”

Michael Seefeldt (1985), in the same volume, explains that adapting to a host

culture is vital to the evaluator who intends to be culturally competent in an

international cultural context. The “conduct of inquiry must be grounded in a

sense of what the values are,” he claims, “since the selection of questions for

inquiry, the problems and potential in inquiry strategies, the use of data, the

impact of findings and even the evaluation timetable are all enmeshed in values

underlying custom.” Merryfield makes a similar point about differences in core

values. Reflecting on a technical assistance program aimed at building schools for

Masai children in East Africa, she reports learning that the children’s parents did

not want their children to attend school “because Western schooling conflicted

with some highly regarded beliefs about work and family relationships.” She

explains that such cultural differences are not peculiar to East Africans:

HISTORICAL OVERVIEW


page 4

While these case reports introduced the difficulties and challenges faced by Western-based

evaluators working in unfamiliar international settings, they also initiated interest in understanding

international and cross-cultural evaluations, as evidenced in the growth of international evaluation

in the U.S. in recent years; the development of a topical interest group in the American

Evaluation Association (AEA), which annually hosts papers and discussions on these topics; and

the growth of international cooperation among evaluation professional associations worldwide.

Extending Patton’s seminal work, others (Ginsberg, 1988; Smith, 1991; Smith, et al., 1993)

have further explicated the importance of recognizing the complexity of evaluation in

international cultural contexts and the unique sets of issues

and considerations faced by evaluators who evaluate

projects and programs cross-culturally. In his review of

current approaches in the evaluation of donor-funded

projects and programs in developing countries, Michael

Bamberger (2000) describes a major challenge: “to reconcile

the information requirements of U.S. and European

governments and funding agencies, and the preferred

evaluation approaches in these countries, with the

development information needs, research traditions, and

Similar misunderstandings can happen when Western evaluators assume

that project staff in another culture hold similar attitudes towards work.

The Western idea of the Protestant work ethic, which views effectiveness

as the result of task-oriented efficiency and concern for achievement,

stands in sharp contrast to government bureaucrats in many nations, who

value work only as it offers them and their families security and stability.

Their beliefs about whether their work can effect change and the value

that they place on accomplishments may heavily influence project

planning and program effectiveness (1985:6).

HISTORICAL OVERVIEW

The Western idea of the Protestant

work ethic, which views effectiveness

as the result of task-oriented efficiency

and concern for achievement, stands in

sharp contrast to government

bureaucrats in many nations.{OVERVIEW OF MULTICULTURAL PROGRAM EVALUATION | Issues, Challenges and Opportunities

page 5

social and political contexts in the host countries.” Smith and colleagues (Smith, Chircop, &

Mukherjee, 1993; Smith & Jang, 2002) echo the importance of reciprocal understandings in

the context of developing culturally-sensitive evaluation standards. To Smith, et al, the

development of new standards, the study of differences in practices, or any analysis of cultural

influences in evaluation inevitably require involvement by members and non-members of

culture groups. Thus, mutually beneficial exchanges between U.S. evaluation practitioners and

colleagues and those from developing and foreign countries is a fundamental step in bridging and

understanding international cross cultural contexts in evaluation.

The Domestic, Multicultural Strand

In the United States, the dominant threads of discussion about multicultural/culturally

competent evaluation have focused on attending to cultural differences among the diverse racial

and ethnic groups in this country. These discussions have been strongly influenced by the

disciplines of education and psychology. To date, these discussions in evaluation have been

most influenced by interdisciplinary perspectives in education and psychology largely due to the

preponderance of educationists/educators and psychologists in the evaluation field.

A number of scholars in educational research (Fuller & Clarke, 1994; Tillman, 2002),

teacher education (Foster, 1995; Gay, 1998; Hollins & Oliver, 1999; Ladson-Billings,

1994, 1990; McAllister & Irvine, 2000; Ogbu, 1982; Pewewardy, 1999; Trumbull, et al,

2000), learning styles (Boutte & Deflorimonte, 1998; Irvine & York, 1995), early

childhood education (Lubeck, Jessup, DeVries, & Post, 2001), human development

(Ogbu, 1981; Slaughter-Defoe, Nakagawa, Takanishi, & Johnson, 1990), assessment

(Hood, 1999; Solano-Flores & Nelson-Barber, 2001), and educational

anthropology/ethnography (Au, 1980; Dumont, 1972; Eisenhart, 2001; Leacock, 1977,

Ogbu, 1982; Philips, 1983; Philips, 1972; Watson-Gegeo, 1988) have all contributed

to the study of cultural context in education and how cultural influences and factors

impinge on teaching and learning. These contributions, important in understanding

the academic achievement and failure of ethnically and culturally diverse learners,

have strong relevance for multiculturally aware evaluation.

HISTORICAL OVERVIEW


page 6

The influence of psychology is evident in the terms “cultural context” and “cultural

competence,” both of which have considerable currency in the psychological literature.

As two of the most discussed terms among scholars and practitioners interested in

ethnic minority issues, the terms are replete in community psychology, cross cultural

and ethnic psychology, counseling psychology and psychotherapy. In explaining the

introduction of cultural competence and cultural context into the discipline, scholars

(c.f. Sue, 1998; Betancourt & Lopez, 1993) in psychology point to ongoing

demographic changes and the ever-increasing ethnic diversity of the U.S., as well as the

important philosophical and paradigm shifts that signal the grown of this subfield

(Graham, 1992; Peng & Nisbett, 1999; Spencer, et al, 2001).

In the evaluation field itself, Anna-Marie Madison’s seminal work, the edited volume New

Directions for Program Evaluation (1992), brought to attention how the development of

evaluations should be sensitive to multicultural peoples and perspectives in an American

setting. Madison’s own paper in the volume underscores the important role that

ethnically and racially diverse program participants play in program planning and

design, beginning with how these groups define social problems relevant to them. Not

only did Madison suggest that conceptual constructs for defining social problems be

framed in the cultural context of these racially and ethnically diverse participants, she

borrows from international cultural understandings suggested by Smith, Cuthbert and

Patton to highlight the relevance and connection between understanding the dynamics

of cultural diversity within the U.S.

Moreover, inherent in Madison’s work, and that of others who address cultural

context in evaluation, is attention to broader, sociopolitical issues that affect non-

dominant groups in this country such as lingering effects and implications of racism,

gender inequality, class stratification and White upper class privilege. For instance,

James Davis’ (1992) work in the Madison volume accentuates and encourages “the

use of a contextual framework in the identification and analysis of race differences.”

His main point — consistent with the other papers in the volume, as well as the

HISTORICAL OVERVIEW


page 7

general focus on multicultural context in evaluation — is that we need to use

program theory and improved evaluation methods to focus on the delivery of services

to racial and ethnic minority populations. In doing so, multicultural and culturally

competent evaluation approaches highlight the need for adequate program conceptualizations,

sensitive measures, appropriate generalization of findings and knowledge of the populations

served (Hilliard, 1989; Kirkhart, 1995).

Karen Kirkhart’s 1994 AEA presidential address laid an important foundation for

multicultural/culturally competent evaluation. She called on evaluators to “turn

our knowledge, attitudes and skills surrounding multicultural communication

inward, reflexively, to our own profession, to promote an open exchange of ideas

across disciplines, epistemologies and areas of application, to advance the

dialogue productively” (1995:8). Kirkhart’s work, which emphasizes how social justice

and equity are inextricably linked with how we capture multiple cultural perspectives in

an accurate and sound manner, may eventually pave the way for an extension of topics

in evaluation, such as those related to oppression, discrimination, power differences,

and democracy (Greene, 1997; Hood, 2000; House and Howe, 1998; Greene,

Millett, & Hopson, 2004 forthcoming; Mertens, 1999; Stanfield, 1999).

Stafford Hood’s (2001, 1998) work on culturally-responsive evaluation has also been

noteworthy in addressing the multicultural context of evaluation, particularly for

placing issues of race and culture at the forefront when evaluators work with and within

communities of color (Frierson, Hood, & Hughes, 2002). Expanding and framing

Robert Stake’s (1973) notions of responsive evaluation, Hood acknowledges the “vital

importance of qualitative data, of shared lived experiences and of responsiveness to

critical concerns and issues of the members of the setting being evaluated.” Hood’s

work advances applications of multicultural/culturally competent evaluation by

recognizing the unique set of skills needed by persons of color who concentrate their

efforts on their own culturally diverse communities. His notions of cultural

responsiveness implies that the direct experiences of evaluators of color are critical to

informing evaluations of programs serving racial and cultural groups and the cultural

HISTORICAL OVERVIEW


page 8

responsive evaluator recognizes the importance of rapport, cultural nuances and

nonverbal communication when conducting observations, interviews and making

interpretations in evaluation data. Embedded in Hood’s discussion of the importance of

culture in evaluation is the concomitant call and challenge for the evaluation community to

attract, embrace and cultivate more evaluators of color in the field.

PROFESSIONAL ORGANIZATIONS AND MEETINGS AS A CATALYST

In addition to the scholarly perspectives advanced by scholarship in evaluation

journals and periodicals, professional organizations and meetings centered on

evaluation have been important vehicles to promote notions of multiple cultural

perspectives in the field. In recent years, the American Evaluation Association, the

Relevance in Culture and Assessment in Evaluation conference at Arizona State University

and the National Science Foundation have played important roles in pushing the cultural

context envelope to include larger numbers of evaluation practitioners and scholars to

deliberate on the relevance and significance of cultural context in evaluation.

One notable example is the American Evaluation Association’s (AEA) initial phase of the

Building Diversity Initiative (BDI) funded by the W.K. Kellogg Foundation. Supported by

the AEA Board, the initiative intends to help community leaders, policymakers, service

providers and other professionals design programs and policies that are culturally sensitive

and relevant and encourage evaluators with the capacity to work across cultures. In

improving the quality and effectiveness of evaluation, a further important goal is to develop

a larger pool of culturally diverse evaluators (American Evaluation Association, 2000).

Another example was The Cultural Context of Educational Evaluation: The Role of

Minority Evaluation Professionals workshop that was sponsored by the National

Science Foundation (NSF) in June 2000. This workshop brought together a select

group of evaluators of color to present conceptual papers and engage in

meaningful dialogue about the importance of cultural context in educational

PROFESSIONAL ORGANIZATIONS


page 9

evaluation. The papers and discussions emphasized national concerns about the

need to build the capacity of the evaluation community to more effectively

engage in evaluations within culturally diverse settings and to increase the

number of trained evaluators of color (National

Science Foundation, 2000). At one significant

level, the proceedings of the workshop provide

impetus for this critical review. Issues, questions and

recommendations garnered from the workshop serve

as a jumping-off point for some topics to be

explored in this project. Namely, one theme that

reverberated in the first session on evaluation of

educational achievement of underrepresented

minorities was the need to examine a litany of

considerations and factors, cultural or otherwise, in

explaining assessment of student achievement and performance. Knowing this,

the participants argued would foster better understanding of program theory,

designs, interventions and evaluations as it relates to cultural context.

A follow-up workshop on cultural context among Native Americans, also sponsored

by NSF, occurred in April 2002. Much like the initial workshop, a host of evaluators

and educators specific to the Native American community reflected on issues relating

to the academic achievement of Native American students, opportunities for Native

American evaluators and ways to expand networks of evaluators in the Native American

community (National Science Foundation, 2002). Acknowledging the potential

interaction of cultural context and evaluation activities, issues raised in both

workshops, one participant acknowledged:

PROFESSIONAL ORGANIZATIONS


page 10

The papers and discussions

emphasized national concerns about

the need to build the capacity of

the evaluation community to more

effectively engage in evaluations

within culturally diverse settings....}

These two efforts by AEA and NSF over the last few years signal an important

beginning to address culture and context within the evaluation field. It is not

fair or accurate to suggest that any one particular approach or theoretical

disposition is responsible for the recent flurry of activity pertaining to these

deliberations in the larger evaluation field. It is possible to consider, however, a

plethora of topical discussions on cultural context help to expand its currency

and contribute to the current climate and timeliness of this movement.

SYNTHESIS OF MULTICULTURAL/CULTURALLY COMPETENT

EVALUATION: FIVE BASIC TENETS

The two strands of multicultural and culturally competent evaluation have in common a

number of key concepts. These have been synthesized by the author into five basic tenets of

multicultural/culturally competent evaluation. They are not to be considered as “guiding

principles” per se, but as a basis for developing our understanding of the components of

multicultural and culturally competent evaluation generally. A brief, hardly exhaustive,

discussion follows the listing of each tenet.

In cases where evaluations are constructed, implemented and/or interpreted across

cultural lines, evaluators must be especially vigilant in exploring the possible

disconnects in foundational knowledge among those who are being evaluated, doing

the evaluation and using the evaluation. The community whose programs are being

evaluated must have a meaningful presence in constructing both the goals of, and the

means to, the evaluation. Moreover, the community context must be clearly

represented to those who will interpret the evaluation, its processes and outcomes to

render decisions around program design and funding (Jolly, 2002).

SYNTHESIS


page 11

(1) The social location of the evaluator matters

An evaluator’s “lived” experiences, as determined by his or her class background, racial

and ethnic identity, educational background and so on, help to shape the assumptions and

frames of reference that he or she brings to the evaluation process (Cuthbert, 1985;

Guzman, 2002; Merryfield, 1985; Hood, 2001). Since these assumptions and frames of

reference have a large impact on how the evaluator constructs knowledge, it is important

for the evaluator to be aware of the influences of his or her social location, and how it

differs from that of people being evaluated or using the evaluation. The social location of

the evaluator also can have an impact on how knowledgeable the evaluator is about the

meanings and issues of race and culture. Davis’ (1992) reflections on the complicated

nature of race and the need for evaluators to understand its explanatory power are helpful

in understanding this aspect of culturally competent evaluation.

(2) Evaluators play a role in furthering social change and social justice

Because evaluators “are more than technicians” (Madison 1992), they have the ability

— some would say the duty — to recognize asymmetric power relations and to

challenge systems and mechanisms of inequity and injustice in hope of dismantling

oppression against cultural groups on the fringes of our larger society. In other words,

evaluators aim to generate inclusive, emancipatory knowledge. This advocacy position

is informed by several evaluation theories and approaches, including transformative

paradigms (Mertens, 1999) and feminist theories (Hood & Cassaro, 2002).

(3) Avoiding ethnocentrism means embracing multiple cultural perspectives

The culturally competent evaluator is able to shift between diverse perspectives, recognizes

the limits of narrow culturocentric standards and ideas, and values reciprocal cultural

understanding (Smith & Jang, 2002). These goals can be realized by selecting team members

who can assist in “translating” evaluation from multiple cultural contexts (Hood, 2000;

Merryfield, 1985; Smith, 2002) or by actively involving people of color in the stakeholder

evaluative process (e.g. Madison’s 1992 primary inclusion concept). Becoming fluent in

SYNTHESIS


page 12

multiple cultural perspectives inevitably challenges Western cultural and epistemological

considerations that suggest diversity is deficiency or difference is pathological (Hilliard, 1989;

Hopson, 2001; Gordon, et al, 1990; Merryfield, 1985; Patton, 1999).

(4) Culture is central to the evaluation process

The culturally competent evaluator understands the profound way in which culture

(including racial and ethnic identity, social origin, class background, etc.) shapes

worldview, values and norms, and thereby impacts the uses of, reactions to, and legitimacy

of, any evaluation. Notions of multicultural validity (c.f. Kirkhart, 1997, 1995) and

culturally responsive evaluation (c.f. Hood, 2001) are two ways of viewing the significance

of culture in evaluation. In these two cases, conceptual constructs for defining social

problems incorporate a cultural dimension and use the lived experiences of diverse cultural

groups to inform and interpret evaluation information. Eric Jolly (2002) suggests that the

culturally competent evaluator ought to have an awareness of the cultural norms and

experiences of people with whom he works so as to have an understanding “how these

norms will play out in the context of evaluation instruments and protocols.”

(5) Culturally and ethnically diverse communities have contributions to

make in redefining the evaluation field

The standards, guidelines, methods and paradigms of the evaluation field need to be

rethought, and underserved and marginalized culturally diverse groups have an

important role to play in this process (Cuthbert, 1985; Hood, 2001; Hopson, 1999;

Madison, 1992; Merryfield, 1985; Seefeldt, 1985). Reflecting on his work in southern

Africa with the San of Namibia, Rodney Hopson (2001) suggests that the process and

structure of evaluation efforts around traditionally underserved communities needs

decolonizing around how evaluation thinking and design needs to better incorporate

various communities’ knowledge in the whole evaluation learning and capacity-building

process. The contributions of these diverse and culturally, and often marginalized groups

ultimately triggers alternative ways for responding to evaluative processes of change and

transformation both locally and globally (Smith, 1999; Stanfield, 1999).

SYNTHESIS


page 13

IMPLICATIONS FOR MOVING THE EVALUATION FIELD FORWARD

The five basic tenets of multicultural/culturally competent evaluation suggest that a

number of practitioners and scholars in the field have already begun to think about the

relevance and importance of culture in evaluation. In thinking about next steps and

ways to advance the evaluation field, a few proposals are suggested below. The

proposals, while intended to take into consideration some efforts already underway in

the evaluation profession via the professional association of evaluators and other

initiatives, reflect the author’s attempt to propose next steps to both strengthen and

deepen notions of multicultural/culturally competent evaluation:

INCLUDE STANDARDS, ETHICS AND GUIDELINES THAT REFLECT AND VALUE

MULTICULTURAL/CULTURALLY COMPETENT EVALUATION

The integration of evaluation standards and guidelines which integrate the centrality

and place of culture is an important advancement in the field, consistent not only

with other disciplines but with the current efforts to build the multicultural/culturally

competent evaluation capacity for those who use and do evaluation. This proposal

presupposes that proponents can both articulate the value added aspects of culture in

evaluation, they can also influence leaders and mechanisms in the field and

profession to consider revising standards, ethics, and guidelines.

EXPAND PIPELINES FOR EVALUATORS OF COLOR THROUGH INTERNSHIPS,

FELLOWSHIPS AND OTHER TRAINING VEHICLES

Although evaluation training programs for the professional evaluator through

summer institutes and the professional association, as well for students in

university degree programs, more efforts are needed both to integrate culturally

competent/multicultural evaluation frameworks and notions in these training

and degree-granting programs, post-baccalaureate and graduate level pipelines

can play critical roles in establishing training sites, building consortiums of

school districts, nonprofit organizations, foundations and others, and creating a

IMPLICATIONS


page 14

pool of culturally and ethnically diverse evaluation faculty/trainers. This

proposal presupposes that resources can be leveraged by foundations,

universities, school districts, non-profit organizations and other stakeholders to

build a swelling of support around these initiatives.

ENCOURAGE JOURNALS, MONOGRAPHS AND OTHER EVALUATION-

ORIENTED OUTLINES TO PUBLISH MULTICULTURAL/CULTURALLY

COMPETENT EVALUATION

One important way to build the knowledge base and exposure to the topics

and themes around multicultural/culturally competent evaluation is through a

variety of publication venues (e.g. special issues, featured articles, reflections in

the field, teaching evaluation methodology, etc…). This proposal presupposes

that editorial advisory boards are both culturally and ethnically diverse in its

representation as they create opportunities within these publication outlets for

discussion and address of culture, race and diversity in evaluation.

CONCLUSION AND SUMMARY

Considering the clear signs that the topic of culture in evaluation is receiving growing

attention, this overview underscores the timeliness and importance of clarifying our

understanding multicultural and culturally competent evaluation and rethinking how the

current generation of evaluation students and practitioners can respond to this trend in the

field. Certainly, the notion of making culture central to the evaluation process has sufficient

roots, both in international cross-cultural work and domestic multicultural evaluation

(despite their unique histories) to serve as a strong foundation for further development. The

attention now being given to multicultural/culturally competent evaluation suggests that we can

expect more courses, training programs and professional development opportunities to be devoted to

multiculturally aware and culturally competent evaluation, and for evaluators to continue to

deliberately integrate this concept in evaluation guidelines, approaches and standards.

CONCLUSION


page 15

REFERENCES

American Evaluation Association. (2000). Proposal for the Initiative for BuildingDiversity Among the Evaluation Community, Phase 1. Magnolia, AR: Author.

Au, K.H. (1980). Participant structure in a reading lesson with Hawaiian children:Analysis of a culturally appropriate instructional event. Anthropology & EducationQuarterly, 11(2), 91-115.

Bamberger, M. (2000). The evaluation of international development programs. A viewfrom the front. American Journal of Evaluation, 21, 95-102.

Betancourt, H. & Lopez, S.R. (1993). The study of culture, ethnicity, and race inAmerican psychology. American Psychologist, 48(6), 629-637.

Boute, G.S. & DeFlorimonte, D. (1998). The complexities of valuing culturaldifferences without overemphasizing them: Taking it to the next level. Equity &Excellence in Education, 31(3), 54-62.

Cuthbert, M. (1985). Evaluation encounters in third world settings: A Caribbeanperspective. In M.Q. Patton (Ed.) Culture and evaluation. New Directions forProgram Evaluation, 25. San Francisco, CA: Jossey Bass.

Davis, J.E. (1992). Reconsidering the use of race as an explanatory variable inprogram evaluation. In A-M. Madison (Ed.), Minority Issues in Evaluation. NewDirections for Program Evaluation, no. 53 (p. 55-67). San Francisco: Jossey-Bass, 1992.

Dumont, R.V. (1972). Learning English and how to be silent: Studies in Sioux andCherokee classrooms. In Cazden, C.B., John, V.P., & D. Hymes (Eds.). Functions oflanguage in the classroom (pp. 344-369). NY: Teachers College Press.

Eisenhart, M. (2001). Changing conceptions of culture and ethnographic methodology:Recent thematic shifts and their implications for research on teaching. InRichardson, V. (Ed.) Handbook of research on teaching, 4th edition. Washington,DC: American Educational Research Association.

Foster, M. (1995). African American teachers and culturally relevant pedagogy. InBanks, J.A., & McGee-Banks, C.A. (Eds.) Handbook of research on multiculturaleducation. NY: Macmillan.

REFERENCES


page 16

Frierson, H, Hood, S., & Hughes, G. (2002). Strategies that address culturallyresponsive evaluation. Frechtling, J. The 2002 User-Friendly Handbook for ProjectEvaluation. Arlington, VA: NSF.

Fuller, B. & Clarke, P. (1994). Raising school effects while ignoring culture? Localconditions and the influence of classroom tools, rules, and pedagogy. Review ofEducational Research, 64(1), 119-157.

Gay, G. (1988). Designing relevant curricula for diverse learners. Education and UrbanSociety, 20(4), 327-340.

Ginsberg, P.E. (1988). Evaluation in cross-cultural perspective. Evaluation and ProgramPlanning, 1(2), 189-195.

Gordon, E., Miller, F., & Rollock, D. (1990). Coping with communicentric bias inknowledge production in the social sciences. Educational Researcher, 19, 14-19.

Graham, S. (1992). “Most of the subjects were white and middle class”: Trends inpublished research on African Americans in selected APA journals, 1970-1989.American Psychologist, 47(5), 629-639.

Greene, J. (1997). Evaluation as advocacy. Evaluation Practice, 18(1), 25-35.

Greene, J., Millet, R., & Hopson, R. (2004, forthcoming). “Evaluation asDemocratizing Practice”. Braverman, M., Constantine, N., & Slater, J.K. (Eds.)Putting evaluation to work for foundations and grantees. San Francisco: Jossey-Bass.

Guzman, B.L. (2002). Examining the role of cultural competency in programevaluation: Visions for new millennium evaluators. In Donaldson, S. & Scriven,M. (Eds.), Evaluating social programs and problems (pp. 167-181) Claremont, CA:Claremont Graduate University.

Hilliard, A.G. (1989). Kemetic (Egyptian) historical revision: Implications for cross-cultural evaluation and research in education. Evaluation Practice, 109, 7-23.

Hollins, E. & Oliver, E.I. (Eds.) (1999). Pathways to success in school: Culturallyresponsive teaching. Mahwah, NJ: Lawrence Erlbaum Associates.

Hood, D.W. & Cassaro, D.A. (2002). Feminist evaluation and the inclusion ofdifference. New Directions for Evaluation, 96, 27-40.

Hood, S. (2001). Nobody knows my name: In praise of African American evaluatorswho were responsive. New Directions for Evaluation, 92, 31-43.

REFERENCES


page 17

Hood, S. (2000). Commentary on Deliberative Democratic Evaluation. In K.E. Ryan & L.DeStefano (eds.). Evaluation as a democratic process: Promoting inclusion, dialogue, anddeliberation. New Directions for Evaluation, no. 85. San Francisco: Jossey-Bass, 2000.

Hood, S. (1999). Culturally responsive performance-based assessment: Conceptual andpsychometric considerations. Journal of Negro Education, 67, 187-196.

Hood, S. (1998). Responsive evaluation Amistad style: Perspectives of one African-American evaluator. In R. Sullivan (ed.) Proceedings of the Stake Symposium onEducational Evaluation. Urbana-Champaign: , IL: University of Illinois atUrbana-Champaign.

Hopson, R. (2001). Global and local conversations on culture, diversity, and socialjustice in evaluation: Issues to consider in a 9/11 era. American Journal ofEvaluation, 22(3), 375-380.

Hopson, R. (1999). Minority issues in evaluation revisited: Re-conceptualizing andcreating opportunities for institutional change. American Journal of Evaluation,20(3), 445-451.

House, E. R. (1991). Evaluation and social justice: Where are we? In M.W. McLaughlinand D.C. Phillips (Eds.) Evaluation and Education: At Quarter Century. NinetiethYearbook of the National Study for the Study of Education. Chicago: TheUniversity of Chicago Press.

House, E. R. and Howe, K.R. (1998). The issue of advocacy in evaluation. AmericanJournal of Evaluation, 19(2), 233-236.

Jolly, E.J. (2002). On the quest for cultural context in evaluation: All things are notequal – they never were. National Science Foundation, Directorate for Educationaland Human Resources. The cultural context of educational evaluation: A Native-American perspective. Arlington, VA: NSF.

Irvine, J.J. & York, D.E. (1995). Learning styles and culturally diverse students: Aliterature review. In Banks, J.A., & McGee-Banks, C.A. (Eds.) Handbook ofresearch on multicultural education. NY: Macmillan.

Kirkhart, K. E. (1997). Racism and issues of multicultural validity in programevaluation theory. Paper presented at the American Psychological Associationannual convention, 15-17 August.

REFERENCES


page 18

Kirkhart, K.E. (1995). Seeking multicultural validity: A postcard from the road.Evaluation Practice, 16, 1-12.

Ladson-Billings, G. (1994). Dreamkeepers: Successful teachers of African Americanchildren. San Francisco: Jossey Bass.

Ladson-Billings, G. (1990). Culturally relevant teaching: Effective instruction for Blackstudents. College Board Review, 155, 20-25.

Leacock, E. (1977). Race and the “we-they dichotomy” in culture and classroom.Anthropology & Education Quarterly, 8(2), 152-159.

Lincoln, Y.S. (1991). The arts and sciences of program evaluation.” Evaluation Practice,12, 1-7.

Lubeck, S., Jessup, P., deVries, M., & J.Post (2001). The role of culture in programimprovement. Early Childhood Research Quarterly, 16, 499-523.

McAllister, G. & Irvine, J.J. (2000). Cross cultural competency and multiculturalteacher education. Review of Educational Research, 70(1), 3-24.

Madison, A-M. (Ed.) (1992). Minority issues in program evaluation. New Directions forProgram Evaluation, 52. San Francisco: Jossey Bass.

Madison, A-M. (1992). Primary inclusion of culturally diverse minority programparticipants in the evaluation process. In A-M. Madison (Ed.), Minority Issues inEvaluation. New Directions for Program Evaluation, no. 53 (p. 35-43). SanFrancisco: Jossey-Bass, 1992.

Merryfield, M. (1985). The challenge of cross-cultural evaluation: Some views from thefield. In M.Q. Patton (Ed.) Culture and evaluation. New Directions for ProgramEvaluation, 25. San Francisco, CA: Jossey Bass.

Mertens, D. (1999). Inclusive evaluation: Implications of transformative theory forevaluation. American Journal of Evaluation. 20, 1-14.

National Science Foundation (2002). The cultural context of educational evaluation: ANative American perspective. Arlington, VA: NSF.

National Science Foundation (2000). The cultural context of educational evaluation: Therole of minority evaluation professionals. Arlington, VA: NSF.

REFERENCES


page 19

Ogbu, J.U. (1982). Cultural discontinuities and schooling. Anthropology & EducationQuarterly, 13(4).

Ogbu, J.U. (1981). Origins of human competence: A cultural-ecological perspective.Child Development, 52, (2), 413-429.

Patton, M.Q. (1999). Some framing questions about racism and evaluation: Thoughtsstimulated by Professor John Stanfield’s “Slipping through the front door”.American Journal of Evaluation, 20(3), 437-443.

Patton, M.Q. (Ed.) (1985). Culture and evaluation. New Directions for ProgramEvaluation, 25. San Francisco, CA: Jossey Bass.

Peng, K. & Nisbett, R.E. (1999). Culture, dialectics, and reasoning aboutcontradiction. American Psychologist, 54(9), 741-754.

Pewewardy, C. (1999). Culturally responsive teaching for American Indian students. InHollins, E. & Oliver, E.I. (Eds.) Pathways to success in school: Culturally responsiveteaching (pp. 85-100). Mahwah, NJ: Lawrence Erlbaum Associates.

Philips, S.U. (1983). The invisible culture: Communication in classroom and community onthe Warm Springs Indian Reservation. NY: Longman.

Philips, S.U. (1972). Participant structures and communicative competence: Warm Springschildren in community and classroom. In Cazden, C.B., John, V.P., & D. Hymes (Eds.).Functions of language in the classroom (pp. 370-394). NY: Teachers College Press.

Reagan, T. (1996). Non-western educational traditions. Alternative approaches toeducational thought and practice. Mahwah, NJ: Lawrence Erlbaum Associates.

Seefeldt, F.M. (1985). Cultural considerations for evaluation consulting in the Egyptiancontext. In M.Q. Patton (Ed.) Culture and evaluation. New Directions for ProgramEvaluation, 25. San Francisco, CA: Jossey Bass.

Slaugher-Defoe, D., Nakagawa, K., Takanishi, R., & D. J. Johnson (1990). Towardcultural/ecological perspectives on schooling and achievement in African- andAsian-American children. Child Development, 61, 363-383.

Smith, L.T. (1999). Decolonizing methodologies: Research and indigenous peoples. London: Zed.

Smith, N. (2002). International students’ reflections on the cultural embeddedness ofevaluation theory. American Journal of Evaluation, 23(4), 481-492.

REFERENCES


page 20

Smith, N. & Jang, S. (2002) Increasing cultural sensitivity in evaluation practice: ASouth Korean illustration. Studies in Educational Evaluation, 28(1), 61-69.

Smith, N.L., Chircop, S., & Mukherjee, P. (1993). Considerations on the development ofculturally relevant evaluation standards. Studies in Educational Evaluation, 19, 1, 3-13.

Smith, N.L. (1991). The context of investigations in cross-cultural evaluations. Studiesin Educational Evaluation, 17, 3-21.

Solano-Flores, G. & Nelson-Barber, S. (2001). On the cultural validity of scienceassessments. International Journal of Research in Science Teaching, 38, 553-573.

Spencer, M.B., Noll, E., Stolzfus, & V. Harpalani. (2001). Identity and school adjustment:Revisiting the “acting white” assumption. Educational Psychologist, 36(1), 21-30.

Stanfield, J.H. (1999). Slipping through the front door: Relevant social sciences in thepeople of color century. American Journal of Evaluation, 20(3), 415-431.

Stake, R.E. (1973). Program evaluation, particularly responsive evaluation. Keynoteaddress at the conference “New Trends in Evaluation”, Institute of Education,University of GÖteborg, Sweden, October.

Sue, S. (1998). In search of cultural competence in psychotherapy and counseling.American Psychologist, 53(4), 440-448.

Tillman, L.C. (2002). Culturally sensitive research approaches: An African-AmericanPerspective. Educational Researcher, 31(9), 3-12.

Trumbull, E., Rothstein-Fisch, C., & Greenfield, P.M. (2000). Bridging cultures in ourschools: New approaches that work. San Francisco: WestEd. [ERIC: DocumentReproduction Service No. ED 440 954].

Watson-Gegeo, K.A. (1988). Ethnography in ESL: Defining the essentials. TESOLQuarterly, 22(4), 575-592.


page 21

REFERENCES

www.calendow.org

The California Endowment21650 Oxnard Street, Suite 1200

Woodland Hills, CA 91367800.449.4149

Established by Blue Cross of California

CM PA/MHEval Overview 5/04

Documents

MULTICULTURAL HEALTH · health-related programs and the like. This shortcoming is unfortunate considering the increasing cultural diversity of communities within the United States