California Reducing Disparities Project, Phase 2

Population Evaluation

GuidelinesFor Native American populations

i

Population Evaluation Guidelines for Native Americans

California Reducing Disparities Project, Phase 2

PIRE Native American Technical Assistance Team: Roland S. Moore, PhD, Anna Pagano, PhD, Juliet P. Lee, PhD, Elizabeth Waiters, PhD, Janet L. King, MSW, Claradina Soto, PhD (USC), Cathleen Willging, PhD, Narinder Dhaliwal, MA (ETR), Jennifer Silver-Herman (ETR), Carlos Recarte, MCP Prepared for: Office of Health Equity California Department of Public Health Under CRDP contract #16-10574, funds made possible by the Mental Health Services Act

Prepared by: HBSA/PIRE (Pacific Institute for Research and Evaluation) 180 Grand Avenue, Suite 1200 Oakland, CA 94612 http://natap.prev.org April 18, 2017

ii

TableofContents

Acknowledgements iii

Introduction 1

A. Community driven evaluation 7

B. Use mixed methods 10

C. Community consent 14

D. Safeguard tradition 16

E. Locally attuned evaluation 19

F. Community reflection 22

G. Advice from the community 24

Makingthecase 26

References cited 28

Appendices: 37

Evaluationinstrumentsthat

Granteesmaychoosetouse

iii

For their insights, we are grateful to the California Reducing Disparities Project (CRDP)

team (Office of Health Equity staff, especially Edward Soto, Rafael Colonna, and

leadership, fellow TAPs, SWE, and above all the Native American Grantees and Native

Vision authors from CRDP Phase 1). We also acknowledge the insights and

contributions of consultants serving this Native American Technical Assistance Provider

contract: Michael E. Bird, MPH, MSW, Roxane Spruce Bly, Nicole R. Bowman-Farrell,

PhD, Raymond Daw, MSW, Bonnie Duran, PhD, and Nadine Tafoya, MSW. With

gratitude, we acknowledge Dr. Kaye Herth, Dr. James Allen and the People Awakening

Team, Dr. Paul Masotti, Dr. Angela Snowshoe, Dr. Les Whitbeck and colleagues for

their survey instruments offered as potential tools in the appendix. We also thank PIRE

librarian Julie Murphy for her assistance with the literature review.

We dedicate this work to the future generations,

and in honor of those who came before us

1

Commissioned by the California Department of Public Health Office of Health

Equity, this booklet presents evaluation guidelines that are specific to the populations

served by the California Reducing Disparities Project (CRDP) Phase 2 Native American

Implementation Pilot Project (IPP) Grantees. In the spirit of community partnership, we

offer the contents of this booklet to the Grantees in support of the Grantees’ efforts to

document, evaluate, document, and disseminate the results of their groundbreaking work.

We aim for the booklet to reflect and respond to the vision and goals of the Grantees’

proposed projects and evaluation plans, and we hope Grantees, their evaluators, and the

communities they serve will find the booklet useful. The booklet is written as a narrative

technical report. Details and examples of research tools (for example, survey measures)

are included in the Appendix.

Defining “Guidelines”

The assignment from the Office of Health Equity is as follows: “Guidelines shall

be focused on three critical factors: 1) Fulfilling the requirement for effectively

incorporating community stakeholders in the full evaluation process; 2) Ensuring the

evaluation is culturally and linguistically appropriate for the individuals that will be

served by the population, including addressing any cross-population issues; and 3)

Ensuring the timeline is compatible with the Contractor’s need to coordinate technical

assistance across seven IPPs.”

As the term “guidelines” has multiple definitions, we discussed it with the

directors and evaluators of the Native American IPP Grantees before and during the

CRDP Kickoff Meeting in March of 2017. It was determined that “guidelines” should

refer to suggestions and considerations (rather than requirements) for the Grantees to use

as they finalize their evaluation plans for their community-defined evidence practices.

These population-specific guidelines complement the overarching evaluation guidelines

offered by the Statewide Evaluator. The guidelines in this booklet may help the Grantees

2

and their evaluators to account for the unique historical, political and cultural contexts

that have shaped the lives and health of American Indians and Alaska Natives.

In this document, we provide evaluation tools, measures, and concepts that

Grantees may adopt if they find them useful. These guidelines consist of suggestions or

options based on the wisdom of those who have conducted evaluations in Tribal

communities in the recent past. However, the collection of evaluation tools placed in our

metaphorical basket of shared offerings may change and grow as we return to the basket

over the course of the next five years. As Technical Assistance Provider, we will

continue collecting and reviewing instruments, including ones specified at the request of

Grantees.

As the CRDP Phase 2 participants (Grantees, Technical Assistance Providers,

Statewide Evaluator, as well as the Office of Health Equity) collaborate over the next five

years, we will gain important new understandings. Therefore, we view these initial

guidelines as “version 1.0,” certain to be revised extensively throughout the evaluations

of the seven funded Native American community-defined evidence practices.

Terminology

We recognize that some Grantees or their community members may prefer the

term “Native American,” some may prefer the term “American Indian/Alaskan Native,”

some may prefer “Indigenous,” and some may prefer to be referred to by the name/s of

their specific tribe/s. Grantees have noted that these different preferences reflect

community members’ different experiences, thoughts, and feelings about their sense of

individual and collective identity, as well as terms they use in interactions with

government agencies, researchers, and funders. The people we serve in the CRDP project

include California Natives and other American Indians and Alaska Native tribes

(whether or not they are Federally or State recognized), within sovereign rural Tribal

settings as well as in urban locations. For the purposes of these Population Evaluation

Guidelines we use these terms interchangeably, acknowledging that we often refer to

“Native Americans” in keeping with the phrase used in Native Vision, the Native

American population report produced in CRDP Phase 1.

3

Native Vision: A Source of Inspiration

The Native Vision report produced by the Native American Stakeholder

Population Workgroup in CRDP Phase I (California Reducing Disparities Project, 2012,

pp. 30-31) was the product of intensive deep thinking, visioning, research, and

conversation by a distinguished panel of consultants, refined through many meetings with

community members throughout California. In shaping this booklet, we follow the

recommendations which are summarized at the conclusion of the Native Vision report

produced by the Native American Stakeholder Population Workgroup in CRDP Phase I

(California Reducing Disparities Project, 2012, pp. 30-31). We include thumbnail

summaries of those recommendations within the relevant sections of this report.

A Summary of Best Practices

There is a large and growing body of literature on evaluation of health promotion

programs for Native American populations. In this document, we discuss examples of

best practices that recur in the literature. Table 1, adapted from Kawakami et al. (2007),

provides recommendations for conducting culturally appropriate program evaluations in

Native American communities. While Kawakami and colleagues’ work is informed

specifically by a Native Hawaiian cultural background, their recommendations align with

those of evaluation scholars from other American Indian and Alaska Native communities,

whose work we review throughout this document. We have included Table 1 because it

summarizes several main points explored below:

Having a community-generated research agenda. To be useful for the

communities involved, an evaluation should begin with a needs assessment. Questions

might include:

How do community members define and understand mental health?

What are their views of mental health services offered in the community?

How could these services be improved?

Community input can be solicited during the evaluation process through a variety of

methods, such as town halls and community advisory boards, which we will discuss

below.

4

Using mixed-methods evaluation approaches. There are many ways to conduct

a needs assessment and gather information for evaluation purposes. Surveys are the most

popular method, but may not always be the best method, especially on their own. It is

often beneficial to use qualitative methods of data collection, such as open-ended

interviews or focus groups, for program evaluation if they seem appropriate for the

team’s research questions and acceptable to participating community members.

Disseminating evaluation findings to community participants as well as

academic/funding agencies. Many community members appreciate being appraised of

evaluation results, especially if they participated in the evaluation. Dissemination can

take place in a variety of venues and formats, such as a debriefing with the local

community to validate and further discuss results, highlighting findings on a local radio

station or in a newsletter, and providing a report with all results/findings to those who

participated.

Using visual, oral and other culturally specific formats for sharing results.

This recommendation can apply to results being shared both with community members

and with funding agencies. For instance, in some communities, storytelling or other

narrative practices may be a customary way to express thoughts about the community’s

historical trauma and ways to heal. Another example is using PhotoVoice or VideoVoice,

in which community members record images from their daily lives that show something

about community mental health or the impact of an intervention.

Linking the data to specific community contexts. There are over 560 federally

recognized Tribes, with many people living not only on their respective reservations but

in urban settings. American Indian and Alaska Native evaluation scholars stress that

research findings cannot be separated from the community in which they were generated.

This concept goes against the grain of “generalizable” data by acknowledging that

interventions will unfold in different ways depending on community characteristics.

Culturally responsive evaluation makes the case that effective interventions must be

adapted to fit the needs and reality of each community in which they are implemented.

Applying evaluation findings to empower communities in resolving

challenges and celebrating community strengths. This is perhaps the most important

general guideline offered by the American Indian and Alaska Native evaluation

5

specialists whose work we discuss throughout this document. If community members are

involved from the beginning of the evaluation process and are aware of the results, there

is a greater chance that they can use the information to make changes and sustain

effective programs.

In this document, we provide evaluation tools and concepts that Grantees may adopt

if they find them useful. The collection of tools placed in our metaphorical basket of

shared offerings may change and grow as we return to the basket over the course of the

next five years. In pulling together these resources we were guided by the Grantees’

proposals, requests, and recommendations, and thus this report does not constitute a

systematic review of the literature. We apologize for any oversights, and welcome

comments and suggestions.

Native American Technical Assistance Provider contact:

Roland S. Moore, PhD roland@prev.org (510) 883-5770 http://natap.prev.org

6

Table 1. A conceptual framework for Indigenous evaluation practice, adapted from Kawakami et al. (2007).

Methodology

Function Primarily Indigenous (includes some

mainstream and adds dimensions)

Primarily widely practiced mainstream

Purpose and goals Set by community agenda. Externally generated.

Driving question

Has the community been affected in a positive way as a result of the program/ project/initiative?

Have proposal goals/objectives been met?

Methodology Quantitative, qualitative, and more.

Primarily quantitative.

Data

Spiritual, cultural, historical, social, emotional, cognitive, theoretical, situated information. Graphics, narratives, culturally created manifestations valid to that place.

Objective decontextualized data. Objective validity and reliability.

Analysis Cultural and environmental significance.

Statistical and practical significance and effect size.

Format for findings

Narratives, stories, relationships, photos, DVDs, CDs, videos.

Written reports, charts, tables, graphs, databases.

Conclusions and recommendations

Shared among project, community, evaluator, and funder. Revised community agenda.

Submitted to funder. Fulfillment of contract.

Impact

Value added, lessons learned, clarity, empowerment.

Revised funding priorities.

7

Native Vision Recommendation 3A. Ensure a community driven evaluation process. Require the use of community-based participatory research methods within each community. It is essential to move beyond "cookie cutter" paper surveys to community members and standardized forms to project staff as methods to evaluate the success of program implementation. … with a strong grassroots evaluation strategy that is driven, literally, from the ground up.

Community-based participatory research (CBPR) approaches are recommended

by many American Indian and Alaska Native and allied researchers, evaluators, service

providers, and community members as the most ethical, equitable, and effective for

conducting evaluation in American Indian and Alaska Native communities (Baydala,

Ruttan, & Starkes, 2015; Jernigan et al., 2015; Starkes & Baydala, 2014). Incorporating

CBPR methods into the evaluation of health interventions among American Indians and

Alaska Natives can take many forms, such as:

Co-design of data collection instruments with community consultants

(Gonzalez & Trickett, 2014; Johansson, Knox-Nicola, & Schmid, 2015;

Perry & Hoffman, 2010);

Empowering community members to collect and analyze evaluation data

(Bowman, Francis, & Tyndall, 2015);

Ensuring joint ownership of data between community members,

researchers, and community-serving organizations (or primary/exclusive

Tribal ownership with data-sharing privileges for researchers and non-

Tribally-affiliated community organizations) (Pahwa et al., 2015);

Participatory manuscript development, in which both academic and

community partners contribute to manuscripts for publication of

evaluation findings (Jernigan et al., 2014);

Cooperatively applying evaluation findings to refine intervention

materials, such as culturally appropriate health education media

8

(Burhansstipanov et al., 2014; Cueva, Cueva, Dignan, & Landis, 2016;

Eschiti et al., 2014; Scott et al., 2016).

Some evaluations of Native American CBPR projects have included a measure of

“project ownership” to assess the degree to which the project is community-driven.

During a 3-year diabetes intervention conducted in Alaska Native communities, Cargo et

al. (2011) measured academic, service provider, and community participants’ perceptions

of primary project ownership. At baseline (Time 1), participants perceived service

providers as having primary ownership. At Times 2 and 3, however, they perceived

service providers and community advisory board members as equally sharing ownership

of the project, thus showing an increase in the degree to which the project was

community-driven (the academic partners were not perceived as having primary

ownership of the project at any point).

Other evaluation efforts have begun with a pre-intervention assessment of

community climate, such as the Community Readiness Model (CRM) (Oetting et al.,

1995) to elicit community members’ perceptions of the health problem to be addressed,

as well as their interest, engagement and potential participation in the planned

intervention. The CRM uses a combination of key informant interviews and a scoring

system to assess various dimensions of community readiness (Donnermeyer, Plested,

Edwards, Oetting, & Littlethunder, 1997; Plested, Smitham, Thurman, Oetting, &

Edwards, 1999; Thurman, Plested, Edwards, Foley, & Burnside, 2003). Its overarching

purpose is to honor and incorporate the community’s views on the most urgent health

issues and what should be done to address them.

The Choctaw Nation of Oklahoma successfully used the CRM for pre-

implementation planning of a cardiovascular disease (CVD) prevention intervention

(Peercy, Gray, Thurman, & Plested, 2010). While previous studies had shown

disproportionately high rates of CVD among Native Americans, the research and

evaluation team wanted to learn more about Tribal members’ perceptions of CVD and

other health problems in their everyday lives. First, they conducted key informant

interviews with Choctaw community leaders representing several community systems (e.g.,

health care, spirituality, social services) to elicit their views on the most pressing health

9

issues in the community. They then used the CRM’s scoring system to gauge the

community’s readiness for a CVD intervention. Through the key informant interviews,

they learned that methamphetamine use was of greater concern to the community than

CVD, one of the distal effects of long-term methamphetamine use. Based on this

information, the project team re-worked their intervention to focus on methamphetamine

use and its many health consequences, including CVD. This is one example of many in the

literature that shows the importance of considering community perceptions and reception of

potential interventions before taking action. Community-partnered evaluation before,

during, and after health interventions can help to increase their resonance and cultural

appropriateness.

Summary:

Community-based participatory research (CBPR) approaches are effective for ensuring community-driven evaluation efforts.

CBPR principles can be incorporated into evaluation in several ways, such as inviting community members to co-create data collection instruments, participate in data collection, co-author reports of evaluation findings, and help create or revise health programs based on results.

Some evaluators recommend measuring community “ownership” of health programs co-created by health care organizations, researchers, and community members.

Tools such as the Community Readiness Model can help evaluators to determine community health concerns and “climate” regarding possible program or policy changes.

10

Native Vision Recommendation 3B. Use mixed methods evaluation to ensure strongest reflection of successes and challenges. Community-based participatory research and evaluation is rapidly becoming the most valid way of reflecting information and priorities from communities; however, in order to ensure the most valid information it is often critical to use a combination of qualitative and quantitative evaluation methods. We strongly encourage the content of all evaluation to be driven by the community through a participatory process to ensure validation from a community and a scientific perspective.

Mixed-methods approaches, which combine both qualitative and quantitative

methods of data collection and analysis, appear frequently in published studies regarding

evaluation of community health programs for American Indians and Alaska Natives.

Although quantitative tools such as survey instruments are used frequently in evaluation,

they can often be strengthened and contextualized through the addition of qualitative

measures. While quantitative data are necessary for describing outcomes in terms of

what was accomplished, qualitative data can be helpful for describing how and why

intervention activities contributed to observed outcomes (the latter is also known as the

theory of change (Weiss, 1997).

Storytelling and other narrative approaches are often recommended in the

program evaluation literature by American Indian and Alaska Native researchers as

culturally appropriate sources of process data (Kawakami et al., 2007; LaFrance, 2004;

LaFrance, Nichols, & Kirkhart, 2012; Lavallée, 2009). Narrative and conversational

methods may also improve the quality of evaluation data by putting respondents at ease

and allowing them the necessary space and time to reflect and express their thoughts

(Bowman et al., 2015). These methods may also provide additional opportunities for

community collaboration, as elders can play a pivotal role in guiding talking circles and

other traditional forms of group dialogue.

11

Key informant interviews, or in-depth interviews with community stakeholders

such as health care or social service providers, spiritual leaders, and others can support

evaluation by illuminating important factors that evaluators might not think to include on

survey instruments. These might include different ways of using tobacco in ceremonial

versus secular contexts (Margalit et al., 2013), for instance, or diverse resilience

strategies for dealing with historical trauma (Reinschmidt, Attakai, Kahn, Whitewater, &

Teufel-Shone, 2016). Key informant interviews are most useful at the outset of an

evaluation process, because they tend to offer a broad view of community systems. Key

informants usually interact with large groups of community members and have unique

perspectives on health and social issues. They can also point the evaluator toward others

who might be able to share information or data relevant to the evaluation.

As an example, The Alaska Native Colorectal Cancer Family Outreach Program

team used key informant interviews to examine program strengths and challenges, such

as outreach response (Redwood et al., 2016). During these interviews, they learned that

community members had mixed reactions to the screening outreach methods in use (e.g.,

cold-calling family members of cancer patients to encourage them to be screened). This

information alerted the team to the need for increased staffing and training of Alaska

Native patient navigators, who could approach the family members personally in a more

sensitive manner.

Focus groups, which feature a facilitator who asks questions and engages in

dialogue with several interviewees simultaneously, are another possibly useful

component of a mixed-method evaluation plan. For example, one team that evaluated a

physical activity intervention for American Indian youth used both a survey developed by

their Community Advisory Board and participant focus groups to collect mixed-methods

data on program effectiveness (Perry & Hoffman, 2010). The focus groups allowed youth

to provide more input into the research process; for example, they described a conceptual

distinction between “exercise” and “sports,” which the researchers incorporated into the

evaluation design. They found that youths’ motivations for engaging in “exercise” versus

“sports” were different, which suggested that they should be addressed in distinct ways

within the intervention. Another evaluation team combined focus groups with field

observations and surveys in order to triangulate (compare) individual/group and self-

12

report/observed data sources regarding the effectiveness of a Native women’s heart

health Talking Circle (Ziabakhsh, Pederson, Prodan-Bhalla, Middagh, & Jinkerson-Brass,

2016).

Qualitative approaches can also be useful for creating, elaborating or

operationalizing survey measures. For instance, measures of cultural identity or

cultural connectedness (Snowshoe, Crooks, Tremblay, Craig, & Hinson, 2015;

Snowshoe, Crooks, Tremblay, & Hinson, 2017) may be included in evaluation of Native

American-specific health programs as part of a decolonizing approach (Smith, 1999) to

counteract the effects of historical trauma by strengthening participants’ connection to

traditional culture. However, cultural identity is a fluid phenomenon that shifts according

to social, spatial, and temporal contexts. Quantitative measures to assess cultural identity

(e.g., knowledge of traditional culture, sense of relationship to American Indian and

Alaska Native communities, Tribal affiliation) may not fully capture its complexity.

Supplementing these with qualitative measures (e.g., in-depth interviews with program

recipients and providers, community stakeholders) may provide a fuller understanding of

how community members view and live their cultural identities on an everyday basis

(Jette & Roberts, 2016).

Mixed methods can enhance not only community data collection, but also team-

based self-assessment. For instance, the Native American Cancer Prevention Model

team (Trotter, Laurila, Alberts, & Huenneke, 2015) incorporated qualitative queries into

their standard logic model—which included inputs/resources, activities, outputs, and

outcomes—to better understand the mechanisms by which their intervention activities led

to desired outcomes. They combined the qualitative queries with periodic tracking and

assessment data to continuously monitor the progress of their program.

For the qualitative component, they convened their community partnership team

(consisting of academic researchers and a community advisory board) and asked the team

questions such as: “Is the program working as planned?” and “To what extent and in

what time frame are the three primary programs being implemented as planned? (Trotter

et al., 2015, p. 3) Their multi-pronged self-assessment approach was designed to

periodically monitor progress and facilitate real-time adjustments in the intervention.

13

Whether evaluating the effectiveness of an intervention or their own team,

evaluators can consider using a variety of methods as needed to honor and reflect the

social, cultural, and political contexts within which they work. While quantitative

methods may be normative within Western scientific discourse and practice, Indigenous

Evaluation Frameworks (LaFrance & Nichols, 2008) call for the elevation of culturally

appropriate qualitative methods to equal status within evaluation research.

Summary:

The use of “mixed methods” (quantitative and qualitative ways of collecting information) can strengthen evaluation findings.

Quantitative data can tell us “how much or how many,” while qualitative data can tell us “why or how.”

Examples of quantitative data collection include surveys, program enrollment counts, and calculating rates of specific health diagnoses from medical records.

Examples of qualitative data collection include individual or focus group interviews, observations, and documentation of oral traditions such as storytelling.

14

Native Vision Recommendation 3C. Gather consent from communities as well as individuals. It is essential to gather consent from the communities where the work occurs. Much akin to the research world's Ethical Review Board, nearly every California Native American community has a panel of elders, council members, or community members who serve in this role within the community. It is important to respect the nature of Native Communities and engage the community leaders to ensure work is in alignment with community priorities. This is particularly relevant as we move toward evaluating best/promising practices that may be culturally based and provoke ethical sensitivities around documentation and evaluation.

The literature on ethics in research conducted with Indigenous communities

stresses that individual and community consent are equally important (Brant Castellano,

2004; Dunbar & Scrimgeour, 2006; Flicker & Worthington, 2012; Wax, 1991).

Community consent, often granted through Tribal research review boards, is an

expression of Tribal sovereignty and Indigenous peoples’ self-determination (LaFrance,

2004). Flicker & Worthington (2012) note that obtaining community-level consent may

be less straightforward for research and evaluation conducted in diverse urban

environments where there is no single entity authorized to represent the community’s

interests vis-à-vis knowledge production. In such cases, research review boards at local

academic institutions or clinical agencies may be able to partner with community

organizations to carry out ethics review of planned evaluation activities (Hicks et al.,

2012).

Some Tribal review boards require that researchers protect the identities of not

only individual study participants but also of the tribe or tribes (Morton et al., 2013),

particularly in research concerning stigmatizing conditions (Norton & Manson, 1996).

Indian Health Service guidelines specify that Tribal communities should not be identified

in results without their explicit consent (Freeman, 2004). Aggregating Tribal health data

is one method that has been used successfully, following Tribal leaders’ input, in order to

preserve the confidentiality of research participants from smaller tribes and avoid directly

identifying the participating tribes (Van Dyke et al., 2016).

15

The process of obtaining community consent provides yet another opportunity to

use a community-driven approach. Prior to drafting consent protocols and presenting

them to Tribal or associated IRBs, researchers and evaluators can consult with

community stakeholders about ethical concerns. Benefits of pre-IRB community

consultation include the following:

Community consultants can alert researchers to areas of ethical concern they may

not have considered, and suggest ways to minimize potential risks and hazards.

Community consultants can recommend ways to enhance potential benefits to

communities that elect to participate in intervention and evaluation activities.

Engaging in community consultation can increase the legitimacy of the informed

consent process by allowing community members to consider and make changes

to the IRB protocol prior to submission.

Community consultation can help to increase community members’ sense of

shared responsibility for the proposed intervention, and perhaps encourage them

to participate in the conduct of research and evaluation activities (Dickert &

Sugarman, 2005).

To ensure that research and evaluation are conducted in an ethical way that benefits the

community, Indigenous researchers recommend obtaining pre- and post-intervention

testimonials from community stakeholders regarding the potential for community benefit

(pre-intervention) and the extent to which these benefits actually accrued to the

community (post-intervention) (Ball & Janyst, 2008).

Summary:

When conducting research with Indigenous communities, obtaining informed consent from the community is often as important as obtaining it from the individual.

If there is no Tribal institutional review board (IRB), evaluators can sometimes partner with academic agencies to have their IRBs review the research plan.

Before submitting materials to an IRB (Tribal or otherwise), evaluators ideally should show their research plans to local community stakeholders and request their input on the appropriateness of planned measures and safeguards for confidentiality.

16

Native Vision Recommendation 3D. Set strict criteria for evaluation of cultural and traditional practices. It is essential to protect the integrity of Native American ceremonial knowledge. For evaluation purposes, when a ceremony is administered it must only report the input and outcomes. The ceremony itself may be described as to the purpose, but not the details. The leadership must set strict criteria for evaluation and description of cultural and traditional practices for entities reporting findings as part of the CRDP project.

American Indian and Alaska Native researchers have reflected extensively on the

different values governing Western/academic versus Native American/traditional cultural

contexts (Deloria, 2003; Hernandez-Avila, 1996). While Western academic and scientific

culture demands detailed empirical descriptions of events with an eye toward

comparative study and replicability, American Indian and Alaska Native scholars

emphasize the need to preserve traditional culture and respect its origins: …even though in the world of academia I might feel I had not done anything improper in describing [a sweat lodge ceremony], I know that in the Native American community, among the elders, I could not say the same thing. …just as there would be readers who would be truly respectful of the information, there are those who would feel that my description of details gave them permission to appropriate [the tradition] (Hernandez-Avila, 1996, p. 331). Asking what can and cannot be included. In the course of demonstrating the

effectiveness of culture-driven health programs, evaluators must describe their

components. If community organizations and/or service providers wish to register these

programs as evidence-based practices (EBP), their components should be defined clearly

enough to facilitate replication in other communities (with appropriate adaptation to the

unique characteristics of each community). However, evaluators must be careful not to

expose sacred knowledge or rites in the process of describing program components. To

that end, early discussions with community stakeholders about what can and cannot be

included in data collection and reports should be a part of inclusive evaluation design.

According to Bowman and colleagues, “…information gleaned in a sacred space like a

sweat lodge or teaching circle may not be available to or shared with outside investigators

17

and the wider world in the way that information from more public ceremonies or

discussions might be” (Bowman et al., 2015, p. 345).

Upholding sovereignty and traditions. This guideline also speaks to the

importance of acknowledging each Tribal community’s unique sovereign knowledge and

traditions, the importance and inimitability of elders’ wisdom, and the need to carefully

adapt program content with community input prior to replication attempts in other

communities. However, the meaning of “tradition,” and the determination of which

traditions are amenable to incorporation and adaptation for public health purposes, are

not clear-cut. Sometimes adaptation of cultural traditions occurs in unexpected ways. In

Australia, for example, Aboriginal providers of substance abuse treatment have widely

adopted the North American sweat lodge rather than incorporating local healing

traditions into their programs (Brady, 1995). The sweat lodge model, argues Brady, is

more easily adapted to group therapy than Australian Aboriginal healing ceremonies,

which tend to be private and involve only the traditional healer and the person seeking

help. She describes how Aboriginal treatment providers have consulted Canadian First

Nations Tribal healers for guidance in the proper incorporation of sweat lodge practices,

and conferred with them regarding respectful adaptation to local community contexts.

Acknowledging intracultural variation. In other instances, cultural symbols and

practices that are meaningful for one community (or a group of communities) might not

resonate with others. Further, there is often a great deal of intracultural variation

(differences within the same culture) in how individual members of the same community

respond to particular cultural symbols (Kumanyika, 2003). An example of intracultural

variation is described by a team of evaluators and program designers who created an

intervention to help prevent diabetes in Southwestern American Indian communities

(Willging, Helitzer, & Thompson, 2006). They designed educational materials that

featured two cultural symbols assumed to be relevant to people from a wide range of

Tribal backgrounds: the Storyteller (present in the traditional lore of many Southwestern

tribes) and the Medicine Wheel (prominent within Northern Plains traditional healing

practices). To ensure cultural relevance, the evaluation team conducted focus groups to

pilot test the educational materials with members of the intended audience: local

American Indian women from diverse Tribal backgrounds. In so doing, they discovered

18

that the women had varied reactions to the symbols which were magnified by

generational, Tribal, and urban/rural differences. Many younger women thought the

Storyteller image reinforced stereotypes about traditional femininity and motherhood

with which they did not identify. The Medicine Wheel did not resonate with many of the

women from Southwestern tribes, while others disliked the fluorescent colors the

designers had used for the Wheel image. With the focus group participants’ input, the

team was able to revise the materials using strategies such as vignettes of local

community members describing how they incorporated healthy eating and exercise into

their daily routines.

Summary:

Evaluators should be careful not to reveal any sacred knowledge or details of sacred ceremonies when documenting cultural practices.

Cultural traditions vary substantially between communities.

Even within the same community, ideas about what constitutes tradition, and which traditions can be incorporated or adapted for public health purposes, may vary among individuals.

It is important to consult with community members, and especially the target audience for a health program, during the planning stages in order to ensure that the program is relevant and relatable from their perspectives.

19

Native Vision Recommendation 3E. Utilize a consultant who is experienced conducting evaluation in Native American communities. Community-based participatory evaluation focuses on involvement, development, participation, and empowerment, where the community is seen as the expert with the best ability to identify issues and solutions. This approach can be time-consuming and requires a unique set of evaluation skills on the part of the evaluation team. It is important that whoever is hired in this capacity has experience working in the Native American community and is familiar with the strong similarities between community-based participatory methods and cultural norms relating to evaluation methods. This approach, coupled with mixed-methods evaluation, will ensure that practice-based evidence is evaluated at the standard of evidence-based practices without sacrificing the integrity and need for community-driven evaluation questions and analysis.

It is important for evaluators of American Indian and Alaska Native CDEPs

(community-defined effective practices) to have a deep understanding not only of

colonial history and abuses vis-à-vis Indigenous communities, but also of the traditions

and values of the communities with which they partner (Johnston-Goodstar, 2012).

Perhaps most importantly, they should be aware of the complex history between

researchers and Indigenous communities. Too often, “helicopter researchers” (so named

because they seemingly fly in and fly out without returning to share their findings) have

collected data without proper informed consent, then used this data in ways that either do

not benefit the community directly, or even cause harm (Oberly & Macedo, 2004;

Robertson, Jorgensen, & Garrow, 2004).

With these complexities in mind, Indigenous researchers have identified a number

of “best practices” for evaluation in Indigenous communities. LaFrance & Nichols

(2008, p. 22) created an Indigenous Evaluation Framework consisting of four key values:

“being people of a place,” “recognizing our gifts,” “honoring family and community,”

and “respecting sovereignty” (see Table 2, below).

The first value refers to situating evaluation within the specific geographic, historical,

social, and cultural context of the community partners. Evaluators should resist the

20

tendency to distill community members’ stories and wisdom into de-contextualized units

of data. Combining vignettes and de-identified interview quotes with survey data can

bring the numbers to life, as can the inclusion of community members’ own photos and

videos to document health program activities (as long as proper consent is obtained to

record or take photographs for evaluation purposes). Rather than focusing on

“universalizing” the program for replicability, evaluators should consider, along with

their community partners, about how and why the program worked in a particular

community context (LaFrance et al., 2012). They might ask, for instance, how the

program reflects core community values or honors aspects of the community’s history or

geographical setting.

The second value in the Indigenous Evaluation Framework is the importance of

recognizing individuals’ unique gifts, accomplishments, and contributions to the

evaluation process. This value has implications for evaluation, because it encourages

holistic methods. For example, rather than isolating a single characteristic such as age, a

holistic approach considers multiple aspects of a person. While statistical techniques

such as regression models “control” for certain variables (i.e., examine an isolated

variable’s effect on outcomes while holding other variables constant), these can be

enriched by adding case studies that tell the story of a participant and how the health

intervention impacted his or her life. The third value is engaging communities, not just

individuals, in the evaluation process. This value speaks not only to gaining community

consent for research, but also inviting community members to participate as equals in the

design and conduct of evaluation. The fourth value emphasizes Tribal/community

ownership of information gleaned from evaluation and advocates for ethical research

practice.

Summary:

Evaluators working with Indigenous communities should be aware of the complex historical relationship between tribes and researchers.

Given this complex relationship, evaluators should take special care to share research findings with community members.

The Indigenous Evaluation Framework (LaFrance & Nichols, 2008) provides best practices to orient evaluators working with Indigenous communities.

21

Table 2. Core Values and Evaluation Practice (LaFrance & Nichols, 2008) Core Values Indigenous Evaluation Practice Indigenous knowledge creation context is critical

o Evaluation itself becomes part of the context; it is not an “external” function

o Evaluators need to attend to the relationships between the

program and community o If specific variables are to be analyzed, care must be

taken to do so without ignoring the contextual situation People of a place o Honor the place-based nature of many of our programs

o Situate the program by describing its relationship to the

community, including its history, current situation, and the individuals affected

o Respect that what occurs in one place may not be easily transferred to other situations or places

Recognizing our gifts — personal sovereignty

o Consider the whole person when assessing merit

o Allow for creativity and self-expression

o Use multiple ways to measure accomplishment

o Make connections to accomplishment and responsibility Centrality of community and family

o Engage the community, not only the program, when planning and implementing an evaluation

o Use participatory practices that engage stakeholders

o Make evaluation processes transparent

o Understand that programs may focus not only on individual achievement, but also on restoring community health and well-being

Tribal sovereignty o Ensure Tribal ownership and control of data o Follow Tribal Institutional Review Board processes

o Build capacity in the community o Secure proper permission if future publishing is expected o Report in ways meaningful to Tribal audiences as well as

to funders

22

Native Vision Recommendation 3F. Ensure that each local community is reflected uniquely in its own evaluation process. Local community driven input and direction should be gathered for each community to reflect the range of values and issues seen as important for mental health prevention and early intervention. Information from each of these communities should be integrated to form a quantitative and qualitative evaluation that can be used statewide.

This guideline reflects the holistic nature of the Native Visions report, in that it

incorporates elements of previous guidelines concerning the importance of community-

driven and mixed-methods approaches to evaluation. The guidelines cannot be

considered in isolation from one another, as they contain overlapping elements that are

linked organically within diverse American Indian and Alaska Native value systems.

Guideline 3F also illustrates a challenge for evaluators of CDEPs: the need to situate

evaluation findings within the context of the community or communities in which the

intervention occurred, versus the expectation within public health practice that effective

interventions can be readily adapted and transferred to other communities. American

Indian and American Indian-collaborating evaluation teams have grappled with this

challenge in various ways.

In their description of a CBPR project designed to prevent alcohol abuse and

suicide, Gonzalez & Trickett (2014) emphasize the importance of incorporating each

community’s unique characteristics into assessment measures. In their project, they

worked with two closely related Alaska Native (Yup’ik) communities with different

prevalence and incidence rates of suicide. One community with no recent suicides feared

that asking direct questions about suicide might empower its spiritual essence and cause

suicide to be revisited on the community. The university/community co-researchers

engaged in careful deliberation to resolve this issue, eventually deciding to replace

suicide risk measures with a Reasons for Life scale (their culturally-informed adaptation

of an existing measure). This decision was practical in that it allowed evaluators to

indirectly assess suicide risk via a related construct that was culturally appropriate across

23

Yup’ik communities. Of course, different approaches might be necessary for assessing

suicide risk for other Native communities, given their social, cultural, and political

contexts. This case illustrates the importance of obtaining community input at the

beginning of evaluation efforts, and avoiding assumptions regarding the cultural views of

communities.

Summary:

Evaluations should reflect the local character of the community.

This involves, for instance, collecting information about social and historical context and how this might affect community members’ reception of the health program being evaluated.

Previous recommendations, such as mixed-methods data collection and engaging community members in the evaluation process, will help to ensure accurate reflection of the local community in evaluation reports.

24

Native Vision Recommendation 3G. Develop a community advisory board to ensure evaluation integrates traditional and culturally based services and ensure appropriate community involvement. Many counties do not have a clear understanding of what Native American culturally based services are and how they relate to Native American mental health, best practices, or even community-based evaluation processes. We recommend Native American organizations/ tribes do their own evaluation without relying on state or county evaluators who may not know about Native American issues. It is important that Native American grantees/ contractors not be forced into a prepackaged evidence-based service delivery system that is top down and culturally disengaged.

Research and evaluation efforts that employ a community-driven approach often

seek the input and guidance of a community advisory board (CAB) or, more specific to

evaluation, an evaluation advisory group (EAG). EAGs may include community elders,

Tribal council members, and representatives of diverse community systems, such as

social or health services, spiritual or religious life, and others depending on the focus of

the program being evaluated. If the program serves a subgroup of community members,

such as youth or LGBTQ, the EAG might also include members of that subgroup to

ensure their voices are heard.

EAGs can aid research and evaluation efforts in American Indian and Alaska

Native communities in several ways (Johnston-Goodstar, 2012). At the beginning of the

project, they can be invaluable in helping to define which health problems are the most

urgent to the community. EAG members can also help to establish legitimacy of the

evaluation efforts vis-à-vis the community, resulting in stronger community buy-in and

participation. Further, they can assist in the design of evaluation instruments by advising

evaluators on the “right” questions to ask and how to ask them; and on appropriate

dissemination of findings via community venues. For example, in their evaluation of a

public safety program undertaken by the Oglala Sioux Tribe, Robertson and colleagues

utilized the local Tribal community radio station to share details of the program and

evaluation findings (Robertson et al., 2004). During the Yup'ik Experiences of Stress and

25

Coping project (Rivkin et al., 2013), the research team followed community guidance on

how best to disseminate findings (via community-wide presentations and discussions)

and include community members in the process of translating research findings into an

intervention (via Community Planning Group meetings). In these and other cases,

assembling an EAG or community advisory board helped to establish a consistent conduit

for dialogue between community stakeholders and evaluators on the best ways to conduct

evaluations in local communities.

Summary:

Whenever possible, American Indian and Alaska Native professionals should conduct evaluation of health interventions in Indigenous communities.

Even when evaluators come from the same community where the research is taking place, they can benefit from the guidance of an Evaluation Advisory Group (EAG).

EAGs consist of community leaders and stakeholders, and can offer advice regarding pressing community health problems, how best to engage community members, and ethical research practice.

26

One of the overarching goals of the CRDP is to acknowledge the value of

community-defined effective practices (CDEPs) for promoting community mental health

and preventing mental health problems. Too often, CDEPs have been overshadowed by

the dominant paradigm within Western scientific discourse of evidence-based practice

(EBP). Although communities are the most well-informed on their specific mental health

needs and which practices work best to meet those needs, prevailing health policy and

funding structures privilege and support the use of EBPs, even when they may not be an

ideal fit for local communities (Echo-Hawk, 2011).

When seeking appropriate support for their CDEPs, American Indian and Alaska

Native-serving health organizations often find themselves obligated to provide evidence

of their programs’ effectiveness in ways that may not honor or reflect Indigenous ways of

knowing and Tribal sovereignty (Cochran et al., 2008; Simonds & Christopher, 2013;

Walker, Whitener, Trupin, & Migliarini, 2015). Indigenous researchers and evaluators

argue that in order to overcome the divide between Western and Indigenous epistemes

(knowledge systems), and the privileging of the former to the detriment of the latter, a

paradigm shift is necessary (Kawakami et al., 2007; LaFrance & Nichols, 2008). In these

guidelines, we have discussed some ways that American Indian and Alaska Native

evaluators and their collaborators have begun to make this paradigm shift, such as

incorporating traditional forms of knowledge sharing (e.g., storytelling) as data sources

within evaluation reports.

Publication is a first step toward “making the case” for CDEP effectiveness.

American Indian and Alaska Native research and evaluation teams have published

extensively within public health journals, describing their interventions as well as their

evaluation designs and findings. Examples in the literature include (among others): The

Canoe Journey, a substance use prevention intervention in Pacific Northwest Tribal

communities (Donovan et al., 2015); FORGE AHEAD, a community-driven quality

27

improvement initiative to improve chronic disease care for Alaska Natives (Hayward,

Paquette-Warren, Harris, Naqshbandi Hayward, & Forge Ahead Program Team, 2016);

Circle of Life, a Native American-specific HIV prevention intervention (Kaufman et al.,

2014); CONNECT, a youth suicide prevention intervention developed with the Cherokee

Nation (Komro et al., 2015); the Parenting in Two Worlds intervention for American

Indian families (Kulis, Ayers, Harthun, & Jager, 2016) the Diabetes Prevention Project

for American Indians and Alaska Natives in urban settings (Rosas et al., 2016); an

entrepreneurship training intervention to prevent substance use and suicide among

American Indian and Alaska Native youth (Tingey et al., 2016); an American Indian-

specific substance abuse intervention combining motivational interviewing and the

Community Reinforcement Approach (MICRA) (Venner et al., 2016); and the Youth

Leaders Program, a school-based intervention to prevent substance use, violence, and

ultimately suicide among Alaska Native youth (Wexler et al., 2016). Although this is

nowhere near an exhaustive list, it provides some examples of where and how research

teams have published (often mixed-methods) data on the effectiveness of interventions

designed specifically for Native communities.

Summary:

Increasingly, funding for community health services depends on the use of “evidence-based practices.”

Although evidence-based practices may be effective for some communities, they may not work as well in all communities.

Many American Indian and Alaska Native community health providers are making the case for their own culturally appropriate CDEPs (community-defined effective practices).

Collecting and publishing evaluation data of CDEPs in public health journals is an important step toward gaining recognition of their effectiveness and funding support in order to sustain them.

28

Ball, J., & Janyst, P. (2008). Enacting Research Ethics in Partnerships with Indigenous

Communities in Canada: “Do it in a Good Way”. Journal of Empirical Research

on Human Research Ethics, 3(2), 33-51.

Baydala, L., Ruttan, L., & Starkes, J. (2015). Community-based participatory research

with Aboriginal children and their communities: Research principles, practice and

the social determinants of health. First Peoples Child & Family Review, 10(2),

82-94.

Bowman, N. R., Francis, C. D., & Tyndall, M. (2015). Culturally Responsive Indigenous

Evaluation: A practical approach for evaluating indigenous projects in tribal

reservation contexts. In S. Hood, R. Hopson & H. Frierson (Eds.), Continuing the

Journey to Reposition Culture and Cultural Context in Evaluation Theory and

Practice (pp. 335–359). Charlotte, NC: Information Age Publishing.

Brady, M. (1995). Culture in treatment, culture as treatment. A critical appraisal of

developments in addictions programs for indigenous North Americans and

Australians. Social Science & Medicine, 41(11), 1487-1498.

Brant Castellano, M. (2004). Ethics of Aboriginal research. International Journal of

Indigenous Health, 1(1), 98.

Burhansstipanov, L., Krebs, L. U., Dignan, M. B., Jones, K., Harjo, L. D., Watanabe-

Galloway, S., Petereit, D. G., Pingatore, N. L., & Isham, D. (2014). Findings from

the Native Navigators and the Cancer Continuum (NNACC) study. Journal of

Cancer Education, 29(3), 420-427.

California Reducing Disparities Project. (2012). Native Vision: A Focus on Improving

Behavioral Health Wellness for California Native Americans. Retrieved from

29

http://www.nativehealth.org/sites/dev.nh.edeloa.net/files/native_vision_report_co

mpressed.pdf

Cargo, M. D., Delormier, T., Lévesque, L., McComber, A. M., & Macaulay, A. C.

(2011). Community capacity as an “inside job”: evolution of perceived ownership

within a university-aboriginal community partnership. American Journal of

Health Promotion, 26(2), 96-100.

Cochran, P. A. L., Marshall, C. A., Garcia-Downing, C., Kendall, E., Cook, D.,

McCubbin, L., & Gover, R. M. S. (2008). Indigenous Ways of Knowing:

Implications for Participatory Research and Community. American Journal of

Public Health, 98(1), 22-27.

Cueva, K., Cueva, M., Dignan, M., & Landis, K. (2016). Print Material in Cancer

Prevention: an Evaluation of Three Booklets Designed with and for Alaska's

Community Health Workers. Journal Of Cancer Education, 31(2), 279-284.

Deloria, V. (2003). God is red: A native view of religion. Golden, CO: Fulcrum

Publishing.

Dickert, N., & Sugarman, J. (2005). Ethical Goals of Community Consultation in

Research. American Journal of Public Health, 95(7), 1123-1127.

Donnermeyer, J. F., Plested, B. A., Edwards, R. W., Oetting, G., & Littlethunder, L.

(1997). Community Readiness and Prevention Programs. Journal of the

Community Development Society, 28(1), 65-83.

Donovan, D. M., Thomas, L. R., Sigo, R. L. W., Price, L., Lonczak, H., Lawrence, N.,

Ahvakana, K., Austin, L., Lawrence, A., Price, J., Purser, A., & Bagley, L.

(2015). Healing of the Canoe: Preliminary results of a culturally grounded

intervention to prevent substance abuse and promote tribal identity for Native

youth in two Pacific Northwest tribe. American Indian and Alaska Native Mental

Health Research, 22(1), 42-76.

30

Dunbar, T., & Scrimgeour, M. (2006). Ethics in Indigenous Research – Connecting with

Community. Journal of Bioethical Inquiry, 3(3), 179-185.

Echo-Hawk, H. (2011). Indigenous communities and evidence building. Journal of

Psychoactive Drugs, 43(4), 269-275.

Eschiti, V., Lauderdale, J., Burhansstipanov, L., Weryackwe-Sanford, S., Weryackwe, L.,

& Flores, Y. (2014). Developing Cancer-Related Educational Content and Goals

Tailored to the Comanche Nation. Clinical Journal of Oncology Nursing, 18(2),

E26-E31.

Flicker, S., & Worthington, C. A. (2012). Public Health Research Involving Aboriginal

Peoples: Research Ethics Board Stakeholders' Reflections on Ethics Principles

and Research Processes. Canadian Journal of Public Health, 103(1), 19-22.

Freeman, W. (2004). The protection of potential individual volunteers and tribal

communities in research involving the Indian Health Service (IHS). Retrieved

from http://www.npaihb.org/images/epicenter_docs/irb/docs/Protections_fr.pdf

Gonzalez, J., & Trickett, E. J. (2014). Collaborative measurement development as a tool

in CBPR: Measurement development and adaptation within the cultures of

communities. American Journal of Community Psychology, 54(1-2), 112-124.

Hayward, M. N., Paquette-Warren, J., Harris, S. B., Naqshbandi Hayward, M., & Forge

Ahead Program Team. (2016). Developing community-driven quality

improvement initiatives to enhance chronic disease care in Indigenous

communities in Canada: the FORGE AHEAD program protocol. Health Research

Policy & Systems, 14, 1-12.

Hernandez-Avila, I. (1996). Mediations of the Spirit: Native American Religious

Traditions and the Ethics of Representation. American Indian Quarterly, 20(3/4),

329-352.

Hicks, S., Duran, B., Wallerstein, N., Avila, M., Belone, L., Lucero, J., Magarati, M.,

Mainer, E., Martin, D., & Muhammad, M. (2012). Evaluating community-based

31

participatory research to improve community-partnered science and community

health. Progress in Community Health Partnerships: Research, Education, and

Action, 6(3), 289.

Jernigan, V. B. B., Brokenleg, I. S., Burkhart, M., Magdalena, C., Sibley, C., & Yepa, K.

(2014). The implementation of a participatory manuscript development process

with Native American tribal awardees as part of the CDC Communities Putting

Prevention to Work initiative: Challenges and opportunities. Preventive Medicine,

67(Suppl 1), S51-S57.

Jernigan, V. B. B., Peercy, M., Branam, D., Saunkeah, B., Wharton, D., Winkleby, M.,

Lowe, J., Salvatore, A. L., Dickerson, D., Belcourt, A., D’Amico, E., Patten, C.

A., Parker, M., Duran, B., Harris, R., & Buchwald, D. (2015). Beyond Health

Equity: Achieving Wellness Within American Indian and Alaska Native

Communities. American Journal of Public Health, 105(S3), S376-S379.

Jette, S., & Roberts, E. B. (2016). ‘We usually just start dancing our Indian dances’:

Urban American Indian (AI) female youths’ negotiation of identity, health and the

body. Sociology of Health & Illness, 38(3), 396-410.

Johansson, P., Knox-Nicola, P., & Schmid, K. (2015). The Waponahki Tribal Health

Assessment: Successfully using CBRP to conduct a comprehensive and baseline

health assessment of Waponahki tribal members. Journal of Health Care for the

Poor and Underserved, 26(3), 889-907.

Johnston-Goodstar, K. (2012). Decolonizing evaluation: The necessity of evaluation

advisory groups in indigenous evaluation. New Directions for Evaluation, 136,

109-117.

Kaufman, C. E., Whitesell, N. R., Keane, E. M., Desserich, J. A., Giago, C., Sam, A., &

Mitchell, C. M. (2014). Effectiveness of Circle of Life, an HIV-preventive

intervention for American Indian middle school youths: A group randomized trial

in a Northern Plains tribe. American Journal of Public Health, 104(6), e106-e112.

32

Kawakami, A. J., Aton, K., Cram, F., Lai, M. K., & Porima, L. (2007). Improving the

Practice of Evaluation Through Indigenous Values and Methods: Decolonizing

Evaluation Practice—Returning the Gaze From Hawaiÿi and Aotearoa. Hulili:

Multidisciplinary Research on Hawaiian Well-Being, 4(1), 319-348.

Komro, K. A., Wagenaar, A. C., Boyd, M., Boyd, B. J., Kominsky, T., Pettigrew, D.,

Tobler, A. L., Lynne-Landsman, S. D., Livingston, M. D., Livingston, B., &

Molina, M. M. M. (2015). Prevention trial in the Cherokee Nation: Design of a

randomized community trial. Prevention Science, 16(2), 291-300.

Kulis, S. S., Ayers, S. L., Harthun, M. L., & Jager, J. (2016). Parenting in 2 worlds:

Effects of a culturally adapted intervention for urban American Indians on

parenting skills and family functioning. Prevention Science, 17(6), 721-731.

Kumanyika, S. K. (2003). Commentary: cultural appropriateness: working our way

toward a practicable framework. Health Education & Behavior, 30(2), 147-150.

LaFrance, J. (2004). Culturally competent evaluation in Indian Country. New Directions

for Evaluation, 102, 39-50.

LaFrance, J., & Nichols, R. (2008). Reframing evaluation: Defining an Indigenous

evaluation framework. The Canadian Journal of Program Evaluation, 23(2), 13.

LaFrance, J., Nichols, R., & Kirkhart, K. E. (2012). Culture writes the script: On the

centrality of context in indigenous evaluation. New Directions for Evaluation,

135, 59-74.

Lavallée, L. F. (2009). Practical Application of an Indigenous Research Framework and

Two Qualitative Indigenous Research Methods: Sharing Circles and Anishnaabe

Symbol-Based Reflection. International Journal of Qualitative Methods, 8(1), 21-

40.

Margalit, R., Watanabe-Galloway, S., Kennedy, F., Lacy, N., Red Shirt, K., Vinson, L.,

& Kills Small, J. (2013). Lakota Elders' Views on Traditional Versus

33

Commercial/Addictive Tobacco Use; Oral History Depicting a Fundamental

Distinction. Journal of Community Health, 38(3), 538-545.

Morton, D. J., Proudfit, J., Calac, D., Portillo, M., Lofton-Fitzsimmons, G., Molina, T.,

Flores, R., Lawson-Risso, B., & Majel-McCauley, R. (2013). Creating research

capacity through a tribally based institutional review board. American Journal of

Public Health, 103(12), 2160-2164.

Norton, I. M., & Manson, S. M. (1996). Research in American Indian and Alaska Native

communities: Navigating the cultural universe of values and process. Journal of

Consulting and Clinical Psychology, 64(5), 856-860.

Oberly, J., & Macedo, J. (2004). The R word in Indian country: Culturally appropriate

commercial tobacco-use research strategies. Health Promotion Practice, 5(4),

355-361.

Oetting, E. R., Donnermeyer, J. F., Plested, B. A., Edwards, R. W., Kelly, K., &

Beauvais, F. (1995). Assessing Community Readiness for Prevention.

International Journal of the Addictions, 30(6), 659-683.

Pahwa, P., Abonyi, S., Karunanayake, C., Rennie, D. C., Janzen, B., Kirychuk, S.,

Lawson, J. A., Katapally, T., McMullin, K., Seeseequasis, J., Naytowhow, A.,

Hagel, L., Dyck, R. F., Fenton, M., Senthilselvan, A., Ramsden, V., King, M.,

Koehncke, N., Marchildon, G., & McBain, L. (2015). A community-based

participatory research methodology to address, redress, and reassess disparities in

respiratory health among First Nations. BMC Research Notes, 8(1), 1-11.

Peercy, M., Gray, J., Thurman, P. J., & Plested, B. (2010). Community readiness: An

effective model for tribal engagement in prevention of cardiovascular disease.

Family & Community Health, 33(3), 238-247.

Perry, C., & Hoffman, B. (2010). Assessing Tribal Youth Physical Activity and

Programming Using a Community-Based Participatory Research Approach.

Public Health Nursing, 27(2), 104-114.

34

Plested, B., Smitham, D. M., Thurman, P. J., Oetting, E. R., & Edwards, R. W. (1999).

Readiness for Drug Use Prevention in Rural Minority Communities. Substance

Use & Misuse, 34(4-5), 521-544.

Redwood, D., Provost, E., Lopez, E. D., Skewes, M., Johnson, R., Christensen, C., Sacco,

F., & Haverkamp, D. (2016). A Process Evaluation of the Alaska Native

Colorectal Cancer Family Outreach Program. Health Education & Behavior,

43(1), 35-42.

Reinschmidt, K. M., Attakai, A., Kahn, C. B., Whitewater, S., & Teufel-Shone, N.

(2016). Shaping a stories of resilience model from urban American Indian elders’

narratives of historical trauma and resilience. American Indian and Alaska Native

Mental Health Research, 23(4), 63-85.

Rivkin, I., Trimble, J., Lopez, E. D. S., Johnson, S., Orr, E., & Allen, J. (2013).

Disseminating research in rural Yup'k communities: challenges and ethical

considerations in moving from discovery to intervention development.

International Journal of Circumpolar Health, 72, 1-8.

Robertson, P., Jorgensen, M., & Garrow, C. E. (2004). Indigenizing evaluation research:

How Lakota methodologies are helping "raise the tipi" in the Oglala Sioux

Nation. American Indian Quarterly, 28(3), 499-526.

Rosas, L. G., Vasquez, J. J., Naderi, R., Jeffery, N., Hedlin, H., Qin, F., LaFromboise, T.,

Megginson, N., Pasqua, C., Flores, O., McClinton-Brown, R., Evans, J., &

Stafford, R. S. (2016). Development and evaluation of an enhanced diabetes

prevention program with psychosocial support for urban American Indians and

Alaska natives: A randomized controlled trial. Contemporary Clinical Trials, 50,

28-36.

Scott, S., D'Silva, J., Hernandez, C., Villaluz, N. T., Martinez, J., & Matter, C. (2016).

The Tribal Tobacco Education and Policy Initiative: Findings From a

Collaborative, Participatory Evaluation. Health Promotion Practice.

35

Simonds, V. W., & Christopher, S. (2013). Adapting Western Research Methods to

Indigenous Ways of Knowing. American Journal of Public Health, 103(12),

2185-2192.

Smith, L. T. (1999). Decolonizing methodologies: Research and indigenous peoples.

London: Zed Books.

Snowshoe, A., Crooks, C. V., Tremblay, P. F., Craig, W. M., & Hinson, R. E. (2015).

Development of a Cultural Connectedness Scale for First Nations youth.

Psychological Assessment, 27(1), 249-259.

Snowshoe, A., Crooks, C. V., Tremblay, P. F., & Hinson, R. E. (2017). Cultural

connectedness and its relation to mental wellness for First Nations youth. The

Journal of Primary Prevention, 38(1-2), 67-86.

Starkes, J. M., & Baydala, L. T. (2014). Health research involving First Nations, Inuit

and Métis children and their communities. Paediatrics & Child Health, 19(2), 99-

102.

Thurman, P. J., Plested, B. A., Edwards, R. W., Foley, R., & Burnside, M. (2003).

Community readiness: the journey to community healing. Journal of Psychoactive

Drugs, 35(1), 27-31.

Tingey, L., Larzelere-Hinton, F., Goklish, N., Ingalls, A., Craft, T., Sprengeler, F.,

McGuire, C., & Barlow, A. (2016). Entrepreneurship education: A strength-based

approach to substance use and suicide prevention for American Indian

adolescents. American Indian & Alaska Native Mental Health Research, 23(3),

248-270.

Trotter, R. T., II, Laurila, K., Alberts, D., & Huenneke, L. F. (2015). A diagnostic

evaluation model for complex research partnerships with community engagement:

The partnership for Native American Cancer Prevention (NACP) model.

Evaluation and Program Planning, 48, 10-20.

36

Van Dyke, E. R., Blacksher, E., Echo-Hawk, A. L., Bassett, D., Harris, R. M., &

Buchwald, D. S. (2016). Health Disparities Research Among Small Tribal

Populations: Describing Appropriate Criteria for Aggregating Tribal Health Data.

American Journal of Epidemiology, 184(1), 1-6.

Venner, K. L., Greenfield, B. L., Hagler, K. J., Simmons, J., Lupee, D., Homer, E.,

Yamutewa, Y., & Smith, J. E. (2016). Pilot outcome results of culturally adapted

evidence-based substance use disorder treatment with a Southwest Tribe.

Addictive Behaviors Reports, 3, 21-27.

Walker, S. C., Whitener, R., Trupin, E. W., & Migliarini, N. (2015). American Indian

perspectives on evidence-based practice implementation: results from a statewide

tribal mental health gathering. Administration and Policy in Mental Health and

Mental Health Services Research, 42(1), 29-39.

Wax, M. L. (1991). The Ethics of Research in American Indian Communities. American

Indian Quarterly, 15(4), 431-456.

Weiss, C. H. (1997). How can theory-based evaluation make greater headway?

Evaluation Review, 21(4), 501-524.

Wexler, L., Poudel-Tandukar, K., Rataj, S., Trout, L., Poudel, K. C., Woods, M., &

Chachamovich, E. (2016). Preliminary evaluation of a school-based youth

leadership and prevention program in rural Alaska Native communities. School

Mental Health, advance online, 1-12.

Willging, C. E., Helitzer, D., & Thompson, J. (2006). ‘Sharing wisdom’: Lessons learned

during the development of a diabetes prevention intervention for urban American

Indian women. Evaluation and Program Planning, 29(2), 130-140.

Ziabakhsh, S., Pederson, A., Prodan-Bhalla, N., Middagh, D., & Jinkerson-Brass, S.

(2016). Women-centered and culturally responsive heart health promotion among

indigenous women in Canada. Health Promotion Practice, 17(6), 814-826.

37

We offer here a series of six evaluation instruments that have been useful for

other projects serving American Indians and Alaska Natives. Some of these instruments

have been used extensively and evaluated with Indigenous populations; others, less so.

We have included the instruments here under three categories: 1) Instruments used

frequently in research with American Indians and Alaska Natives, 2) Instruments

suggested by Office of Health Equity Native American Implementation Pilot Project

grantees, and 3) Instruments used by our network of colleagues doing similar work with

American Indian and Alaska Native populations. This is not intended to be a

comprehensive list, but rather a list of instruments that may be useful for evaluating your

projects.

Category 1: Instruments used frequently in research with Native Americans

1. and 2. Historical Losses and Historical Losses Associated Symptoms Scales (Whitbeck et al., 2004.)

Category 2: Items suggested by OHE NA IPP Grantees

3. Cultural Connectedness Scale-CA (localized from Snowshoe, [2015] by the Native American Health Center for use with California Native American populations)

4. Herth Hope Index (Abbreviated Herth Hope Index) and Scoring Instructions (Herth, 1991; copyrighted and permission from the author must be obtained prior to use.)

Category 3: From network of colleagues doing similar work

5. Reasons for Life Scale (Positive measures opposed to suicide factors. Designed for Alaska Yup’ik Natives by James Allen and the People Awakening Team)

6. Perceived Discrimination Measures (Whitbeck et al., 2001)

38

References

Allen, J. and People Awakening Team Reasons for Life Scale.

Herth, K. (1991). Development and refinement of an instrument to measure hope. Research and Theory for Nursing Practice, 5(1), 39-51.

Herth, K. (1992). An abbreviated instrument to measure hope: Development and psychometric evaluation. Journal of Advanced Nursing, 17, 1251-1259.

Native American Health Center. Cultural Connectedness Scale-CA.

Snowshoe, A. (2015). “The Cultural Connectedness Scale and its Relation to Positive Mental Health among First Nations Youth.” Electronic Thesis and Dissertation Repository. 3107. http://ir.lib.uwo.ca/etd/3107

Whitbeck, L. B., Adams, G. W., Hoyt, D. R., & Chen, X. (2004). Conceptualizing and measuring historical trauma among American Indian people. American Journal of Community Psychology, 33, 119–130. http://dx.doi.org/10.1023/B:AJCP.0000027000.77357.31

Whitbeck, L., Hoyt, D., McMorris, B., Chen, X., & Stubben, J. (2001). Perceived Discrimination and Early Substance Abuse among American Indian Children. Journal of Health and Social Behavior, 42, 405-424.

39

1. Historical Losses and 2. Historical Losses Associated Symptoms Scales

Whitbeck, L. B., Adams, G. W., Hoyt, D. R., & Chen, X. (2004). Conceptualizing and measuring historical trauma among American Indian people. American Journal of Community Psychology, 33, 119–130. http://dx.doi.org/10.1023/B:AJCP.0000027000.77357.31

40

Table A1: Historical Losses Scale, Whitbeck et al., 2004, p. 128 HG1. American Indian people have experienced many losses since we came into contact with Europeans (Whites). Please read the types of losses that people have mentioned to us (the scale developers), and I would like you to circle how often you think of these losses, from never thinking about them to thinking about them several times a day. DK/REF means you don’t know or refuse to answer that particular item.

Table A2: Historical Losses Associated Symptoms Scale, Whitbeck et al., 2004, p. 129

HG2. Now, I would like to ask you about how you feel when you think about these losses. How often do you feel…

Source: Whitbeck, L. B., Adams, G. W., Hoyt, D. R., & Chen, X. (2004). Conceptualizing and measuring

historical trauma among American Indian people. American Journal of Community Psychology, 33, 119–130. http://dx.doi.org/10.1023/B:AJCP.0000027000.77357.31

41

3.1 Cultural Connectedness Scale (CA)

Snowshoe, A. (2015). The Cultural Connectedness Scale and its Relation to Positive Mental Health among First Nations Youth. Electronic Thesis and Dissertation Repository. 3107. http://ir.lib.uwo.ca/etd/3107

Adaptation: Native American Health Center. Cultural Connectedness Scale-CA

42

Cultural Connectedness Scale – Urban California_ (Revised January 26, 2017) Age: _______ Gender: ______________ Tribe(s): ____________________________________________________ Please place an ‘X’ next to the most accurate statement for each category relating to the Tribal Affiliation you indicated above. (Pick only one statement per category)

Self ___ I have not lived in the geographic location where my tribe is from. ___ I have lived in the geographic location where my tribe is from for 15 years or less. [Please indicate number of years _______] ___ I have lived in the geographic location where my tribe is from for 16 or more years. [Please indicate number of years _______] Parents ___ I do not know if my parents lived in the geographic location where my tribe is from. ___ One or both of my parents lived in the geographic location where my tribe is from.

I do not know how long they lived there. ___ One or both of my parents lived in the geographic location where my tribe is from for 15 years or Less. ___ One or both of my parents lived in the geographic location where my tribe is from for 16 years or

More.

Grandparents ___ I do not know if my grandparents lived in the geographic location where my tribe is from. ___ One or more of my grandparents lived in the geographic location where my tribe is from.

I do not know how long they lived there. ___ One or more of my grandparents lived in the geographic location where my tribe is from for 15 years or

Less. ___ One or more of my grandparents lived in the geographic location where my tribe is from for 16 years or

More.

43

[For Questions 1 – 11, circle Yes, NO or NA]

1. I know my cultural, spirit, Indian or Traditional name. Yes No (Not Applicable. We don’t use these names.) NA

2. I can understand some of my Native American/Indigenous words or languages. Yes No

3. I believe things like animals, rocks (and all nature) have a spirit like Native American Yes No /Indigenous People.

4. I use ceremonial/traditional medicines (See Examples List 1) for guidance or prayer or Yes No other reasons. (See Examples List 2)

5. I have participated in a traditional/cultural ceremony or activity. (See Examples List #3) Yes No

6. I have helped prepare for a traditional/cultural ceremony or activity in my family Yes No or community. (See Examples List #3)

7. I have shared a meal with community, offered food or fed my ancestors for a Yes No

traditional/cultural or spiritual reason. (See Example List #4)

8. Someone in my family or someone I am close with attends traditional/cultural Yes No ceremonies or activities. (See Examples List #3)

9. I plan on attending a traditional/cultural ceremony or activity in the future. Yes No (See Examples List #3)

10. I plan on trying to find out more about my Native American/Indigenous culture, such Yes No as its history, Tribal identity, traditions, customs, arts and language.

11. I have a traditional person, elder or other person who I talk to. Yes No (See Examples List #5)

[Please go to next page]

44

[For Questions 12 – 29 on the next two pages, place an X in the appropriate circle.]

Strongly Disagree Do Not Agree Strongly Disagree Agree or Agree Disagree

12. I have spent time trying to find out more about being Native American/Indigenous, such as its history, Tribal identity, traditions, language and customs.

13. I have a strong sense of belonging to

my Native American/Indigenous family, community, Tribe or Nation.

14. I have done things that will help me

understand my Native American/Indigenous background better.

15. I have talked to community members or

other people (See Example List #5) in order to learn more about being Native American/Indigenous.

16. When I learn something about my Native

American/Indigenous culture, history or ceremonies, I will ask someone, research it, look it up, or find resources to learn more about it.

17. I feel a strong connection/attachment towards

my Native American community or Tribe.

18. If a traditional person, counsellor or Elder who is knowledgeable about my culture, spoke to me about being Native American/Indigenous, I would listen to them carefully. (See List #5)

19. I feel a strong connection to my ancestors

and those that came before me.

20. Being Native American/Indigenous means I sometimes have a different perception or way of looking at the world.

21. The eagle feather (or other feathers)

has a lot of traditional meaning for me. (See Examples List #6)

[Please go to next page]

45

Strongly Disagree Do Not Agree Strongly Disagree Agree or Agree Disagree

22. It is important to me that I know my Native

American/Indigenous or Tribal language(s).

23. When I am physically ill, I look to my Native American/Indigenous culture or community for help.

24. When I am overwhelmed with my emotions,

I look to my Native American/Indigenous culture or community for help.

25. When I need to make a decision about

something, I look to my Native American/Indigenous culture or community for help.

26. When I am feeling spiritually ill or disconnected,

I look to my Native American/Indigenous culture or community for help.

Never Once/ Every Every Every Twice in Month Week Day Past Year

27. How often do you offer a

ceremonial/traditional medicine for cultural/traditional purposes? (See Examples List 1)

28. How often do you use ceremonial/traditional medicines? (See Example List #1)

29. How often does someone in your family or someone you are close to use ceremonial/traditional medicines? (See Examples List #1)

[Please go to next page]

46

EXAMPLE LISTS 1 – 6

Cultural Connectivity Scale – Urban California [Revised January 16, 2017] #1 Ceremonial & Traditional Medicines

Angelica Root Bear Root Cedar Corn Pollen Copal Greasewood Jimson Milk Weed Mountain Tea Mugwort Palo de Santo, Peyote Sage Sweetgrass Tobacco Women’s Tea

#2 Uses of Ceremonial & Traditional Medicines

Asking for a blessing in a sacred manner,

Calmness Cultural

connections Gifting to

show respect Give thanks Guidance Help Sleeping To honor Personal

Healing Prayer Smudge Spiritual

connections Spiritual

Offerings Steady Mind Talk to the

creator Keep bad

spirits away

#3 Traditional, Tribal & Cultural Ceremonies or Activities

Acorn Ceremony Beading Class Bear Dance, Sun Dance,

Round Dance or other Cultural Dance

Big Time Burning of Clothes Coming of Age Deer Gathering Drumming Feast Giveaway Fiesta (South of Kern

Valley) GONA Longhouse Moon Ceremony New Years Pot Latch Pow Wow Puberty Ceremony Repatriation Running is my High Spring Ceremony Story Telling Sunrise Ceremony Sun Rise (Alcatraz) Sweat Lodge Traditional Tattoo Washing of the Face Wiping of Tears Young Men’s Ceremony Yuwipi

#4 Cultural Uses of Food

Spirit Plate Thank You

Ceremony Special

Feast Community

Feed Other….

#5 Traditional Persons, Elders & Leaders

Ceremonial Leader

Cultural Teacher Doctor Elder Father Feather Man Feather Woman God Father God Mother Head Heir Head Man Head Woman Medicine People Mother Mother Bear Regalia Leader Spiritual Person Timiiwal Top Doc

Example List #6 Eagle Feather Condor Flicker Humming Bird Raven Hawk Turkey Quail Woodpecker

47

4. Herth Hope Index (Abbreviated Herth Hope Index) and Scoring Instructions

(Permission from the author must be obtained prior to use)

Herth, K. (1992). An abbreviated instrument to measure hope: Development and psychometric evaluation. Journal of Advanced Nursing, 17, 1251-1259.

48

Study No.

HERTH HOPE INDEX Listed below are a number of statements. Read each statement and place an [X] in the box that describes how much you agree with that statement right now. Strongly

DisagreeDisagree Agree Strongly

Agree1. I have a positive outlook toward

life.

2. I have short and/or long range goals.

3. I feel all alone.

4. I can see possibilities in the midst of difficulties.

5. I have a faith that gives me comfort.

6. I feel scared about my future.

7. I can recall happy/joyful times.

8. I have deep inner strength.

9. I am able to give and receive caring/love.

10. I have a sense of direction.

11. I believe that each day has potential.

12. I feel my life has value and worth.

© 1989 Kaye Herth 1999 items 2 & 4 reworded

49

SCORING INFORMATION FOR THE HERTH HOPE INDEX (HHI)

Scoring consists of summing the points for the subscale and for the total scale. Subscales are based on the three factors (see Table 2 in 1992 publication). Total possible points on the total scale is 48 points. The higher the score the higher the level of hope. Note the following items need to be reversed scored: 3, 6. Score items as follows: Strongly Disagree = 1 Disagree = 2 Agree = 3 Strongly Agree = 4 HHI has been translated into Arabic, Bangla, Brazilian, Chinese, Dutch, Filipino, French, German, Hebrew, Icelandic, Italian, Japanese, Korean, Malay, Norwegian, Persian, Polish, Portuguese, Russian, Slovenian, Spanish, Swahili, Swedish, Tai, Turkish, and Urdu. Herth, K. (1992). Abbreviated instrument to measure hope: Development and psychometric

evaluation. Journal of Advanced Nursing, 17, 1251-1259. Seven major instrument textbooks including Simmons, C. & Lehmann, P. (2013). Tools for Strengths-Based Assessment and Evaluation. New York, NY: Springer Publishing Co., Elsevier volume on Measures of Personality and Social Psychological Constructs by Fred Bryant and Patrick Harrison, and Schutte, N. and Malouff, J. (2014). Assessment of Emotional Intelligence. TIME: Toolkit of Instruments to Measure End-of-Life Care. http://www.chcr.brown.edu/pcoc/toolkit.htm

International Centre for Socioeconomic Research Compendium of Quality of Life Instruments, the International Complementary and Alternative Medicine (CAM) Outcome Measures Data Base http://www.IN-CAMoutcomesdatabase

Pocket-sized reference book for physicians and other healthcare professionals edited by C. Porter

Storey, MN titled: UNIPAC OR: A Quick Reference to the Hospice and Palliative Care Training for Physicians.

e-version of UNIPAC 2: Alleviating Psychological and Spiritual Pain. American Psychological Association’s PsycTESTS database.

Update 9/28/16

50

5. Reasons for Life Scale

Allen, J. Reasons for Life Scale.

51

Reasons for Life (Yuuyaraqegtaar: “A way to live a very good, beautiful life”)–(12 items, previous 9 item version, =.79). This measure is an extension of constructs tapped in the Brief Reasons for Living Inventory for Adolescents (1), itself a modification of an adult measure, the Reasons for Living Inventory (2). Reasons for Life assess beliefs and experiences that make life enjoyable, worthwhile, and provide meaning. Subscales tap Others’ Assessment of Me, Cultural and Spiritual Beliefs, and Personal Efficacy, and 2 new items are added to increase reliability. The measure provides a positive psychology approach to assessing Alaska Native cultural values associated with protection from suicide (3). [Self-report measure]

1. Osman A, Kopper BA, Barrios FX, Osman JR. The Brief Reasons for Living Inventory for adolescents (BRFL-A). Journal of Abnormal Child Psychology: An official publication of the International Society for Research in Child and Adolescent Psychopathology. 1996 Aug;24(4):433-43. PubMed PMID: 1996-01625-003. English.

2. Linehan MM, Goodstein AJ, Nielsen SL, Chiles JA. Reasons for staying alive when you are thinking about killing yourself: The reasons for living inventory. Journal of Consulting and Clinical Psychology. 1983;51:276-86.

3. Allen, J., Mohatt, G.V., Fok, C.C.T., Henry, D., Burkett, R., & People Awakening Team. A

protective factors model for alcohol abuse and suicide prevention among Alaska Native youth. American Journal of Community Psychology. 2014;54(1-2):125-139.

52

Variables Type

People saw I live my life in a good way. Slider People saw I live my life in a Yup'ik way. Slider People saw me do good things to help others. Slider People saw that I am strong and care about others. Slider My Yup'ik Elders taught me that my life is valuable. Slider I believed I must live to be an Elder. Slider No matter how hard things got, I believed God wanted me to live. Slider My religion taught me that my life is valuable. Slider I had the courage to face life's hardest moments. Slider I believed I can help others fix their problems. Slider I believed I can fix my problems. Slider I believed I can make things work out for the best even when life gets difficult. Slider

53

6. Perceived Discrimination Measures (Whitbeck et al., 2001)

Whitbeck, L., Hoyt, D., McMorris, B., Chen, X., & Stubben, J. (2001). Perceived Discrimination and Early Substance Abuse among American Indian Children. Journal of Health and Social Behavior, 42, 405-424.

54

Perceived Discrimination (response to each item 1 “never” through 3 “always”)

Global Discrimination

1. How often have other kids said something bad or insulting to you because you are a

Native American?

2. How often have other kids ignored you or excluded you from some activities because you

are a Native American?

3. How often has someone yelled a racial slur or racial insult at you?

4. How often has someone threatened to harm you physically because you are a Native

American?

5. How often have other kids treated you unfairly because you are Native American?

Authority Discrimination

1. How often has a store owner, sales clerk, or someone working at a place of business

treated you in a disrespectful way because you are a Native American?

2. How often have adults suspected you of doing something wrong because you are a Native

American?

3. How often have the police hassled you because you are a Native American?

School Discrimination

1. How often have you encountered teachers who are surprised that you as a Native

American person did something really well?

2. How often have you encountered teachers who didn’t expect you to do well because you

are a Native American?

Prevention Research CenterPacific Institute for Research and Evaluation

180 Grand Avenue, Suite 1200, Oakland, CA 94612 Phone: (510) 486-1111