Evaluation of the Tribal HPOG Program OPRE 2013-55 | FEBRUARY 2014

Practice Brief

Tribal Health Profession Opportunity Grants (HPOG) Program Implementation & Evolution Michael Meit, MA, MPH, Katherine Meyer, MPP, MA, Tess Gilbert, MHS, Sara Levintow, BS, Heather Langerman, BS, Carol Hafford, PhD, Alana Knudson, PhD, Aleena Hernandez, MPH, Theresa Carino, M.Ed., Paul Allis, MEd

This practice brief is part of a series of practice briefs being developed by the Tribal HPOG evaluation team, comprised of NORC at the University of Chicago, Red Star Innovations and the National Indian Health Board (NIHB). The briefs will be used to disseminate important lessons learned and findings from the Evaluation of the Tribal Health Profession Opportunity Grants (HPOG) program, which is funded by the Office of Planning, Research and Evaluation (OPRE) within the Administration for Children and Families (ACF). The HPOG program is funded by the Affordable Care Act (ACA) to support 32 demonstration projects, including five Tribal Organizations and Colleges, to train Temporary Assistance for Needy Families (TANF) recipients and other low-income individuals as healthcare professionals. The Tribal HPOG program aims to meet local healthcare demands by increasing the number of well-trained health professionals in tribal communities. The program is designed using a career pathways approach where students advance through related trainings that build on each other to deepen students’ healthcare knowledge and skills. This practice brief presents qualitative data collected on program implementation as part of the Tribal HPOG evaluation. It discusses key Tribal HPOG program elements that have facilitated program implementation according to program staff and students; provides site specific examples of program implementation; discusses challenges encountered during program implementation and lessons learned; and describes how the Tribal HPOG sites and their programs are evolving.

National shortages in the health workforce coupled with growing demands for healthcare have prompted action to develop and sustain a health workforce that meets high standards for education, certification, and professional development.1 To meet these critical needs, the Health Profession Opportunity Grants (HPOG) program provides education and training in the healthcare field to Temporary Assistance for Needy Families (TANF) recipients and other low-income individuals for occupations in the healthcare field that pay well and are expected to either experience labor shortages or be in high demand. Funded through the Affordable Care Act (ACA) and administered by the Administration for Children and Families (ACF) Office of Family Assistance (OFA), the HPOG program has made grant awards to 32 five-year demonstration projects across 23 states. Of the 32 demonstration projects, five were awarded to tribal organizations and tribal colleges— Blackfeet Community College (Browning, MT), Cankdeska Cikana Community College (Fort Totten, ND), College of Menominee Nation (Keshena, WI), Cook Inlet Tribal Council, Inc. (Anchorage, AK), and Turtle Mountain Community College (Belcourt, ND). The tribal grantees’ programs are distinct from those of non-tribal grantees in that they aim to integrate health professions training programs with culturally-informed models of learning and practice, such as cooperative learning and mentoring, to nurture and educate low-income individuals from American Indian/Alaska Native populations into healthcare careers. The Tribal HPOG programs are designed to benefit underserved members of tribal communities who are eligible for or receive TANF or have incomes that fall below the federal poverty level. They specifically recruit prospective students who reside on or near tribal reservations, with the goal

Tribal HPOG Grantees ¬ Blackfeet Community College Location: Browning, MT (Glacier County) Project: Issksiniip Project: Meeting the Holistic

Health and Education Needs of the Niitsitapi ¬ Cankdeska Cikana Community College Location: Fort Totten, ND (Benson County) Project: Next Steps: An Empowerment Model for

Native People Entering the Health Professions ¬ College of Menominee Nation Location: Keshena, WI (Menominee County) Project: College of Menominee’s CNA to RN Career

Ladder Program ¬ Cook Inlet Tribal Council, Inc. Location: Anchorage, AK (Anchorage County) Project: Cook Inlet Tribal Council Health Professions

Opportunity Program ¬ Turtle Mountain Community College Location: Belcourt, ND (Rolette County) Project: Project CHOICE: Choosing Health

Opportunities for Indian Career Enhancement

2 Evaluation of the Tribal HPOG Program • FEBRUARY 2014 OPRE 2013-55

Tribal Health Profession Opportunity Grants (HPOG) Program Implementation & Evolution

of meeting local healthcare demands by increasing the number of well-trained, culturally competent health professionals in tribal communities. NORC at the University of Chicago has collaborated with tribal partners at the National Indian Health Board (NIHB) and Red Star Innovations, LLC, to conduct a culturally responsive evaluation of the Tribal HPOG program.

This practice brief provides an overview of the strategies that Tribal HPOG grantees have used to implement the HPOG program, challenges encountered during implementation, lessons learned, and ongoing program evolution and adaptation to address unique tribal cultural and programmatic needs. It is important to learn about and better understand diverse programmatic approaches to health professions trainings serving the tribal population (e.g., cultural adaptations) in order to enhance program sustainability and to inform the broader field about how to best set up similar programs for tribal populations.

KEY PROGRAM ELEMENTS Across Tribal HPOG programs, several program elements have facilitated program implementation, according to HPOG program staff and students. The following program elements were identified as contributing to HPOG program implementation: (1) dedicated staff/strong leadership; (2) application and screening process for program acceptance; (3) thorough assessment of student needs for program success; (4) social supports to help retain students in their programs; and (5) instilling pride and a sense of accomplishment among students.

To illustrate the ways in which these program elements facilitated program implementation, this practice brief draws upon data collected from the first and second year of evaluation activities with the Tribal HPOG programs. Information presented in this brief is drawn from site visits, which included interviews with administrative and implementation staff, focus groups with current students, and telephone interviews with program completers and non-completers.2

Dedicated Program Staff/Strong Leadership Each Tribal HPOG site reported that dedicated program staff and/or strong leadership were key elements for the successful implementation of their program. Many of the staff across programs are tribal members who have had similar experiences as the students with whom they are working. Several staff mentioned that they are able to detect and understand challenges that the HPOG students are encountering and can not only offer suggestions and resources to help students, but can also relate to and share their stories with students as examples of success.

Turtle Mountain Community College (TMCC) and Blackfeet Community College (BCC) were able to hire and retain dedicated staff and faculty to support HPOG efforts. The TMCC students recognize and acknowledge the support and understanding that the TMCC and HPOG program staff provides. As one student said, “The instructors are very good, dedicated. They have a lot of experience, like to teach, and never miss a class. They are always there to listen and are very helpful for whatever we need.” Another TMCC student said, “You can really tell that they (HPOG staff and faculty) actually care what is going to happen to us. It makes you want to try harder.” Similarly at BCC, implementation staff is dedicated to helping students. They feel that since most staff is native, they are able to empathize with students and that students feel more comfortable approaching staff with personal or academic issues. Cankdeska

Cikana Community College (CCCC) uses a mentor approach where each site is led by a University of North Dakota RAIN (Recruitment/Retention of American Indians into Nursing) program mentor. The mentors are located at each implementation site to ensure that they are easily accessible to HPOG students. These examples illustrate how many HPOG program staff members have made efforts to be available and accessible to HPOG students, and how this effort is recognized by the students and encourages them to invest further and do well in their programs.

Staff Dedication to Students

At Blackfeet CommunityCollege, most administrative and implementation staff areNative, with experiences similar to those of the students. Program staff feels these commonalitiesallow them to empathize with students and facilitates students’ comfort in approaching staff withpersonal or academic issues. Asone implementation staff shared,“Part of the reason I came on is because I have experience on the reservation. The impact is on your whole world when you go to a university. When they’re coming off the reservation there is a sense of culture shock...I can spotthe challenges within students andI try to work with that.”

Application & Screening Process for Program Acceptance All Tribal HPOG programs engage in an application and screening process for student acceptance into the program; however, the type of screening process varies by program from meeting basic eligibility requirements to engaging in an intensive review process. According to the grantees, it is important to have a robust application and screening process to ensure that students are eligible for HPOG services and poised for success. For the Next Steps program at CCCC, there are three main eligibility criteria that students must fulfill which include having low-income level (200% of the Federal Poverty Level); enrollment in a federally recognized tribe; and entering in a degree or certificate program at a college or university. If a student fills out the CCCC Next Steps application, which consists of standard application materials including forms to verify tribal enrollment and income level, and the student meets the eligibility criteria mentioned above, then they are accepted into the Next Steps program.

Cook Inlet Tribal Council, Inc. (CITC) and TMCC Tribal HPOG programs engage in a more rigorous application and screening process and cite this process as a key strategy for student retention. For acceptance in the CITC HPOG program, applicants must fulfill basic eligibility criteria (Alaska Native, TANF or low income, GED or high school diploma, and Anchorage resident), achieve the minimum reading and math skill levels, pass a background check, and express interest in a healthcare profession (as demonstrated on a resume, essay, and letters of reference). In particular, CITC staff reported improvement in their screening process by including the background check. A CITC staff member gave the example of students who were highly successful in the training program, only to be stopped at the state board of nursing certification because of a previous criminal charge that had not been cleared from his or her record.3

The Project CHOICE Tribal HPOG program at TMCC employs an intensive screening process, which staff believe ensures the commitment of admitted students and ultimately boosts student retention in the training programs. Prior to the start of the academic year, Project CHOICE convenes a screening committee – consisting

OPRE 2013-55 Evaluation of the Tribal HPOG Program • FEBRUARY 2014 3

Tribal Health Profession Opportunity Grants (HPOG) Program Implementation & Evolution

of the Project CHOICEDirector, Allied Health Career Program Facilitators, TMCC instructors, and StudentServices staff – to review applications (which includeverification of income level, academic transcript, and attendance records) anddecide which students shouldbe interviewed for admission. The screening committee theninterviews selected students and the staff believes that the

interview is a strong indication of student preparedness for the program.

TMCC Application & Screening Process

Program staff report that an important component of the interview is to communicate the ultimate purpose of Project CHOICE: “We explain that they have to be committed since employment is the goal. Once you finish training, you have to find a job.”

Thorough Assessment of Student Needs for Program Success The theory of change for career pathways consists of four hallmark strategies: (1) Comprehensive and well-designed approaches to assessment; (2) Promising and innovative approaches to instruction and occupational training; (3) Academic and non-academic supports to promote success; and (4) Approaches for connecting students with career-track employment opportunities.4 Tribal HPOG grantee assessment activities align with this theory of change, and they are seen as an important strategy to support student retention in their HPOG programs. Specifically, assessments are used to evaluate academic achievement to date and current academic and social support needs in order to develop a success plan. At the College of Menominee Nation (CMN), the primary goal of the assessment process is to develop the Student Success Plan (SSP). The SSP is developed when the student enters their respective program, with the student, and is an overall plan on how the student will achieve their ultimate goal – completion of their academic program. During this initial meeting, academic and supportive needs are discussed and it is an opportunity to identify any barriers that may hinder student success. The SSP is updated each semester, in order to foster regular communication, and to develop new short-term goals and identify remaining academic requirements. As one student described her SSP experience, “I went in to set up my student plan. It included budgeting and was very informative. We discussed what the expectations of the program are and what I need in order to meet those expectations. The staff has been very helpful in walking us through everything and explaining what we need to do in order to be successful.”

At CITC, student needs are initially assessed during intake and are continuously monitored throughout enrollment. The program coordinator serves as the point of contact for all student needs, and program staff and students report feeling comfortable reaching out to the coordinator should any issues arise. The ongoing communication between the program coordinator and Alaska Vocational Technical Center (AVTEC) staff has been critical to meet all of the participant needs, especially needs that develop over the course of the program. Examples of basic living needs of students include housing, transportation and childcare, and examples of academic supports include monetary support for tuition and book fees, as well as tutoring. An interesting feature of the program at CITC is that the supportive services coverage for students is so seamless that many instructors are unaware which of their students are involved with HPOG. In this way, AVTEC staff feel that the Tribal HPOG program at CITC has leveled the playing field for the HPOG students.

Social Supports to Help Retain Students Supportive services (social and academic) are a required feature of the HPOG program model, where all HPOG programs must offer an array of supportive services for the participants. The supportive services offered to students, primarily social supports, were referenced multiple times across sites as one of the key components of the program and necessary for some students to continue to attend class and obtain their degree. The supportive services that comprehensively address students’ basic living needs (i.e., housing, transportation, childcare and food) have often been reported by program administrators and faculty as the difference between a student being able to stay enrolled in the program and having to drop out. As noted by one CITC staff member, “I feel very strongly that the supportive services offered by the HPOG program wouldn’t otherwise be possible [without HPOG funding]. When this came up, I was just thrilled. You have to have support in place in order to succeed in any training.”

Across the Tribal HPOG programs, students are assessed regularly to determine which supportive services students require and the program provides access and/or linkage to those services. For example, at CMN, the HPOG program assesses student needs at intake and continuously during their enrollment. This intake process was established to understand the barriers, assets, and needs for each student, and to help develop an individual success plan. The range of areas discussed during the intake process is vast; as one student explained, “They try to get as much information as they can, so they know what you need and how they can help you.” Specific areas that are assessed include reading and math skills, computer knowledge and typing skills, and nursing skills, as well as nonacademic factors, such as family composition and support, living arrangements and employment status.

Similar to CMN, TMCC requires Tribal HPOG students to meet with a support service specialist on a monthly basis. Students reported knowing who they need to talk to should an emergency arise or if they need something, while program staff reported that they seek out students if they haven’t seen them in a month for a check in. Beyond the supportive services that the Tribal HPOG grantees are providing, students reported that programs are also providing a sense of community in that they feel supported and valued by the program staff.

HPOG Helps Students Overcome Barriers

The primary needs of thestudents at TMCC include mileage reimbursement andchildcare reimbursement. As onestudent stated, “the commute is abarrier, getting that mileage paid for really helps.”

Instilling Pride & a Sense of Accomplishment When discussing their programs students displayed both pride and accomplishment, which they identified as motivating factors in working hard to succeed. In addition, students reported that they frequently refer family and friends to the programs, thereby helping with student recruitment. HPOG students often described the program as “life changing,” and said that without the opportunity to enter the training programs and receive the supportive services, they feel that they would be unemployed or underemployed, and unable to pursue their interests in healthcare. Many of the students who are parents noted that their pride is heightened by the opportunity to be good role models for their children.

4 Evaluation of the Tribal HPOG Program • FEBRUARY 2014 OPRE 2013-55

Tribal Health Profession Opportunity Grants (HPOG) Program Implementation & Evolution

As reported by one BCC student, “My goal is to graduate with my two-year degree in two years and HPOG allows me to not have to work so I can devote time to my studies and do that. I worked in the medical field in the military; spent 20 years in the military and found a lot of success there that I would like to replicate in the civilian world. I want that sense of achievement and worth. I don’t want to go back to checking groceries or bartending. My goal is to realize my potential.”

LESSONS LEARNED

Program Implementation in Rural Settings With the exception of one grantee, the Tribal HPOG programs are being implemented in rural areas. This type of setting presents a unique set of challenges for many grantees. For instance, two grantees experienced challenges related to program staffing. CMN and TMCC mentioned that it was difficult to find qualified personnel that were able to acclimate to the community. BCC experienced difficulty in ensuring that all necessary program staff positions were filled at the launch of the program.

An additional obstacle faced across rural sites is the limited availability of technology, specifically internet access. For example, BCC mentioned that there is no college-wide student email service and students have limited internet access, which makes it difficult for supportive services staff to connect with students. In order to address these issues, many sites have internet-connected computers available for student use. Another barrier faced by students at rural sites is that many of them live far away, which results in a costly commute. Sites have overcome this barrier by providing transportation support, such as gas cards, transit passes and mileage reimbursement.

Rural sites also mentioned a lack of health-related employment opportunities available in the communities making it difficult for recent program completers to find jobs. Increased efforts to partner with local employers have been undertaken by multiple sites with varying degrees of success. BCC has partnered with eight organizations, with six of these partnerships being formalized by MOUs with agreements to train students and/or for employment opportunities upon graduation.

Program Start Up Challenges Developing and incorporating policies and procedures for the HPOG program was a reported challenge encountered at the time of program start-up. At CMN, a site that had the advantage of having an existing curricula to build upon, policies and procedures specific to the HPOG program needed to be established (e.g., the process for seeking approval for the administration of supportive services). The process for establishing the policies and procedures had to be completed at the same time as program implementation as opposed to before the program was implemented. This timing challenge resulted in the program having to overcome procedural issues as they arose and some processes took longer to establish than anticipated. CMN indicated that they have successfully overcome these challenges and the program’s policies and procedures have been fully implemented.

Another aspect of program start-up that Tribal HPOG grantees struggled with was TANF participant recruitment. An important source of participant recruitment is the TANF program; however, the program administrators at CMN reported that they were not receiving the expected number of TANF referrals from the community organization that is responsible for these referrals.

It was reported that one of the issues with TANF referrals at CMN was that a number of TANF recipients were unable to pass their caregiver background check, which would be a barrier to employment in the healthcare field. Therefore, these individuals were not admitted to the Tribal HPOG program at CMN (as one of the criteria for admission into the HPOG program is that individuals need to clear a caregiver background check). Additionally, CITC found that prospective students may face barriers in preparing the program application materials, particularly if English is not their first language. To help students overcome this barrier, candidates are given assistance in writing their application essay.

Student Retention and Success Tribal HPOG grantees often reported that students were underprepared for college coursework, which affected student attrition and overall success in the program. In order to facilitate student success in the program, CITC raised the reading level requirement for program enrollment. Should a student not meet the reading level requirement, the student would be referred to academic services to improve their reading level, and would re-apply to be able to enter their respective academic program. Administrators and staff from CMN identified appropriate benchmarks for students, in terms of knowledge and skills, at each step of the nursing career ladder. The idea is that a student needs to meet the identified benchmarks at each step of the nursing career ladder prior to entering that program or step of the ladder. Examples of placement tests that CMN students complete prior to program placement include the Accuplacer test,5 the Test of Adult Basic Education (TABE),6 and the Test of Essential Academic Skills (TEAS).7 CMN administrators and staff believe these benchmarks have improved program success and retention by ensuring that students only continue with their coursework when they are ready. If they do not meet a specific benchmark, students would not have to leave the program; rather, they would receive enhanced support (i.e., tutoring, remediation) from the program. Staff also noted that individuals for whom English is a second language may face particular challenges in their coursework and struggle to complete the program. They believe that these students would benefit from additional remediation, particularly related to medical terminology education. CMN is working towards implementing these services, including the identification of innovative teaching strategies.

BCC staff believes that some student attrition is a result of students lacking basic college readiness skills. The staff has been thinking about how to better prepare students for rigorous academic programs and mentioned that BCC is currently developing a campus-wide initiative to address this challenge.

To increase student retention in and completion of the program, TMCC requires students to sign a written agreement stating that they will comply with program requirements, which include seeing a support service specialist once a month. In the event that a student does not check in with the case manager, staff will call to check in with that student. The case manager ensures that students are receiving the support that they need from the HPOG program and refers students to appropriate community resources, if necessary. Students indicated that they have found this constant support to be very helpful; they view it as motivation to stay and try to succeed in the program.

Employment after Graduation By establishing connections with local employers, HPOG grantees help facilitate employment in the community for program completers. However, grantees reported difficulties in

OPRE 2013-55 Evaluation of the Tribal HPOG Program • FEBRUARY 2014 5

Tribal Health Profession Opportunity Grants (HPOG) Program Implementation & Evolution

partnering with employers or, in general, finding local employment opportunities for students. CITC found that employers desire relevant work experience in addition to a health profession licensure or certification. While the HPOG program includes a job shadowing component prior to the start of coursework, program staff felt that finding opportunities for post-graduate on-the-job training would help graduates find employment while being responsive to employers’ preferences. CITC staff plan to establish partnerships with local employers to create more on-the-job training opportunities and facilitate employment for program completers. However, staff has found that some Human Resources offices are not willing to partner or allow for any alternative hiring processes. Nonetheless, CITC staff is currently in the process of setting up a Memorandum of Agreement with the Indian Health Service to allow program participants to complete an externship at the Alaska Native Medical Center. Leadership at CITC have also begun thinking about adding additional types of training to address further needs of the local healthcare industry.

To address employment challenges for students, BCC implementation staff noted that they are working towards strengthening relationships with employers in the Blackfeet service area to increase the potential for student employment, job shadowing and internships, and to learn more about the credentials employers seek in prospective employees. Specifically, BCC is currently exploring a partnership with Benefis Health Systems, a comprehensive acute care hospital in Great Falls, MT. This partnership will expand the possibility for students to gain experience in the healthcare field prior to graduation since Benefis has offered to host students for unpaid internships during the summer months.

TMCC has addressed the issue of finding employment for students by recently hiring a job placement coordinator who will be engaging employers in the community to establish successful partnerships. The coordinator plans to meet with local hospitals and clinics as well as pharmacies in the area to establish a relationship and hopes to be able to successfully link their HPOG program students with employment upon graduation and licensure receipt.

CONCLUSION The Tribal HPOG grantees identified several key strategies that facilitated successful program implementation while also identifying areas to enhance as programs continue to evolve and mature. Key to successful implementation for all sites was having a dedicated staff and strong leadership for the program. Likewise, utilizing an application and screening process and a thorough assessment of student needs coupled with providing strong social supports enhanced students’ probability for success. Tribal HPOG grantees also identified areas to enhance as they continue to support their students’ successful transition to healthcare professionals, including work experience opportunities and relationships with employers. Valuable insights gleaned from the Tribal HPOG grantees will help to inform program improvements and guide the implementation of similar tribal health workforce training programs in the future.

REFERENCES 1 National Center for Health Workforce Analysis. (n.d.). Retrieved January 4, 2013 at

http://bhpr.hrsa.gov/healthworkforce/ 2 Site visits were conducted at Blackfeet Community College, College of Menominee

Nation, Cook Inlet Tribal Council, Inc., Turtle Mountain Community College and Cankdeska Cikana Community College from February-November 2012 and from February to October 2013.

3 The inability to pass a criminal background check was discussed across Tribal HPOG sites as this precludes individuals from obtaining employment in an allied health position.

4 Fein, David J. (2012). Career Pathways as a Framework for Program Design and Evaluation: A Working Paper from the Innovative Strategies for Increasing Self-Sufficiency (ISIS) Project. OPRE Report # 2012-30, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

5 ACCUPLACER is a suite of tests that determines your knowledge in math, reading and writing as you prepare to enroll in college-level courses. ACCUPLACER is used to identify your strengths and weaknesses in each subject area and to help you improve your skills through interactive online learning tools (http://accuplacer.collegeboard.org/).

6 The TABE is a placement test that covers the basics of reading, English and math. Based on test scores, a decision is made on whether students need to take remedial courses in some areas prior to taking college credit courses or whether a student is ready forcollege credit courses (http://www.testprepreview.com/tabe_practice.htm).

7 The TEAS test includes assessment tools to pre-determine and measure students’ readiness for nursing school as well as a curriculum management program to ensure for long term success (https://www.atitesting.com/Solutions/PreNursingSchool.aspx).

This report is in the public domain. Permission to reproduce is not necessary. Submitted to: Michael Dubinsky, Project Officer Hilary Forster, Project Officer Office of Planning, Research and Evaluation Administration for Children and Families U.S. Department of Health and Human Services Contract Number: HHSP23320095647W Project Director: Michael Meit, MA, MPH NORC at the University of Chicago 4350 East West Highway, Suite 800 Bethesda, MD 20814 Acknowledgments: We would like to acknowledge the Tribal HPOG grantees for their review and contributions to this practice brief. Suggested citation: Meit, Michael, et al. (2013). OPRE Report 2013-55. Washington, DC; Office of Planning, Research and Evaluation. Administration for Children and Families, U.S. Department of Health and Human Services: NORC at the University of Chicago, Red Star Innovations, National Indian Health Board. Disclaimer The views expressed in this publication do not necessarily reflect the views or policies of the Office of Planning, Research and Evaluation, the Administration for Children and Families, or the U.S. Department of Health and Human Services. This report and other reports sponsored by the Office of Planning, Research and Evaluation are available at http://www.acf.hhs.gov/programs/opre/index.html.