Evaluating Evaluating Mental Mental Health Health

ServicesServices

Christine B. Kleinpeter, MSW., Psy.D., Professor CSU, Long Beach

Rob Bachmann, RN, MN, DirectorGolden West College

Presentation OverviewPresentation Overview

I. Current Study II. Prior Studies

III. Recommendations

I. Current ProjectI. Current Project

Stages of Change and Therapeutic Alliance

among College Students Seeking

Mental Health Services in an

Integrated Health Center

GWC Student GWC Student DemographicsDemographics

N=15,798 Students Males=46.4% Females=53.0% African-American=1.9% Asian=27.5% White=32.5% Hispanic=27.6% Multicultural=4.0% Unreported=3.8% Mean age=24.7%

Mental Health Mental Health DemographicsDemographics

N=130 (2012-2013) Males=28.7% Females=71.3% Asian=27.0% Hispanic=18.3% White=52.2% Other=2.6% Mean age=26.2

Purpose of the StudyPurpose of the Study

Describe who we treat and what type of treatment they receive?

Compare our students in the health center to the general population of GWC students to see if we are serving our campus in all of its diversity?

To compare our current year (2012-2013) data to prior years for program improvement

To study the relationship between therapeutic bond and treatment progress

MethodsMethods

Exploratory/descriptive study Closed case review of mental health data

(2012-2013)using a data retrieval form GWC data was provided by University

Research Treatment progress was measured by the

Rhode Island Change Assessment (URICA) administered pre and post treatment

The therapeutic bond between clients and therapists was measured by the Working Alliance Inventory (WAI)at discharge

SPSS was used to analyze data

Results: DemographicsResults: Demographics

MH Students are female (71.3%) and slightly older (26.2 vs 24.7)

Whites are over-represented in the MH group (52.2% vs 32.5%)

Asians are similar in both groups (27.5% vs 27.0%)

Latinos were under-represented in the MH group (18.3% vs 27.6%)

Results: Clinical Results: Clinical

MH Students received M=4.2 sessions, range (0-11) 0=nurse intake only

MH Students presented with depression (24.%), anxiety (15.0%),

and stress (13%). 6.9% Dropped out after the nurse intake, 18.5% after 1

session /Therapist Students reported strong therapeutic bond with therapists,

WAI (M=77) for women and (M=80) for men Students show improvement in Action Stage (URICA) from

intake to discharge (t=5.82, p=.001) Students who reported strong therapeutic bonds with their

counselors had higher levels of treatment progress: Bond (r=.29, p=.02),

Task (r=.40, p=.001), and Goal (r=.44, p=.001)

DiscussionDiscussion

Outreach needs to focus on men and Latinos The total drop out rate is 25.4%, however the

therapists’ drop out rate is 18.5% The study findings are consistent with the

literature; that is, clients who have a strong therapeutic alliance are more likely to stay in treatment and have positive therapy outcomes.

Using Motivational Interviewing by both nurses and therapists is consistent with other studies that have seen positive outcomes with MI in school settings.

In an academic environment where accountability is a key administrative issue, the chart audit provides continuous feedback for program improvement. In addition to reviewing our current program services we have also added new services where we discovered service gaps.

Administrative Administrative ImplicationsImplications

II. Prior Studies: GWCII. Prior Studies: GWC

3 publications 1 current study under review

(URICA) and (WAI) 1 study in progress (PHQ-9)

health and mental health samples

Baseline AuditBaseline Audit

Baseline audit revealed lack of consistent documentation: DSM diagnosis Short-term, measurable goals Active interventions Client response to treatment Case disposition

High drop out rate (44%) Uneven standard of care by

provider

Administrative Response Administrative Response #1#1

Uniform charting expectations Case disposition follow-up calls Staff training in Motivational

Interviewing Staff supervision regarding

deficiencies Nurses intake includes client

motivation and goal setting Front desk staff make reminder calls

to clients

Second Audit: Replication Second Audit: Replication StudyStudy

Drop out rate decreased to 25% Most charts contained: diagnosis, goals,

treatment plan, and active interventions Interventions of evidenced-based practice

were clearly documented Outcomes were measured: goals met,

partially met, or not met A symptom checklist was given pre and

post, documenting a decrease in symptoms

We served 146 clients, up from 118 last year

Administrative Response Administrative Response #2#2 Improving outcomes was rewarding

to the staff We identified 2 gaps in services in

this audit: medication evaluation and groups

Our physician agreed to meet the need for medication evaluation and management

We began a Veteran’s Support Group and a Student Nurses Support Group

Third AuditThird Audit

Low drop out rate of 15.9% Served 132 students High Client Satisfaction (CSQ-8)

(29.79 out of 32 possible) Students need help with positive

coping skills (CSI) (High Avoidance)

Administrative Response Administrative Response #3#3 The efforts made in reducing no- shows

and cancellations is holding as seen in the low drop out rate

The documentation continues to be complete and accurate

Coping skills lectures have been provided by Health Center staff in individual classrooms

The need for a platform to teach coping skills to the larger student body is needed

SummarySummary

The audit provides important information about service gaps and areas in need of improvement

The staff receives feedback that leads to improved patient care

The program improves by ensuring that services offered are current and evidenced-based

The students benefit when we monitor client satisfaction and ask for their input

III. RecommendationsIII. Recommendations

Useful research questions Practical Instruments/surveys

Research that improves services

First StepsFirst Steps

Take a baseline audit of your charts: Number of students seen per

year Demographics of students seen Number of therapy sessions per

client Drop out rate Number of service hours

provided each year by the clinic

Comparison of your dataComparison of your data

Comparing your MH students’ demographics to your college demographics to demonstrate you serve all students on your campus

Comparing your data with prior years to monitor program improvements over the years

Comparing your data to other colleges to learn from each other

Use Standardized Use Standardized InstrumentsInstruments

Using standardized instruments means that they are valid and reliable. This allows you to compare your data with other colleges and universities

Using client checklists is an easy way to get started because they take little time to score and clients can complete them while they wait for their appointment

Research QuestionsResearch Questions

Client satisfaction is a very good place to start. We used the Client Satisfaction Questionnaire (CSQ-8), as a measure of consumer satisfaction.

Therapeutic Alliance is a good measure of the client/counselor bond, we used the Working Alliance Inventory (WAI) (12 items).

Measuring outcome can be done with a pre and post test. We used the Symptoms Checklist (20 items) (Bartone, et., al., 1989)

Measuring outcome can be done with the URICA, using Action subscale pre and post treatment. (32-item, no easy to score)

What about specific aspects What about specific aspects of my program?of my program?

If you have specific questions, you can design a survey that focuses on your program, for example a group for recovering substance users:

There are standardized surveys written for many types of therapy treatments and can be used for objective questions (e.g., Were the objectives of the group met?)

You can design open-ended questions to cover things like: “What two new things did you learn in this group?”

What about outcomes with What about outcomes with certain types of clients?certain types of clients?

You could choose to look at how well you are doing when you treat depressed clients. If so, you can use the Beck Inventory (BDI) 21-items or (CES-D) 20-items, or Zung (20- item). All are self-report checklists. (BDI requires a Psychologist)

Future ResearchFuture Research

Clinical outcomes that are specific and standardized have been adopted

Larger samples including multiple sites to determine the generalizability of the findings

Studies that include the health clients as well as mental health clients in an effort to become more integrated in our practice

Next year: The PHQ-9 Study for all SHC clients, Would you like to participate??

Thank You!

Christine B. Kleinpeter, MSW., Psy.D., ProfessorCSU, Long Beach

Rob Bachmann, RN, MN, Director,

GWC, Huntington Beach