1

Evaluation of Hawai‘i Clubhouses

Anna R. Smith, MA Joy Agner, MS John P. Barile, PhD Ecological Determinants Lab University of Hawai‘i at Mānoa

2

Table of Contents

Evaluation Summary p. 3 Clubhouse Overview p. 4 Evaluation Activities Overview p. 6 Clubhouse Membership p. 7 Clubhouse Attendance p. 9 Clubhouse Impacts p. 10 Member Challenges p. 16 Clubhouse Activities p. 17 Member Ideas for Growth & Improvement p. 20 Gaps in Data p. 21 Program Evaluation Plan p. 23 Appendix A: Logic Model p. 24 Appendix B: Interview Guide p. 25

3

Evaluation Summary

Clubhouse Participation • Ten Clubhouses across the state of Hawai’i served close to one thousand members (988). • Hawaii Clubhouses had 205 new members in the past year. • The highest growth was at the Hale ‘Oluea, with 14 new members in the past year. • The Waipahu Aloha Clubhouse reported the highest attendance, averaging 66 members per day. • Clubhouse members participated in 485 employment opportunities in the previous year, including

supported employment (161), independent employment (140), transitional employment (123), and group placement (51); earning an average of $8.46 per hour.

Clubhouse Environment • Interviews with Clubhouse members suggest that Clubhouses increase their access to social support,

housing, employment, and advocates. • Members at the Clubhouses report that staff have served as advocates, helped with housing, and

employment support. • Members report that the warm and welcoming Clubhouse environment has helped them develop

friendships and find meaningful work. • Clubhouse members participate in weekly decision-making meetings – this nurtures a sense of ownership

and buy-in for Clubhouse members. Clubhouse Services • The most common services provided by the Hawai’i Clubhouses are wellness support, dual recovery

supports, community support services and transportation supports. These services help enable Clubhouse members to develop job skills and maintain their employment opportunities.

• Clubhouse members received outreach when they were going through hard times or were absent from the Clubhouse.

4

Clubhouse Overview The Clubhouses are psycho-social rehabilitation centers for individuals with a history of serious mental illness. They were adopted from the original Fountain House facility established in New York in 1948 (Fountain House, 1999). Clubhouses are intentional communities composed of consumers, referred to as members, who work side by side with staff performing the jobs essential to the operation of the Clubhouse (Fountain House, 1999). Being a member of a Clubhouse comes with four primary rights: 1) a right to a place to come; 2) a right to meaningful relationships; 3) a right to meaningful work; and 4) a right to a place to return. Clubhouses are designed to empower members to become increasingly autonomous while still being part of a community. The Clubhouse model seeks to remove mental health labels in order to elicit qualities of the individual that can contribute to the productivity of the Clubhouse. Traditional mental health services largely rely on psychopharmacology, individual therapy, and case management. Unlike traditional treatment, which largely is conducted in isolation, Clubhouses are purposely designed to create an interdependency among members. This form of mutual help fosters the sense of community iconic of the Clubhouse Model. Importantly, this sense of community is what counteracts the feelings of social isolation and marginalization frequently experienced by individuals with a history of serious mental illness. The Clubhouses in Hawai‘i are funded by the State of Hawai‘i Adult Mental Health Division (AMHD) and accredited by Clubhouse International and adhere to the International Standards for Clubhouse Programs (http://www.iccd.org/quality.html). These standards include the inclusion of a work-ordered day. Work ordered days typically include member participation in Clubhouse “units” which are small work groups specifically designed to build employable skill with a supported environment. These skills (e.g., clerical, food service, etc.) can then be used in transitional, supported, and independent employment opportunities my available to Clubhouse members.

Clubhouse Member Rights 1) a right to a place to come

2) a right to meaningful relationships

3) a right to meaningful work

4) a right to a place to return

5

Hawai‘i Clubhouses

Clubhouse International states Clubhouses are designed to help members stay out of hospitals while achieving social, financial, and vocational goals (Clubhouse International, 2016). The goals of this evaluation are to assess whether the 10 Clubhouses in Hawai‘i are conducting services and activities that align with this mission, detect the impact of these activities, when possible, and provide an opportunity for members to express their experiences in their Clubhouse and in the wider community. A logic model, which details the activities of the Clubhouse, goals, and potential impacts are presented in Appendix A.

Clubhouses in the State of Hawai‘i

Hawaii • HALE ‘OLUEA

Hilo, HI • THE KONA PARADISE CLUB

Kailua Kona, HI Kauai

• FRIENDSHIP HOUSE Kapa‘a, HI

Maui • HALE o LANAKILA

Wailuku, HI

Molokai • HANA KALIMA

Kaunakakai, HI Oahu

• DIAMOND HEAD CLUBHOUSE Honolulu, HI

• HALE O HONOLULU Honolulu, HI

• KAUHALE LAHILAHI, THE MAKAHA CLUBHOUSE Waianae, HI

• KO‘OLAU CLUBHOUSE Kaneohe, HI

• WAIPAHU ALOHA CLUBHOUSE Waipahu, HI

6

Evaluation Activities Overview Analysis of the Clubhouse Performance Indicators collected by AMHD and Clubhouse Staff:

• Detail the services provided by Clubhouses

• Describe Clubhouse member characteristics

Interviews conducted with Clubhouse members

• Purpose: To add qualitative data and deeper understanding of Clubhouse members’ experiences.

• Questions were reviewed by the AMHD Service Coordinator, and distributed to all Clubhouse directors on

O‘ahu; see Appendix B for list of interview questions.

• 8 Semi-structured interviews were completed at Hale O and Ko’olau Clubhouses on O‘ahu.

• Duration was 30 minutes to an hour each.

• Members who participated were briefed on the following:

o Participation was completely voluntary and members could end interview at any time with no

pressure to continue.

o Anonymity: quotes were stripped of identifying information, and either attributed using

pseudonyms, or as “Clubhouse member.”

Identify gaps in Performance Indicators and interview data

Outline future evaluation plans

7

Clubhouse Membership Performance Indicators, Reporting Period 7/1/15 - 6/30/16

23% 21% 17%

16% 13% 9%

8% 6% 3%

Hale O Honolulu

Waipahu

Ko’olau

Hale ‘Oluea Diamond Head

Friendship House

Kauhale Lahilahi The Kona Paradise Club

Hale o Lanikila

Percent of total members by Clubhouse.

Hale O Honolulu has

the largest percentage, with 23% of the total 848 active

members.

848 total active

members as of July 2016

8

-12

-5

-4

-3

0

2

5

12

14

-15 -10 -5 0 5 10 15 20

Ko‘olau

New members: 205

196 Inactivated members

Total Growth: 9 members

Total Growth by Clubhouse

Hale ‘Oluea Diamond Head

The Kona Paradise Club Kauhale Lahilahi

Hale o Lanakila

Friendship House

Waipahu Aloha

Hale O Honolulu

New Member Perspective I just started coming in July. I was at HSH and had friends that started coming to Clubhouse. They just told me that they checked the internet here. So, I didn’t really know what Clubhouse was. What are your first impressions? It has a homey feel here. They have a lot of soul food here. What do you mean by that? I mean it nourishes the spirit. The food is good as well (laughs). There is the heart of family here. Everyone has been really friendly. This is probably the most friendly place on Oahu. Do you think the Clubhouse contributes to the larger community?

Yeah, it reminds people to share in aloha. From the lei on the board when you walk in, that’s pretty much what the motto of Clubhouse is. It’s about aloha.

9

Clubhouse Attendance Performance Indicators, Reporting Period 7/1/15 - 6/30/16

30%

Total Clubhouse attendance for all islands in 2015-16 = 988 O‘ahu Clubhouses account for 70% of overall attendance.

The highest daily average attendance was 66 at Waipahu Aloha in 2015-16 and 49 at Hale ‘Oluea on the Big Island in 2014-15.

Daily Average attendance fluctuated very little between

2014-15 and 2015-16.

70%

0102030405060

Daily Average Attendance 2014-2016

2014-15

2015-16

14

15

22

24

24

43

49

52

66

0 10 20 30 40 50 60 70

Hale o Lanakila

The Kona Paradise Club

Kauhale Lahilahi

Friendship House

Diamond Head

Hale ‘Oluea

Hale O Honolulu

Ko'olau

Waipahu Aloha

Average Daily Attendance 2015-16

10

Clubhouse Impacts

Interviews with staff and members showed that Hawai‘i Clubhhouses impacted both members and the larger community. • Members indicated that Clubhouses increased access to social support, housing, employment and advocates.

• Clubhouses also impacted the community through its service provision, community education, and advocacy.

By offering a safe, welcoming, and non-judgmental environment, Clubhouses increased members’ wellness and

wellbeing and educated the surrounding community on mental illness.

From the Members: On How Clubhouses Help

-- “I’ve been supported by other members more than one time. When I’ve been depressed or ill. There is one person who has had a lot of good advice for me. When my schizophrenia or depression has affected me, she tries to help me. She’s helped me in many in many ways.”

-- “Through the Clubhouse I got my housing. I am living independently. They helped me get my rent subsidy. They meet with me twice a month. One of the staff members helped me. I was homeless in my van before that for 20 years. I was staying on the beach on the North Shore, and one day I decided to detox. Then I went to rehab. I’ve been living independently two years in August.”

-- “I’ve done both supported and transitional employment. Right now my supported employment is as a peer coach and I have an independent position working in a nursery with babies. And the pay is pretty good, much higher than the minimum wage. The cost of living is so high here…”

-- “We’ve got someone right now in a domestic violence situation and she is in a shelter. Staff has supported her in that, and in getting a restraining order. People who are mentally ill don’t always know how to self-advocate.”

11

On Clubhouse Value to Members On Clubhouse Value to the Larger Community

“One of the biggest things is that [the Clubhouse] is a place to go. It’s safe. It’s non-judgmental. I don’t have to feel or perform a certain way. It still has the structure of going to a job, through the work ordered day. I know several members who used to just sit at home and watch TV all day. Instead, we are needed. We make friends. We’re at all different levels of work experience and ability.”

“I think we grow every day. Every single one of us. Change happens. As a Clubhouse we still grow. We still move. No matter what situation comes to us, we still pick ourselves up. We still move. We smile, and we laugh. We have disagreements. We keep on. Our Clubhouse makes us… I wouldn’t say make noise. Our Clubhouse stands for our rights when it comes to things. Definitely.”

“Many homeless people are mentally ill, and some of our members are homeless. Programs for the homeless are priceless for the community. Anything from decreased crime, helping them find housing, supporting them in recovery.... “

“Telling [the larger community] about mental illness is important. Not just through Clubhouse but other programs and events. We try to really let them know what people with mental illness are. People with mental illness are people, just like them. And we have a lot of the same experiences as others. Feeling depression, suicidal – other words may describe people and they may not use those words to add up to a diagnosis, but they are things many people experience.”

“At the state and federal level we have an advocacy committee. We call and visit regarding bills that support the mentally ill. We recently advocated for a bill on advocacy for hiring disabled people. I am trying to join the neighborhood committee here.”

12

Story of a Long-Term Member

Clubhouse as a Safety Zone

“The Clubhouse is kind of like having a family, which is something I’ve never really had. My dad was a construction worker; we lived in a trailer and moved around. Slept in the same bed, just parked it different places. That’s the reason I didn’t have friends growing up. I had an abusive mother, and I was an unwanted child. I didn’t get any breaks. The Clubhouse is a place to go -- a kind of a safe zone. I will be 73 in October, and I’ve been coming [to the Clubhouse] for about 20 years. There is kind of a code of ethics, just basic civil rights. You aren’t pressured into any activities. I enjoy doing things like hanging the blackboard. The barbeque kit was failing and I fixed it. Made it sturdy and strong. I like doing things like that, mechanical duties. Some of us have talents in different fields. They have breakfast and lunch, which is good for people who can’t cook or don’t cook. I kind of like the things like their job club, helping with the supported employment. And, if they do need help with the housing they can provide some help. They help me with my case manager, keeping in contact. [The Clubhouse] keeps me from sitting around stewing in my own juice if you know what I mean. There’s something to do. I’ve seen people go to the hospital because of physical things, but there aren’t that many crises here. I think if they are at the Clubhouse they don’t have these problems, as frequently as if they weren’t here. I think because there is more understanding and tolerance. I don’t experience discrimination here. I hear it rarely. But it isn’t like it is on the streets.

Like I said, it’s kind of like a safety zone.”

13

Member Employment A total of 485 members were employed in 2015-16. The majority of members were employed in supported employment. While making up only 21% of total membership, Waipahu Aloha makes up 66% of all employed members and 41% of its daily attendance consists of employed members.

485 members employed in 2015-16

66%

Percent of Total Employed Members

Waipahu Aloha 66%

14

Average Hourly Wage: $8.46

Most Common Types of Employment

Supported Employment

34%

Independent Employment

29%

Transitional Employment

26%

Group Placement

11%

Total Member Earnings 2015-16: $605,227

15

Member Wellness

Member Perspectives on Wellness Q: Do you think the Clubhouse helps you be more healthy?

“I think it does. Its helping me get better with my speech. I was isolating myself before. This has helped me come out of that. I tried a few Clubhouses and this is my favorite. I used to be a smoker and they helped me quit. “ “Well, we just started a hula group. Hula is very difficult to me; we’re breaking it down and practicing. It’s a 4,000 year-old chant and dance. You can take it as deep as you want to, or not. I’m struggling with that but I think it’s good to learn. It’s just a sense of relief whenever you practice hula.“

“Just coming here, socializing, having something to do makes me healthier.”

Members indicated that attending Clubhouses increased their overall

mental and physical health wellness.

16

Percent of Total Challenging Events by Clubhouse, 2015-16

Overall, the total number of members’ challenging events was low, with psychiatric hospitalizations being the most common event (n=58). Ko‘olau & Diamond Clubhouses had the most psychiatric hospitalizations, each making up 17% of total psychiatric hospitalizations.

Member Challenges Performance Indicators, Reporting Period 7/1/15 - 6/30/16

Despite the benefits of Clubhouses, members still faced many challenges, including incarceration, medical & psychiatric hospitalization, homelessness, and membership suspension. However, Clubhouses offered many support systems to help members through these crises.

Total Number of Challenging Events, 2015-16

Having only 16% of total active Hawaii Clubhouse members, Hale ‘Oluea experienced a disproportionate amount of challenging events, with:

• 51% of member suspensions,

• 40% of member incarcerations,

• 39% experiences of homelessness,

• 34% of medical hospitalizations. 16% 16%15%

13%

13%

17%

9%7%

17%

26%

13%

13%10%

6%

17%

6%

39%

40%

51% 14%

34%

7%7%

9%

20%14%

12% 9%

Homeless Incarceration Suspensions PsychiatricHospitalization

MedicalHospitalization

Hale o LanakilaFriendship HouseThe Kona Paradise ClubHale ‘Oluea Diamond HeadHonolulu HaleKo'olauKauhale LahilahiWaipahu

15

3443 45

58

010203040506070

Incarceration MedicalHospitalization

Homelessness MemberSuspensions

PsychiatricHospitalization

17

14% 12%

56%29% 28%

23%42%

8%

10%15%

51%

11%

Hale ‘Oluea WaipahuAloha

Ko'olau

Types of Services at Clubhouses with most Supports, 2015-16

Community SupportServices Interventions

Transportation Supports

Wellness Supports

Dual Recovery Supports

Clubhouse Activities Performance Indicators, Reporting Period 7/1/15 - 6/30/16

In order to help members navigate life challenges, Clubhouses offered transportation, community support interventions, dual recovery, and wellness supports. Wellness supports were the most frequently provided services overall. Member Rehabilitation Planning & Supported Education

Total Supports Provided by Clubhouse, 2015-16

Ko‘olau offered the most total supports followed closely by Waipahu Aloha and Hale ‘Oluea.

• 56% of Ko’olau’s services were dual recovery supports

• 51% of Waipahu’s services were community support interventions

• 42% of Hale ‘Oluea were transportation services

100271 311

724876 939 1012

1531 1602

0200400600800

10001200140016001800

2125

1952

1767

1522

0 500 1000 1500 2000 2500

Wellness Supports

Dual Recovery Supports

Community Support ServicesInterventions

Transportation Supports

Types of Support Provided, 2015-16

18

Clubhouse Outreach In order to continue to impact the larger community and to impact current members, Clubhouses engage in continual outreach. Members were outreached via phone (76%), face-to-face (5%), and by mail (19%).

A total of 10, 021 outreach attempts were made in 2015-16.

On Outreach “Well, for me, I went through a real serious depression, and the Clubhouse members were very good about calling me and telling me they would like to see me. Generally, if we haven’t seen anyone for a while, we give them a call and tell them we miss them. Recently, I was headed for a manic episode and they recognized that and helped me stay calm, remember my breathing, and avoid it.

Also, I went through breast cancer and I recently had another lump and one of the staff went to the biopsy with me and also went to get the results. Which was very supportive. I’ve also seen them do mobile outreach where they actually go to someone’s place of living. Take them something, see how they are doing. One lady had been in her house for a while and needed cat litter so they brought it to her. It’s a simple thing, but it means so much. It’s pretty powerful.”

“[If someone has a crisis] we try to find them. We try to go visit them. Find a way to get to them. Ask staff if they can take us. It depends where they end up, and what their reaction is. We have to be careful because people may not want to see us if they are really down. We may feel sad or down and we may want to cry if we see them really hurting, not how they usually are. “

A total of 2,663 people were outreached

in 2015-16

76% of these people were

called

19

House Functions

Clubhouse members participate in decision-making planning meetings as well as in day-to-day activities like meals. Members also participate in Clubhouse-sponsored recreational events and activities.

1,263 members participated in social or recreational activities, with the highest participation from Waipahu Aloha (57% of total member participants were from Waipahu Aloha). It is unclear if this higher participation is due to a difference in number events or commitment to Clubhouse.

37

20

19

14

14

13

12

11

8

0 5 10 15 20 25 30 35 40

Ko'olau

Hale O Honolulu

Waipahu Aloha

Hale ‘Oluea

Friendship House

The Kona Paradise Club

Hale o Lanakila

Kauhale Lahilahi

Diamond Head

Average Number of Members Participating in Weekly Decision-making Meetings, 2015-16

Those Clubhouses with the highest average number of members attending decision-making meetings were those with the highest percentage of membership, suggesting that participation remains consistent across Clubhouses.

73,007 Meals

served in 2015-16

20

Member Ideas for Growth and Improvement

-- “I guess I would like the community to know a lot more about us. To help more the mentally ill, particularly the homeless, and to get more support from the community. Especially through employment opportunities. Because they really do help integrate mentally ill people back into the community. It reduces the stigma. That’s our job.” “We really need a lot more Transitional Employment positions. They are all rather physical.”

-- “It would be nice if we could grow the board, see more involvement from the board. That is where we have to fundraise. I don’t know if you see the goal, but we are fundraising to send a staff member and a Clubhouse member to training. Our board is great, it’s just very small.”

-- “I think the staff should gather some members and take them to UH and go to the classroom. All the Clubhouses should do that. Take members out to the community and let people in the class ask questions. Maybe for an hour or half hour. They can ask the students questions, the students ask them questions. Not just one, all of us. We have our names. We have our name tags. It would help you guys to get to know us.”

-- “Well, the facilities are really hurting. The men don’t have a bathroom. They have to use a key and go outside. They have a broken down air conditioner that they say they’ve been going to replace for years. It’s a little small for the 50 people or so that come here every day. They could use an upgrade in the kitchen.”

21

Gaps in Data

• Individual-level indicators o Indicators that measure Clubhouse impacts on quality of life o Demographic variables such as age, gender, and race

• Organizational-level indicators

o Number of events held per Clubhouse o Number of staff o Number of active members linked to their demographic information

• Qualitative data across Clubhouses

o Interviews confined to Oahu Clubhouses thus far

• Community-level indicators o Indicators of impact on community attitudes toward and knowledge of mental illness

• Longitudinal data

o Measuring of above variables at different time points to determine impacts of Clubhouse over time.

22

Future Evaluation Plans Month 1:

• Meet with directors, staff and Clubhouse members at each O‘ahu Clubhouse to identify which Clubhouses are interested in participating in research, and potentially creating a “research unit”

o Decide on Clubhouse for survey pilot o Identify 3 Clubhouses interested in research

• Provide consultation on performance indicators. Review with the AMHD Service Coordinator considering

the following: o Contract monitoring requirements o Clubhouse accreditation standards o Medicaid billing requirements o Last evaluation report

Month 2: • Pilot Clubhouse Member Survey at 1 Clubhouse to answer the following questions:

o What is the best format to conduct the survey at Clubhouses? o Will Clubhouse members administer the survey to one another or will we it be conducted by staff or

UH research team? o Should the survey be modified? o Should anything be added to the evaluation plan?

Month 3: • Update Clubhouse Member Survey as appropriate • Roll-out survey at 2nd Clubhouse • Provide support and follow-up to both Clubhouses

23

Month 4: • Look at initial outcomes from survey data, modify survey or process if necessary • Roll-out survey at 3rd Clubhouse • Provide support and follow-up to 3 Clubhouses (with the goal of collecting monthly data)

Month 5:

• Provide support and follow-up • Conduct interviews at 1-3 Clubhouses to supplement survey data • Discuss feasibility and interest with Clubhouse members on continuing or developing research units

Month 6:

• Submit written report • Meet to review findings with AMHD • Present findings at an open celebration for Clubhouse members and staff

24

Appendix A: Clubhouse Logic Model

25

Appendix B: Interview Guide for Exploratory Interviews

Initial Interview with Clubhouse Members

The purpose of this interview is to help us learn more about your experience at the Clubhouse. Participation is completely voluntary! You can stop at any time with no pressure to continue, and you can skip questions or add to the questions if you think we are missing something important. All responses will be made anonymous. We will not collect names or share any information that identifies you. Thank you for your time!

1) Please tell me about your involvement in the Clubhouse. How long have you been coming and how did you start?

2) What are some things that you do at the Clubhouse?

3) Have you learned anything new since coming to the Clubhouse?

4) What do you like about the Clubhouse?

5) Can you tell me about a time when the Clubhouse provided support for you or one of the other members?

6) Where do you think you would you spend your days if you weren’t at the Clubhouse?

7) What would you like to see change at the Clubhouse? Is anything hard about it, or is there a way you would like to see it grow?

8) Do you think the Clubhouse adds value to the larger community? If so, how?

9) Is there anything else you would like to share?