Recovery Oriented System Indicators (ROSI) Survey FY 2015 ROSI Survey Results Virginia Department of Behavioral Health and Developmental Services September,

Recovery Oriented SystemIndicators (ROSI) Survey

FY 2015 ROSI Survey ResultsFY 2015 ROSI Survey ResultsVirginia Department of Behavioral Health

and Developmental Services

September, 2015

FY 2015 ROSI Survey Method

• In Fiscal Year 2015, 40 CSBs conducted the ROSI survey with 4,084 adult recipients of mental health services.

• 16 CSBs – used online SurveyMonkey®

• 6 CSBs – used paper version and local data entry

• 18 CSBs – used combination of paper version and online version

Respondents by Category (n=4,084)Frequency Percent Frequency Percent

Female 1,998 48.9 Ages 1-17 16 .4

Male 1,864 45.6 Ages 18-34 859 21.0

Missing 222 5.4 Ages 35-64 2,470 60.5

Ages 65+ 223 5.5Missing 516 12.6

Frequency Percent

Less than 1 year 329 8.1

1 to 2 years 335 8.2 Frequency Percent

3 to 5 years 533 13.1 Less than High School 746 18.3

More than 5 years 2,587 63.3 High School/GED 1,771 43.4

Missing300 7.3

College/Technical Training

975 23.9

Graduate School 204 5.0

Other 34 0.8

Frequency Percent Missing 351 8.6

American Indian/ Alaska Native 59 1.4Asian 60 1.5

Black or African American 1,303 31.9 Frequency Percent

Native Hawaiian / Other Pacific 10 .2 Hispanic 128 3.1

White/Caucasian 2,181 53.4 Not Hispanic 228 5.6

More than one race 108 2.6 Missing 3,728 90.9

Other 43 1.1

Missing 320 7.8

Age Groups

Education

Hispanic

Time in Treatment

Racials Groups

Gender

Survey Analysis• Using MS Excel, Access, and SPSS

– Average scores from 1 to 4• 1 = strongly disagree• 2 = disagree• 3 = agree• 4 = strongly agree

– % responding• Yes = agree or strongly agree• No = disagree or strongly disagree• N/A = does not apply to me• Blank = no response

8 ROSI Recovery Domains Ranked

Average Score / 4 = “Grade”• Formal Service Staff 3.395 / 4 = 84.9

• Formal Services 3.353 / 4 = 83.8• Social Relationships 3.347 / 4 = 83.7

• Choice 3.260 / 4 = 81.5 • Basic Material Resources 3.047 / 4 = 76.2

• Meaningful Activities 3.045 / 4 = 76.1• Peer Support 3.003 / 4 = 75.1

• Self/Holism 2.945 / 4 = 73.6

Findings: Formal Service Staff

Formal Service Staff – Avg. Score = 3.395– the critical roles formal service staff play in helping or

hindering the recovery process roles

% ‘Yes’: Survey Item (average score)• 90%: Staff respect me as a whole person. (3.4)

• 85%: Staff treat me with respect regarding my cultural background. (3.6)

• 84%: Staff believe that I can grow, change and recover. (3.5)

• 83%: Staff listen carefully to what I say. (3.5)

Findings: Formal Service Staff

Formal Service Staff continued

% ‘Yes’: Survey Item (average score)• 79%: Staff see me as an equal partner in my treatment

program (3.4) • 75%: My treatment plan goals are stated in my own

words. (3.4)

• 18%: Staff do not understand my experience as a person with mental health problems. (1.9)

• 21%: Staff lack up-to-date knowledge on the most effective treatments. (1.8)

= Increase of 2 or more percentage points from FY 2013

Findings: Formal ServicesFormal Services – Avg. Score = 3.353

– the systems’ culture, organization, structure, funding, access, choice, quality, range, continuity and other characteristics can help or hinder the process of recovery

% ‘Yes’: Survey Item (average score)• 81%: Mental health staff support my self-care or

wellness. (3.5) • 78%: Mental health staff help me build on my strengths.

(3.4)• 77%: The doctor worked with me to get on medications

that were most helpful for me. (3.5) • 75%: I have information and/or guidance to get the

services and supports I need. (3.3)• 68%: I can see a therapist when I need to. (3.2)

Findings: Formal Services Formal Services continued

% ‘Yes’: Survey Item (average score)• 47%: My family gets the education or supports they need

to be helpful to me. (2.9)

• 5%: Staff use pressure, threats or force in my treatment. (1.2)

• 10%: Mental health services have caused me emotional or physical harm. (1.7)

• 12%: The mental health staff ignore my physical health. (1.8)

• 16%: I cannot get the services I need when I need them. (1.9)

Findings: Social Relationships

Social Relationships – Avg. Score = 3.347– the roles social and personal relationships play in

facilitating recovery

% ‘Yes’: Survey Item (average score)• 92%: There is at least one person who believes

in me. (3.4)

• 9%: Mental health staff interfere with my personal relationships. (1.4)

• 21%: I do not have the support I need to function in the roles I want in my community. (2.0)

Findings: Choice

Choice – Avg. Score = 3.260– having choices, as well as support in the process of making

choices, regarding housing, work, social, service, treatment as well as other areas of life facilitate recovery

% ‘Yes’: Survey Item (average score)• 81%: Staff give me complete information in words I

understand before I consent to treatment or medication. (3. 5)

• 69%: I have a say in what happens to me when I am in crisis. (3.2)

• 68%: My right to refuse treatment is respected. (3.3)

• 22%: I do not have enough good service options to choose from. (2.0)

= Decline of 2 or more percentage points from FY 2013

Findings: Basic Material Resources

Basic Material Resources – Avg. Score = 3.047– (recovery from mental illness is incumbent on basic

material resource needs being met).

% ‘Yes’: Survey Item (average score)• 84%: I have a place to live that feels like a

comfortable home to me (3.2) • 78%: Staff stood up for me to get the services

and resources I needed (3.4)

• 72%: Mental health services helped me get medical benefits that meet my needs. (3.1)

Findings: Basic Material Resources

Basic Material Resources continued

% ‘Yes’: Survey Item (average score)• 65%: I have reliable transportation to get where I

need to go. (3.1)

• 60%: I have housing that I can afford. (3.1)

• 49%: Mental health services helped me get housing in a place I feel safe. (2.9)

• 47%: I have enough income to live on. (2.4)

= Decline of 2 or more percentage points from FY 2013

Findings: Meaningful Activities

Meaningful Activities – Avg. Score = 3.045– (work, education, voluntary and/or group advocacy

activities that are meaningful to the individual facilitate recovery).

%‘Yes’: Survey Item (average score)• 82%: Staff encourage me to do things that are

meaningful to me. (3.5)

• 49%: I have a chance to advance my education if I want to. (2.9)

• 25%: Mental health services helped me get or keep employment. (2.5)

Findings: Peer Support

Peer Support – Avg. Score = 3.003– peer support and consumer operated services in a

myriad of forms facilitate recovery

% ‘Yes’: Survey Item (average score)• 77%: I am encouraged to use consumer-run programs

(e.g., support groups, drop-in centers, etc.) (3.2) • 48%: There was a consumer peer advocate to turn to

when I needed one. (2.9)

• 44%: There are consumers working as paid employees in the mental health agency where I receive services. (2.9)

=Decline of 2 or more percentage points from FY 2013

Findings: Self/HolismSelf/Holism – Avg. Score = 2.945

– characteristics that relate to one’s sense of self, such as self-reliance, as well as having a holistic and human rights focus can facilitate recovery and other such characteristics, such as low self-esteem, can hinder recovery

% ‘Yes’: Survey Item (average score)• 81%: Services help me develop the skills I need. (3.1)

• 36%: Mental health services led me to be more dependent, not independent. (2.3)

• 20%: I lack the information or resources I need to uphold my client and basic human rights. (2.0)

ROSI Score ComparisonsROSI Score Comparisons53% scored their CSB’s recovery orientation above the average

score of 3.19

What are the characteristics of respondents who rate their CSB above average in the overall score?


Score ComparisonsScore Comparisons

Above Average Score of 3.19 (n=1,999)

% Scoring above average: significant differences by gender, race, and education

– Gender: Male 51% Female 57%

– Race: White 59% Black 50%

= Increase of 2 or more percentage points from FY 2013 = Decline of 2 or more percentage points from FY 2013

Education Level: < High School

High School

CollegeGrad.

School

% Scoring Above Average 52% 55% 56% 49%

Rated BelowRated Above

Above or Below Average by Gender

Above or Below Average by Race



Above or Below Average by Education Level

Above Average Score of 3.19 (n=1,999)% Scoring above average: significant differences by CSB region, community, and housing type

– Community: Urban 47% Rural 59%Housing Type: homeless

boarding home

residential facility

supervised apartment

own home

% Scoring Above Average

24% 35% 55% 49% 60%


Region % Above Region % Above

1 56% 5 50%

2 45% 6 51%

3 76% 7 51%

4 45% All 53%

Above or Below Average by Partnership Planning Region


Above or Below Average by Community Type Rural vs. Urban CSBs



Above or Below Average by Housing Type

Above Average Score of 3.19 (n=1,999)

% Scoring above average: significant differences by years in service, number and type of services

– In service five years or less: 50%– In service more than five years: 56%

– Number of services received:• One: 43% Two:

53%• Three to Four: 57% More than Four: 58%

– Service received:• Medication: 58% Counseling: 55%• Case Management: 58%


Above or Below Average by Time in CSB Services > 5 Years


No Yes

Above or Below Average by Number of CSB Services Received


Above or Below Average by Receiving Medication Service


No Yes

Above or Below Average by Receiving Counseling Service


No Yes

Above or Below Average by Receiving Case Management Service


No Yes

ROSI Score ComparisonsROSI Score Comparisons

Which survey items showed the biggest difference between above and below overall average scores?


Score ComparisonsScore Comparisons

Which items make the most difference?

The largest difference in responses between the percent of respondents who scored their CSB above average

vs. those who scored their CSB below average:

• % Above – Below by Survey Item

– 73% - 33% Help getting/keeping employment– 82% - 45%: Having Peer Advocates to turn to– 82% - 46%: Chance to advance my education– 81% - 47% Family gets education or supports– 92% - 60%: Having a say in what happens in crisis

Above or Below Avg. by Employment Assistance


StronglyDisagre

e

StronglyAgree

Disagree

Agree

Above or Below Average by Peer Advocate


StronglyDisagre

e

StronglyAgreeDisagree Agree

Above or Below Average by Chance to Advance Education


StronglyDisagre

e

StronglyAgreeDisagree Agree

Above or Below Average by Family Education and Supports


StronglyDisagre

e

StronglyAgree

Disagree

Agree

Above or Below Average by Having a Say when in Crisis


StronglyAgree

Disagree AgreeStrongly Disagree

CSB Provider Survey

• CSBs that reported sharing the results of last year’s ROSI survey with staff and the individuals receiving services had a higher percentage of respondents rating them above average.

And• CSBs that reported making changes based

on their ROSI survey responses had a higher percentage of respondents rating them above average in their Recovery Orientation.

Above or Below Average by Whether ROSI Results are Shared*


No Yes

* With staff and individuals receiving services

Examples of How FY 15 Survey Results Were Shared

“Results were initially shared with program managers and staff, who then shared the results with individuals at the times they were at the programs receiving services.”

“Survey results were shared with the consumer advisory council and sent to a mailing list of over 300 individuals, which includes service recipients, peer-providers, family members and service providers.”

“Results were reviewed with program staff for discussion and planning as well as with individuals receiving services through psychosocial rehabilitation.”

“A copy of the ROSI survey results was distributed to each staff member…Also, a copy of the ROSI survey was left in the reception area for general population perusal of results.”

“We reviewed the results of the last survey with individuals at our psychosocial rehabilitation program during a group meeting.”

Above or Below Average by Made Changes based on Survey*


No Yes

* Based on ROSI survey results

Examples of Actions Taken Based on Survey Results

• “We began offering WRAP seminars this year.”• “Created and filled a Peer Support Specialist position.”• “Developed and distributed recovery-oriented handbook for consumers regarding

services.”• “A peer mentor program was developed to assist individuals on pass from the state

hospital to help in adjusting to the clubhouse environment.”• “Greater emphasis was placed on developing priorities, setting program benchmarks

and formalizing Peer Recovery Services (PRS) as a value-added service. Some examples of this work include finalizing the PRS brochure, scheduling structured monthly peer supervision meetings, setting-up in-service trainings about PRS to market services internally and filling the need for direct PRS services.”

• “MH and SA staff have increased the focus on improving medical health of individuals and assisted consumers in understanding insurance changes.”

• “Developed strategies to make improvements and followed up on those in smaller teams. This included the general training and reminders about language used with clients, the need to remind clients of rights and not assume they are aware, etc.”

Documents

Recovery Oriented System Indicators (ROSI) Survey FY 2015 ROSI Survey Results Virginia Department of Behavioral Health and Developmental Services September,