Measuring the Recovery Process:Measuring the Recovery Process:The Milestones of Recovery ScaleThe Milestones of Recovery Scale

(MORS)(MORS)

Dave Pilon, Ph.D., C.P.R.P.Dave Pilon, Ph.D., C.P.R.P.

Mental Health America of Los Mental Health America of Los AngelesAngeles

dpilon@mhala.orgdpilon@mhala.org

February 7, 2008February 7, 2008

Goals of the TrainingGoals of the Training

Discuss difficulties in the Discuss difficulties in the measurement of “Recovery”measurement of “Recovery”

Describe our attempts to create an Describe our attempts to create an instrument that classifies consumers instrument that classifies consumers according to their current “milestone according to their current “milestone of recovery”of recovery”

Begin the discussion of how the Begin the discussion of how the Milestones can be used create “flow” Milestones can be used create “flow” through the system and increase through the system and increase program and system accountability.program and system accountability.

Recovery as the Basis for Recovery as the Basis for Services under MHSA (From Services under MHSA (From

Section 7)Section 7)““Planning for services shall be consistent Planning for services shall be consistent

with the philosophy, principles, and with the philosophy, principles, and practices of the Recovery Vision for practices of the Recovery Vision for mental health consumers:mental health consumers:

(1) To promote concepts key to the (1) To promote concepts key to the recovery for individuals who have recovery for individuals who have mental illness: hope, personal mental illness: hope, personal empowerment, respect, social empowerment, respect, social connections, self-responsibility, and connections, self-responsibility, and self-determination.”self-determination.”

What is Recovery?What is Recovery?

Many consumers speak of recovery in Many consumers speak of recovery in terms of their own internal experience – terms of their own internal experience – often phrased in such terms as “becoming often phrased in such terms as “becoming empowered,” “taking charge of their own empowered,” “taking charge of their own lives,” “improving their self-esteem,” or lives,” “improving their self-esteem,” or “becoming responsible for themselves.”“becoming responsible for themselves.”

The mitigation of psychiatric symptoms The mitigation of psychiatric symptoms (or symptom distress) and improvement (or symptom distress) and improvement in functioning.in functioning.

Identifying and taking on meaningful roles Identifying and taking on meaningful roles in one’s life.in one’s life.

Recovery Definition MatrixRecovery Definition MatrixService Provision/PracticesService Provision/Practices OutcomesOutcomes

ServiceServiceRecipientRecipientPerspectivePerspective

Recovery Self-Assessment (RSA)Recovery Self-Assessment (RSA)““Staff at this agency listen to and follow my Staff at this agency listen to and follow my

choices and preferences.”choices and preferences.”Consumer Recovery Outcome System Consumer Recovery Outcome System

(CROS) “How do I feel about the (CROS) “How do I feel about the choices I get about my care?”choices I get about my care?”

Recovery Oriented System IndicatorsRecovery Oriented System Indicators(ROSI) “I do not have enough good service (ROSI) “I do not have enough good service

options to choose from.”options to choose from.”

Internal Experience of the Client Internal Experience of the Client Consumer Recovery Outcome System Consumer Recovery Outcome System

(CROS) “I am coping better in my life” (CROS) “I am coping better in my life” Recovery Measurement Tool (RMT)Recovery Measurement Tool (RMT)““I participate in meaningful activities”I participate in meaningful activities”Spirituality Index of Well Being (SIWB)Spirituality Index of Well Being (SIWB)““There is not much I can do to make a There is not much I can do to make a

difference in my life”difference in my life”

ServiceServiceProvider/Provider/FamilyFamilyMember/Member/SystemSystemPerspectivePerspective

Fidelity to Specific PracticesFidelity to Specific PracticesEvidence-Based PracticesEvidence-Based PracticesClinical Strategies Implementation ScaleClinical Strategies Implementation ScaleAssertive Community Treatment (ACT)Assertive Community Treatment (ACT)Supported Employment (SE)Supported Employment (SE)

Symptom ReductionSymptom ReductionImprovement in FunctioningImprovement in FunctioningReductions in Adverse Impact Reductions in Adverse Impact

(hospitalization, incarceration, (hospitalization, incarceration, homelessness, mortality)homelessness, mortality)

Improved Quality of Life (Increases in Improved Quality of Life (Increases in independent living, employment, independent living, employment, education rates, more supportive social education rates, more supportive social network)network)

Milestones of Recovery ProjectMilestones of Recovery Project

In 1997, with a grant from the Center for In 1997, with a grant from the Center for Mental Health Services, the California Mental Health Services, the California Association of Social Rehabilitation Association of Social Rehabilitation Agencies (CASRA) convened a group of 50 Agencies (CASRA) convened a group of 50 administrators, clinicians and consumers.administrators, clinicians and consumers.

This group was given the task of creating This group was given the task of creating a system that would classify consumers in a system that would classify consumers in particular clusters according to their needs particular clusters according to their needs in a way that would enable the providers in a way that would enable the providers of services to be held accountable for the of services to be held accountable for the outcomes of their services.outcomes of their services.

CASRA CSP PROJECTCASRA CSP PROJECT Working independently in small groups, the Working independently in small groups, the

participants all came to a similar conclusion: participants all came to a similar conclusion: consumers could be assigned to clusters consumers could be assigned to clusters based on their level of risk, their level of based on their level of risk, their level of coping skills, and their level of engagement coping skills, and their level of engagement with the mental health system. with the mental health system.

The participants concluded that the The participants concluded that the movement of consumers from one group or movement of consumers from one group or cluster to another could itself be viewed as an cluster to another could itself be viewed as an outcome.outcome.

It also appeared to the participants that It also appeared to the participants that movement from one group to another could movement from one group to another could be reasonably seen as a description of “the be reasonably seen as a description of “the process of recovery.”process of recovery.”

Components and Milestones of Components and Milestones of RecoveryRecovery

Components of RecoveryComponents of Recovery

1.1. Level of RiskLevel of Risk2.2. Level of EngagementLevel of Engagement3.3. Level of Skills and Level of Skills and

SupportsSupports

Milestones of RecoveryMilestones of Recovery

1.1. Extreme RiskExtreme Risk2.2. High Risk/UnengagedHigh Risk/Unengaged3.3. High Risk/EngagedHigh Risk/Engaged4.4. Poorly Coping/ Poorly Coping/

UnengagedUnengaged5.5. Poorly Poorly

Coping/EngagedCoping/Engaged6.6. Coping/RehabilitatingCoping/Rehabilitating7.7. Early RecoveryEarly Recovery8.8. Advanced RecoveryAdvanced Recovery

Early RecoveryEarly RecoveryThese individuals are actively managing their mental These individuals are actively managing their mental health treatment to the extent that mental health staff health treatment to the extent that mental health staff rarely need to anticipate or respond to problems with rarely need to anticipate or respond to problems with them. They are rarely using hospitals and are not being them. They are rarely using hospitals and are not being taken to jails. They are abstinent or have minimal taken to jails. They are abstinent or have minimal impairment from drugs or alcohol and they are managing impairment from drugs or alcohol and they are managing their symptom distress. their symptom distress. With minimal support from staffWith minimal support from staff, , they are setting, pursuing and achieving many quality of they are setting, pursuing and achieving many quality of life goals (e.g., work and education) and have established life goals (e.g., work and education) and have established roles in the greater (non-disabled) community. They are roles in the greater (non-disabled) community. They are actively managing any physical health disabilities or actively managing any physical health disabilities or disorders they may have (e.g., HIV, diabetes). They are disorders they may have (e.g., HIV, diabetes). They are functioning in many life areas and are very self-supporting functioning in many life areas and are very self-supporting or productive in meaningful roles. They usually have a or productive in meaningful roles. They usually have a well-defined social support network including friends well-defined social support network including friends and/or family.and/or family.

The Underlying DimensionsThe Underlying Dimensions

What do we mean by:What do we mean by:

RISKRISK ENGAGEMENTENGAGEMENT SKILLS AND SUPPORTSSKILLS AND SUPPORTS

RISKRISK

The consumer’s The consumer’s LEVEL OF RISKLEVEL OF RISK is is comprised of three primary factors:comprised of three primary factors:

1) the consumer’s likelihood of causing 1) the consumer’s likelihood of causing physical harm to self or others,physical harm to self or others,

2) the consumer’s level of participation 2) the consumer’s level of participation in risky or unsafe behaviors, andin risky or unsafe behaviors, and

3) the consumer’s level of co-occurring 3) the consumer’s level of co-occurring disorders.disorders.

ENGAGEMENTENGAGEMENT

The consumer’s The consumer’s LEVEL OF LEVEL OF ENGAGEMENTENGAGEMENT is the level of is the level of “connection” between the consumer “connection” between the consumer and the mental health service and the mental health service system.system.

What Engagement ISN’TWhat Engagement ISN’T

Engagement DOES NOT mean Engagement DOES NOT mean medication compliancemedication compliance

Engagement DOES NOT mean Engagement DOES NOT mean “insight” into or “acceptance” of one’s “insight” into or “acceptance” of one’s mental illnessmental illness

Engagement DOES NOT mean total Engagement DOES NOT mean total agreement with the service and agreement with the service and treatment approaches of stafftreatment approaches of staff

Engagement DOES NOT mean the Engagement DOES NOT mean the total amount of services receivedtotal amount of services received

What Engagement ISWhat Engagement IS

Engagement is about the quality of Engagement is about the quality of the relationship (“connectedness”) the relationship (“connectedness”) between consumer and staff and between consumer and staff and does not require the consumer to does not require the consumer to accept that s/he has a mental illness. accept that s/he has a mental illness. It merely requires that the consumer It merely requires that the consumer is “tolerant” of the presence of staff is “tolerant” of the presence of staff in his/her life. in his/her life.

SKILLS AND SUPPORTSSKILLS AND SUPPORTS

The consumer’s The consumer’s LEVEL OF SKILLS LEVEL OF SKILLS AND SUPPORTSAND SUPPORTS should be viewed should be viewed as the combination of the member’s as the combination of the member’s abilities and support network(s) and abilities and support network(s) and the the level to which the consumer level to which the consumer needs staff support to meet his/her needs staff support to meet his/her needsneeds..

Inter-rater ReliabilityInter-rater Reliability(MHALA Village)(MHALA Village)

Intra-class correlation coefficientIntra-class correlation coefficient All analysis done in SAS 9.01.03 All analysis done in SAS 9.01.03

using PROC MIXED (Littell, et al., using PROC MIXED (Littell, et al., 2006)2006)

r r = .85= .85 .70 used as meeting acceptable .70 used as meeting acceptable

reliability (Nunnally & Bernstein, reliability (Nunnally & Bernstein, 1994)1994)

Inter-rater Reliability Inter-rater Reliability (Vinfen)(Vinfen)

105 clients rated at two points in 105 clients rated at two points in time (one client missing second time time (one client missing second time point)point)

rr = .86, 95%CI = .80, .90 = .86, 95%CI = .80, .90

Level of Care Utilization Level of Care Utilization SystemSystem(LOCUS)(LOCUS)

Underlying subscales includeUnderlying subscales include– Risk of harmRisk of harm– Functional statusFunctional status– Medical, Addictive, and Psychiatric Co-Medical, Addictive, and Psychiatric Co-

morbiditymorbidity– Recovery Environment—Level of StressRecovery Environment—Level of Stress– Recovery Environment—Level of Recovery Environment—Level of

SupportSupport– Treatment and Recovery HistoryTreatment and Recovery History– EngagementEngagement

LOCUS Validity CoefficientsLOCUS Validity CoefficientsRisk of Risk of Harm Harm RatingRating

FunctionFunctional Status al Status RatingRating

MAP MAP RatingRating

Level of Level of Stress Stress RatingRating

Level of Level of Support Support RatingRating

TreatmenTreatment and t and Recovery Recovery HistoryHistory

Engage-Engage-

ment ment RatingRating

MORS MORS RatingRating

.72.72

.66, .76.66, .76.67.67

.62, .72.62, .72.50.50

.42, .56.42, .56.59.59

.52, .65.52, .65.43.43

.35, .50.35, .50.65.65

.59, .70.59, .70.61.61

.55, .66.55, .66

Stability of Ratings within the Stability of Ratings within the Village ISA January 2005 – March Village ISA January 2005 – March

20052005

January 2005January 2005 February 2005February 2005 March 2005March 2005

SORSOR Freq.Freq. %% Freq.Freq. %% Freq.Freq. %%

11 1818 4.24.2 2121 4.64.6 1919 4.64.6

22 2626 6.06.0 2222 4.94.9 2424 5.85.8

33 4949 11.411.4 4747 10.410.4 3434 8.28.2

44 4646 10.710.7 4545 9.99.9 2626 6.36.3

55 108108 25.125.1 127127 28.028.0 112112 26.926.9

66 122122 28.428.4 130130 28.728.7 139139 33.433.4

77 5050 11.611.6 5252 11.511.5 5252 12.512.5

88 1111 2.62.6 99 2.02.0 1010 2.42.4

TotalsTotals 430430 100.0100.0 453453 100.0100.0 423423 100.0100.0

MORS ClientMORS ClientFemale, African American, DOB 1-15-Female, African American, DOB 1-15-

7575

1

2

3

4

5

6

7

8

1/1/2005

2/1/2005

3/1/2005

4/1/2005

5/1/2005

6/1/2005

7/1/2005

8/1/2005

9/1/2005

10/1/2005

MORS ClientMORS ClientMale, Caucasian, DOB 5-20-40Male, Caucasian, DOB 5-20-40

1

2

3

4

5

6

7

8

1/1/2005

2/1/2005

3/1/2005

4/1/2005

5/1/2005

6/1/2005

7/1/2005

8/1/2005

9/1/2005

10/1/2005

Future Directions and Future Directions and QuestionsQuestions

What is the “typical” path of a person in recovery? What is the “typical” path of a person in recovery? If a person enters our system as “high risk, If a person enters our system as “high risk, unengaged,” how long (on average) will it take for unengaged,” how long (on average) will it take for her to become “coping/rehabilitating?” How long her to become “coping/rehabilitating?” How long until she is in “advanced recovery.”until she is in “advanced recovery.”

Are different services more or less effective at Are different services more or less effective at different milestones of recovery? Can we use the different milestones of recovery? Can we use the milestones to assign consumers to different types milestones to assign consumers to different types of care?of care?

Is it possible to establish “benchmarks” for service Is it possible to establish “benchmarks” for service providers? Can we hold service providers providers? Can we hold service providers accountable for moving people through the accountable for moving people through the milestones? Should we set expectations for milestones? Should we set expectations for service providers to move certain percentages of service providers to move certain percentages of their consumers to higher milestones over a set their consumers to higher milestones over a set amount of time?amount of time?

Program AccountabilityProgram Accountability Data indicate that consumers at the Village Data indicate that consumers at the Village

have a 10% chance of remaining at “extreme have a 10% chance of remaining at “extreme risk” after 10 monthsrisk” after 10 months

Data indicate that consumers at the Village Data indicate that consumers at the Village have a 6% chance of remaining at “extreme have a 6% chance of remaining at “extreme risk” after 20 monthsrisk” after 20 months

From an initial stage of “unengaged,” it takes From an initial stage of “unengaged,” it takes about 1.5 years before a Village member has a about 1.5 years before a Village member has a 50% chance of being in the 50% chance of being in the “coping/rehabilitating stage“coping/rehabilitating stage

From an initial stage of “unengaged,” it takes From an initial stage of “unengaged,” it takes about 5 years before a Village member has a about 5 years before a Village member has a 50% chance of being in the “self responsible” 50% chance of being in the “self responsible” stagestage

Recovery-Based Service Recovery-Based Service DeliveryDelivery

Stage 0: Extreme RiskStage 0: Extreme Risk(Milestone 1)(Milestone 1)

Stage 1: UnengagedStage 1: Unengaged(Milestones 2 and 4)(Milestones 2 and 4)

Stage 2: Engaged, but poorly self-Stage 2: Engaged, but poorly self-directeddirected(Milestones 3, 5 and 6)(Milestones 3, 5 and 6)

Stage 3: Self-responsibleStage 3: Self-responsible(Milestones 6, 7, and 8)(Milestones 6, 7, and 8)

Milestones of Recovery Levels of ServiceMilestones of Recovery Levels of Service(Recovery Based Spectrum of Care)(Recovery Based Spectrum of Care)

Extreme riskExtreme risk UnengagedUnengaged Engaged, but not self coordinatingEngaged, but not self coordinating Self-responsibleSelf-responsible

Locked settings Locked settings (State Hospital, (State Hospital, IMDs, etc.)IMDs, etc.)

Outreach and Outreach and engagementengagement

Drop-in Drop-in centercenter

Intensive case Intensive case managementmanagement

Case Case managementmanagement

teamteam

Appointment Appointment based clinicbased clinic

Wellness Wellness centercenter

Extreme riskExtreme risk(1)(1)

High risk, unengaged High risk, unengaged (2)(2)

Poorly coping, unengaged Poorly coping, unengaged (4)(4)

High risk, High risk, engaged engaged

(3)(3)

Poorly coping, Poorly coping, engaged engaged

(5)(5)Coping, Coping,

rehabilitating rehabilitating (6)(6)

Coping, rehabilitatingCoping, rehabilitating(6)(6)

Early recovery Early recovery (7)(7)

1:1 supervision1:1 supervisionLegal interventionsLegal interventions

Community Community protectionprotection

Acute treatmentAcute treatmentEngagementEngagement

Welcoming/CharityWelcoming/CharityEvaluation and triageEvaluation and triage

DocumentationDocumentationBenefits assistanceBenefits assistance

Accessible MedicationsAccessible MedicationsDrop-in servicesDrop-in services

Case management Case management Full Service PartnershipFull Service PartnershipAccessible medicationsAccessible medications

Supportive servicesSupportive services(Supported Housing, Employment, (Supported Housing, Employment,

Education)Education)Direct subsidiesDirect subsidiesRehabilitationRehabilitation

Appointment based therapyAppointment based therapy““Medications only”Medications only”

Wellness activities (WRAP)Wellness activities (WRAP)Self-helpSelf-help

Peer supportPeer supportCommunity integrationCommunity integration

Service Differentiation by Stage of Service Differentiation by Stage of Recovery (Employment)Recovery (Employment)

Unengaged (Stage 1): day labor, “work for a Unengaged (Stage 1): day labor, “work for a day”day”

Engaged but Poorly Self-Directed (Stage Engaged but Poorly Self-Directed (Stage 2): agency businesses, supported 2): agency businesses, supported employment including job development employment including job development and coaching, group placements, and coaching, group placements, supported mental health employment supported mental health employment (peer provider)(peer provider)

Self-Responsible (Stage 3): non-disclosure Self-Responsible (Stage 3): non-disclosure competitive employment job competitive employment job development, competitive mental health development, competitive mental health employment (regular staff)employment (regular staff)

THANK YOU!THANK YOU!