Los Angeles County Los Angeles County Evaluation SystemEvaluation System

Accomplishments and Next Steps Accomplishments and Next Steps toward Outcomes and toward Outcomes and

Performance Measurement of Performance Measurement of Treatment Services in LA CountyTreatment Services in LA County

20082008

Why was LACES Implemented?Why was LACES Implemented?

LA County Commissions on Alcoholism and Narcotics LA County Commissions on Alcoholism and Narcotics and Dangerous Drugs recommended to ADPA that a and Dangerous Drugs recommended to ADPA that a system for measuring the effectiveness of LA County system for measuring the effectiveness of LA County AOD Treatment services should be developed. AOD Treatment services should be developed.

This sentiment was reinforced by the LA County Board of This sentiment was reinforced by the LA County Board of Supervisors and LACES was funded and initiated.Supervisors and LACES was funded and initiated.

SAMHSA instituted the NOMSSAMHSA instituted the NOMS California instituted CalOMSCalifornia instituted CalOMS Data required to support SACPA and NORAData required to support SACPA and NORA LA Board of Supervisors Resolution to require health LA Board of Supervisors Resolution to require health

services in LA County to move provider contracts to a services in LA County to move provider contracts to a performance based platform.performance based platform.

Overview of LA County Overview of LA County Evaluation System: LACESEvaluation System: LACES

LACES is an eight year collaborative partnership LACES is an eight year collaborative partnership between ADPA, contracted providers, UCLA/ISAP and between ADPA, contracted providers, UCLA/ISAP and other county and state representatives.other county and state representatives.

The purpose of LACES is to design, implement, and The purpose of LACES is to design, implement, and conduct an evaluation of the adult alcohol and other drug conduct an evaluation of the adult alcohol and other drug treatment/recovery system in Los Angeles County.treatment/recovery system in Los Angeles County.

Goals of LACESGoals of LACES

Gather information on the alcohol and other drug Gather information on the alcohol and other drug treatment systemtreatment system

Obtain additional insight into the clients who receive Obtain additional insight into the clients who receive treatment at ADPA contracted sites.treatment at ADPA contracted sites.

Develop a system-wide data collection system to Develop a system-wide data collection system to evaluate the outcomes for individuals in treatment in evaluate the outcomes for individuals in treatment in AOD LA County and to assess performance of LA AOD AOD LA County and to assess performance of LA AOD treatment service agencies.treatment service agencies.

Use data to influence the future development of the AOD Use data to influence the future development of the AOD system.system.

AccomplishmentsAccomplishments of LACES of LACES

To date LACES has:To date LACES has:1.1. Laid the foundation and developed many of Laid the foundation and developed many of

the fundamentals of the ongoing evaluation the fundamentals of the ongoing evaluation system.system.

2.2. Designed and implemented evidence-based, Designed and implemented evidence-based, standardized evaluation procedures.standardized evaluation procedures.

• Collects client data at admission and discharge.Collects client data at admission and discharge.

3.3. Demonstrates treatment benefits to Demonstrates treatment benefits to individuals who receive treatment.individuals who receive treatment.

• Reported findings in LACES Status Report.Reported findings in LACES Status Report.

LACES LACES AccomplishmentsAccomplishments, cont’d., cont’d.

4.4. Led the efforts to review and improve existing Led the efforts to review and improve existing treatment forms and instrument.treatment forms and instrument.

• Revisions were required to bring LA County Revisions were required to bring LA County compliant with the federal mandates compliant with the federal mandates (NOMS/PPG).(NOMS/PPG).

• Revisions adopted by California (CalOMS).Revisions adopted by California (CalOMS).

5.5. Conducted extensive training of Conducted extensive training of treatment/recovery staff.treatment/recovery staff.

• Provided instruction to staff to improve validity and Provided instruction to staff to improve validity and reliability of LACPRS information.reliability of LACPRS information.

• Instructed over 1500 staff in various trainings. Instructed over 1500 staff in various trainings.

Additional AccomplishmentsAdditional Accomplishments

6.6. Designed and implemented LACES reports.Designed and implemented LACES reports.A.A. LACES Clinical Narrative Report.LACES Clinical Narrative Report.

• Encourages and builds off of accurate completion of Encourages and builds off of accurate completion of admission information by treatment/recovery staff.admission information by treatment/recovery staff.

• A summary of clinical related information.A summary of clinical related information.

B.B. LACES Discharge Outcome Report LACES Discharge Outcome Report • Admission & discharge information on a single individual.Admission & discharge information on a single individual.• Useful report treatment outcomes to others.Useful report treatment outcomes to others.

C.C. LACES Site Report.LACES Site Report.

What is a Site Report?What is a Site Report?

Brief – Six pagesBrief – Six pages Provides demographic, client outcome, and Provides demographic, client outcome, and

program performance measures. program performance measures. Information is gathered from the admission & Information is gathered from the admission &

discharge forms.discharge forms. Includes site specific information as well as Includes site specific information as well as

information from other sites of the same type.information from other sites of the same type. Reports provided to all programs on an Reports provided to all programs on an

annual basis and to those programs with high annual basis and to those programs with high admissions every quarter.admissions every quarter.

What is the Purpose of the Site What is the Purpose of the Site Report?Report?

To provide information concerning how To provide information concerning how participants are benefiting from treatment.participants are benefiting from treatment.

To provide information concerning areas of To provide information concerning areas of program functioning and short-term (admission program functioning and short-term (admission to discharge) treatment outcome.to discharge) treatment outcome.

To identify provider performance strengths and To identify provider performance strengths and areas needing improvement.areas needing improvement.

Program Functioning & Treatment Program Functioning & Treatment Outcomes Outcomes

Areas included in the Site Reports include:Areas included in the Site Reports include: Program PerformanceProgram Performance

Number of admissions and discharges.Number of admissions and discharges. Percent of cases with complete discharge information Percent of cases with complete discharge information

collected.collected. Participants who completed treatment.Participants who completed treatment.

Treatment OutcomesTreatment Outcomes Retention in treatment (average days).Retention in treatment (average days). Changes in alcohol and other drug use.Changes in alcohol and other drug use. Changes in other areas including health problems, Changes in other areas including health problems,

employment, and social support. employment, and social support.

Performance Measurement and Outcomes Performance Measurement and Outcomes Monitoring in the US Monitoring in the US A Two Day MeetingA Two Day Meeting

Sacramento, CaliforniaSacramento, CaliforniaNovember 15, 16 2007November 15, 16 2007

Develop a set of performance and outcome Develop a set of performance and outcome measures using CalOMS data to improve measures using CalOMS data to improve accountability and treatment quality and meet accountability and treatment quality and meet national standards.national standards.

Review and synthesize literature on Review and synthesize literature on performance measurement and outcomes performance measurement and outcomes monitoring.monitoring.

Bottom line: Develop some models to define Bottom line: Develop some models to define and measure program performance.and measure program performance.

Objective 1: Major ActivitiesObjective 1: Major Activities

What do we want to accomplish? What do we want to accomplish?

Some specific issues for discussionSome specific issues for discussion Which CalOMS data elements, alone and in Which CalOMS data elements, alone and in

combination, provide meaningful performance and combination, provide meaningful performance and outcomes measures?outcomes measures?

Are there data elements that need to be added to Are there data elements that need to be added to CalOMS data set to optimally measure performance CalOMS data set to optimally measure performance and outcomes?and outcomes?

Is it possible to operationally define “treatment Is it possible to operationally define “treatment success” and how can this issue be communicated to success” and how can this issue be communicated to the public and policy makers?the public and policy makers?

What do we want to accomplish?What do we want to accomplish?

Determine how to adequately measure access Determine how to adequately measure access to care?to care?

Determine how feedback from clients can be Determine how feedback from clients can be used to improve program performance and used to improve program performance and treatment outcomes (e.g., satisfaction surveys)?treatment outcomes (e.g., satisfaction surveys)?

What is the role of “Evidence-based Practices” in What is the role of “Evidence-based Practices” in improving performance and outcomes?improving performance and outcomes? How are these defined? How is their implementation How are these defined? How is their implementation

measured?measured?

Objective 2: ActivitiesObjective 2: Activities

• A 2-day meeting in Spring of 2008 with experts A 2-day meeting in Spring of 2008 with experts and stakeholders to understand promising and stakeholders to understand promising performance improvement strategies.performance improvement strategies.

• Provide trainings for county administrators and Provide trainings for county administrators and treatment providers on how to use CalOMS data treatment providers on how to use CalOMS data for performance and outcome improvement.for performance and outcome improvement.

• Develop CalOMS updates to facilitate Develop CalOMS updates to facilitate communication between State and counties.communication between State and counties.

Summit: Using Summit: Using Performance and Performance and

Outcomes Measures to Outcomes Measures to Improve TreatmentImprove Treatment

Los Angeles SheratonLos Angeles Sheraton

March 20, 21, 2008March 20, 21, 2008

LACES Next StepsLACES Next Steps

Provide input to shape the way Provide input to shape the way performance is measured in LA Countyperformance is measured in LA County

Refine data collection proceduresRefine data collection procedures Advise on technical assistance support Advise on technical assistance support

activities neededactivities needed Provide input into models for improving Provide input into models for improving

program performance with data.program performance with data.

The EndThe End