Embed Size (px)
Citation preview
Kara Taguchi, Psy.D.
Mental Health Clinical Program Head
MHSA Implementation and Outcomes Division
RAND/SRI Data Systems Workgroup
*Los Angeles County PEI Data Collection
*Where It All Started
*Approach to Outcomes Evaluation
*Built upon CiMH support already in place
*General measure and focus of treatment specific measure administered at the beginning and at the end of treatment
*Scale scores or total scores entered in CiMH data shell or PEI Outcome Measures Application (OMA)
*Measures selection based on:
*CiMH Palette of Measures
*Developer requirements
*Cost
*Length
*Language translations
*In general, one specific measure for each foci of treatment
*Review with providers of service at PEI Clinical Implementation meetings
*Things to Consider
*The depth and breadth of the information you want to collect
*Using tools with a copyright vs. public domain instruments
*Valid, reliable measures vs. creating your own tools
*How many users of your system
*Planning for potential growth
*Workflow and evolving processes
*PEI OMA
*Features of PEI OMA
*Secure Log on based on associations in our billing system
*Clients must be selected from our billing system
*Once Focus of Treatment and Practice are selected, questionnaires are shown based on age
*Front end validation
*Entering data for multiple practices in same focus
*Ordering of the assessments (Pre, Update, Post)
*Required fields
*Practices and Outcomes: CiMH and
LACDMHCiMH LACDMH
Triple P All other PEI Practices
Trauma Focused CBT
MAP
*PEI Outcome Measures
*Outcomes Through a Wide Angle Lens
*Clinical and administrative utility of collecting outcomes
*Levels of evaluation
*Quality improvement
*Reports
*Creating provider-to-provider learning
*Volume of Data Collected in PEI OMA
*Almost 26,000 treatment cycles for more than 23,000 clients
*Data for 31 different EBPs
*Data from 111 Legal Entities and 286 different provider site locations.
*381 users
*Approximately 103,000 questionnaires saved
*Data Collected
*Demographic client data and provider data
*Beginning of Treatment information
*Date of First Session
*Starting Diagnosis
*Clinician
*Questionnaires (Pre, Updates, Post)
*Scaled scores and/or total scores
*Questionnaire administration date
*Unable to collect and reason
*End of Treatment Information
*Date of Last session
*Completed EBP
*Measure of success
*Disposition of Client
*Number of Sessions
*Ending Diagnosis
*Types of Analyses/ Reports
*Data collected can be used in a variety of ways
*Pre/Post percent improvement in scores
*Clients above and below clinical cut off scores
*Duration of treatment
*Percent completed vs. Drop outs
*Outcome differences based on gender, ethnicity, primary language, age, focus, or EBP
*Unable to collect reasons
*Reliable Change
*Using Outcome Data to Improve Quality and
Enhance Practice
*CiMH Community Development Team (CDT) Model for CiMH supported practices
*PEI Technical Assistance Site Visits
*Practice Learning Networks
*Service Area/Regional Outcome Data Workgroups
*PEI Technical Assistance Site Visits
*Varying degrees of data entry challenges or communication around outcomes
*The more outcomes are incorporated into normal routines the greater the compliance
*Helping clinicians understand how data is valuable in interventions and as an assessment tool is important
*Discussing outcomes in supervision sets a tone
*Great variability in outcomes protocol and expectations from site to site
*Practice Networks
*Child Parent Psychotherapy (CPP)
*Incredible Years (IY)
*Group CBT for Major Depression (Group CBT)
*Seeking Safety (SS)
*Aggression Replacement Training (ART)
*Crisis Oriented Recovery Services (CORS)
*Families OverComing Under Stress (FOCUS)
*Practice Networks
*Recent Topics
*Strategies for increasing data collection
*Feedback loop of data to clinicians
*Incorporating Outcomes into Supervision
*Sharing Outcomes with Families
*Availability of Training
*General Technical Assistance Questions
*Understanding the aggregate reports compared to site specific data.
*Using Reports and Data: Service Area/Regional Outcome Data
Workgroups
*Funded through PEI Training, Technical Assistance and Capacity Building funds through a contract with CalMHSA, RAND Corporation facilitates workgroups comprised of providers and DMH administrative staff interested in reviewing Service Area outcome data for purposes of improving service quality and for developing strategies to make decisions based on the outcome data. *Goals of the workgroups include:*Provide a forum for Service Area providers to learn from each
other in terms of practices that improve data quality and improve the use of data to make systems as well as clinical decisions within a Service Area.*Provide a forum for DMH administrative Service Area staff to
develop their skills in utilizing reports to inform decision-making at the local level.*Strengthen Service Area capacity to formulate and conduct
evaluations and to use appropriate data and analytic techniques to inform planning and resource allocations.*To develop strategies to review outcome data across programs
and to use the analysis to help inform Service Area level decision-making.
*Improving Data Quality and User
Support
Direct Support:
*Outcome Measure Application (OMA) LabsOffered bi-weekly: all OMA staff are available in a one-stop lab to address user issues and train on reports. Providers drive content and focus.
*OMA Users Group (In person and via Webinar) Meets Bi-Monthly: all Outcomes staff as well as CIOB and others meet and update interested users. The Users Group is also broadcast via WebEx for those who want to participate from their sites.
*Improving Data Quality and User Support
Publications and Broadcasts:
*Egroups (OMAChat, PEI Alert, FCCS Alert) Specialized email announcement/ discussion groups for important and timely announcements, including system alerts.
*OMA NewsletterQuarterly reports on the state of the OMA offering tips and features about new aspects of the Outcomes effort.
*“Canned” Users Groups and TrainingsPre-recorded Users Group sessions and special presentations are available through WebEx, Adobe Captivate, or as Windows Movie files.
*Improving Data Quality and User Support
References and Aids:*DMHOMA Wiki
Project Website, covering all aspects of Outcomes (FSP, FCCS, SFC, Wraparound, PEI, and MHIP). As a “wiki” it is editable by any staff member.
http://dmhoma.pbworks.com
*Quick GuidesReference handouts, featuring tips for
completing PEI questionnaires.
*Worksheets, Spreadsheets, and other aidesFillable PDF and Excel files that help users
calculate needed values and record dates and scores correctly.
*Improving Data Quality and User Support
Accessing the Data and Reports (Next Steps):
*Internet ReportsDMH-developed application to deliver specialized outcome data reports for providers on demand.
*Electronic File Transfer: Legal Entity ExtractsA secure transfer of a Legal Entity’s outcome data allowing providers to analyze and manipulate their own data.
*Contact Information
Kara Taguchi, Psy.D.
Los Angeles County Department of Mental Health
MHSA Implementation & Outcomes Division
213) 251-6818
http ://dmhoma.pbworks.com
