Embed Size (px)
Citation preview
1
On Track Advanced Topics
Getting the Most Out of Your Outcomes Data
Eric Hamilton, M.S.Vice President of Clinical Informatics, ValueOptions
Jeb Brown, Ph.D.Director, Center for Clinical Informatics
2
Overview
1. What can outcomes data add to my practice?
2. A closer look at the Client Feedback Form
3. What is MyToolkit and what do all those numbers mean?
4. Putting data into practice
5. Discussion
3
What can outcomes data add to my practice? • Do you always know which patients
struggling in therapy?• Do some of your clients stop therapy before
you can establish a good relationship?• Can you show clients evidence of their
progress compared to benchmarks?• Can you show prospective clients,
colleagues, or managed care companies evidence of your clinical outcomes?
4
How good are clinicians at identifying clients at risk for treatment failure?
• Lambert and colleagues compared clinician predictions to predictive algorithms using early client self-assessment data
• Only 3 out of 550 cases predicted to have poor outcome• Of the 40 that had a poor outcome, only 1 had been predicted
by a clinician• Algorithms based on early assessments identified 85% of
poor outcome cases – but also identified some that did well
0
100
200
300
400
500
600
Positive No Change Deteriorated
TherapistPredictedOutcome
ActualtreatmentOutcome
Source: Hannan, et al (2005), JCLP
Number of cases
5
What do the predictive algorithms mean?
• Statistical model that compares early client progress to a “typical profile,” adjusting for case characteristics such as intake severity
• Poor early progress does not “predict” a likelihood of failure – rather it is an indicator of heightened drop-out risk
• “Off-track” clients who remain engaged generally get good outcomes – it just might take them longer
0
5
10
15
20
25
30
35
40
45
Intake 3 6 9 12 15 18+
Weeks
WA
-Adu
lt S
core
Actual score
85% percentile
Predicted Score
15th percentile
Clinical cuttoff
Global DistressScore
6
Client Feedback Form (CFF) - Adult
Item Groups • Global Distress: 1-10
– Core scale– Sensitive to change over time– Depression, anxiety, social functioning
• Risk of self-harm: 5– Risk indicator
• Substance use: 11-13– Risk indicator
• Work productivity: 14-15 – Indicator of functioning
• Therapeutic alliance: 16-18– Support tool for therapist
• Background items: 19-20– For case-mix analysis and identification of co-
morbidities
7
Tips for Using the CFF
• Print multiple copies before you need them• Have available for the member to complete
before the session begins• Show you value the data – when giving the
CFF and when it is returned• Review scored results online before the
next session • Measure early and measure often!
8
CFF Results: What do all those numbers mean?
Access On Track results
9
Outcomes based on the most recent CFF
High scores in red
10
You decide which variables
to display
11
Results: Key things to remember• “Effect Size” is a way of looking at the combined
results of cases with 2+ data points, compared to normative data
• Links at the top of each column take you to more information
• All scores are presented as means– Range is from zero to four
• Higher severity results appear in Red• GDS = Global distress score (Questions 1-10)• Change Score = Difference in the mean GDS score
from first assessment to the most recent• Benchmark Score = How much better or worse the
change score is compared to norms
12
Interpreting Adult Global Distress Scores• Scores of 0 to 1.5
– Typical of community (non-treatment) samples
• Scores of 1.6 to 2.5– Moderate distress– Only 20% of community samples would be expected to score
in this range; about half of individuals seeking treatment score in this range
• Scores of 2.6 to 4.0– Severe distress– 25% of individuals seeking MH services score in this range
13
The Trajectory of Change Graph
Case is “off-track” compared to the benchmark projection
14
Putting Data Into Practice
• Most cases don’t require a lot of review of the data, look for the high risk results
• Talk with your clients about their results, especially when they share something new or surprising
• An “off-track” cases does not mean a big change is needed – some encouragement to stay engaged may be all that is needed
• Be sensitive to small changes in the alliance questions
• If your effect size is low at first, keep measuring and reviewing the data
• Alliance items are a great place to look for clues for improving effectiveness
15
Questions and Discussion Resources for Questions
– Frequently Asked Questions
On the web site, near bottom of the ValueOptions page– Technical/Data/Web:
Email to [email protected]
– General comments or questions: Email to [email protected] or Call On Track Customer Service 866-476-9796
