Embed Size (px)
Citation preview
Total Clinical Outcomes Management:Theory and Method
Richard A. Epstein, Ph.D., M.P.H.Michael J. Cull, Ph.D., M.S.N.
Department of Psychiatry
2
Road Map
• The importance of shared vision• Managing information asymmetry• The role of standardized assessment• Implementation examples
3
“To organize the world’s information and make it universally accessible and useful.”
4
“Our investment in data-mining is part of our drive to deliver what our customers want: the item, at the right store, at the right time, at the right price.”
“To organize the world’s information and make it universally accessible and useful.”
5
“Our investment in data-mining is part of our drive to deliver what our customers want: the item, at the right store, at the right time, at the right price.”
“The behavioral health system, at all levels, should always make decisions based on the needs and well-being of the people served.”
“To organize the world’s information and make it universally accessible and useful.”
6
Step 1: Sit downStep 2: Hold on for 8 seconds
7
Landing a passenger jet on the Hudson
8
A Successful Landing• Capt. Sullenberger credited teamwork,
preparation and strict adherence to protocols for the successful landing.
9
Success in Health Care
10
International Comparison, 2004
2.8 2.8 3.1 3.2 3.3
4.4
5.3
6.8
Source: Commonwealth Fund National Scorecard on U.S. Health System Performance, 2008
Infant Mortality Rate
Deaths per 1,000 births
11
Ambulatory Care Visits for Adverse Drug Effects
20
22
18
21
17
9
28
1716
22
0
10
20
30
Total
Mal
e
Femal
e
White
Black
Other
Northea
st
Mid
west
SouthW
est
Visits per 1,000 population per year
Source: Commonwealth Fund National Scorecard on U.S. Health System Performance, 2008
11.312.9
15.3
11.3
17.4
19.8
1999 2000 2001* 2002 2003 2004
Total
Physician Office
Hospital Emergency Department Visits
Hospital Outpatient Department Visits
By Gender, Race, and Region, 2004 Annual Averages, by Care Setting
12
13
Health Care Failures
1. Ignorance – science has given us incomplete information
2. Ineptitude – science has provided a solution, we fail to apply it correctly
Atul Gwande, 2009
14
Ignorance
• Evidence-based practice
• Practice-based evidence
15
Ineptitude
• Organizing information
• Standardize practice
• Support rational decision making
16
The cost of ineptitude• Poor Quality• Inefficiency• Poor resource allocation• Excess costs
17
Combating ineptitude
• Bias• Complexity• Goal incongruence
18
Bias• Stereotyping• Moral hazard• Treatment Framing• Probability• Confirmation bias
19
Complexity• Co-morbidity• Treatment options• Governance and financing
20
Goal Incongruence• Multiple perspectives• Information asymmetry• Motive/”agenda”
21
The Case for Decision Support• Eliminates bias• Manages complexity• Supports creation of a shared vision
22
Shared Vision: Child/Family• Child and family centric• Strengths based• Culturally and contextually sensitive
23
“12 Angry Men” Video Clip
24
“12 Angry Men”• Multiple perspectives?• Group think? Strong-arming?• Disagreement?• Bias?• How will they discuss their disagreement?• Can they reach a shared vision?
25
Managing Complexity: Aviation• Checklists developed in 1935 to manage complexities
of the Boeing model 299 long-range bomber
• In 1968 NTSB recommended a re-evaluation of checklists after the crash of an American World Airways B-707
• Pre-flight checklists are now “standard of care”
26
Reducing Bias & Complexity: Medicine• Critical care
– Pronovost et al. (2003) – Use of “daily goals” checklists associated with improved communication and reduced length of ICU stay.
• Catheter related infections– Pronovost et al. (2006) – Use of checklists and team
empowerment associated with 66% reduction in catheter-related infections.
• Surgery– Makary et al. (2007) – Use of operating room briefing tool
improved team communication and reduced risk of “wrong-side” surgeries
– Haynes et al. (2009) – Use of checklists associated with reduced risk of death and other surgery outcomes
27
Example: Surgical Safety• Surgery is integral to global health
– 234 million operations each year• Many complications are preventable• Evaluated the effectiveness of improving
team communication on outcomes
28
Hospitals• Toronto, Canada• New Delhi, India• Amman, Jordan• Auckland, New Zealand• Manila, Philippines• Ifakara, Tanzania• Long, England• Seattle, Washington
29
Intervention• Measured outcomes before and after
intervention• Intervention was a 19-point checklist• Checklist had three parts:
– Sign in– Time out– Sign out
30
Sign In• Before anesthesia, team verbally confirms:
– Verification of patient identity, surgical site, procedure, and consent
– Surgical site is appropriately marked– Pulse oximeter is on the patient and functioning– All team members are aware of patient allergies– Patient risk of airway obstruction and aspiration
has been evaluated and appropriate equipment is available
– Blood and fluids are available
31
Time out• Before incision, team verbally confirms:
– All team members are introduced by name and role
– Patient identity, surgical site, and procedure– Review of anticipated critical events– Administration of prophylactic antibiotics
before incision or that they are not indicated
– Essential imaging results are displayed
32
Sign Out• Before patient leaves the OR:
– Nurse reviews aloud with the team:• Name of the procedure as recorded• That needle, sponge, and instrument counts are
complete• That specimen is correctly labeled• Whether there are any equipment issues
– Team reviews aloud key concerns for recovery and patient care
33
Complications Before After p value
Surgical site infection 6.2 3.4 < .001
Death 1.5 0.8 < .01
Process Measures
Prophylactic antibiotics given 56.1 82.6 < .001
Orally confirm patient’s identity 54.4 92.3 < .001
Results
34
Summary• Introducing surgical checklist:
– Reduced surgical site infections and death by ½
– Increased compliance with safety process measures
• What did the checklist accomplish?– Ensured similar information for the team – Improved team communication– Supported shared vision
35
Break
36
Road Map• The importance of shared vision• Managing information asymmetry• The role of standardized assessment• Implementation examples
37
Data-driven decision making
38
Service planning exercise• Case vignette• System-level service plan• Importance of measurement and
communication
39
“Bender” Video Clip 1
40
Identify 3 service plan needs
41
“Bender” Video Clip 2
42
Okay, now what?
43
Measurement as Communication• Purpose of all measurement is to communicate
– Time (e.g., days), Temperature (e.g., Fo, C
o), Value
(e.g., dollars)
• Physical sciences– Concern phenomena accessible via instrumentation– Instrumentation improves; measurement stays the
same
• Human and social sciences– Sometimes concern phenomena similarly measured– Many times involves phenomena not readily
accessible
44
Social Science Measurement• Logical empiricist perspective
– Measurement is application of procedure to show verifiable truth from observation
– Implication is that some real phenomenon exists and that this “truth” can be revealed
• Classical test and item response theories are based on a similar philosophies
45
Communication Theory• Traditional perspective also empiricist
– Knowledge transfers from one mind to another
• Main critique of the “transfer” model– Overly linear– A B with no room for impact of B on A and
information coming from A• Constitutive / constructivist model
– Communication as shared meaning making
46
Shared Meaning Making
47
Communication and Measurement• From a communication / constructivist
perspective, measurement is:– Less about “revealing truth”– More about “creating shared meaning”
• Classical test theory – Multiple items; internal consistency
reliability
48
Internal Consistency Reliability• Cronbach’s alpha is the statistic used to define
internal consistency reliability• The equation is as follows:
49
Classical Test and Communication Theories
• Complex systems require clear and concise communication about multiple constructs
• Classical test theory: reliability = redundancy• Inter-rater reliability is therefore both the
– Relevant measure of reliability– Concept most related to shared meaning– Directly facilitates decision support
50
• Reduce bias• Integrate multiple perspectives• Attend to all relevant information• Communicate and use information at all
system levels
Measurement Helps
51
TCOM Grid of ActivitiesFamily & Youth
Program System
Decision Support
Service Planning
Eligibility Resource Management
Quality Improvement
Case Management & Supervision
AccreditationCQI
TransformationSystem Evolution
Outcome Monitoring
Service Transitions & Celebrations
Evaluation Performance Contracting
52
Computer
User
CANS Assessment
Interactive Training
Environment
Public house
Agency
Group
Rater Groups
System
AnalyticsServer
Trained Rater
Child and Family
Using assessment data to support decision-making
rater data creates training opportunities
53
Tennessee DCS Implementation• Case managers are annually trained to reliability• Children are assessed at several points during a custody
episode– Entry and discharge– Placement transitions and regular intervals
• Support is provided by network of Centers of Excellence (COEs)– Training, consultation, independent 3rd party review– Data management and analytics
• Data used to:– Monitor child outcomes; Support placement
decisions– Inform treatment plans; Plan array of services
54
Training: January 2008 – April 2009Training % Completed
Initial 100
Recertification 95
PBC Agencies 85
• Initial and recertification trainings are ongoing• Increasingly use CANS for decision support• Initiated provider trainings and “Guide Program”
55
Assessment Volume: July - April 2009Month No. of completed assessments
July 1185
August 1310
September 1219
October 1349
November 1127
December 1230
January 1642
February 1560
March 1607
Total 12,229
56
Algorithm Services Intensity Recommendation: As of April 2009 (Initial assessments = 5119)
Services intensity recommendation
Frequency Percent
1 2391 46.7
2 1632 31.9
3 840 16.4
4 256 5.0
57
Secondary data utilization• Review of high-risk children• Placement quality team review• Regional utilization review support• Well being team meetings• Unified Assessment process• Assessment of Services Quality (ASQ)
58
Special projects• Restructuring the continuum model project (DCS)• Transition age youth (GOCCC)• Ad-hoc region level reports (DCS)• Surveillance of foster care youth with scores of ‘2’ or
‘3’ on sexual aggression, sexually reactive, and danger to others items (TAC)
• TFACTS placement matching project (DCS)• Substance abuse (T-ACT, GOCCC)• Sexually problematic behaviors (SPB Task Force,
GOCCC)• OJJDP juvenile court screening pilot
59
Characteristics of DCS children, age 5 years or more,
assessed between 2/18/08 – 2/31/09 Demographics # %Age in years
5 – 12 2663 29.8 13 – 16 3790 42.5 17 – 19 2473 27.7Race / ethnicity
African-American 2737 30.7 Non-Hispanic White 5699 63.8 Other 241 2.7Gender
Female 3723 41.7 Male 5203 58.3Adjudication Status
Dependent / Neglect / Unruly 5168 57.9 Delinquent 2074 23.23 regions with the most assessments
East Tennessee 1433 16.1 Shelby 1010 11.3 Mid Cumberland 994 11.1*Totals may not add up to 100% due to missing data
60
Three most prevalent Risk Behaviors by ageActionable Risk Behaviors 5 – 12 13 – 16 17 – 19
Suicide 1.3 4.4 2.5
Self Mutilation 1.5 3.6 1.7
Other Self Harm 2.0 7.9 5.9
Danger to Others 5.1 14.0 8.5
Runaway 0.8 13.4 15.7
Firesetting 0.6 0.9 0.3
Social Behavior 14.8 37.8 27.8
Sexually Reactive Behavior 7.0 5.7 3.5
Sexual Aggression 1.6 4.1 2.3
Delinquency 2.2 28.4 24.2
Substance Use 0.2 14.4 18.1
61
Age at which each risk behavior is most prevalent
Actionable Risk Behaviors 5 – 12 13 – 16 17 – 19
Suicide 1.3 4.4 2.5
Self Mutilation 1.5 3.6 1.7
Other Self Harm 2.0 7.9 5.9
Danger to Others 5.1 14.0 8.5
Runaway 0.8 13.4 15.7
Firesetting 0.6 0.9 0.3
Social Behavior 14.8 37.8 27.8
Sexually Reactive Behavior 7.0 5.7 3.5
Sexual Aggression 1.6 4.1 2.3
Delinquency 2.2 28.4 24.2
Substance Use 0.2 14.4 18.1
62
Three most prevalent Behav. / Emo. Needs by age
Actionable Behavioral and Emotional Needs
5 – 12 13 – 16 17 – 19
Psychosis 1.1 2.1 1.7
Impulsivity 21.3 26.5 18.6
Depression 4.8 17.5 13.8
Anxiety 6.7 11.1 7.6
Oppositional 6.2 21.3 15.7
Adjustment to Trauma 12.5 14.3 7.9
Attachment 4.4 4.8 2.8
Anger Control 12.4 32.6 22.2
Emotional Control 10.5 23.2 16.0
63
Age group in which each Behav. / Emo. Need is most prevalent
Actionable Behavioral and Emotional Needs
5 – 12 13 – 16 17 – 19
Psychosis 1.1 2.1 1.7
Impulsivity 21.3 26.5 18.6
Depression 4.8 17.5 13.8
Anxiety 6.7 11.1 7.6
Oppositional 6.2 21.3 15.7
Adjustment to Trauma 12.5 14.3 7.9
Attachment 4.4 4.8 2.8
Anger Control 12.4 32.6 22.2
Emotional Control 10.5 23.2 16.0
64
Difficulties with adjustment to trauma by SPB category
0
20
40
60
80
100
No SPB SR SA SR +SA
No difficulties with adjustment to trauma
Difficulties with adjustment to trauma
Perc
enta
ge
65
7
7.5
8
8.5
9
9.5
10
10.5
-1 -0.5 0 0.5 1 1.5 2
Years (vs Start Date)
Item
Ave
rage
(x
10)
TOT (ALL) YCM CMO TRH GRH PCR RES
Start
Hinge analysis of outcome trajectories prior to and after program initiation
66
Figure 5.2 Survival analysis of time to placement disruption for children/youth whose placement matches CANS recommendations (Match=0), those whose placed is at a lower intensity than recommended (match=1) and those
whose placement is more intensive than recommended (match=-1).
67
Modeling crisis decisions (Leon et al., 1999)
• A 5 item model predicted 78% of crisis decisions• Items - Suicide Risk, Danger to Others, Impulsivity, Emotional
Disturbance, Behavioral Disturbance• Replicated by He et al. (2004)
Predicted, # (%)
Observed, # (%) Hospitalize Deflect Total
Hospitalize 667(70)
285(30)
952(43)
Deflect 203(16)
1050(84)
1253(57)
Total 870(39)
1335(61)
2205(100)
68
Illinois Department of Children and Family Services
DCFS Wards
69
Mental Health Services
Illinois Department of Children and Family Services
70
Mental Health ServicesDCFS Wards
Illinois Department of Children and Family Services
71
Figure 5.1 Proportion of low-risk youth hospitalized by race over the duration of an decision support initiative
00.050.10.150.20.25
0.30.350.40.450.5
1998 1999 2000 2001 2002
72
Change in Total CSPI Score by Intervention and Hospitalization Risk Level (FY06)
51.2
34.134.231.0
24.4
17.5
47.4
35.2
26.4
22.1 24.218.0
0
10
20
30
40
50
60
SASS Assessment End of SASSEpisode
Mea
n C
SP
I S
core
HOSP (high riskgroup)
ICT (high risk group)
HOSP (medium riskgroup)
ICT (medium riskgroup)
HOSP (low riskgroup)
ICT (low risk group)
73
Summary
• TCOM can help systems:– Reduce bias– Integrate multiple perspectives/shared vision– Attend to all relevant needs and strengths– Communicate and use information at all system levels– Allocate resources and measure/improve outcomes
74
Selected references1. Gawande A. (2010). The Checklist Manifesto: How to Get Things Right. New York: Metropolitan Books.2. The Commonwealth Fund National Scorecard on U.S. Health System Performance. (2008).3. U.S. Government Accountability Office (http://www.gao.gov).4. Lyons JS. (2004). Redressing the Emperor: Improving Our Children’s Public Mental Health System. Westport, CT: Praeger.5. Lyons JS & Weiner DA. (2009). Behavioral Healthcare: Assessment, Service Planning, and Total Clinical Outcomes Management. Kingston, NJ: Civic Research
Institute.6. Lyons JS. (2010). Communimetrics: A Communication Theory of Measurement in Human Service Settings. New York: Springer.7. Lyons JS. (2008). Total Clinical Outcomes Management. Presentation at the Annual Child and Adolescent Needs and Strengths (CANS) Conference. Nashville, TN.8. Lyons JS. (2009). Total Clinical Outcomes Management in the service of children with behavioral and emotional needs: an update. Presentation to the Tennessee
Council on Children’s Mental Health. Nashville, TN.9. Shiv B & Fedorikhin A. (1999). Heart and mind in conflict: the interplay of affect and cognition in consumer decision making. Journal of Consumer Research, 26(3),
278-292.10. Pronovost P, Berenholtz S, Dorman T, et al. (2003). Improving communication in the ICU using daily goals. Journal of Critical Care, 18, 71-75.11. Rawal PH, Anderson TR, Romansky JR & Lyons JS. (2008). Using decision support to address racial disparities in mental health service utilization. Residential
Treatment for Children and Youth, 25(1), 73-84.12. Pronovost P, Needham D, Berenholtz S., et al. (2006). An intervention to decrease catheter-related bloodstream infections in the ICU. New England Journal of
Medicine, 355, 2725-2732.13. Makary MA, Mukherjee A, Sexton B, et al. (2007). Operating room briefings and wrong-site surgery. Journal of the American College of Surgeons, 204(2), 236-243.14. Haynes AB, Weiser TG, Berry WR et al. (2009). A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of
Medicine, 360(5), 491-499.15. Keysar B, Barr DJ, Balin JA & Brauner JS. (2000). Taking perspective in conversation: the role of mutual knowledge in comprehension. Psychological Science, 11(1),
32-38.16. Video clips are from: “12 Angry Men” and “The Breakfast Club”17. Epstein RA. (2008). Needs and strengths of children entering state custody. Presentation at the Annual Child and Adolescent Needs and Strengths (CANS)
Conference. Nashville, TN.18. Epstein RA & Cull M. (2009). Tennessee Department of Children’s Services CANS Implementation. Presentation to the Tennessee Council on Children’s Mental
Health. Nashville, TN.19. Leon SC, Lyons JS, Uziel-Miller ND, Tracy P. (1999). Psychiatric hospital utilization of children and adolescents in state custody. Journal of the American Academy of
Child and Adolescent Psychiatry, 38, 305-310.
