Embed Size (px)
Citation preview
To Err is Human and Costly: Problem Residents and Predictors of Who Will
Become One!
Andrew A. Gonzalez, JD2010 MD/MPH CandidateUIC College of Medicine and School of Public Health
04/18/2023
2
Overview
Predictors of Future Bad BehaviorAcademic MisrepresentationProfessionalism in the Age of Online Social
NetworksRecommendations
04/18/2023Yao & Wright, 284 JAMA (2000) at 1099.
3
Definition of a Problem Resident
“a trainee who demonstrates a significant enough problem that requires intervention by someone of authority, usually the program
director or chief resident.”
-American Board of Internal Medicine
04/18/2023
4
Risk management = gatekeeping
Primary Prevention – uncontrolableGatekeeping – most controllableRemediation – a discussion for another
time
04/18/2023
5
Overlapping Gatekeeper Checkpoints
College Medical schoolDuring medical school (4yrs)Medical school residency (application
phase)During residency (more risky)Residency fellowship (application phase)
04/18/2023
6
Part I – Unprofessional Behavior
1. Importance2. Stakeholders Values3. Traditional Predictors4. California Personality Inventory (CPI)5. Bad Medical School Behavior
04/18/2023
7
Unprofessional Behavior
Low frequency 1% of IM diplomats Overall Rate (2003) is approx 0.3%
High impact event Impact on patient trust Hospital/university’s reputation Ft. Hood tragedy
Papadakis MA, 353(25) NEJM (2005) at 2680; Papadakis MA, 148 Ann. Internal Med. (2008) at 872.
04/18/2023
8
Stakeholders
Program DirectorsMedical BoardsPatientsResidents
04/18/2023Yao & Wright, 284 JAMA (2000) at 1100-01.
9
Program Directors Values
Attributes of the “problem resident”75% of US Internal Medicine Program
Directors Insufficient medical knowledge (~50%)Poor clinical judgment (~45%) Inefficient use of time (~45%)
04/18/2023
10
Medical Board ValuesTable 1. Basis for Disciplinary Actions Taken by State Licensing Boards. All 50 states; “‡ Violation directly related to substandard quality or safety of patient care.”
Papadakis et al, 148 Ann Intern Med (2008) at 871.
Negligence4%
04/18/2023
11
Behavior PrevalenceFailure to Meet CME 20%
Fraud/Billing/Tax Irregularities 9%Inappropriate/Excess Prescribing 8%
Examination/license Irregularities 7%Substance Abuse 6%
Sexual Misconduct 5%
Papadakis et al, 148 Ann Intern Med (2008) at 871.
Most Common Causes of Medical Board Disciplinary Action, 1990-2006
04/18/2023
12
Patient Values
≥1 Complaint≥1 Retained Complaint
Attitude / Communication 12% 9%Quality of Care 8% 5%Professionalism 2% 1%
3,424 MDs; Ontario & Quebec(1993-2005) Practice years 2-12; 22,585 combined practice years
Tamblyn et al 298(9) JAMA at 998
04/18/2023Arora VM et al, 300(10) JAMA (2008) at 1133.
13
BehaviorOverall Score
(1-5 Scale)Participation
by InternsReporting patient information as normal when uncertain of truth 1+ 10%
Falsification of patient records 1+ 13%Making fun of patients to
colleagues 1+ 17%
Resident ValuesSurvey of 110 Internal Medicine Interns at Univ. of Chicago and Northwestern
1 - unprofessional 5 - professional
04/18/2023
14
Traditional Pre-residency Screening Measures
MCAT ScoresPreclinical Grades (objective) USMLE Step 1-2 ScoresRotation evaluations (subjective)Medical School Record (MSPE aka “Dean’s
letter”)
04/18/2023
15
Grades & Test Scores vs. Subsequent ACTUAL Disciplinary Action
NBME 1 or USMLE Step 1 Scores No correlation – Papadakis I; +/- Papadakis II
Did not pass at least 1 course in medical school No Correlation – Papadakis I
MCAT score Papadakis I – lowest quartile NOT predictive Papadakis II – correlated
04/18/2023
16
California Psychological Inventory (CPI)
434-item (T/F) questionnaire 20 folk concept scales2 dbases – CPI and professionalism studies
7 w/record of unprofessional behavior during medical school
19 did not (controls) 100% males; UCSF graduates 1960-65
Hodgson et al., 82(10, Suppl.) Acad Med (2007) at S5.
04/18/2023
17
Unprofessional Medical Students & CPI Indices
Results: level of medical student professionalism correlated with 4 scales responsibility (r = 0.53) communality (r = 0.50) well-being (r = 0.46), rule-respecting (r = 0.65)
Hodgson et al., 82(10, Suppl.) Acad Med (2007) at S6.
04/18/2023
18
Hodgson et al., 82(10, Suppl.) Acad Med (2007) at S4-5.
Scale Attribute of Low Score
Responsibility Non-conforming, rebellious, cynical
Socialization Self-indulgent, undisciplined, lacks self-improvement
Self-control Strong feelings & emotions, problems with impulsivity
Communality Sees self as different than others, moody, lazy
Well-being tends to complain about being treated unfairly or inconsiderately, pessimistic, nervous
04/18/2023
19
Bad behavior in Medical School
Is there a correlation?Which behaviors have strongest
correlation?Any problems with existing research?
04/18/2023Papadakis MA, 79(3) Acad Med (2004) at 247.
20
Is there a correlation? (Papadakis I)
Negative comments re: professionalism Sens: 38% Spec: 81%
04/18/2023
21
Which Specific Behaviors?
Teherani et al; ~250 MDs; UCSF;1990-2000 Poor Reliability, Irresponsibility Lack of Self-Improvement Poor Initiative
Papadakis II (NEJM) ~700 MDs; 1970-2000 UCSF, Thomas Jefferson, UMich
04/18/2023
22
Colliver Critique
Papadakis I/II Prevalence of negative
comments ~40% of subsequently
disciplined MDs ~20% of control group
Conclusion: “twice as likely to be disciplined”
Colliver Suggests comparison of
PPV Sensitivity vs. 1 –
specificity NOT PREDICTIVE!!
Prevalence = 1% PPV = 2% 1 – NPV = 1%
Colliver, 19(3) Teach Learn Med (2007) at 213.
04/18/2023
23
NNT = The Bottom Line!
98% of students with unprofessional remarks would be dismissed even though they would never be subject to disciplinary action
Mathemagic NNT = 80
Colliver, 19(3) Teach Learn Med (2007) at 214.
04/18/2023Reed et al., 3.00(11) JAMA (2008) at 1326-27
24
Behaviors of Highly Professional Residents
~150, PGY-1, internal medicine residents at Mayo (Rochester)
Yardstick: Top 20% of Residents Instrument: 360 degree evaluations
04/18/2023Reed et al., 3.00(11) JAMA (2008) at 1328-29.
25
Reed et al: Independent Variables
In-training Exam Mini-Clinical Evaluation Exercise % evaluations completed by resident Conference attendance Warning or probationary status
04/18/2023
26
Associated with Higher Professionalism
Attribute OR, p < 0.05Higher Score on Clinical Exam 4.64
Higher Score on In-service Exam 1.07More peer/faculty Evaluations Completed 1.07
Reed et al., 3.00(11) JAMA (2008) at 1330-31.
Conference attendance NOT associated with higher professionalism scores
20% of State Medical Board Action for failure to meet CME Attendance
04/18/2023
Part IIAcademic Misrepresentation
271. DDx Misrepresentation2. Scope3. The Hopkins Argument – Look Harder
04/18/2023
DDx of Misrepresentation
Academic Misrepresentation aka “Ghost Publications” Traditional plagiarism
“non-authorship of an existing article” Simple Fabrication
“claimed authorship of a non-existent article”
Konstantakos et al.
28
04/18/2023Berger E. Ann of EM. 2009;54(6):16A-17A
29
r/o Aggravated Fabrication
30
Scope
Author (year)Specialty, n (rounded)
% Unverified (of published applicants)
% Unverified (of all applicants)
Gurudevan (1996) EM, 350 20% 7%
Blige (1998) Peds, 2000 20% 1.5%
Dale (1999) Ortho, 200 17% 5%
Cohen-Gadol (2003) Neurosurg, 100 6% 8%
Kuo (2005) Gen Surg, 500 33% 10%
Yang (2006) Rad Onc, 120 22% 9%
Nosnik (2010) Urology, 150 20% 10%
04/18/2023
31
Associated FactorsYang, Rad Onc Kuo, Gen Surg Nosnik, Urology
USMLE > 235 (OR = 8) Advanced Age, IMG No correlation with demographic variables
04/18/2023Hebert RS, 138 Ann Intern Med (2003) at 391.
32
But is it overstated?
Hopkins study of Internal Medicine interviewees; ~500 interviewees ~200 interviewees with ~600 articles total 99% verified
04/18/2023Hebert RS, 138 Ann Intern Med (2003) at 391.
33
Hopkins Argument – A Problem of Low Sensitivity
Search 1: MEDLINE; last name and first initial +/- PMID
Search 3: Ulrich’s International Periodicals Directory; medical librarian utilizing
SciFinder Scholar, INSPEC, Article First, Genome Database, National Technical Information Service , PsychINFO, and National Library of Medicine’s LocatorPlus
Google search of academic websites (coauthors, professional society, and journal Web sites)
04/18/2023
34
Is Hopkins Study Generalizable?
Across institutions? Across specialties?Against manifest weight of the
evidence
04/18/2023
Part IVRecommendations for Best Practices
35
04/18/2023
36
Issue 1 – Gatekeeping Problem Residents
Recommendation 1 Greater weight to visiting student rotations
Recommendation 2 More research!!!
Recommendation 3 Check the facebook, myspace, friendster
pages of your “possible to match” list
04/18/2023
37
04/18/2023
38
60% of US medical schools had “incidents” Confidentiality 13% Profanity 52% Frankly discriminatory language 48% Intoxication 39% Sexually suggestive material 38%
04/18/2023
39
Not HIPAA violation but Unprofessional?
04/18/2023
40
I’m checking Facebook but …Unprofessional?
On student’s facebook?
Downloaded from residency website?
Residency unprofessional?
04/18/2023
41
Issue 2 – Academic Misrepresentation
Step 1 – Verification of Submission Burden shift to applicants
Include papers/abstracts in application Submission letter for manuscripts under consideration Acceptance letters for “in press” citations
Streamline / Cost Efficiency ONLY Verify PMIDs of interviewee pool
Backup Require applicants to bring publications to interview
04/18/2023
42
Academic Misrepresentation con’t
Step 2 – Punitive Measures Necessary?
4/6 neurosurgery applicants who inaccurately reported manuscripts subsequently matched to a residency
Make match violation Contact applicant first Offer opportunity to rehabilitate
Contact applicant’s medical school Clearing house
Cohen-Gadol AA, 60 Surg Neurol (2003) at 280.
