Networks’ Experiences
Connecticut Center for Primary Care: ProHealth Physicians Practice Based Research Network (PI: John Lynch, MPH)
The Oregon Rural Practice-based Research Network (ORPRN) (PI: LJ Fagnan, MD)
CenTexNet (PI: Samuel N. Forjuoh, M.D., M.P.H., Dr.P.H)
LA Net (Lyndee Knox, PHD)
Connecticut
The Connecticut Center for Primary Care (CCPC) is an independent 501c3 non-profit research and education foundation.
On incorporation in 2002, CCPC assumed management of the ProHealth Physicians Practice Based Research Network, a PBRN affiliated with the AAFP Federation of Practice Based Research Networks.
PBRN Setting: ProHealth Physicians CT’s largest primary care group practice 75 practice sites throughout Hartford,
Middlesex, Litchfield, Tolland, and New Haven Counties (9 practice sites participated in MEADERS)
221 primary care providers 166 MD/DO, 55 APRN/PA (72 Family Practice,
73 Pediatric, 75 Internal Medicine, and 3 specialists)
Patient population 349,000 (approx 10% of CT) Patient Contacts: over 750,000 encounters/yr
CCPC MEADERS 9 Study Sites1) Pediatric – 9 clinicians2) Pediatric – 6 clinicians3) Family Practice – 7 clinicians4) Family Practice – 2 clinicians5) Internal Medicine – 4 clinicians6) Internal Medicine – 3 clinicians7) Internal Medicine – 3 clinicians8) Internal Medicine – 2 clinician9) Internal Medicine – 1 clinician
CCPC Results continued
Responses by type of event 61.6% (n=77) Adverse drug events 20.0% (n=25) Med Error – provider 8.8% (n=11) Med error patient 4.0% (n=5) Pharmacy 5.6% (n=7)other (MCO, manufacturer,
multiple, …)
CCPC Results continued
When happened: 9.6% Today 44.8% Past week 26.4% Past month 18.4% Over month ago 0.8% Don’t know
Discussion: Experience
Recruiting – easy 20 practices for 10 spots
Extended wait – OMB Lost one site – left ProHealth Sites didn’t remember they had signed
up System – Surprisingly easy/straightforward Retention – consistency by week
Discussion: Lessons
Pilot test quirks Participants say they completed many
more forms Form utility
Comfort reporting adverse drug events Patient/pharmacist/MCO failed to…
Blame culture – CQI opportunityNeed for better education/support
Legal concerns Need for Patient Safety Organization
The Oregon Rural Practice-based Research
Network (ORPRN)
Participating ORPRN Clinics
Scappoose
The Dalles
John Day
Baker City
Enterprise
38.4%46.5%
15.1%19.1%
10.0%
70.9%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Primary CareClinicians
Medical Assistantsor Nurse
Other
Job Category
Pe
rce
nta
ge
ORPRN
Nationally
Participant Job Categories
Reporting Facilitators
Adequate Training Reminders & Feedback
Weekly “Chalk Talk” reports Access to online error report page Occasional “fun” reminders
MEADERS Desktop Icon User-friendly Program Active engagement of lead clinician & staffDid you ask about patient medications?
Qualitative Findings in ORPRN1) Rural Family Practice Clinicians and
office staff will report to MEADERS as evidenced during a 10 week reporting trial
“It was painless.” “It went well. All participated and
seemed to enjoy it. It did not take an excessive amount of time.”
“People in the practice picked up on the type of events to report quite quickly and required little direction. Melinda helped a lot too.”
Qualitative Findings in ORPRN2) Barriers to reporting included:
making the time to report, breadth of reporting options, and confusion regarding what to report.
“It was more difficult for the staff it seemed because it was one more thing that they had to do.”
“I am assuming that some errors did not get reported because the criteria were too broad and clinicians didn’t think that they were important.”
Qualitative Findings in ORPRN3) Some practices used MEADERS data
for internal Quality Improvement “Overall, I think that the MEADERS reporting
system was helpful to staff and providers to remind us that errors occur and we should be mindful of them and work to prevent them in the future…my MA and I worked especially hard with one patient who kept taking the wrong dose of her Coumadin… Her care improved because of MEADERS.”
“The biggest thing it changed was making us more attentive to documentation and keeping med lists up to date ‑ especially between institutions like LTC and hospitals and hospices.”
Qualitative Findings in ORPRN4) Some clinics report a desire to
continue using MEADERS while others do not.
“It was easy to use, we would like to continue using it in our practice.”
“I doubt the practice would be excited about reporting events on a regular basis…This will be a significant obstacle to over come if this or a similar system becomes required in clinical practice.”
ORPRN MEADERS Conclusion
“The overall thing is that the direct access made the process much easier - most of our staff had never made any reports regarding adverse drug reactions so in my eyes - if the goal is to improve the process by which reports are made - this is a huge improvement.”Family Physician, Clinic 2
CenTexNet Practices
28 Scott & White Regional Clinics in 22 Locations + Brazos Family Medicine Residency Clinic, Bryan + Family Practice Residency Clinic, Waco
GatesvilleKilleen
WaenTempleTaylor
Follow-up Phone Call about Status of Survey Response
“Clinic has been really busy and just hasn’t had time to complete survey. Will try to get to it ASAP.”
“Clinic is short of nurses this week. Will try to get to it TODAY.”
“Contact has been out sick and just got back to work today. Will try to get to it.”
No responses from two clinics
Facilitators & Barriers
Facilitators Quality & Safety
Officers Clinic staff & MD
champions Adequacy of training Part of network
mission Staff familiarity with
IT
Barriers Time constraints Flu season Staff turnover Workflow
conflict
LA Net
www.lanetpbrn.net
Reducing Health Care Disparities in Southern California
Through Participatory Research Partnerships With the Health Care Community
LA Net - 5 FQHCsClinica Msr. Oscar A. Romero Alvarado St.
Clinica Msr. Oscar A. Romero Marengo St.
QueensCare, Echo Park
Cleaver Family Wellness Center
East Valley Community Health Center West Covina
East Valley Community Health Center Pomona
LA Net- Sites
Site MDs NPs/PAs
Pharmacy
Clinica Msr. Oscar A. Romero Alvarado St.
5 4 1.8
Clinica Msr. Oscar A. Romero Marengo St.
4 1
Cleaver Family Wellness Center
1 2
East Valley Community Health Center West Covina
8 7 3
East Valley Community Health Center Pomona
6 4 2
0
2
4
6
8
10
12
Week of Study
Nu
mb
er
of
Su
bm
issio
ns
Cleaver
OR Alvarado Street
OR Marengo Street
EVCHC Pomona
EVCHC West Covina
QueensCare EP
Note: QueensCare dropped out week 5 and Cleaver joined at week 6
Weekly Data Submissions by Clinic
Reporting Facilitators
Agreed w/ concept reporting/tracking is important
Liked MEADERS system Easy to use Fast - 20 seconds to enter
Said project increased their awareness
Reporting Barriers: Not about time
Culture: Administrative significance not enough-if not clinically significant, why report?
Ergonomics/Work flow Interrupted paper-based work flow for
clinicians Dispensary uses computer but also interrupted
process Translation to QA processes not easy
Feedback function came on-line toward end When did, not used in QI Duplicates existing paper based ME systems
for County and State (logs) Likely will be a key motivator in future
Learning & recommendations (LA Net)
Need training & implementation supports for program that aid: Identifying administrative value of detecting
& reporting non-harm ADEs and MEs Translation of data into existing QI
processes On-demand reports that meet existing reporting
requirements Multiple options for entering data:
Electronic (Electronic ICON integrated into EHR - No site had EHR or eRxing capabilities at time of study but all are either considering or in process of transition at present)
Paper (Paper to electronic)
Total Submissions by Week
0
10
20
30
40
50
Week
# of
Sub
mis
sion
s
CT OPRN CenTex LA Net
CT 14 14 9 15 13 12 15 7 13 13
OPRN 14 12 13 9 11 4 13 7 13 7
CenTex 48 13 12 13 18 22 12 13 18 9
LA Net 0 7 21 20 11 5 9 12 9 7
1 2 3 4 5 6 7 8 9 10
Total Submissions by Network/Site
0
20
40
60
80
Site
# of
Sub
mis
sion
s
CT 44 31 19 14 6 5 4 2 0
OPRN 37 21 20 13 12
CenTex 83 49 40 4 2
LA Net 35 32 16 11 7 3
1 2 3 4 5 6 7 8 9
Post-reporting survey of the experience of the participantsA survey was sent to the 220 clinicians
and staff members who agreed to participate in the project
Purpose: to learn about participants experience with MEADERS
164 completed the survey, a 75% response rate
Practice role of the 164 respondents
Role in the practice Number (%)
MD,DO 51 (31%)
NP,PA 31 (19%)
RN, LPN,MA, Lab Tech 55 (33%)
Pharmacist 2 (1%)
Office Manager 16 (10%)
Front Office Staff 8 (5%)
Billing/Administrative Staff 1 (1%)
Demographics of 164 respondents
Gender Number (%)Male 40 (25%)
Female 120 (75%)Age
18-37 50 (33%)38-47 35 (23%)48-57 52 (34%)58-87 15 (10%)
Demographics of 164 respondents
Race Number (%)American Indian/Alaska Native 1 (1%)
African American 1 (1%)Asian 2 (1%)
Hispanic/Latino 11 (6%)Pacific Islander/Native Hawaiian 1 (1%)
White, not Hispanic 129 (79%)Multi-racial 3 (2%)No response 16 (9%)
Training for MEADERS
Percent Trained on Use of MEADERS Number (%)
Trained 156 (96%)Untrained 7 (4%)
Effectiveness of Training in Preparation
Effective 76 (48%)Somewhat effective 23 (15%)
Not effective 59 (37%)
Extent of understanding on what events to report
Understand 133 (82%)Somewhat Understand 26 (16%)
Do Not Understand 3 (2%)
Difficulty using MEADERS
Have you reported any events in the past 3 months using MEADERS?
Number (%)
Yes 93 (57%)
No 71 (43%)
Degree of difficulty accessing the electronic reporting form and submitting a report
No or little difficulty 101 (62%)
Moderate to a great deal of difficulty 3 (2%)
Unable to access system 1 (1%)
Did not make a report in MEADERS 58 (35%)
Experience using MEADERS-1
Experience AgreeNeither Agree or Disagree
Disagree
It is easy to use 90% 9% 1%It allows me to be candid when reporting errors
79% 18% 3%
It protected my anonymity when I was filing reports
70% 28% 2%
It encourages me to learn from my mistakes and the mistakes of others
64% 29% 7%
It is viewed positively by this practice 60% 38% 2%It has increased my own awareness of how errors affect patient care
61% 29% 10%
Experience using MEADERS-2
Experience Agree
Neither Agree or Disagree
Disagree
It has helped me to improve patient care at my practice
41% 44% 16%
It has increased the fear of repercussion in the practice
36% 47% 17%
It takes too much time to submit a report 26% 20% 54%
It has led to changes in how we practice medicine
25% 54% 21%
It made me feel like I was informing on my co-workers
10% 20% 70%
It has not worked in this practice 9% 26% 65%
Did you have any concerns about. . .
Concerns Number (%)
The time it took to complete a report 29 (33%)
Knowing what should be reported 26 (29%)
The privacy of your reports 12 (13%)
Knowing how to correctly make a report 10 (11%)
The layout or format of the reporting system 9 (10%)
Others in your practice finding out you had made a report 6 (7%)
The type of questions asked 5 (6%)
How your practice might react to your making a report 5 (6%)
Forwarding event data to MedWatch
Were there ADVERSE DRUG EVENTS or MEDICATION ERRORS that you reported in MEADERS that you did not forward to the FDA MedWatch database?
Number (%)
Yes 38 (40%)What were the reasons you elected not to forward an
adverse drug event report to the MedWatch?Did not see any benefit to myself or my practice by reporting an error or adverse event to MedWatch
25 (42%)
Unclear on the procedure for reporting to MedWatch 13 (22%)Uncomfortable about the possibility that the FDA could contact me about the report I filed
6 (10%)
Concerned about repercussions of reporting to MedWatch 4 (7%)Worried about provider confidentiality (mine and/or others) 3 (5%)Worried about patient confidentiality 3 (5%)Other 5 (9%)
Under what circumstance might you have reported more events?
Instances LikelyNo
differenceUnlikely
If more errors occurred 78% 19% 3%If there were a greater awareness of the system’s benefits
50% 36% 14%
If I had more time or opportunity to access the system
50% 30% 20%
If it took less time to file a report 45% 37% 18%If I had a better understanding of what should be reported
25% 57% 18%
If I had more assurance of the system’s confidentiality
13% 63% 24%
If there was a change in the attitudes of my practice towards reporting
11% 59% 30%
Did the study affect you personally?
Has your participation in this study affected you personally?
Number (%)
Yes 24(26%)How has your use of this system affected you?
More cognizant of potential errors and adverse events. More vigilant about not repeating them.More aware of medication errors and the importance of careful med review/prescribing practices.More aware of all the different types of medication errors that go on in day to day practice.Made me more aware of the number of errors and made me more careful when doing my job.Made me less complacent about refilling prescriptionsIt has taken up a fair amount of my time that I would typically use on different tasks.
I’m more cautious in giving shots and med samples.
Conclusions
The MEADERS tool was used by practices and felt to be of value
FDA MedWatch is rarely used by clinicians Reporting on medication errors is was not considered
“safe” by many participants. They expressed concerns about repercussions of reporting and a few people were uncomfortable reporting on the mistakes of others
Comments of participants note that MEADERS improved the quality of their care; they learned from mistakes and changed how they did their job
Time is money: the sustainability of using a reporting tool such as MEADERS is unclear. Health systems might consider implementing MEADERS across their organization.
DISCUSSION