Networks’ Experiences

Connecticut Center for Primary Care: ProHealth Physicians Practice Based Research Network (PI: John Lynch, MPH)

The Oregon Rural Practice-based Research Network (ORPRN) (PI: LJ Fagnan, MD)

CenTexNet (PI: Samuel N. Forjuoh, M.D., M.P.H., Dr.P.H)

LA Net (Lyndee Knox, PHD)

Connecticut

The Connecticut Center for Primary Care (CCPC) is an independent 501c3 non-profit research and education foundation.

On incorporation in 2002, CCPC assumed management of the ProHealth Physicians Practice Based Research Network, a PBRN affiliated with the AAFP Federation of Practice Based Research Networks.

PBRN Setting: ProHealth Physicians CT’s largest primary care group practice 75 practice sites throughout Hartford,

Middlesex, Litchfield, Tolland, and New Haven Counties (9 practice sites participated in MEADERS)

221 primary care providers 166 MD/DO, 55 APRN/PA (72 Family Practice,

73 Pediatric, 75 Internal Medicine, and 3 specialists)

Patient population 349,000 (approx 10% of CT) Patient Contacts: over 750,000 encounters/yr

CCPC MEADERS 9 Study Sites1) Pediatric – 9 clinicians2) Pediatric – 6 clinicians3) Family Practice – 7 clinicians4) Family Practice – 2 clinicians5) Internal Medicine – 4 clinicians6) Internal Medicine – 3 clinicians7) Internal Medicine – 3 clinicians8) Internal Medicine – 2 clinician9) Internal Medicine – 1 clinician

CCPC Results continued

Responses by type of event 61.6% (n=77) Adverse drug events 20.0% (n=25) Med Error – provider 8.8% (n=11) Med error patient 4.0% (n=5) Pharmacy 5.6% (n=7)other (MCO, manufacturer,

multiple, …)

CCPC Results continued

When happened: 9.6% Today 44.8% Past week 26.4% Past month 18.4% Over month ago 0.8% Don’t know

Discussion: Experience

Recruiting – easy 20 practices for 10 spots

Extended wait – OMB Lost one site – left ProHealth Sites didn’t remember they had signed

up System – Surprisingly easy/straightforward Retention – consistency by week

Discussion: Lessons

Pilot test quirks Participants say they completed many

more forms Form utility

Comfort reporting adverse drug events Patient/pharmacist/MCO failed to…

Blame culture – CQI opportunityNeed for better education/support

Legal concerns Need for Patient Safety Organization

The Oregon Rural Practice-based Research

Network (ORPRN)

Participating ORPRN Clinics

Scappoose

The Dalles

John Day

Baker City

Enterprise

38.4%46.5%

15.1%19.1%

10.0%

70.9%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Primary CareClinicians

Medical Assistantsor Nurse

Other

Job Category

Pe

rce

nta

ge

ORPRN

Nationally

Participant Job Categories

Reporting Facilitators

Adequate Training Reminders & Feedback

Weekly “Chalk Talk” reports Access to online error report page Occasional “fun” reminders

MEADERS Desktop Icon User-friendly Program Active engagement of lead clinician & staffDid you ask about patient medications?

Qualitative Findings in ORPRN1) Rural Family Practice Clinicians and

office staff will report to MEADERS as evidenced during a 10 week reporting trial

“It was painless.” “It went well. All participated and

seemed to enjoy it. It did not take an excessive amount of time.”

“People in the practice picked up on the type of events to report quite quickly and required little direction. Melinda helped a lot too.”

Qualitative Findings in ORPRN2) Barriers to reporting included:

making the time to report, breadth of reporting options, and confusion regarding what to report.

“It was more difficult for the staff it seemed because it was one more thing that they had to do.”

“I am assuming that some errors did not get reported because the criteria were too broad and clinicians didn’t think that they were important.”

Qualitative Findings in ORPRN3) Some practices used MEADERS data

for internal Quality Improvement “Overall, I think that the MEADERS reporting

system was helpful to staff and providers to remind us that errors occur and we should be mindful of them and work to prevent them in the future…my MA and I worked especially hard with one patient who kept taking the wrong dose of her Coumadin… Her care improved because of MEADERS.”

“The biggest thing it changed was making us more attentive to documentation and keeping med lists up to date ‑ especially between institutions like LTC and hospitals and hospices.”

Qualitative Findings in ORPRN4) Some clinics report a desire to

continue using MEADERS while others do not.

“It was easy to use, we would like to continue using it in our practice.”

“I doubt the practice would be excited about reporting events on a regular basis…This will be a significant obstacle to over come if this or a similar system becomes required in clinical practice.”

ORPRN MEADERS Conclusion

“The overall thing is that the direct access made the process much easier - most of our staff had never made any reports regarding adverse drug reactions so in my eyes - if the goal is to improve the process by which reports are made - this is a huge improvement.”Family Physician, Clinic 2

CenTexNet Practices

28 Scott & White Regional Clinics in 22 Locations + Brazos Family Medicine Residency Clinic, Bryan + Family Practice Residency Clinic, Waco

GatesvilleKilleen

WaenTempleTaylor

Follow-up Phone Call about Status of Survey Response

“Clinic has been really busy and just hasn’t had time to complete survey. Will try to get to it ASAP.”

“Clinic is short of nurses this week. Will try to get to it TODAY.”

“Contact has been out sick and just got back to work today. Will try to get to it.”

No responses from two clinics

Facilitators & Barriers

Facilitators Quality & Safety

Officers Clinic staff & MD

champions Adequacy of training Part of network

mission Staff familiarity with

IT

Barriers Time constraints Flu season Staff turnover Workflow

conflict

LA Net

www.lanetpbrn.net

Reducing Health Care Disparities in Southern California

Through Participatory Research Partnerships With the Health Care Community

LA Net - 5 FQHCsClinica Msr. Oscar A. Romero Alvarado St.

Clinica Msr. Oscar A. Romero Marengo St.

QueensCare, Echo Park

Cleaver Family Wellness Center

East Valley Community Health Center West Covina

East Valley Community Health Center Pomona

LA Net- Sites

Site MDs NPs/PAs

Pharmacy

Clinica Msr. Oscar A. Romero Alvarado St.

5 4 1.8

Clinica Msr. Oscar A. Romero Marengo St.

4 1

Cleaver Family Wellness Center

1 2

East Valley Community Health Center West Covina

8 7 3

East Valley Community Health Center Pomona

6 4 2

0

2

4

6

8

10

12

Week of Study

Nu

mb

er

of

Su

bm

issio

ns

Cleaver

OR Alvarado Street

OR Marengo Street

EVCHC Pomona

EVCHC West Covina

QueensCare EP

Note: QueensCare dropped out week 5 and Cleaver joined at week 6

Weekly Data Submissions by Clinic

Reporting Facilitators

Agreed w/ concept reporting/tracking is important

Liked MEADERS system Easy to use Fast - 20 seconds to enter

Said project increased their awareness

Reporting Barriers: Not about time

Culture: Administrative significance not enough-if not clinically significant, why report?

Ergonomics/Work flow Interrupted paper-based work flow for

clinicians Dispensary uses computer but also interrupted

process Translation to QA processes not easy

Feedback function came on-line toward end When did, not used in QI Duplicates existing paper based ME systems

for County and State (logs) Likely will be a key motivator in future

Learning & recommendations (LA Net)

Need training & implementation supports for program that aid: Identifying administrative value of detecting

& reporting non-harm ADEs and MEs Translation of data into existing QI

processes On-demand reports that meet existing reporting

requirements Multiple options for entering data:

Electronic (Electronic ICON integrated into EHR - No site had EHR or eRxing capabilities at time of study but all are either considering or in process of transition at present)

Paper (Paper to electronic)

Total Submissions by Week

0

10

20

30

40

50

Week

# of

Sub

mis

sion

s

CT OPRN CenTex LA Net

CT 14 14 9 15 13 12 15 7 13 13

OPRN 14 12 13 9 11 4 13 7 13 7

CenTex 48 13 12 13 18 22 12 13 18 9

LA Net 0 7 21 20 11 5 9 12 9 7

1 2 3 4 5 6 7 8 9 10

Total Submissions by Network/Site

0

20

40

60

80

Site

# of

Sub

mis

sion

s

CT 44 31 19 14 6 5 4 2 0

OPRN 37 21 20 13 12

CenTex 83 49 40 4 2

LA Net 35 32 16 11 7 3

1 2 3 4 5 6 7 8 9

Post-reporting survey of the experience of the participantsA survey was sent to the 220 clinicians

and staff members who agreed to participate in the project

Purpose: to learn about participants experience with MEADERS

164 completed the survey, a 75% response rate

Practice role of the 164 respondents

Role in the practice Number (%)

MD,DO 51 (31%)

NP,PA 31 (19%)

RN, LPN,MA, Lab Tech 55 (33%)

Pharmacist 2 (1%)

Office Manager 16 (10%)

Front Office Staff 8 (5%)

Billing/Administrative Staff 1 (1%)

Demographics of 164 respondents

Gender Number (%)Male 40 (25%)

Female 120 (75%)Age

18-37 50 (33%)38-47 35 (23%)48-57 52 (34%)58-87 15 (10%)

Demographics of 164 respondents

Race Number (%)American Indian/Alaska Native 1 (1%)

African American 1 (1%)Asian 2 (1%)

Hispanic/Latino 11 (6%)Pacific Islander/Native Hawaiian 1 (1%)

White, not Hispanic 129 (79%)Multi-racial 3 (2%)No response 16 (9%)

Training for MEADERS

Percent Trained on Use of MEADERS Number (%)

Trained 156 (96%)Untrained 7 (4%)

Effectiveness of Training in Preparation

Effective 76 (48%)Somewhat effective 23 (15%)

Not effective 59 (37%)

Extent of understanding on what events to report

Understand 133 (82%)Somewhat Understand 26 (16%)

Do Not Understand 3 (2%)

Difficulty using MEADERS

Have you reported any events in the past 3 months using MEADERS?

Number (%)

Yes 93 (57%)

No 71 (43%)

Degree of difficulty accessing the electronic reporting form and submitting a report

No or little difficulty 101 (62%)

Moderate to a great deal of difficulty 3 (2%)

Unable to access system 1 (1%)

Did not make a report in MEADERS 58 (35%)

Experience using MEADERS-1

Experience AgreeNeither Agree or Disagree

Disagree

It is easy to use 90% 9% 1%It allows me to be candid when reporting errors

79% 18% 3%

It protected my anonymity when I was filing reports

70% 28% 2%

It encourages me to learn from my mistakes and the mistakes of others

64% 29% 7%

It is viewed positively by this practice 60% 38% 2%It has increased my own awareness of how errors affect patient care

61% 29% 10%

Experience using MEADERS-2

Experience Agree

Neither Agree or Disagree

Disagree

It has helped me to improve patient care at my practice

41% 44% 16%

It has increased the fear of repercussion in the practice

36% 47% 17%

It takes too much time to submit a report 26% 20% 54%

It has led to changes in how we practice medicine

25% 54% 21%

It made me feel like I was informing on my co-workers

10% 20% 70%

It has not worked in this practice 9% 26% 65%

Did you have any concerns about. . .

Concerns Number (%)

The time it took to complete a report 29 (33%)

Knowing what should be reported 26 (29%)

The privacy of your reports 12 (13%)

Knowing how to correctly make a report 10 (11%)

The layout or format of the reporting system 9 (10%)

Others in your practice finding out you had made a report 6 (7%)

The type of questions asked 5 (6%)

How your practice might react to your making a report 5 (6%)

Forwarding event data to MedWatch

Were there ADVERSE DRUG EVENTS or MEDICATION ERRORS that you reported in MEADERS that you did not forward to the FDA MedWatch database?

Number (%)

Yes 38 (40%)What were the reasons you elected not to forward an

adverse drug event report to the MedWatch?Did not see any benefit to myself or my practice by reporting an error or adverse event to MedWatch

25 (42%)

Unclear on the procedure for reporting to MedWatch 13 (22%)Uncomfortable about the possibility that the FDA could contact me about the report I filed

6 (10%)

Concerned about repercussions of reporting to MedWatch 4 (7%)Worried about provider confidentiality (mine and/or others) 3 (5%)Worried about patient confidentiality 3 (5%)Other 5 (9%)

Under what circumstance might you have reported more events?

Instances LikelyNo

differenceUnlikely

If more errors occurred 78% 19% 3%If there were a greater awareness of the system’s benefits

50% 36% 14%

If I had more time or opportunity to access the system

50% 30% 20%

If it took less time to file a report 45% 37% 18%If I had a better understanding of what should be reported

25% 57% 18%

If I had more assurance of the system’s confidentiality

13% 63% 24%

If there was a change in the attitudes of my practice towards reporting

11% 59% 30%

Did the study affect you personally?

Has your participation in this study affected you personally?

Number (%)

Yes 24(26%)How has your use of this system affected you?

More cognizant of potential errors and adverse events. More vigilant about not repeating them.More aware of medication errors and the importance of careful med review/prescribing practices.More aware of all the different types of medication errors that go on in day to day practice.Made me more aware of the number of errors and made me more careful when doing my job.Made me less complacent about refilling prescriptionsIt has taken up a fair amount of my time that I would typically use on different tasks.

I’m more cautious in giving shots and med samples.

Conclusions

The MEADERS tool was used by practices and felt to be of value

FDA MedWatch is rarely used by clinicians Reporting on medication errors is was not considered

“safe” by many participants. They expressed concerns about repercussions of reporting and a few people were uncomfortable reporting on the mistakes of others

Comments of participants note that MEADERS improved the quality of their care; they learned from mistakes and changed how they did their job

Time is money: the sustainability of using a reporting tool such as MEADERS is unclear. Health systems might consider implementing MEADERS across their organization.

DISCUSSION