Evaluating Foraging Tools for Keeping Up with New, Relevant and Valid Information

1

A Bigger Problem?

“It’s not what you don’t know that hurts you (your patients), it’s what you think you know that’s not so”

Important to answer practice-based questions with best source

Equally important to make sure the necessary questions are being asked

Sorting Out InformationThe Usefulness Equation

Usefulness = Relevance x Validity

of any source Work

Shaughnessy AF, Slawson DC, Bennett JH. Becoming an Information Master: A Guidebook to the Medical Information Jungle. The Journal of Family Practice 1994;39(5):489-99.

Two Tools Needed to Master Information A method of being alerted to new information (a “foraging”

tool) A tool for finding the information again when you need it. (a

“hunting” tool) Without both:

• You don’t know that new info. is available

• You can’t find it when you do Clinical example- Riboflavin for migrainesShaughnessy AF, Slawson DC. Are we providing doctors with the training and tools for lifelong

learning? British Medical Journal 1999 (13 Nov): www.bmj.com. (http://bmj.com/cgi/reprint/319/7220/1280.pdf)

Characteristics of an Ideal Clinical Awareness System

Specialty-specific Comprehensive Coordinated hunting and foraging tools Specific and reproducible criteria for relevance and

validity Available at the point-of-care All backed up by levels of evidence

Information Overload

Foraging clip

Real World Medicine

Two patients in first week with mild COPD, minimal symptoms, history of CAD/MI, secondary prevention.

Currently on Spiriva- should I stop it? Doc not seeing study would never ask

question!!! (no known link)

Bottom line: Adults with chronic obstructive pulmonary disease (COPD) treated

with inhaled anticholinergics, including ipratropium (Atrovent) and tiotropium

(Spiriva), are at an increased risk of adverse major cardiovascular events

including myocardial infarction (MI) and cardiovascular death. However,

anticholinergics do improve the important patient oriented outcome of quality of

life while not increasing the risk of all-cause mortality. Clinicians should assess

the individual risk and benefit of treatment for each patient (e.g. withhold

anticholinergics from patients with mild to moderate symptoms of COPD at high

risk of CVD and strongly consider treating patients with life-altering symptoms

from COPD at medium or low risk of CVD). (Common POEM)

Chest 2010: Celli B, et al. Cardiovascular safety of tiotropium in patients with COPD.

Bottom lineThis study finds some support for the safety of tiotropium (Spiriva) in patients with chronic obstructive pulmonary disease (COPD). However, an important limitation of the study was that the authors only looked at studies sponsored by the manufacturer, and the results were heavily weighted by a single large, long study that excluded patients with recent evidence of heart disease. Studies of ipratropium have found different results (Chest 2010;137(1):13-19), and it is unclear why there would be an important difference in risk given the similarity of these drugs.

Singh S, Loke YK, Enright PL, et al.Mortality associated with tiotropium mist inhaler in patients with chronic obstructive pulmonary disease: systematic review and

meta-analysis of randomised controlled trials. BMJ 2011 Jun 14;342:d3215.

RESULTS: Five randomised controlled trials were eligible for inclusion. Tiotropium mist inhaler was associated with a significantly increased risk of mortality (90/3686 v 47/2836; relative risk 1.52, 95% confidence interval, 1.06 to 2.16; P=0.02; I(2)=0%). Both 10 microg (2.15, 1.03 to 4.51; P=0.04; I(2)=9%) and 5 microg (1.46, 1.01 to 2.10; P=0.04; I(2)=0%) doses of tiotropium mist inhaler were associated with an increased risk of mortality. The overall estimates were not substantially changed by sensitivity analysis of the fixed effect analysis of the five trials combined using the random effects model (1.45, 1.02 to 2.07; P=0.04), limiting the analysis to three trials of one year`s duration each (1.50, 1.05 to 2.15), or the inclusion of additional data on tiotropium mist inhaler from another investigational drug programme (1.42, 1.01 to 2.00). The number needed to treat for a year with the 5 microg dose to see one additional death was estimated to be 124 (95% confidence interval 52 to 5682) based on the average control event rate from the long term trials.

CONCLUSIONS: This meta-analysis explains safety concerns by regulatory agencies and indicates a 52% increased risk of mortality associated with tiotropium mist inhaler in patients with chronic obstructive pulmonary disease.

Foraging Tool Work Sheet

Key Points:• Specialty specific, POC (work)

• Disease vs Patient Oriented (relevance)

• LOE rating, best if SORT (validity)

• Coordinated with HQ hunting tool

Reflections/Questions

Quality First-Alert Systems

1. How is the information filtered?• Patient- vs disease- oriented?

• Specialty-specific?

• Comprehensive? Which journals?

• Does it matter (change my practice?) or is it simply news?

2. Is the information valid?• must have levels of evidence (LOE) labels

• Beware “Trojan Horse”!

Quality First-Alert Systems

3. How well is information summarized?• 2000 - 3000 words accurately in 200 words

4. Is the information placed into context?• Much more than abstracts

• “Translational Validity”

First-Alert SystemRisks

“Spyware”: Doc Alerts “Trojan Horse”: who’s paying when it’s free? Abstracts only: Journal Watch,Tips from

other Journals, ClinicalUpdates, • No relevance/ validity filter

You can have information “free” and you can have it “uncensored”, but you can’t have it both ways. No Free Lunch!

Clinical Quandry

Black box warning on Avandia (rosiglitazone) NEJM, then JAMA

Blood sugar still too high, what about Actos (pioglitazone) ?

Should I still be recommending Actos? (did my foraging tool keep me UTD?)

1. www.google.com

2. www.bmjupdates.com (user ID: dcs6e; password: marnie

3. www.medscape.com (user: slawson44; pass: andrew

Cochrane Review

Twenty-two trials which randomised approximately 6200 people to pioglitazone treatment were identified. Longest duration of therapy was 34.5 months. Published studies of at least 24 weeks pioglitazone treatment in people with type 2 diabetes mellitus did not provide convincing evidence that patient-oriented outcomes like mortality, morbidity, adverse effects, costs and health-related quality of life are positively influenced by this compound. Metabolic control measured by glycosylated haemoglobin A1c (HbA1c) as a surrogate endpoint did not demonstrate clinically relevant differences to other oral antidiabetic drugs. Occurrence of oedema was significantly raised. The results of the single trial with relevant clinical endpoints (Prospective Pioglitazone Clinical Trial In Macrovascular Events - PROactive study) have to be regarded as hypothesis-generating and need confirmation.

Foraging tool overview

Tool Less work More workACP Journal Club Specialty specific (IM) Validity assessment but no LOE

Relevance: No POE vs. DOE, no “matters” factor

No hunting tool

Journal Watch Specialty specific (various) Validity: No assessment, no LOE

Relevance: No POE vs. DOE, no “matters” factor

No hunting tool

Dynamed Alerts Specialty specific

Validity assessment, LOE

Relevance: Focuses on evidence that matters

Coordinated hunting tool,

Foraging tool overview

Tool Less work More workMedscape Specialty specific (various) Validity: No assessment, no LOE

Relevance: No POE vs. DOE, no “matters” factor

No hunting tool

BMJ Updates Specialty specific (various) Validity assessment but no LOE

Relevance: No POE vs. DOE, no “matters” factor

(example: breast size=DM2 risk)

No hunting tool

Comparison of Various Email Alert Services for Clinical Knowledge Updates

Stacy Hom MD, Scott Strayer MD MPH,

and David Slawson MD

Email Alert Services

free subscription alerts sent out at least monthly in frequency automatic push service (user did not have to

take additional steps to receive email updates) Measured Translational Accuracy (MTA): Time

to Diffusion and Quality of Assessment.

Emails were collected continuously from September 2008 to September 2010

We using the search terms “Tiotropium” and “Spiriva” Search results were reviewed and only those related to

Tiotropium’s impact on cardiovascular health were included

This was confirmed by checking online archives of email updates sent

We limited our search to a six month period from the date of article publication

Email Alert Services Compared

1. BMJ Evidence Updates2. Doctor’s Guide3. Dynamed4. Essential Evidence POEMs5. Global Family Doctor6. Peerview Institute7. Physician’s First Watch8. Cochrane Pearls

Time to Diffusion

Did NOT send out 2010 article

1. Doctor’s Guide

2. Dynamed

3. Peerview Institute

4. Cochrane Pearls

5. Essential Evidence POEMs: 6 months later

Quality of Assessment

Results

• 6 out of 8 email services looked at were missing

updates on one or both articles.

• Physician’s First Watch & Wonca Global Family

Doctor alerted clinicians on both articles.

• Average time to update was 12 days for the 2008

study, and 37 days for the 2010 study

Quality of Assessment

• Essential Evidence was the only email service to

make the distinction that the 2010 study largely

excluded patients with recent heart disease.

Measured Translational Accuracy (MTA) of Foraging Tools

Time to Diffusion Quality of Assessment Just like getting to Fenway Park for Game 7

World Series