The performance of adverse effects search filters in medline and embase

DOI:10.1111/j.1471-1842.2012.00980.x

The performance of adverse effects search filtersin MEDLINE and EMBASE

Su Golder* & Yoon K. Loke†*Centre for Reviews and Dissemination (CRD), University of York, York, UK and †School of Medicine, Health Policy and

Practice, University of East Anglia, Norwich, UK

Abstract

Background: Search filters can potentially improve the efficiency of searches involving electronic databasessuch as MEDLINE and EMBASE. Although search filters have been developed for identifying records thatcontain adverse effects data, little is known about the sensitivity of such filters.Objectives: This study measured the sensitivity of using available adverse effects filters to retrieve paperswith adverse effects data.Methods: A total of 233 included studies from 26 systematic reviews of adverse effects were used foranalysis. Search filters from MEDLINE and EMBASE were tested for their sensitivity in retrieving the recordsincluded in these reviews. In addition, the sensitivity of each individual search term used in at least onesearch filter was measured.Results: Subheadings proved the most useful search terms in both MEDLINE and EMBASE. No indexing termsin MEDLINE achieved over 12% sensitivity. The sensitivity of published search filters varied in MEDLINE

from 3% to 93% and in EMBASE from 57% to 97%. Whether this level of sensitivity is acceptable will bedependent on the purpose of the search.Conclusions: Although no adverse effects search filter captured all the relevant records, high sensitivitycould be achieved. Search filters may therefore be useful in retrieving adverse effects data.

Keywords: adverse effects, information storage and retrieval, MEDLINE, safety, toxicity.

Key Messages

Implications for Practice

• A more sensitive search for adverse effects data can be constructed with greater ease in EMBASE

than in MEDLINE.• Subheadings are useful in MEDLINE and EMBASE when searching for adverse effects.• No indexing terms were particularly useful in MEDLINE.• Adverse effects search filters can be used for focused searches of MEDLINE and EMBASE with reason-

able sensitivity.

Implications for Policy

• The performance of adverse effects search filters in terms of precision should be measured in awide range of prospectively conducted studies in order to determine the effectiveness of such filtersin practice.

• The indexing of records with adverse effects data needs improvement.

Correspondence: Su Golder, Research Fellow in Health Services

Research, Centre for Reviews and Dissemination (CRD), University

of York, York YO10 5DD, UK. E-mail: [email protected]

ª 2012 The authors. Health Information and Libraries Journal ªHealth Information and Libraries Journal

Introduction

Search filters (predefined combinations of searchterms) have been developed to capture records fornumerous different study designs [such as randomised

2012 Health Libraries Group

1

Adverse effects search filters, Su Golder & Yoon K. Loke

controlled trials (RCTs) and systematic reviews]and topic areas (such as by age category or disease).Search filters are useful for searchers as they canimprove the effectiveness of searches and reducethe time needed to create search strings. Filters canbe developed via expert opinion or throughobjective research methods, or a combination ofboth.

However, it can be difficult to create searchfilters that achieve an acceptable sensitivity andprecision. One area that is particularly problematicis the development of adverse effects filters. Anadverse effect is defined as ‘A harmful or undesir-able outcome that occurs during or after the use ofa drug or intervention for which there is at least areasonable possibility of a causal relation’.1

Adverse effects are an important consideration forpatients, clinicians and other decision makers (suchas NICE) to make informed balanced decisions, onwhether the potential benefits of an interventionoutweigh its potential harm.2,3 Identifying appro-priate information on adverse effects, however, canbe challenging, and attempts have been made toease this process with the creation of adverseeffects search filters.

A key difficulty in creating adverse effectssearch filters stems from the absence of adverseeffects terms in the title, abstract and indexing ofrelevant articles. In 2001, Derry et al.4 evaluatedtrials that reported adverse effects data and foundthat about 23% of such trials had no adverseeffects terms (either generic, such as adverse effector side effect, or specific, such as headache orrash) in the title, abstract or indexing of records inMEDLINE or EMBASE. Despite a lack of reportedadverse effects terms, a number of adverse effectsfilters have still been proposed for MEDLINE

5–11 andEMBASE.7,8 However, the sensitivity of using suchfilters is of concern given that nearly a quarter ofrelevant papers do not have any adverse effect–related terms. This lack of confidence in adverseeffects search filters has led to current guidancethat emphasises the use of non-specific searches(without relying on adverse effects search filters)as well as the need to check full-text versions ofretrieved articles.12 Unfortunately, implementationof such guidance in systematic reviews is onerousand time-consuming compared to running morespecific adverse effect–based searches.

ª 2012 The authors. Health Info

In this study, we planned to measure the perfor-mance, in terms of sensitivity, of published searchfilters for MEDLINE and EMBASE as well as individualsearch terms in identifying papers with adverseeffects data.

Methods

Selection of studies containing adverse effects

data

A collection of papers that report data on the fre-quency of adverse effects was sought from thestudies of systematic reviews of adverse effects.The use of included papers from systematicreviews has been shown to be an effective way ofidentifying a standard set of records for use instudies evaluating search strategies.13

The included reviews may have evaluatedadverse effects in different ways, for instance, sin-gling out specific named adverse clinical outcomesof interest (such as fractures or myocardial infarc-tion), or they may have taken a broader look atthe general safety or tolerability profile withoutlimiting their review to particular adverseoutcomes.

Identification of systematic reviews of adverse

effects

Systematic reviews of adverse effects were identi-fied through browsing the Database of Abstracts ofReviews of Effects (DARE) (http://www.crd.york.ac.uk/crdweb/). A systematic review was con-sidered eligible for inclusion if it was publishedbetween 2007 and 2010 andd Adverse drug effects were the primary outcome.d Adverse effects search terms or specified ⁄namedadverse effects had not been used by the reviewauthors. This enabled us to build an unselectedcohort, that is, where relevant articles had notalready been chosen because of the presence ofspecific adverse effects terms. Typically, suchreviews would have relied on search terms forthe population ⁄ condition and intervention only,such as type II diabetes and rosiglitazone.

d The search included either hand searching orreference checking in addition to databasesearches.

rmation and Libraries Journal ª 2012 Health Libraries Group

Health Information and Libraries Journal

Adverse effects search filters, Su Golder & Yoon K. Loke 3

Selection of relevant primary studies with

adverse effects data

The included references in each systematic reviewwere checked for papers published in English afterthe year 2000, so that a contemporaneous studycohort could be obtained. Non-English languagepapers were excluded as we were concerned aboutobtaining valid matches for adverse effects termsin different languages. Full-text articles werechecked to confirm the presence of adverse effectsdata that had been used in the systematic review.The papers were then de-duplicated, in order toremove copies of papers that had been included inmore than one systematic review.

Analysis

Availability in selected databases. The first stageof the analysis was to check whether each paperwas listed and indexed in MEDLINE or EMBASE. Inorder to ascertain this, several iterations usingauthor names and words from the title were used.

Box 1 Published adverse effects filters

Search Strategies excluding specified named adverse effects te

MEDLINE

Badgett5,6

(ae OR co OR po OR de).fs OR CASE REPORT ⁄ and HUMAN ⁄Golder7

Most sensitive search strategy excluding use of specified name

(ae OR co OR de).fs OR (safe OR safety OR side effect* OR u

OR toxicity OR adrs OR (adverse adj2 (effect OR effects OR re

OR outcomes)).ti,ab

EMBASE

Golder7

Most sensitive search strategy excluding use of specified name

DRUG ⁄ ae, to OR (safe OR safety OR side effect* OR undesira

toxicity OR adrs OR (adverse adj2 (effect OR effects OR reacti

outcomes))).ti,ab

Search Strategies including specified named adverse effects te

MEDLINE

BMJ Clinical Evidence8

Specified named adverse effect AND (ae OR to OR po OR co).

((adverse OR undesirable OR harm$ OR serious OR toxic) adj3

OR exp product surveillance, postmarketing ⁄ OR exp adverse d

iv ⁄ OR exp poisoning ⁄ OR exp substance-related disorders ⁄ OR e

exp drug monitoring ⁄ OR exp drug hypersensitivity ⁄ OR (toxicity

exp Postoperative Complications ⁄ OR exp Intraoperative Compl

Buckingham9

Step 1. Source of harm with possible subheadings


Published search filters. We then checkedwhether the papers available on MEDLINE and EM-

BASE would have been identifiable using a pub-lished search filter. Each filter was tested in turnand the sensitivity of using each filter recorded.Owing to the logistical constraints of repeatingsearch strategies, it was not possible to measurethe precision of the search filters. However, itcould be safely assumed that precision will behigher with a search filter as opposed to without afilter, as fewer records would be retrieved with afilter. The MEDLINE search filters tested were Badg-ett et al.,5,6 Golder et al.,7 BMJ Clinical Evi-dence,8 and Buckingham et al.,9 and the EMBASE

filters; Golder et al.,7 and BMJ Clinical Evidence2006 (Box 1).8 The PubMed filters by Wielandand Dickersin10,11 were not tested as these filterswere designed specifically for a review on oralcontraceptives and breast cancer (Box 1).

The search filters by Badgett et al.5,6 and Golderet al.7 do not contain any specific named adverseeffects terms and could therefore be tested on allthe included papers. The other filters (including a

rms

(no need to preselect specific ADRS)

d adverse effects

ndesirable effect* OR treatment emergent OR tolerability

action OR reactions OR event OR events OR outcome

d adverse effects

ble effect* OR treatment emergent OR tolerability OR

on OR reactions OR event OR events OR outcome OR

rms

fs. OR (safe OR safety).ti,ab. OR side effect$.ti,ab. OR

(effect$ OR reaction$ OR event$ OR outcome$)).ti,ab.

rug reaction reporting systems ⁄ OR exp clinical trials, phase

xp drug toxicity ⁄ OR exp abnormalities, drug induced ⁄ OR

OR complication$ OR noxious OR tolerability).ti,ab. OR

ications ⁄


Box 1 Continued.

Adverse Effects (AE), Mortality (MO) (not always an option), Contraindications (CT)

Poisoning (PO), Toxicity (TO)

Step 2. Disease or disorder ⁄ outcome with possible subheadings

Abnormalities (AB), Chemically Induced (CI), Epidemiology (EP), Etiology (ET)

Mortality (MO), Transmission (TM)

Step 3. Combine Previous Searches with a Quality Filter

Combine the first two search statements, using the boolean operator ‘and’ and combine the final subject statement with

the quick filter (hedge) below

Case control studies ⁄ OR cohort studies ⁄ OR risk ⁄

Golder7

Most sensitive search strategy

Specified named adverse effects ⁄ ci OR (ae OR co OR de).fs OR (safe OR safety OR side effect* OR undesirable effect*

OR treatment emergent OR tolerability OR toxicity OR adrs OR (adverse adj2 (effect OR effects OR reaction OR

reactions OR event OR events OR outcome OR outcomes)).af

Wieland10,11

Exploding MeSH term search

1966:1995 [dp] AND ‘human’ [MESH] AND journal article [pt] AND breast neoplasms [mh] AND contraceptives, oral

[mh] AND (risk [mh] OR follow-up studies [mh] OR case-control studies [mh])

Maximise Precision: MeSH term search with major topics and subheadings

1966:1995 [dp] AND ‘human’ [MESH] AND journal article [pt] AND ‘breast neoplasms’ [majr:noexp] AND

(contraceptives, oral [mh:noexp] OR contraceptives, oral ⁄ pharmacology [mh] OR contraceptives, oral ⁄ therapeutic use

[mh] OR estrogens ⁄ therapeutic use [mh] OR contraceptives, oral ⁄ adverse effects [mh]) AND (risk [mh:noexp] OR risk

factors [mh:noexp] OR follow-up studies [mh:noexp] OR odds ratio [mh:noexp])

MeSH term search without study methodology terms

1966:1995 [dp] AND ‘human’ [MESH] AND journal article [pt] AND ‘breast neoplasms’ [majr:noexp] AND

(contraceptives, oral [mh:noexp] OR contraceptives, oral ⁄ pharmacology [mh] OR estrogens ⁄ therapeutic use [mh] OR

contraceptives, oral ⁄ therapeutic use [mh] OR contraceptives, oral ⁄ adverse effects [mh])

Maximise Sensitivity: MeSH term search without intervention terms

1966:1995 [dp] AND ‘human’ [MESH] AND journal article [pt] AND ‘breast neoplasms’ [majr:noexp] AND (risk

[mh:noexp] OR risk factors [mh:noexp] OR follow-up studies [mh:noexp] OR odds ratio [mh:noexp])

Text word search with automatic term mapping

1966:1995 [dp] AND ‘human’ [MESH] AND journal article [pt] AND breast cancer AND (oral contraceptive OR oral

contraceptives OR estrogen OR estrogens OR hormones OR hormonal) AND (risk OR follow-up OR epidemiologic)

Text word search with truncation and double quotes

1966:1995 [dp] AND ‘human’ [MESH] AND journal article [pt] AND ‘breast cancer’ AND (oral contraceptive* OR

‘estrogen’ OR ‘hormones’ OR ‘hormonal’) AND (‘risk’ OR ‘epidemiologic’)

Text word search without study methodology text words

1966:1995 [dp] AND ‘human’ [MESH] AND journal article [pt] AND ‘breast cancer’ AND (oral contraceptive* OR

‘estrogen’ OR ‘hormones’ OR ‘hormonal’)

Maximise Sensitivity: Text word search without intervention text words

1966:1995 [dp] AND ‘human’ [MESH] AND journal article [pt] AND ‘breast cancer’ AND (‘risk’ OR epidemiolog*)

EMBASE

BMJ Clinical Evidence8

Specified named adverse effect AND (ae OR si OR to OR co).fs. OR (safe OR safety).ti,ab. OR side effect$.ti,ab. OR

((adverse OR undesirable OR harm$ OR serious OR toxic) adj3 (effect$ OR reaction$ OR event$ OR outcome$)).ti,ab.

OR exp adverse drug reaction ⁄ OR exp drug toxicity ⁄ OR exp intoxication ⁄ OR exp drug safety ⁄ OR exp drug

monitoring ⁄ OR exp drug hypersensitivity ⁄ OR exp postmarketing surveillance ⁄ OR exp drug surveillance program ⁄ OR exp

phase iv clinical trial ⁄ OR (toxicity OR complication$ OR noxious OR tolerability).ti,ab. OR exp postoperative

complication ⁄ OR exp Peroperative Complication ⁄Golder7

Most sensitive search strategy

Specified named adverse effects OR (safe OR safety OR side effect* OR undesirable effect* OR treatment emergent OR

tolerability OR toxicity OR adrs OR (adverse adj2 (effect OR effects OR reaction OR reactions OR event OR events OR

outcome OR outcomes))).ti,ab


ª 2012 The authors. Health Information and Libraries Journal ª 2012 Health Libraries Group



second filter by Golder et al.7) include specificnamed adverse effects terms and were thereforeonly tested on those included papers identifiedfrom a systematic review of a named adverseeffect of interest, and not those reviews that aimedto carry out a generalised broad evaluation ofsafety.

In addition to the search combinations proposedin the published search filters, each term includedin at least one search filter was tested individuallyand the sensitivity measured (this included termsfrom Wieland and Dickersin10,11). This was carriedout to ascertain whether specific terms included inthe search filters are particularly useful in retriev-ing papers containing adverse effects data.

Results

Systematic reviews

Twenty-six systematic reviews met the inclusioncriteria (Table 2).14–39 Half were concernedwith the safety profile for an interven-tion,14,15,20,23–28,31–33,36 whereas the other halfwere limited to a named ⁄ specific adverse effectwith an intervention.16–19,21,22,29,30,34,35,37–39

Primary studies

A total of 474 papers were initially found withinthe systematic reviews. Subsequently, 241 paperswere excluded from the analysis; 140 werepublished before 2001, eight were published in anon-English language, 35 were not indexed oneither MEDLINE or EMBASE, 25 were duplicate papers(contained in more than one systematic review),and 33 papers did not contain adverse effects datain the full text relevant to the systematic review inwhich they were cited. Those references notidentified in any of the databases were 15company reports, eight conference papers, twoFood and Drug Administration (FDA) reports andone book.

A total of 233 papers were eligible for use inthe analysis. The majority of the references wereRCTs (90%, 209 ⁄233); however, there were also12 case series, five chart reviews, three casereports, two cohort studies, one non-RCT studyand one uncontrolled study.


Availability of records on MEDLINE and EMBASE

A total of 231 references were indexed on MED-

LINE; 119 papers were from reviews on a specificnamed adverse effect and 117 from safety profilereviews (five references were included in bothtypes of review). A total of 222 references wereindexed on EMBASE (two of which were notindexed on MEDLINE), 114 papers from reviews ona specific named adverse effect and 113 fromsafety profile reviews (five references wereincluded in both types of review).

Published search filters (excluding specific

named adverse effects terms)

Search filters that are based on generic adverseeffects terms (such as ‘adverse effect’ or ‘sideeffect’ or ‘tolerability’ or ‘toxicity’) can potentiallybe used in any systematic review and were testedon all the included studies. If the Badgett et al.5,6

and Golder et al.7 search filters had been appliedin MEDLINE, 28% (65 ⁄231) and 13% (29 ⁄231) ofpapers would have been missed and 12% (26 ⁄222)of papers would have been missed if the Golderet al.7 search filter had been applied in EMBASE

(Table 1).

Published search filters (including specific

named adverse effects terms)

Those filters dictating the use of specific namedadverse effects terms were tested only with theincluded papers from systematic reviews of spe-cific named adverse effects. Here, 77% (92 ⁄119),93% (111 ⁄119), 97% (116 ⁄119) and 7% (8 ⁄119)of papers would have been missed if the BMJClinical Evidence,8 the Buckingham et al.9 vari-ants and Golder et al.7 search filters had beenapplied in MEDLINE, and 43% (49 ⁄114) and 4%(5 ⁄114) of papers would have been missed if theBMJ Clinical Evidence8 and Golder et al.7 searchfilter had been applied in EMBASE (Table 1).

The sensitivity of the BMJ Clinical Evidence8

filter would have been improved if the specificnamed adverse effects terms had been ORedinstead of ANDed with the generic adverse effectsterms. A total of 109 papers would have beenretrieved in MEDLINE with a sensitivity of 92%


Table 1 Sensitivity of published MEDLINE and EMBASE search

strategies for adverse effects

Number of

relevant

records

Sensitivity

(%)

Search strategies excluding

specified named adverse

effects terms

MEDLINE (n = 231)

Badgett5,6 166 72

Golder7 202 87

EMBASE (n = 222)

Golder7 195 88

Search strategies including

specified named adverse

effects terms

MEDLINE (n = 119)

BMJ Clinical Evidence8 27 23

Buckingham9 Without

the quick filter (hedge)

8 7

Buckingham9 With the

quick filter (hedge)

3 3

Golder7 111 93

EMBASE (n = 114)

BMJ clinical evidence8 65 57

Golder7 109 96


(109 ⁄119) and 109 in EMBASE with a sensitivity of96% (109 ⁄114).

Different study designs

Of the 24 non-RCT studies in MEDLINE, all wereretrieved by the BMJ Clinical Evidence8, and theGolder et al.7 search strategies and the searchstrategy by Badgett et al.5,6 retrieved all but threechart reviews and one non-RCT. Twenty threenon-RCT studies were available in EMBASE, and all23 were retrieved by the BMJ Clinical Evidence8

and the Golder et al.7 search strategies.

Individual search terms

Table 2 shows the number of relevant recordsretrieved using each individual generic search termcontained in at least one published search filter.

In MEDLINE, the most commonly used terms werethe subheadings ‘therapeutic use (tu)’ (77%,177 ⁄231) and ‘adverse effects (ae)’ (51%, 117 ⁄231)followed by the terms ‘adverse adj3 event$’ (32%,75 ⁄231), ‘safety’ (31%, 71 ⁄231), ‘adverse adj2


events’ (29%, 67 ⁄231) and ‘risk’ (28%, 64 ⁄231) inthe title or abstract, and the subheadings ‘drugeffects (de)’ (27%, 62 ⁄231) and ‘complications(co)’ (18%, 41 ⁄231). None of the indexing termsretrieved a large proportion of the relevant records.

In EMBASE, the highest number of records fromEMBASE was retrieved with the floating subheadings,‘adverse drug reaction’ (ae) (83%, 185 ⁄222) and‘side effect’ (si) (83%, 185 ⁄222), this was followedby the EMTREE term exp drug safety ⁄ (38%,85 ⁄222) and the terms ‘adverse adj3 event$’ (32%,71 ⁄222), ‘safety’ (28%, 63 ⁄222), ‘adverse adj2events’ (28%, 63 ⁄222) and ‘risk’ (27%, 61 ⁄222).

Discussion

The search filters by Badgett et al.5,6 and Golderet al.7 achieved fairly high sensitivity, and the fil-ter by BMJ Clinical Evidence8 would also haveachieved high sensitivity had the specific namedadverse effects been ORed instead of ANDed withthe generic adverse effects terms. Whether thislevel of sensitivity is acceptable for a systematicreview search will depend on the topic under eval-uation, resources available and anticipated gain inprecision. For example, if the search filters reducedthe numbers of records needed to sift from anunmanageable set of tens of thousands of records,these search filters may be worth using. High pre-cision is often at the loss of sensitivity though. Forinstance, the Buckingham filters9 aim to achievehigh precision and it is therefore not surprisingthat these filters fared poorly with respect to sensi-tivity. The inevitable trade-off between sensitivityand precision may or may not be problematicdepending on the particular review, for example,in situations where the search filters reduce thenumbers of records needed to sift by only ahandful, then any loss in sensitivity may not beacceptable.

Evaluation of adverse effects may involve theinclusion of a range of different study designs,such as non-randomised pharmacoepidemiologicalstudies. Hence, the ability to accurately identifyrelevant observational studies would be a poten-tially useful feature of a search filter. Althoughbased on a smaller sample, the results here indicatethat the BMJ Clinical Evidence8 and the Golderet al.7 search strategies may be particularly useful



Table 2 Performance of individual search terms from published search filters in MEDLINE and EMBASE

MEDLINE

Number of

Relevant

Records

(n = 231) EMBASE

Number of

Relevant

Records

(n = 222)

Floating subheadings (n = 231) Floating subheadings

Adverse effects (ae) 117 (51%) Adverse drug reaction (ae) 185 (83%)

Complications (co) 41 (18%) Complication (co) 24 (11%)

Drug effects (de) 62 (27%) Drug toxicity (to) 2 (1%)

Poisoning (po) 0 Side effect (si) 185 (83%)

Toxicity (to) 0

Subheadings attached to intervention (n = 231) Subheadings attached to intervention

Adverse effects (ae) 117 (51%) Adverse drug reaction (ae) 185 (83%)

Contraindications (ct) 0 Drug toxicity (to) 2 (1%)

Mortality (mo) 0

Pharmacology (pd) 22 (10%)

Poisoning (po) 0

Therapeutic use (tu) 177 (77%)

Toxicity (to) 0

Subheadings attached to disease or disorder ⁄ outcome (n = 119)

Abnormalities (ab) 0

Chemically induced (ci) 7 (%)

Epidemiology (ep) 2 (1%)

Etiology (et) 0

Mortality (mo) 3 (%)

Transmission (tm) 0

MeSH terms EMTREE terms

Exp abnormalities, drug induced ⁄ 0 exp adverse drug reaction ⁄ 43 (19%)

Exp adverse drug reaction reporting systems ⁄ 0 exp drug hypersensitivity ⁄ 4 (2%)

Case control studies ⁄ 1 (0.5%) exp drug monitoring ⁄ 0

Exp case control studies ⁄ 8 (3%) exp drug safety ⁄ 85 (38%)

Case report ⁄ 0 exp drug surveillance program ⁄ 0

Exp clinical trials, phase iv as topic ⁄ 0 exp drug toxicity ⁄ 0

Cohort studies ⁄ 3 (1%) exp intoxication ⁄ 0

Exp drug hypersensitivity ⁄ 2 (1%)

Exp drug monitoring ⁄ 0

Exp drug toxicity ⁄ 3 (1%)

Exp follow-up studies ⁄ 24 (10%)

Human ⁄ 231 (100%)

Exp Intraoperative Complications ⁄ 3 (1%)

odds ratio ⁄ 1 (0.5%)

Exp poisoning ⁄ 4 (2%)

Exp Postoperative Complications ⁄ 1 (0.5%)

Exp product surveillance, postmarketing ⁄ 0

Risk ⁄ 6 (3%)

Exp Risk ⁄ 28 (12%)

Risk factors ⁄ 19 (8%)

Exp Substance-related disorders ⁄ 6 (3%)

Terms in the title or abstract Terms in the title or abstract

adrs 0 adrs 0

adverse adj2 effect 4 (2%) adverse adj2 effect 4 (2%)

adverse adj2 effects 17 (%) adverse adj2 effects 15 (7%)

adverse adj3 effect$ 19 (8%) adverse adj3 effect$ 19 (9%)

adverse adj2 event 14 (6%) adverse adj2 event 13 (6%)

adverse adj2 events 67 (29%) adverse adj2 events 63 (28%)


ª 2012 The authors. Health Information and Libraries Journal ª 2012 Health Libraries Group


7

Table 2 Continued.

MEDLINE

Number of

Relevant

Records

(n = 231) EMBASE

Number of

Relevant

Records

(n = 222)

adverse adj3 event$ 75 (32%) adverse adj3 event$ 71 (32%)

adverse adj2 outcome 0 adverse adj2 outcome 0

adverse adj2 outcomes 0 adverse adj2 outcomes 0

adverse adj3 outcome$ 0 adverse adj3 outcome$ 0

adverse adj2 reaction 0 adverse adj2 reaction 0

adverse adj2 reactions 3 (1%) adverse adj2 reactions 3 (1%)

adverse adj3 reaction$ 3 (1%) adverse adj3 reaction$ 3 (1%)

complication$ 7 (3%) complication$ 7 (3%)

epidemiolog* 0 epidemiolog* 0

epidemiologic 0 epidemiologic 0

follow-up 29 (13%) follow-up 28 (13%)

harm$ adj3 effect$ 0 harm$ adj3 effect$ 0

harm$ adj3 event$ 0 harm$ adj3 event$ 0

harm$ adj3 outcome$ 0 harm$ adj3 outcome$ 0

harm$ adj3 reaction$ 0 harm$ adj3 reaction$ 0

noxious 0 noxious 0

risk 64 (27%) risk 61 (27%)

safe 17 (7%) safe 17 (8%)

safety 71 (31%) safety 63 (28%)

serious adj3 effect$ 1 (0.5%) serious adj3 effect$ 1 (0.5%)

serious adj3 event$ 15 (6%) serious adj3 event$ 12 (5%)

serious adj3 outcome$ 0 serious adj3 outcome$ 0

serious adj3 reaction$ 2 (1%) serious adj3 reaction$ 2 (1%)

side effect$ 22 (10%) side effect$ 23 (10%)

tolerability 23 (10%) tolerability 24 (11%)

toxic adj3 effect$ 2 (1%) toxic adj3 effect$ 2 (1%)

toxic adj3 event$ 0 toxic adj3 event$ 0

toxic adj3 outcome$ 0 toxic adj3 outcome$ 0

toxic adj3 reaction$ 0 toxic adj3 reaction$ 0

toxicity 11 (5%) toxicity 11 (5%)

treatment emergent 0 treatment emergent 0

undesirable effect* 0 undesirable effect* 0

undesirable adj3 effect$ 0 undesirable adj3 effect$ 0

undesirable adj3 event$ 0 undesirable adj3 event$ 0

undesirable adj3 outcome$ 0 undesirable adj3 outcome$ 0

undesirable adj3 reaction$ 0 undesirable adj3 reaction$ 0


in retrieving non-randomised studies. This may bebecause many observational studies are focused onadverse effects as primary rather than secondaryoutcomes and therefore easier to retrieve.

To narrow things down further, our analysis ofthe utility of particular search terms can help inthe design of new filters or in guiding the modifi-cation of existing ones for improved performance.Here, we found the most useful individual searchterms in both MEDLINE and EMBASE were adverseeffects subheadings. These tended to retrieve more


relevant records than either text words or indexingterms. In the common scenario where a customisedsearch strategy for a particular topic needs to bebuilt from scratch, it would be sensible to designthe search with a greater emphasis on the inclusionof relevant subheadings (based on examples inTable 2), rather than rely on picking up specificterms in the title or abstract. Subheadings arelikely to be more useful as there is a dearth ofappropriate indexing terms available. In addition,adverse effects terms are less likely to appear in




the title and abstract, and when they do, there is alack of consistency in the terms used.

Limitations

The main limitation of this research is that the pre-cision of search terms and search filters could notbe measured. Although this study indicates whichsearch terms are most effective in retrieving paperswith adverse effects data, these terms may stillretrieve a high proportion of irrelevant records (butless so than similar searches with no adverseeffects terms at all). In developing any search strat-egy, it is important to obtain a balance betweensensitivity and precision. The most sensitivesearches may also be those with the lowest preci-sion. Caution should therefore be applied whenusing the results of this study, especially as someof the search terms appeared to have only a vagueconnection to adverse effects (such as ‘therapeuticuse’, ‘pharmacology’ ‘follow-up’ or ‘risk’).Although systematic review searches aim to be ascomprehensive as possible, such a search may yieldseveral thousand articles and only a small fractionof them will be relevant. Search results should aimto be manageable in terms of the feasibility ofscreening all the potentially relevant studies, partic-ularly as previous recommendations have suggestedthat checking full-text articles may be necessary.

Conclusions

In our evaluation of search terms for adverseeffects data, subheadings (either free floating orwith subject heading) seem to be the most usefulmeans of identifying relevant papers in both MED-

LINE and EMBASE. Few indexing terms exist foradverse effects, and the sensitivity of those that doexist is low, particularly in MEDLINE. While somefree text terms for adverse effects in the title andabstract may be useful, we recommend that freetext terms should only be applied concomitantlywith other terms such as subheadings. Thesensitivity of adverse effects search filters variedconsiderably, and although high sensitivity couldbe achieved in both MEDLINE and EMBASE, 100%sensitivity was not achievable.

Further research that also measures the precisionof the search terms and filters is required to


establish the full value of using adverse effectsterms in search strategies. In the meantime,improvements in the allocation of subheadings tobibliographic records could ease the workload ofthe systematic reviewer.

Acknowledgements

We would like to thank Professor Lesley Stewart,CRD for comments on an earlier draft.

Source of Funding

This research was undertaken by Su Golder as partof a Medical Research Council (MRC) fellowship.The views expressed in this presentation are thoseof the authors and not necessarily those of theMRC. The funders had no role in study design,data collection and analysis, decision to publish orpreparation of the manuscript.

Conflict of Interest

No conflicts of interests are declared by theauthors.

References

1 Chou, R., Aronson, N., Atkins, D., Ismaila, A. S.,Santaguida, P., Smith, D. H., Whitlock, E., Wilt, T. J. &Moher, D. AHRQ Series Paper 4: Assessing harms whencomparing medical interventions: AHRQ and the EffectiveHealth-Care Program. Journal of Clinical Epidemiology2010, 63, 502–512.

2 Keech, A. C., Wonders, S. M., Cook, D. I. & Gebski, V. J.Balancing the outcomes: reporting adverse events. MedicalJournal of Australia 2004, 181, 215–218.

3 Strom, B. L. What is pharmacoepidemiology? In: Strom, B.L. (ed.). Pharmacoepidemiology. Chichester, PA: JohnWiley, 2006: 3–11.

4 Derry, S., Loke, Y. K. & Aronson, J. K. Incompleteevidence: the inadequacy of databases in tracing publishedadverse drug reactions in clinical trials. BMC MedicalResearch Methodology 2001, 1, 7.

5 Badgett, R., Chiquette, E., Anagnostelis, B. & Mulrow, C.Locating Reports of Serious Adverse Drug Reactions(PowerPoint Presentation), 1999. Accessible at: http://medinformatics.uthscsa.edu/#FILTERS, accessed 7 January2005.

6 Badgett, R., Chiquette, E., Anagnostelis, B. & Mulrow, C.Locating reports of serious adverse drug reactions. 7thAnnual Cochrane Colloquium Abstracts, October 1999.



Rome, Italy, 1999. Accessible at: http://www.imbi.uni-freiburg.de/OJS/cca/index.php?journal=cca&page=article&op=view&path[]=4565, accessed 15 February 2012.

7 Golder, S., McIntosh, H. M., Duffy, S. & Glanville, J.Developing efficient search strategies to identify reports ofadverse effects in MEDLINE and EMBASE. HealthInformation and Libraries Journal 2006, 23, 3–12.

8 BMJ Clinical Evidence. Search Filters. BMJ, 2006.Accessible at: http://clinicalevidence.bmj.com/x/set/static/ebm/learn/665076.html, accessed 14 February 2012.

9 Buckingham, J., Fisher, B. & Saunders, D. OvidMedline ⁄Pubmed Search Strategies for Articles onCausation ⁄Harm ⁄Etiology. Alberta: Evidence BasedMedicine Tool Kit, 2005. Accessible at: http://www.ebm.med.ualberta.ca/HarmSearch.html, accessed 13 May 2005.

10 Wieland, S. & Dickersin, K. Selective exposure reportingand Medline indexing limited the search sensitivity forobservational studies of the adverse effects of oralcontraceptives. Journal of Clinical Epidemiology 2005, 58,560–567.

11 Wieland, S. & Dickersin, K. Erratum: selective exposurereporting and Medline indexing limited the search sensitivityfor observational studies of the adverse effects of oralcontraceptives (Journal of Clinical Epidemiology (June2005) 58:6 (560–567)). Journal of Clinical Epidemiology2005, 58, 1077.

12 Higgins, J. P. T. & Green, S. (eds). Cochrane Handbook forSystematic Reviews of Interventions 5.1.0 [updated March2011]. The Cochrane Collaboration, 2011. Accessible at:http://www.cochrane-handbook.org.

13 Sampson, M., Zhang, L., Morrison, A., Barrowman, N. J.,Clifford, T. J., Platt, R. W., Klassen, T. P. & Moher, D. Analternative to the hand searching gold standard: validatingmethodological search filters using relative recall. BMCMedical Research Methodology 2006, 6, 33.

14 Agbabiaka, T., Pittler, M., Wider, B. & Ernst, E. Serenoarepens (saw palmetto): a systematic review of adverseevents. Drug Safety 2009, 32, 637–647.

15 Albavera-Hernandez, C., Rodrıguez, J. & Idrovo, A. Safetyof botulinum toxin type A among children with spasticitysecondary to cerebral palsy: a systematic review ofrandomized clinical trials. Clinical Rehabilitation 2009, 23,394–407.

16 Alghamdi, A. A., Moussa, F. & Fremes, S. E. Does the useof preoperative aspirin increase the risk of bleeding inpatients undergoing coronary artery bypass grafting surgery?Systematic review and meta-analysis. Journal of CardiacSurgery 2007, 22, 247–256.

17 Berlie, H., Kalus, J. & Jaber, L. Thiazolidinediones and therisk of edema: a meta-analysis. Diabetes Research andClinical Practice 2007, 76, 279–289.

18 Bonovas, S. & Sitaras, N. Does pravastatin promote cancer inelderly patients? A meta-analysis. CMAJ 2007, 176, 649–654.

19 Cao, Y., Liu, L., Liao, C., Tan, A. & Gao, F. Meta-analysisof incidence and risk of hypokalemia with cetuximab-basedtherapy for advanced cancer. Cancer Chemotherapy andPharmacology 2010, 66, 37–42.


20 Chavez-Tapia, N., Tellez-Avila, F., Bedogni, G., Croce,L., Masutti, F. & Tiribelli, C. Systematic review andmeta-analysis on the adverse events of rimonabanttreatment: considerations for its potential use inhepatology. BMC Gastroenterology 2009, 9, 75.

21 Chen, L. & Ashcroft, D. Risk of myocardial infarctionassociated with selective COX-2 inhibitors: meta-analysis ofrandomised controlled trials. Pharmacoepidemiology andDrug Safety 2007, 16, 762–772.

22 Correll, C. Weight gain and metabolic effects of moodstabilizers and antipsychotics in pediatric bipolar disorder: asystematic review and pooled analysis of short-term trials.Journal of the American Academy of Child and AdolescentPsychiatry 2007, 46, 687–700.

23 Dugoua, J., Machado, M., Zhu, X., Chen, X., Koren, G. &Einarson, T. Probiotic safety in pregnancy: a systematicreview and meta-analysis of randomized controlled trials ofLactobacillus, Bifidobacterium, and Saccharomyces spp.Journal of Obstetrics and Gynaecology, Canada 2009, 31,542–552.

24 Ford, A., Malfertheiner, P., Giguere, M., Santana, J., Khan,M. & Moayyedi, P. Adverse events with bismuth salts forHelicobacter pylori eradication: systematic review andmeta-analysis. World Journal of Gastroenterology 2008, 14,7361–7370.

25 Gehling, M. & Tryba, M. Riks and side-effects ofintrathecal morphine combined with spinal anaesthesia: ameta-analysis. Anaesthesia 2009, 64, 643–651.

26 Johansson, K., Neovius, K., DeSantis, S., Rossner, S. &Neovius, M. Discontinuation due to adverse events inrandomized trials of orlistat, sibutramine and rimonabant: ameta-analysis. Obesity Reviews 2009, 10, 564–575.

27 Lakhdar, R., Al-Mallah, M. & Lanfear, D. Safety andtolerability of angiotensin-converting enzyme inhibitorversus the combination of angiotensin-converting enzymeinhibitor and angiotensin receptor blocker in patients withleft ventricular dysfunction: a systematic review andmeta-analysis of randomized controlled trials. Journal ofCardiac Failure 2008, 14, 181–188.

28 Luykx, J., Mason, M., Ferrari, M. & Carpay, J. Aremigraineurs at increased risk of adverse drug responses? Ameta-analytic comparison of topiramate-related adverse drugreactions in epilepsy and migraine. Clinical Pharmacologyand Therapeutics 2009, 85, 283–288.

29 Mauri, D., Valachis, A., Polyzos, I., Polyzos, N.,Kamposioras, K. & Pesce, L. Osteonecrosis of the jaw anduse of bisphosphonates in adjuvant breast cancer treatment:a meta-analysis. Breast Cancer Research and Treatment2009, 116, 433–439.

30 Morris, T., Castrejon, S., Devendra, G. & Gamst, A. Nodifference in risk for thrombocytopenia during treatment ofpulmonary embolism and deep venous thrombosis witheither low-molecular-weight heparin or unfractionatedheparin: a metaanalysis. Chest 2007, 132, 1131–1139.

31 Phillips, C., Kashani, A., Ko, D., Francis, G. & Krumholz,H. Adverse effects of combination angiotensin II receptorblockers plus angiotensin-converting enzyme inhibitors for




left ventricular dysfunction: a quantitative review of datafrom randomized clinical trials. Archives of InternalMedicine 2007, 167, 1930–1936.

32 Ravindran, V., Rachapalli, S. & Choy, E. Safety ofmedium- to long-term glucocorticoid therapy in rheumatoidarthritis: a meta-analysis. Rheumatology 2009, 48, 807–811.

33 Rodrigo, G., Nannini, L. & Rodrıguez-Roisin, R. Safety oflong-acting beta-agonists in stable COPD: a systematicreview. Chest 2008, 133, 1079–1087.

34 Rodrigo, G., Castro-Rodriguez, J., Nannini, L., Plaza Moral,V. & Schiavi, E. Tiotropium and risk for fatal and nonfatalcardiovascular events in patients with chronic obstructivepulmonary disease: systematic review with meta-analysis.Respiratory Medicine 2009, 103, 1421–1429.

35 Siegel, C., Marden, S., Persing, S., Larson, R. & Sands, B.Risk of lymphoma associated with combination anti-tumornecrosis factor and immunomodulator therapy for thetreatment of Crohn’s disease: a meta-analysis. ClinicalGastroenterology and Hepatology 2009, 7, 874–881.


36 Silva, M., Matthews, M. L., Jarvis, C., Nolan, N. M.,Belliveau, P., Malloy, M. & Gandhi, P. Meta-analysis ofdrug-induced adverse events associated with intensive-dosestatin therapy. Clinical Therapeutics 2007, 29, 253–260.

37 Singh, S., Loke, Y. K. & Furberg, C. D. Long-term risk ofcardiovascular events with rosiglitazone: a meta-analysis.JAMA 2007, 298, 1189–1195.

38 Singh, S., Loke, Y. K. & Furberg, C. D. Inhaledanticholinergics and risk of major adverse cardiovascularevents in patients with chronic obstructive pulmonarydisease: a systematic review and meta-analysis. JAMA 2008,300, 1439–1450.

39 Singh, S., Amin, A. V. & Loke, Y. K. Long-term use ofinhaled corticosteroids and the risk of pneumonia in chronicobstructive pulmonary disease: a meta-analysis. Archives ofInternal Medicine 2009, 169, 219–229.

Received 27 July 2011; Accepted 24 January 2012


Documents

The performance of adverse effects search filters in medline and embase