InterCESS: How EBM principles apply to the

Internet

Andrew Booth & Louise Falzon

The Ingredients of EBM• Focused questions• Hierarchy of

evidence• Filters• Problem based

learning– Scenarios– Anatomy of the

question– Shared expertise

Focusing the question

• Helping requester to break information need down into EBM anatomy clarifies the originating question.

• “OR” within categories and “AND” between categories.

Understanding study design• Adds to your

ability to recognise “good studies”and “bad studies”.

• Helps you to identify the question being answered by an article.

Using methodological filters• One line, sensitive and specific

versions• rely on publication types or occurrence

of textwords to identify methodologically superior studies

• are also used to compile pre-filtered database products e.g.. Cochrane Library (clinical trials), DARE (reviews) & NHS EED (economic evaluations)

EBM and the Internet

Why is the Internet important for EBM?

• New paradigm - rapid but simultaneous learning

• Publishing medium• Delivery mechanism• Database access• Teaching packages

What are the dangers of the Internet for EBM?

• quality of information

• lack of context• no responsibility for

revision• size and lack of

control

Challenges for today

• Can the Internet be used to answer patient focused scenarios?

• Are there valuable documents that can supplement our hard copy collections?

• Is “focusing the question” applicable to the WWW?

• Can we devise the equivalent of “filters”?• Can we use the WWW for teaching EBM?

The Programme

• Databases for Clinically Effective Practice

• Full-text reports on the Web• One-stop shops• Focusing the question using

general search engines

Which databases?

• For Therapy questions - Trials Registers• For Guidelines - Guidelines Databases• For Reviews - DARE, Cochrane, -

HealthSTAR• For Economic Studies - NHS EED, CDC• For ALL question types - PubMed,

Research Registers, Health Technology Assessment Reports

Full-Text reports on the WebIssues:• How do you locate?• How do you prioritise

sites to check?• Executive summary?• Order facility?• All of the document?• All documents?

• Document Types:– HTML (In pieces?)– Word 6.0 (Viewer

available from Microsoft site)

– Word 8.0– Adobe Acrobat

(pdf) (Acroread viewer available from Adobe)

Issues around Report contents• Whole report on a

general subject but chapter on your topic e.g. Drug therapy for MS

• Generic names instead of proprietary names

• Classes of drugs e.g lipid-lowering agents

• Solutions: Do some general Internet searching on topic first. What is it? What else is it called? Is there patient information on topic? Has there been FDA approval (drugs)?etc.

Issues around report producers• Who are they?

– National HTA agencies

– Regional HTA agencies

– National R&D projects or programmes

– Academic centres

• But also?– Drug company

sponsored research units/health deliverers

– Non-profit making or health maintenance organisations (but...) e.g. Group Health Northwest

Two tools to help you

HTA Database NHSCRD York

• Records for reports of all INAHTA members. Some of these on the Web, others sent on demand.

• Search Engine/ Common CRD interface

• http://nhscrd.york.ac.uk/

INAHTA Current Projects in Progress

• Brief title entry for all projects being undertaken by INAHTA members.

• Crude search engine• Requires follow-up to Web

sites• http://

nzhta.chmeds.ac.nz/ inahta/inahta.htm

One-stop shops

• TRIP Database – http://www.tripdatabase.com/

• SUMSearch – http://sumsearch.uthscsa.edu/

• NELH PC – http://www.nelh-pc.nhs.uk/

One Stop Shops cont

• Medline Pro– http://www.medlinepro.com/

• PubMed Clinical Queries – http://www.ncbi.nlm.nih.gov/entrez/

query/ static/clinical.html

Using a general search engine

Focusing the question

A cautionary tale I:

• Systematic search using Yahoo and Excite search engines for sites relating to feverish children.

• Of 41 Web pages identified only four compared favourably with recommendations of published guidelines

• Impicciatore et al BMJ 1997; 314: 1875-1877.

Search Engines

General• AltaVista• Excite• HotBot• Infoseek• Lycos• Northern Lights • Yahoo

Multi-search• Inference Find• Megasearch• Meta Crawleretcetera

A Cautionary tale - II

• The Web contains an estimated 320 million pages of information. HotBot covers 34 %, AltaVista 28 %, Northern Light 20 %, Excite 14 %, Infoseek 10 % and Lycos 3 %.

• "Combining six engines in this study covered about 3.5 times more of the Web".

• Steve Lawrence and C. Lee Giles. Searching the World Wide Web," Science, 280, April 3, 1998, p.98-100.

An Evidence Based Conclusion• "Given that the coverage of any one

search engine is limited, the simplest means of improving the coverage of Web searches is to combine the results of multiple engines, as is done with new search engines such as MetaCrawler (www.metacrawler.com)," they wrote.

Cp. Also Megasearch engine at www.dogpile.com

However...........

• Mega-search engines do not usually use Boolean logic. They are good for a search on a single distinctive name e.g. celecoxib

• For a focused question it is better to use a single search engine with Boolean e.g. Alta Vista Advanced (www.altavista.com)

• Use Intervention AND Population (then Filter, then Outcomes/Comparisons)

Example

Effectiveness/Cost effectiveness of Propentofylline in Vascular and Alzheimer's Disease

• POPULATION= AD/Vascular Disease • INTERVENTION= Propentofylline [28]• FILTERS=“technology assessment”[2],

“economic*”[4], “clinical trial*” [9]

Tips & Tricks [Alta Vista]• Finding HTA

materials– Combine with a filter

such as “technology assessment” or “clinical trials”

• Finding images– Combine with image

command:

image:hernia

• Limiting to a specific domain e.g. only sites at a UK academic institution– Combine with host

command: host:ac.uk

Another filtered approach• Use Publication

Type field in www.omni.ac.uk

• Accessed via MeSH• Includes:

– Practice Guideline– Clinical Trial– Meta-analysis– RCT

In Conclusion

• It is possible to practice EBM using the Internet

• But it is much cruder and more time consuming

• Nevertheless plucking a jewel from the sewage may just make someone’s day!