SSC in Evidence Based Medicine - Evaluating the evidence

Paula [email protected]

Senior Academic Liaison Librarian (Medicine and Dentistry)

Guide to

Evaluating the Evidence

Why?

To weigh up how valid and useful

the research will be

Why? – to save time

• In order to keep up to date, clinicians would have to read 17 articles a day, 365 days a year

• Research is of variable quality• Only an estimated 1% is

judged clinically relevant• Need to find the 1%

Publication bias

Papers with “interesting” results are more likely to be:

• Submitted and accepted for publication• Published in a major journal • Published in English• Quoted by authors• Quoted in newspapers

Brainstorm

What factors should you be bearing in mind when reading an article?

Think about• the research described• how it is reported

RCT checklist

How are the results presented?

• Number needed to treat (NNT)• Odds Ratio• Relative risk• Mean difference

Odds and risk

Odds of winning

1:9

You versus the rest

Risk of winning

1:10

You versus all the runners

10 horses running, you bet on 1 horse

Forest plots

more than 1less than 1 1

Forest plots


Line of no effect

Forest plots


Line of no effect

Forest plots


Line of no effect

Best estimate

Forest plots


Line of no effect

Best estimate

Confidence interval

Forest plots


Line of no effect

Best estimate

Confidence interval

Forest plots


Line of no effect

Best estimate

Confidence interval

Pooled result

P-value

Could the result have occurred by chance?

p = 0.001 (1 in 1000)

p = 0.2 (1 in 5)

A p-value of less than 0.05 (1 in 20) is considered to be statistically significant

How it works

• Involves answering a short questionnaire• We use the CASP questionnaires at

http://www.sph.nhs.uk/what-we-do/public-health-workforce/resources/critical-appraisals-skills-programme

• The questionnaires were devised by doctors for doctors

Summary

Validity

Is it trustworthy?

Results

What does it say?

Relevance

Will it help?

Group critical appraisal

1) Did the review address a clearly-focused question?


2) Did the authors look for the appropriate sort of papers?


Is it worth continuing?


3) Do you think the important, relevant studies were included?


4) Did the reviewers do enough to assess the quality of the included studies?


5) If the results of the studies have been combined, was it reasonable to do so?


6) What are the overall result of the reviews?


7) How precise are these results?


8) Can the results be applied to the local population?


9) Were all important outcomes considered?


10) Are the benefits worth the harms and costs?

Health & Medicine

SSC in Evidence Based Medicine - Evaluating the evidence