EVIDENCE BASED MEDICINE

Effectiveness of therapy

Ross Lawrenson

Critically appraising a therapy paper

Critical Appraisal of a therapy paper - methodology

• When critically appraising a paper ask yourself three questions:– Are the results valid?

– What are the results?

– Will the results help me in caring for my patients?

• Go to the therapy worksheet for the complete list of questions

I. Are the results valid?

• In other words was this a well designed study in a relevant population. The best study design to answer a therapy question is a randomised controlled trial.

• Go through the worksheet questions 1- 6 to help you decide whether you are likely to believe the results of the paper you are considering.

1. Did the study address a clearly focused question?

• Can you define

– The population they studied

– The intervention

– The comparison group

– The outcomes

2. Was the assignment of patients randomised?

Efficacy versus effectiveness

Efficacy versus effectiveness

• Efficacy - does receiving treatment work under ideal conditions?

Efficacy versus effectiveness

• Efficacy - does receiving treatment work under ideal conditions?

• Effectiveness - does offering treatment help under ordinary circumstances?

Observation versus experimental studies.

Observation versus experimental studies

• A study population of 2000 patients with acute coronary heart disease of whom half receive a certain intervention and the other half do not. Of the 2000 patients, 700 have arrhythmia "X" and 1300 do not.

Observation versus experimental studies

• A study population of 2000 patients with acute coronaries of whom half receive a certain intervention and the other half do not. Of the 2000 patients, 700 have arrhythmia "X" and 1300 do not.

• X(+) = Patients with arrhythmia "X" have a mortality of 50%

• X(-) = Patients without arrhythmia have a mortality of 10%.

Observational study

Intervention No intervention

X(-)800

X(+)200

X(-)500

X(+)500

Deaths 180 300

Relative risk = 0.6

Randomised controlled trial

Intervention No intervention

X(-)650

X(+)350

X(-)650

X(+)350

Deaths 240 240

Relative risk = 1

Absolute risk

• Incidence rate of the outcome in the population (can be the treated or the untreated population).

Relative risk

• Relative risk (RR) is the absolute risk in the treated group divided by the absolute risk in the untreated group (or vice versa)

Randomised controlled trials• Because the randomised trial removes selection bias

the result of the study should be believed over the evidence from the observational study i.e. the Relative risk is 1 (no difference in treatment) not 0.6 (which suggested a benefit from treatment.)

• An example of this would be the use of HRT and the reduction in cardiovascular risk. Observational studies have shown a 50% reduction in CHD but the RCT showed no benefit. (References)

3. Were all patients who entered the trial properly accounted for and attributed at its conclusion?

(a) Was the follow up complete? - selection bias

(b) Were the patients analysed in the groups to which they were randomised? - intention to treat analysis.

Selection biasRandomised controlled trials

Outcomes Outcomes

SamplePopulation

Treatment 1 Treatment 2

(a) Selection of study population

sample

sample trial population

unsuitable (excluded)

(Should be representativeof the general population to ensure external validity)

sample trial population

unsuitable (excluded)

intervention trial completed

adverse events/lost to follow up

sample trial population

unsuitable (excluded)

intervention trial completed

adverse events/lost to follow up

= Possible bias

Sources of selection bias

Non random sample is selected. e.g. Volunteers. Healthy worker. Hospital patients.

Sources of selection biasNon random sample is selected. e.g. Volunteers. Healthy worker. Hospital patients.

Unsuitable patients excluded

Sources of selection bias

Non random sample is selected. Volunteers. Healthy worker. Hospital patients.Unsuitable patients excludedHard to trace people are omitted.

Sources of selection bias

Non random sample is selected. Volunteers. Healthy worker. Hospital patients.Unsuitable patients excludedHard to trace people are omitted.Large number of refusals in the selected population.

Sources of selection bias

Non random sample is selected. Volunteers. Healthy worker. Hospital patients.Unsuitable patients excludedHard to trace people are omitted.Large number of refusals in the selected population.Large number of people dropping out of the study or lost to follow up.

(b) Intention to treat analysis.