Compliance

Original Study Design

Randomised

Surgical care Medical care

Reality - unplanned cross overs

Surgical care Medical care

Refuse surgery

Require surgery

Intention to treat analysis• In this example of unplanned cross overs it may be that elderly or high

risk patients are refused surgery whilst some of the fitter, younger patients, although randomised to medical treatment actually end up having the surgical intervention. This will cause a bias - there will be more elderly patients in the medical group and they will have a worse prognosis. By analysing the results using intention to treat this bias will be avoided. If there is still a treatment effect then this is likely to be a true effect. It is still worth while analysing by actual treatment groups - this should reveal an even better outcome with treatment. However if the intention to treat shows no benefit, and the analysis by treatment group shows a positive effect then the reviewer should question whether the result is due to bias and loss of randomisation.

Reality - poor compliance

Medical care Placebo

Refuse treatment

Refuse treatment

Intention to treat analysis• With placebo controlled trials it has been shown that compliant patients

who take their placebo have a better outcome (up to 30% better) than the non-compliant patients. If there is a large drop out in both the active and placebo arms of the trial it is attractive to analyse only those who received the active treatment (discarding the non-compliant patients in the active arm) but include all the patients entered into the placebo arm to increase the precision of the results. If the “active” treatment is actually of no benefit, because the non-compliant patients (who have worse outcomes) are only included in the placebo arm then the ”active” treatment may falsely appear to be of benefit.. Intention to treat analysis removes this bias.

4. Were patients, health workers and study personnel “blind” to

treatment?• In a well designed randomised trial the

person giving one of two (or more) possible treatments should not know which treatment the patient is receiving. In a double blind trial the patient should also not know which treatment they are receiving.

5. Were the groups similar at the start of the trial?

• In the paper there should be a table showing the characteristics of the two treatment groups. Sometimes by chance, particularly in small studies, the groups may be unequal (e.g. more men in one group) and this can cause bias.

• The larger the study the more likely the groups are to be similar and the less likely the difference between the groups will be due to chance. If 20 characteristics are looked at then by chance at 0.05 level, we would expect a significant difference in one characteristic between the groups.

Study size

• The larger the study the more likely the groups are to be similar and the less likely the difference between the groups will be due to chance. Thus big studies (mega trials) are to be preferred. This will also help avoid Type 1 and Type 2 error.

Type 1 and Type 2 ErrorGroups aredifferent

Groups are notdifferent

Concludegroups aredifferent

CorrectDecision

Type 2 error

Concludegroups are notdifferent

Type 1 error Correctdecision

Statistical Power

• Relative frequency with which a true difference of specified size between populations would be detected by the proposed study.

Statistical Power

• Relative frequency with which a true difference of specified size between populations would be detected by the proposed study.

• It is equal to 1 minus the probability of Type 2 error.

Sample Size• Difference in response rates to be determined• An estimate of the response rate in one of the

groups• Level of statistical significance• The value of the power desired• Whether the test should be one-sided or two-

sided

6. Aside from the experimental interventions, were the groups

treated equally?• This can sometimes be a problem,

particularly if one treatment group is followed up more intensively. The better outcomes may then be due to something that is occurring in the follow up consultations rather than be due to the original intervention.

II. What are the results?

1. What are the overall results of the study?

II. What are the results?

1. What are the overall results of the study?– Look at the Relative Risk (RR) of the

main outcome in the two groups.

II. What are the results?

1. What are the overall results of the study?– Look at the Relative Risk (RR) of the

main outcome in the two groups.– What about sub-group analyses?

What about sub-group analyses?• First look at the intention to treat analysis.• You may also want to look at the results

in the groups that actually received the treatment.

• Is the result the same in men and women? For different age groups? Smokers and non-smokers etc.

II. What are the results?

1. What are the overall results of the study?– Look at the Relative Risk (RR) of the main

outcome in the two groups.– What about sub-group analyses?– Can you calculate the Number Needed to

Treat (NNT) from the results presented?

Number needed to treat

• NNT is 1/ARR• ARR = Absolute risk reduction

Absolute risk reduction

• Absolute risk reduction (ARR) is the absolute risk in the untreated group minus the absolute risk in the treated group

• (see example)

II. What are the results?

2. How precise are the results?

II. What are the results?

• How precise are the results?• Give both p values and confidence

intervals for each result.

Confidence intervals• A 95% confidence interval (95% CI) is the

range within which, were the study to be repeated the true result would occur 95% of the time. When looking at a relative risk, if the 95% CI contains 1 then the results are not significantly different. A confidence interval is equal to + or - 1.96 times the standard error

p-values

• p-values indicate the likelihood that the ‘Null hypothesis’ is true. i.e. that there is no difference between the results. A p-value less than 0.05 is by convention considered significant but it gives you no idea of the range of the likely true result.

III. Will the results help me in caring for my patients?

1. Can the results be applied to my patient care?

Will the results help me in caring for my patients?

1. Can the results be applied to my patient care? Clinical significance.

• Refer back to the clinical problem• Are the studies generalisable to our patient?• Age, ethnicity, community or hospital

patients etc?

Will the results help me in caring for my patients?

2. Were all the clinically relevant outcomes considered?

Will the results help me in caring for my patients?

2. Were all the clinically relevant outcomes considered?

• What about other outcomes - particularly harm.

• What about quality of life issues?

Will the results help me in caring for my patients?

3. Are the benefits worth the harms and costs?

Will the results help me in caring for my patients?

3. Are the benefits worth the harms and costs?• Cost differences in treatments.• Greater benefits and less side effects?

Example

• If 2000 patients with mild hypertension are randomly allocated to treatment or placebo and 4 patients in the placebo group have had a CVA at the end of the year and only 2 in the treated group have suffered a CVA what are:

Absolute riskTreated groupAbsolute riskuntreated groupRelative risk(RR)Absolute risk reduction(ARR)Number needed to treat(NNT)

Absolute riskTreated group

2/1000

Absolute riskuntreated group

4/1000

Relative risk(RR)

0.5

Absolute risk reduction(ARR)

2/1000

Number needed to treat(NNT)

500