Workshop 5: Journal Club & Review

LIFE SCIENCES LIBRARY

Summer/Fall 2012

Chantal Cassis, MDRobin Featherstone, MLISFrancesca Frati, MLISRoland Grad, MDCM, MSc, FCFP

Jewish General Hospital Library

Workshop 5 - Objectives

By the end of the workshop, you will be able to:1. Present clinical EBM summaries to your

peers2. Critically reflect on the practical application

of a clinical study3. Describe benefits and challenges to

integrating EBM into clinical practice

Workshops

July 25 - Introduction to EBM for Haematology

Aug 8 - Hands-on Searching WorkshopsAug 22 - Critical AppraisalSept 5 - Resident PresentationsSept 19 - More Resident Presentations +

Review

EBM Process

Appraising the Evidence

Incorporating evidence into decision-making

Evaluating the Process

Formulating the clinical question

Searching the Evidence

Your patient for whom you are uncertain about therapy, diagnosis, or prognosis

Workshop1

Workshop2

Workshop3Workshop

4

Workshop5

Journal Club

Scenario – What was your clinical scenario?PICO - What was your PICO? What type of question is it? What type of study will best answer this question?Search strategy – Which of the 7 resources from workshop 2 did you search? What was your strategy? What study did you select? Critical appraisal – How well did your selected study fare when judged according to the criteria on your appraisal worksheet? Did the study answer your PICOImplication for practice – Would you use this evidence to inform your practice? If yes, then how? If no, then how else could you answer your question?

Therapy (RCTs)1

1. Are the results valid?

A. Did intervention and control groups start with the same prognosis? Were patients randomized? Was group allocation concealed? Were patients in the study groups similar with respect to known

prognostic variables?

B. Was prognostic balance maintained as the study progressed? To what extent was the study blinded?

C. Were the groups prognostically balanced as the study progressed? Was follow-up complete? Were patients analyzed in the groups to which they were first allocated? Was the trial stopped early? 1. http://jamaevidence.com/criticalAppraisalWorksheet/27

Therapy (RCTs), cont.

2. What are the results?

A. How large was the treatment effect? What was the relative risk reduction? What was the absolute risk reduction?

B. How precise was the estimate of the treatment effect? What were the confidence intervals?

Therapy (RCTs), cont.

3. How can I apply the results to patient care?

A. Were the study patients similar to my population of interest? Does your population match the study inclusion criteria ? If not, are there compelling reasons why the results should not apply to

your population?

B. Were all clinically important outcomes considered? What were the primary and secondary endpoints studied? Were surrogate endpoints used?

C. Are the likely treatment benefits worth the potential harm and costs? What is the number needed to treat (NNT) to prevent 1 adverse

outcome or produce 1 positive outcome? Is the retention of clinical endpoints worth the increase of cost and risk

of harm?

Harm (Cohort Studies, Case-Control Studies)2

1. Are the results valid?

Cohort Studies: Aside from the exposure of interest, did the exposed and control groups start and finish with the same risk for the outcome?

Were patients similar for prognostic factors known to be associated with the outcome (or was statistical adjustment done)?

Were the circumstances and methods for detecting the outcome similar? Was the follow-up sufficiently complete?

Case-Control Studies: Did the cases and control group have the same rise (chance) for being exposed in the past?

Were cases and controls similar with respect to the indication or circumstances that would lead to exposure?

Were the circumstances and methods for determining exposure similar for cases and controls?

2. http://jamaevidence.com/criticalAppraisalWorksheet/23

Harm (Cohort Studies, Case-Control Studies), cont.

2. What are the results?

A. How strong is the association between exposure and outcome?

What is the risk or odds ratio? Is there a dose-response relationship between exposure and outcome?

B. How precise was the estimate of the risk? What is the confidence interval for the relative risk or odds ratio?

Harm (Cohort Studies, Case-Control Studies), cont.

3. How can I apply the results to patient care?

A. Were the study subjects similar to your patients or population?

Is your patient so different from those included in the study that the results may not apply?

B. Was the follow-up sufficiently long? Were study participants followed-up long enough for important harmful

effects to be detected?

C. Is the exposure similar to what might occur in your patient? Are there important differences in exposures (dose, duration, etc.) for

your patients?

D. What is the magnitude of the risk? What level of baseline risk for the harm is amplified by the exposure

studied?

E. Are there any benefits known to be associated with the exposure?

What is the balance between benefits and harms for patients like yours?

Summarizing the Evidence (Systematic Reviews)3

1. Are the results valid?A. Did the review explicitly address a sensible clinical

question? Is the underlying biology or sociology such that, across the range of

interventions and outcomes included, the effect should be similar? Did the review include explicit and appropriate eligibility criteria?

B. Was the search for relevant studies detailed and exhaustive? Were sources of evidence and search strategies specified in sufficient

detail for replication Was the likelihood and direction of publication bias considered?

C. Were the primary studies of high methodologic quality? Were clear methodological selection criteria specified? Were all included studies accessed by these criteria?

D. Were selection and assessments of studies reproducible? Was an explicit approach used to select and extract data from all

included studies? Was study selection and assessment validated by a blinded second

observer? 3. http://jamaevidence.com/criticalAppraisalWorksheet/26

Summarizing the Evidence (Systematic Reviews), cont.

2. What are the results?

A. Were the results similar from study to study? How similar were the point estimates? Do confidence intervals overlap between studies?

B. What are the overall results of the review? Were results weighted both quantitatively and qualitatively in summary

estimates?

C. How precise were the results? What is the confidence interval for the summary or cumulative effect

size?

Summarizing the Evidence (Systematic Reviews), cont.

3. How can I apply the results to patient care?

A. Were all patient-important outcomes considered? Did the review omit outcomes that could change decisions?

B. Are any postulated subgroup effects credible? Were subgroup differences postulated before data analysis? Were subgroup differences consistent across studies?

C. What is the overall quality of the evidence? Were prevailing study design, size, and conduct reflected in a summary

of the quality of evidence?

D. Are the benefits worth the costs and potential risks Does the cumulative effect size cross a test or therapeutic threshold?

Review

Discussion Questions:1. How did EBM improve your practice? 2. What challenges did you face integrating EBM

into your practice? 3. Will EBM continue to influence your practice?

Slides available: http://www.slideshare.net/featherr