Upload
christopher-bird
View
220
Download
3
Tags:
Embed Size (px)
Citation preview
META-ANALYSES: APPROPRIATE GROWTH META-ANALYSES: APPROPRIATE GROWTH OR MALIGNANT TUMOR? OR MALIGNANT TUMOR?
Ian ShrierIan Shrier MD, PhD, Dip Sport Med (FACSM)MD, PhD, Dip Sport Med (FACSM)
• Associate Professor, Dep’t of Fam Med, McGill UniversityAssociate Professor, Dep’t of Fam Med, McGill University• Centre for Clinical Epidemiology and Community Studies, Centre for Clinical Epidemiology and Community Studies,
SMBD-Jewish General Hospital and McGill UniversitySMBD-Jewish General Hospital and McGill University• Past-president, Canadian Academy of Sport MedicinePast-president, Canadian Academy of Sport Medicine
Canadian Academy of Sport MedicineL’Académie Canadienne de Médecine du Sport
GROWTH VS QUALITY?
• Cochrane Collaboration – 10 pointsCochrane Collaboration – 10 points collaborationcollaboration building on the enthusiasm of individualsbuilding on the enthusiasm of individuals avoiding duplicationavoiding duplication minimizing biasminimizing bias keeping up to date keeping up to date striving for relevance striving for relevance promoting access promoting access ensuring qualityensuring quality continuity continuity enabling wide participationenabling wide participation
• Cochrane Collaboration – 10 pointsCochrane Collaboration – 10 points collaborationcollaboration building on the enthusiasm of individualsbuilding on the enthusiasm of individuals avoiding duplicationavoiding duplication minimizing biasminimizing bias keeping up to date keeping up to date striving for relevance striving for relevance promoting access promoting access ensuring qualityensuring quality continuity continuity enabling wide participationenabling wide participation
GrowthGrowthGrowthGrowth
GROWTH VS QUALITY?
• Cochrane Collaboration – 10 pointsCochrane Collaboration – 10 points collaborationcollaboration building on the enthusiasm of individualsbuilding on the enthusiasm of individuals avoiding duplicationavoiding duplication minimizing biasminimizing bias keeping up to date keeping up to date striving for relevance striving for relevance promoting access promoting access ensuring qualityensuring quality continuity continuity enabling wide participationenabling wide participation
• Cochrane Collaboration – 10 pointsCochrane Collaboration – 10 points collaborationcollaboration building on the enthusiasm of individualsbuilding on the enthusiasm of individuals avoiding duplicationavoiding duplication minimizing biasminimizing bias keeping up to date keeping up to date striving for relevance striving for relevance promoting access promoting access ensuring qualityensuring quality continuity continuity enabling wide participationenabling wide participation
GrowthGrowthGrowthGrowth
QualityQualityQualityQuality
XX XX
DATA ENTRY
OBJECTIVE?
A Meta-analysis is an Observational Study, A Meta-analysis is an Observational Study,
Subject to Selection Bias, Confounding, Subject to Selection Bias, Confounding,
and Measurement Error Like Any Other!and Measurement Error Like Any Other!
A Meta-analysis is an Observational Study, A Meta-analysis is an Observational Study,
Subject to Selection Bias, Confounding, Subject to Selection Bias, Confounding,
and Measurement Error Like Any Other!and Measurement Error Like Any Other!
Platt, personal communicationPlatt, personal communication
Computer programs and standardized scoring make it Computer programs and standardized scoring make it very easy to make mistakes that are difficult for the very easy to make mistakes that are difficult for the reviewer to pick up.reviewer to pick up.
Computer programs and standardized scoring make it Computer programs and standardized scoring make it very easy to make mistakes that are difficult for the very easy to make mistakes that are difficult for the reviewer to pick up.reviewer to pick up.
OVERVIEW
• Inclusion Criteria / Data abstractionInclusion Criteria / Data abstraction
• AnalysesAnalyses
• Interpretation of EvidenceInterpretation of Evidence
• Inclusion Criteria / Data abstractionInclusion Criteria / Data abstraction
• AnalysesAnalyses
• Interpretation of EvidenceInterpretation of Evidence
INCLUSION CRITERIA - SEARCHINCLUSION CRITERIA - SEARCH
• Search StrategySearch Strategy ““Does Vitamin C prevent death due to cancer in the Does Vitamin C prevent death due to cancer in the
general population”general population”
• Search StrategySearch Strategy ““Does Vitamin C prevent death due to cancer in the Does Vitamin C prevent death due to cancer in the
general population”general population”
33,30033,300Vitamin CVitamin C#1#1
HitsHitsTermsTermsSearchSearch
34,41534,415Ascorbic AcidAscorbic Acid#2#2
9,5459,545AscorbateAscorbate#3#3
4,3764,376#3 NOT #2#3 NOT #2#4#4
42,09842,098#1 OR #2 OR #3#1 OR #2 OR #3#5#5
INCLUSION CRITERIA – RCTs?
Good (56%)
Poor (44%)
INCLUSION CRITERIA – RCTs?
Furlan, 2006Furlan, 2006Furlan, 2006Furlan, 2006
DATA ABSTRACTION
• Peer-review for stretching article:Peer-review for stretching article:
““The RR 0.79 … was based on the article in Pope The RR 0.79 … was based on the article in Pope 2000.”2000.”
• Peer-review for stretching article:Peer-review for stretching article:
““The RR 0.79 … was based on the article in Pope The RR 0.79 … was based on the article in Pope 2000.”2000.”
… … multivariate analysis HR=1.04 (0.82,1.33)multivariate analysis HR=1.04 (0.82,1.33) … … multivariate analysis HR=1.04 (0.82,1.33)multivariate analysis HR=1.04 (0.82,1.33)
• Actual articleActual article
The X article … Cox regression … differing The X article … Cox regression … differing exposure times for different subjects. …univariate exposure times for different subjects. …univariate hazard ratio of 0.95 (CI: 0.77,1.18)hazard ratio of 0.95 (CI: 0.77,1.18)
• Actual articleActual article
The X article … Cox regression … differing The X article … Cox regression … differing exposure times for different subjects. …univariate exposure times for different subjects. …univariate hazard ratio of 0.95 (CI: 0.77,1.18)hazard ratio of 0.95 (CI: 0.77,1.18)
OVERVIEW
• Inclusion Criteria / Data abstractionInclusion Criteria / Data abstraction
• AnalysesAnalyses
• Interpretation of EvidenceInterpretation of Evidence
• Inclusion Criteria / Data abstractionInclusion Criteria / Data abstraction
• AnalysesAnalyses
• Interpretation of EvidenceInterpretation of Evidence
ANALYSESANALYSES
• Effect ModifiersEffect Modifiers Study-level confoundersStudy-level confounders Subject-level confoundersSubject-level confounders
• Publication BiasPublication Bias Papers with significant results are more Papers with significant results are more
likely to be published (i.e. selected)likely to be published (i.e. selected)
• Reporting BiasReporting Bias Outcomes that are statistically significant Outcomes that are statistically significant
are more likely to be reported in publications are more likely to be reported in publications (i.e. selected)(i.e. selected)
• Effect ModifiersEffect Modifiers Study-level confoundersStudy-level confounders Subject-level confoundersSubject-level confounders
• Publication BiasPublication Bias Papers with significant results are more Papers with significant results are more
likely to be published (i.e. selected)likely to be published (i.e. selected)
• Reporting BiasReporting Bias Outcomes that are statistically significant Outcomes that are statistically significant
are more likely to be reported in publications are more likely to be reported in publications (i.e. selected)(i.e. selected)
ANALYSESANALYSES
Fixed vs RandomFixed vs RandomFixed vs RandomFixed vs Random
Effect EstimateEffect EstimateRR vs ORRR vs OR
Effect EstimateEffect EstimateRR vs ORRR vs OR
ANALYSESANALYSES
Effect EstimateEffect EstimateRR vs RDRR vs RD
Effect EstimateEffect EstimateRR vs RDRR vs RD
OVERVIEW
• Inclusion Criteria / Data abstractionInclusion Criteria / Data abstraction
• AnalysesAnalyses
• Interpretation of EvidenceInterpretation of Evidence
• Inclusion Criteria / Data abstractionInclusion Criteria / Data abstraction
• AnalysesAnalyses
• Interpretation of EvidenceInterpretation of Evidence
GRADE SCALE DECISIONS
Atkins. BMC Health Services Research 2005Atkins. BMC Health Services Research 2005Atkins. BMC Health Services Research 2005Atkins. BMC Health Services Research 2005
4
3
2
1
RecommendationBalance Benefit vs HarmRater
8
7
6
5
12
11
10
9
13
17
16
15
Toss up
Do it
Don't do it
Probably do it
Do it
Do it
Do it
Probably do it
Probably do it
Probably do it
Probably do it
Do it
Toss up
Don't do it
Toss up
Probably do it
Net benefit
Net benefit probably
Uncertain net benefit
Net benefit
Net benefit
Net benefit
Net benefit
Trade offs
Trade offs
Net benefit
Net benefit
Net benefit
Net benefit
Not net benefit
Uncertain net benefit
Net benefit 14
Reasons:• Lack of information in
summaries• Missing baseline risk• What is sparse information?
Reasons:• Lack of information in
summaries• Missing baseline risk• What is sparse information?
INTERPRETATIONSINTERPRETATIONS
I believe magnesium has now been shown to be beneficial for I believe magnesium has now been shown to be beneficial for patients during the post-MI periodpatients during the post-MI period (SD-SA) (SD-SA)
Rev 8Rev 8
Rev 7Rev 7
Rev 6Rev 6
Rev 5Rev 5
Rev 4Rev 4
Rev 3Rev 3
Rev 2Rev 2
Ag
DA
Ag
Ag
DA
DA
SA
Ag
DA
-
Ag
Ag
DA
-
SA
Ag
SD
-
Ag
Ag
DA
-
Ag
AgRev 1Rev 1
II22
Rand. ORRand. OR
Fixed ORFixed OR
Ag
DA
Ag
Ag
SD
SD
Ag
Ag
59%59%
0.75 (0.61-0.92)0.75 (0.61-0.92)
1.01 (0.96-1.07)1.01 (0.96-1.07)
69,50569,505
59%59%
0.65 (0.48-0.87)0.65 (0.48-0.87)
1.02 (0.96-1.08)1.02 (0.96-1.08)
63,04763,047
-
DA
Ag
Ag
Ag
-
SA
Ag
21%21%
0.66 (0.53-0.81)0.66 (0.53-0.81)
0.64 (0.52-0.79)0.64 (0.52-0.79)
3,6853,685
56%56%
0.38 (0.21-0.66)0.38 (0.21-0.66)
0.40 (0.28-0.61)0.40 (0.28-0.61)
597597
0%0%
0.40 (0.18-0.86)0.40 (0.18-0.86)
0.40 (0.19-0.83)0.40 (0.19-0.83)
415415
SD
DA
-
-
SD
SD
Ag
SD
n/an/a
n/an/a
n/an/a
111111NN
1-231-231-201-201-101-101-51-51-31-311# RCTs# RCTs
CONCLUSION• The explosion of meta-analyses has occurred The explosion of meta-analyses has occurred
because of “apparently simple recipes”because of “apparently simple recipes”
• However, a meta-analysis is an observational study However, a meta-analysis is an observational study with potential errors at many levelswith potential errors at many levels
Authors of meta-analyses need to be appropriately Authors of meta-analyses need to be appropriately trainedtrained
Readers need to be Readers need to be appropriatelyappropriately skeptical skeptical
• The benefits of meta-analyses may be lost if current The benefits of meta-analyses may be lost if current growth remains unrestricted and becomes a growth remains unrestricted and becomes a malignancymalignancy
• The explosion of meta-analyses has occurred The explosion of meta-analyses has occurred because of “apparently simple recipes”because of “apparently simple recipes”
• However, a meta-analysis is an observational study However, a meta-analysis is an observational study with potential errors at many levelswith potential errors at many levels
Authors of meta-analyses need to be appropriately Authors of meta-analyses need to be appropriately trainedtrained
Readers need to be Readers need to be appropriatelyappropriately skeptical skeptical
• The benefits of meta-analyses may be lost if current The benefits of meta-analyses may be lost if current growth remains unrestricted and becomes a growth remains unrestricted and becomes a malignancymalignancy
Canadian Academy of Sport MedicineL’Académie Canadienne de Médecine du Sport