Department of Epidemiology & Public Health

  • View
    23

  • Download
    0

Embed Size (px)

DESCRIPTION

Department of Epidemiology & Public Health. Criticisms and concerns regarding the STROBE statement: a review of commentaries, editorials and letters Patrizia Frei, PhD Swiss Tropical and Public Health Institute, Basel. Overview. Review of 34 commentaries, editorials, letters on STROBE - PowerPoint PPT Presentation

Text of Department of Epidemiology & Public Health

  • Department of Epidemiology & Public HealthCriticisms and concerns regarding the STROBE statement: a review of commentaries, editorials and letters

    Patrizia Frei, PhDSwiss Tropical and Public Health Institute, Basel

    Prsentationstitel

  • Overview

    Review of 34 commentaries, editorials, letters on STROBE

    Focus on the criticism/concerns regarding STROBEGeneral Specific

    Prsentationstitel

  • STROBE: judging study quality?Concern that STROBE will be used as a tool for judging the quality of a study (Editors of Epidemiology, 2007; Keiser and Egger, 2008; Meerpohl et al., 2007)STROBE should expressly discourage the use of its guidelines for the evaluation of studies (Editors of Epidemiology, 2007)

    Strobe ventures into the matter of study conduct (Editors of Epidemiology, 2007)It should restrict its function to the improvement of reports on epidemiological studies (Editors of Epidemiology, 2007)

    Prsentationstitel

  • STROBE: recommendations becoming a law?Concern that STROBE may be considered prescriptive or dogmatic and that journals/decision-making authorities will turn checklist into requirements (Rothman and Poole, 2007; Kohlmann, 2008)Guidelines should be formulated more as suggestions than as rigid rules (Rothman, 2007)

    Prsentationstitel

  • Constraint to scientific creativityConcerns about any effort to formalize reporting in a field as heterogeneous as observational epidemiology (Editors of Epidemiology, 2007; Nijsten et al., 2008)STROBE represents a breach in the capacity for scientific creativity; standardization of research (Potvin, 2008; Meerpohl, 2009)STROBE may eventually make our field of research more boring (Potvin, 2008)

    STROBE should give clear emphasis to the importance of good judgement over any specific criteria. We should not allow the blindly applied rule to trump the creative exception (Editors of Epidemiology, 2007)Incorporate expiration dates into STROBE (Rothman and Poole, 2007)

    Prsentationstitel

  • Polished appleThe basic principles of epidemiology should not be introduced only at the time of reporting (MacMahon and Weiss, 2007)Some authors might be tempted to describe what should have been done, rather than what was done (MacMahon and Weiss, 2007; Battegay, 2008; Keiser and Egger, 2008)There is always a risk that poorly designed studies will be made more difficult to spot by superficial improvements in the way they are reported (Rothwell and Bhatia, 2007)

    Prsentationstitel

  • Other general points (1)The suggestions are rather general, which can be confusing (Ebrahim and Clarke, 2007; Nijsten et al., 2008)There are odd remnants of clinical-trial thinking in STROBE, STROBE should acknowledge the differences between RCTs and observational studies (Editors of Epidemology, 2007)More thorough purging of inapplicable RTC concepts, and a clear statement of the crucial distinctions between RCTs and observational studies (CONSORT-STROBE) (Editors of Epidemiology, 2007)Negative effects on the review process (MacMahon and Weiss, 2007; Bagettay, 2008)More discussion on how observational epidemiology can be made more robust (Ebrahim and Clarke, 2007)

    Prsentationstitel

  • Other general points (2)The criteria for selecting items that were ultimately included in the STROBE checklist are not clear (Nijsten et al., 2008)Inclusion of only three study designs (Ebrahim and Clarke, 2007; Knotterus and Tugwell, 2008)Statement about extent of any input from people with relevant clinical expertise might be an additional future STROBE recommendation (Rothwell and Bhatia, 2007)STROBE guidelines nevertheless incorporate a reliance on statistical significance testing into the canon of reporting (by examples) (Item 12b: were explored by testing for biologic interaction according to Rothman) (Rothman and Poole, 2007)

    Prsentationstitel

  • Specific criticism: Title and AbstractTitle and Abstract

    Item 1a: TitleRecommendation to include study design in the title. Given the use of key words for indexing, this is not essential (Sellers, 2008)

    Item 1b: Abstract, E&ELimitations are mentioned in the abstract. But abstracts are usually very short, therefore it is not practical (Sellers, 2008)

    Prsentationstitel

  • Specific criticism: Introduction/MethodsIntroductionNot enough emphasis on presenting a clear definition of the hypothesis in the introduction, its biologic rationale, and its implication to the health of the public (Kuller and Goldstein, 2007)

    MethodsIt is not mentioned that a careful definition of host factors is essential, including a clear statement of results specific to race, sex and ethnicity (Kuller and Goldstein, 2007) The definition and prespecification of outcomes is crucial, particularly in cohort studies, where composite outcomes are often used to increase statistical power (Rothwell and Bhatia, 2007)

    Prsentationstitel

  • Specific criticism: MethodsItem 7: Variables (E&E)STROBE recommends an inclusion of an appendix with a list of variables. Although this is helpful, it adds excess pages, with consequent effects on cost and number of articles we can publish. Maybe better idea: creation of study website (Sellers, 2008)

    Item 10: Study sizeExplain how the study size was arrived at not bother readers with post hoc justifications for study size or retrospective power calculations (E&E); this is contradictory (Rothman and Poole, 2007)

    Prsentationstitel

  • Specific criticism: MethodsItem 12cMaybe the STROBE group could take an initiative promoting a more systematic approach to missing values in observational studies? (Thelle, 2009)

    Prsentationstitel

  • Specific criticism: ResultsItem 16c: Main resultsConsider translating estimates of relative risk into absolute risks for a meaningful time period instead of absolute risk, it should say risk difference or absolute change in risk (Rothman and Poole, 2007)

    Item 19: LimitationsSTROBE does not say anything about strengths, only limitations (EpiMonitor, 2007)

    Prsentationstitel

  • Specific criticism: ResultsStudies should be required to present the actual rates or number of events, as well as HRs and relative risk. Multiple regression analyses should provide information about the number of individuals and events related to each of the independent variables (Kuller and Goldstein, 2007)It should be stated that the p values should be restricted only to those hypotheses that were generated prior to the data analysis (Kuller and Goldstein, 2007)Importance of reporting data on the precision of measurement of the exposure(s) under study deserves particular emphasis (Rothwell and Bhatia, 2007)

    Prsentationstitel

  • Specific criticism: DiscussionItem 20: InterpretationInstead of Cautious overall interpretation of results: give an interpretation of the results appropriate to the available data (Ebrahim and Clarke, 2007)Richard Doll: If you find something that is unexpected and is going to be of social significance you have a responsibility to be sure that youre right before you publicize your results to the rest of the world. This does at least require repeating some of your observations (instead of cautious overall interpretation) (Ebrahim and Clarke, 2007)

    Prsentationstitel

  • Specific criticism: DiscussionItem 21: Generalisability (E&E)A discussion of the existing external evidence is particularly important for studies reporting small increases in risk -> but studies with small risks are not nearly as dangerous as reports of large increases/decreases in risk! More external evidence is required in order to protect the public from false claims if high risks are presented. (Ebrahim and Clarke, 2007)

    Prsentationstitel

  • Specific criticism: DiscussionSTROBE does not include that a health warning or recommendations should accompany observational publications (Lloyd, 2007)

    The evidence to be considered in the discussion should include for and against the biologic plausibility of the findings (incubation period, temporal relationship between associations) (Bradford Hill on biologic plausibility -> not mentioned in the STROBE checklist, only in 1 paragraph in the full STROBE document) (Kuller and Goldstein, 2007)

    Prsentationstitel

    Study conduct (example): It is unfortunate that STROBE ventures into matters of study conduct, with (for example) debatable opinions on the conditions under which matching should be done. Reporting guidelines should avoid attempts to define what constitutes good epidemiology.-> vor allem Methoden beschrieben wie auszuwerten wenn so oder so gematched. Nicht: so soll man genau matchen.Rothman and Poole: We have always abhorred (verabscheuen) the thought of conducting or reporting science by checklist. Most worrisome is the ease with with journals and other decision-making authorities will be tempted to turn checklists into requirements the template for all your future publications. We hope that journals resist this regimentation. Nevertheless, understanding that it may happen, as it did with the CONSORT guidelines, we suggest that guidelines should be formulated more as suggestions than as rigid rules.Cannot be compared to the narrowly constructed clinical trials (CONSORT). STROBE may be too prescriptive for a field as heterogenerous as observational studies or that it may have counterproductive effects on authors and reviewers of observational studies It s possible that rigid applications of the principles underlying observational studies will lead to losses in the discovery potential of those methods and in the richness in the possibilities they open up for future investigations (19) Jede Leitlinie stellt zunchst den Konsensus einer Gruppe von Experten zu einem be