Peer review – how to deal with with reviewers?

Ana Marušić

Editor in chief, Journal of Global HealthEditor emerita, Croatian Medical Journal

University of Split School of Medicine, Split, Croatia

Workshop: Editorial Process

How to find good reviwers and deal with them?How to obtain better performance from

reviewers?How to deal with completely different views

about the same paper?

Peer review

Ann Emerg Med. 2002 Sep;40(3):323-8.Effect of structured workshop training on subsequent performance of

journal peer reviewers.Callaham ML, Schriger DL.

CONCLUSIONS:• Among invited peer reviewers, voluntary attendance at a highly structured and

interactive workshop was low and did not improve the quality of subsequent reviews, contrary to the predictions of attendees. Efforts to aggressively recruit average reviewers to a second workshop were time consuming, had low success rates, and showed a similar lack of effect on ratings, despite improvement in scores on a test instrument. Workshop teaching formats, although traditional, are of unproven efficacy.

Is training of peer reviewers effective?

PLoS Med. 2007 Jan;4(1):e40.The relationship of previous training and experience of journal peer reviewers to

subsequent review quality.Callaham ML, Tercier J.

CONCLUSIONS:• Our study confirms that there are no easily identifiable types of formal training or

experience that predict reviewer performance. Skill in scientific peer review may be as ill defined and hard to impart as is "common sense." Without a better understanding of those skills, it seems unlikely journals and editors will be successful in systematically improving their selection of reviewers. This inability to predict performance makes it imperative that all but the smallest journals implement routine review ratings systems to routinely monitor the quality of their reviews (and thus the quality of the science they publish).

Is training of peer reviewers effective?

Ann Emerg Med. 2011 Feb;57(2):141-8. Epub 2010 Nov 12.Longitudinal trends in the performance of scientific peer reviewers.Callaham M, McCulloch C.

CONCLUSIONS:• This study, one of few tracking expert performance longitudinally,

demonstrates that most journal peer reviewers received lower quality scores for article assessment over the years. This could be due to deteriorating performance (caused by either cognitive changes or competing priorities) or, to a partial degree, escalating expectations; other explanations were ruled out. This makes monitoring reviewer quality even more crucial to maintain the mission of scientific journals.

Is training of peer reviewers effective?

BMC Med Educ. 2012 Aug 28;12(1):83. [Epub ahead of print]Does mentoring new peer reviewers improve review quality? A randomized

trial.Callaham M, Green S, Houry D.

CONCLUSIONS:• A structured training intervention of pairing newly recruited medical

journal peer reviewers with senior reviewer mentors did not improve the quality of their subsequent reviews.

Is training of peer reviewers effective?

J Clin Epidemiol. 2012 Mar;65(3):247-52. Epub 2011 Nov 8.Medical journal editors lacked familiarity with scientific publication issues

despite training and regular exposure.Wong VS, Callaham ML.

CONCLUSIONS:• Our study presents a current look at editors of major clinical medical

journals. Most editors reported training in medical editing topics, saw ethical issues regularly, and were aware of scientific publication organizations, but their knowledge of four common and well-disseminated publication ethics topics appears poor.

Is training of peer reviewers effective?

Integrity of peer review

World Association of Medical Editors

EUROPEAN ASSOCIATION OF SCIENCE EDITORS

www.councilscienceeditors.org

www.ease.org.uk

www.icmje.org

www.wame.org

www.publicationethics.org.uk

Authors and Reviewers

• Manuscripts must be reviewed with due respect for authors’ confidentiality. In submitting their manuscripts for review, authors entrust editors with the results of their scientific work and creative effort, on which their reputation and career may depend. Authors’ rights may be violated by disclosure of the confidential details during review of their manuscript. Reviewers also have rights to confidentiality, which must be respected by the editor. Confidentiality may have to be breached if dishonesty or fraud is alleged but otherwise must be honored.

• Editors must not disclose information about manuscripts (including their receipt, content, status in the reviewing process, criticism by reviewers, or ultimate fate) to anyone other than the authors and reviewers. This includes requests to use the materials for legal proceedings.

Authors and Reviewers

• Editors must make clear to their reviewers that manuscripts sent for review are privileged communications and are the private property of the authors. Therefore, reviewers and members of the editorial staff must respect the authors’ rights by not publicly discussing the authors’ work or appropriating their ideas before the manuscript is published. Reviewers must not be allowed to make copies of the manuscript for their files and must be prohibited from sharing it with others, except with the editor’s permission. Reviewers should return or destroy copies of manuscripts after submitting reviews. Editors should not keep copies of rejected manuscripts.

• Reviewer comments should not be published or otherwise publicized without permission of the reviewer, author, and editor.

11

Guidelines for reviewers?

• Good resource: BMJ

Resources for reviewers:http://www.bmj.com/about-bmj/resources-reviewers

12http://www.cmj.hr/2005/46/2/15849858.htm

Einstein’s answer to Physical Review in 1936

Obrigada!

ana.marusic@mefst.hr

http://3.bp.blogspot.com/-oLwi2iC6-pk/UAbBLqUQR_I/AAAAAAAABS0/BmcmHauPTtE/s1600/poor-review-and-peer-review.jpg