How to Review a Manuscript_Slides_JS_02.24.2014

Southern California Clinical and Translational Science Institute Translating Science into Solutions for Better Health

How to Review a Manuscript*

Jonathan M. Samet, MD, MS Professor and Flora L. Thornton Chair

Department of Preventive Medicine Director, ECDE, SC CTSI

Career Development Seminar Series Feb 24, 2014

*And respond to reviewers’ comments


Things we did not talk about last week

•  Selecting a journal •  Authorship issues •  Cover letters •  Recommending/not recommending

reviewers •  Acknowledgements •  Conflict-of-interest

SELECTING A JOURNAL


Selecting a Journal •  Should have potential readership and

journal in mind as writing begins – Who will care about your findings? – For whom are they relevant?

•  The impact consideration •  Tiers of journals

– First level: NEJM etc. – Second level: Good ones in your field – Third level: the “undesirables and new

journals


Selecting a journal

•  The tiers: – The big ones: NEJM, Lancet, JAMA, and so on. – The big ones in your field: the American Journal

of whatever and so on. – The little ones in your field: the American

Journal of your particular niche. – The latest on-line creation (beware “predatory

publishing”)


General Guidance

•  If you have something that may be “big” then give the first tier journals a try. They reject quickly (and often without review).

•  Then turn to the best journals in your field—and write for them.

•  And if rejected, start down the hierarchy. •  Should you ever give up? Remember, the

papers are “forever” on your CV.


Impact Factor •  Definition: “In a given year, the impact

factor of a journal is the average number of citations received per paper published in that journal during the two preceding years.”

•  Should you care? Unfortunately, you probably should.

•  Citation analysis increasingly part of promotion process.


Impact Factor: Important? •  Sadly---YES •  For promotion at USC, we are submitting

impact factors for journals along with citation counts

•  In some countries, CVs list journal impact factors and citation counts

•  High impact journals receive great weight at USC


Journal Impact Factor •  Journal Impact Factors are calculated on a yearly basis

by the Journal Citation Reports database (http://wokinfo.com/products_tools/analytical/jcr/)

•  The Impact Factor of a journal is the average number of times that articles published in that journal in a two year period have been cited in the following “JCR year”

2012 Impact Factor

Total citations made in 2012 to papers published in that journal in 2010 & 2011

Total papers published in that journal in 2010 & 2011

=

Impact Factor – All Journals

Impact Factor – Journals by specialty fields (e.g., respiratory system)


H-Index •  The h-index of a publication is the largest

number h such that at least h articles in that publication were cited at least h times each. For example, a publication with five articles cited by, respectively, 17, 9, 6, 3, and 2, has the h-index of 3.

•  The h5-index is the h-index of only those of its articles that were published in the last five complete calendar years.


H-Index for author


Citation analysis for author (e.g., Samet JM)


Citation analysis for author (e.g., Samet JM)


AUTHORSHIP


Authorship Issues •  Who should be an author? •  What about author order? •  Multiple first and senior authors •  Dealing with student authors •  Dealing with disputes about authorship •  Consortium papers

http://www.icmje.org/


ICMJE: Who is an Author? 1. Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND 2. Drafting the work or revising it critically for important intellectual content; AND 3. Final approval of the version to be published; AND 4. Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.


Author Order

•  You might be first author if: – The project is your work – You are the PI and wrote the paper – You are a trainee and carried out the work and

wrote the paper •  You might be the senior author if:

– You are the PI – Your trainee is the first author – A junior colleague is first author


Some examples of authorship •  Pineles BL, Park E, Samet JM. Systematic review and meta-

analysis of miscarriage and maternal exposure to tobacco smoke during pregnancy. Am J Epidemiol. 2014; in press.

•  Peng RD, Samoli E, Pham L, Dominici F, Touloumi G, Ramsay T, Burnett RT, Krewski D, Le Tertre A, Cohen A, Atkinson RW, Anderson HR, Katsouyanni K, Samet JM. Acute effects of ambient ozone on mortality in Europe and North America: results from the APHENA study. Air Qual Atmos Health. 2013; 6(2): 445-453.

•  Giovino GA, Mirza SA, Samet JM, Gupta PC, Jarvis MJ, Bhala N, Peto R, Zatonski W, Hsia J, Morton J, Palipudi KM, Asma S; GATS Collaborative Group. Tobacco use in 3 billion individuals from 16 countries: an analysis of nationally representative cross-sectional household surveys. Lancet 2012; 380(9842): 668-79.


Consortium Papers

•  An emerging problem reflecting team science and consortium mechanisms

•  Issues with consortia: – Positioning yourself on writing committees – Multiple/multiple authorship

•  How will they count for recognition of scholarship and advancement?



Cover Letters •  Not so important •  Opportunity to say why paper is important,

but probably not receive much attention •  With regard to suggesting reviewers:

–  If there is someone to be avoided, then reasonable to let the editor know and

– Giving suggestions for reviewers may be helpful, particularly if the article is in a particular niche


Conflict-of-interest

“A conflict of interest exists when professional judgment concerning a primary interest (such as patients’ welfare or the validity of research) may be influenced by a secondary interest (such as financial gain). Perceptions of conflict of interest are as important as actual conflicts of interest.” (ICJME 2013)


COI and Pat Buffler In 2010, FMC paid Buffler nearly $200,000 in cash and stock. Securities and Exchange Commission records show that when she sold the stock the company gave her, mostly in 2010, Buffler made more than $2 million. A review of public records shows that in publishing her results in scientific journals or in applying for government funding from the National Institutes of Health, Buffler did not disclose that she owned stock in FMC or served as one of its directors.

PEER REVIEW


Peer Review: The Ideal

•  Evaluation of scholarly work by several peers with appropriate expertise.

•  Careful consideration of the comments by an editor who has read the manuscript.

•  Transmission of comments and suggestions for revision to the authors.

•  Revision followed by acceptance


Peer Review: The Reality

•  There are too many journals and too many papers to be reviewed.

•  Typically, editors struggle to identify peer reviewers.

•  Peer reviewers may not do their jobs well. •  Comments from editors may be limited

and give too little guidance.


Peer Review Comments

•  General comments: – Significance of the findings – Methodological issues – Generalizability – Links to other studies – Implications


Peer Review Comments

•  Specific comments: – The gamut – Minor editorial stuff – Points of disagreement – Methodological fine points – Errors


Things I Say

•  Important and makes a contribution (unimportant and repetitive)

•  Substantive methodological concerns (give it up)

•  Additional analyses needed (data are not well analyzed)

•  Findings over-interpreted (Not much here—but!)


Things that annoy me! •  Most annoying: priority claims (This is the

first study of______ or whatever). •  Excessive reliance on statistical

significance. •  Failure to discuss limitations and

generalizability. •  Silly discussions of implications of findings. •  MRIN (more research is needed)


Comments to the editor

•  Not much new here (hint: reject) •  Fundamental problems with the data (hint:

be careful) •  Much needed to make this ready for

publication (hint: be careful) •  Writing needs careful editing (hint: this

manuscript needs a rescue mission) •  New findings and well written (hint: accept!)

An example of my review: Obstructive sleep apnea and systemic hypertension

FIRST REVIEW

SECOND REVIEW


THIRD (AND FINAL) REVIEW


Another example of my review: Chlamydia pneumoniae and asthma

Another example of my review (cont.): Chlamydia pneumoniae and asthma

COMMENTS AND LETTERS FROM THE EDITOR


What I do as editor

•  Initially scan the manuscript to find reviewers.

•  When comments come, carefully read them and go back to the manuscript and read carefully.

•  Make a decision and justify it in the letter. •  If revision, the letter highlights the points

that must be addressed.


Letter from the Editor

•  Should provide a roadmap for interpreting the comments.

•  Provide an indication of those comments that are most critical.

•  Address conflicting comments. •  Provide the editor’s own insights.


Responding to Comments

•  Stay calm and be respectful. •  Express gratitude/thanks. •  Do NOT be combative, even if justified. •  Respond to every comment systematically. •  Provide analyses to support responses. •  Accept minor comments unless wrong to

show your responsiveness.


Responses to Comments

•  Respond to each comment, identified in some systematic way.

•  Embed responses into comments. •  Provide additional analyses and tables,

and consider use of supplement to address concerns.

Example #1: Response to Review – Smoking and TB

Example #1: Manuscript Revision – Smoking and TB

48

Example #2: Response to Review – Smoking and DM

Example #2: Manuscript Revision – Smoking and DM

50

Example #2: Manuscript Revision – Smoking and DM


What to do if rejected! •  Don’t take it personally. •  Do not call the editor or send an email

(generally). •  Look through the comments—take

advantage of the review. •  Quickly set out a strategy for revision and

submission elsewhere. •  Remember that rejection happens to great

work.


Why would your manuscript be rejected?

•  Not notable or newsworthy: the “big journals.”

•  Not novel: any journal. •  Not well done: any journal. •  Not well written: any journal. •  Badly written: any journal.

LETTERS TO THE EDITOR (FROM READERS OF ARTICLE)

Samet, Straif, Schuz, Saracci. Epidemiology 2014;25:23-7


Letters to the Editor


Letters to the Editor – Author’s Response


Letters to the Editor – Author’s Response (cont.)


Exercises


You receive the following comments on your

manuscript. Write responses:


1. While the study findings are of interest, the generalizability of these findings based on Hispanic patients at LAC, is uncertain.


2. This is an elegant experimental model of addiction. However, I hardly see any relevance to alcoholism in people.


3. The main finding, increased risk for diabetes in those drinking more coffee is of interest (RR = 1.8, 95% CI 0.98-2.70) but the p-value (p=0.06) is not significant.

GENERAL RESOURCES

http://www.icmje.org/

http://www.equator-network.org/reporting-guidelines/

Education

How to Review a Manuscript_Slides_JS_02.24.2014