Is Peer Review Peerless?

Tony EklofJames Joyce Library

University College DublinLIR Annual Seminar 2009

Peer Review

• What it is • The peer review process• Why peer review• Criticisms• Peer review in the

sciences• Peer review in the

humanities• Conclusions

The Frescoes of Domenico Ghirlandaio:A Study in High Renaissance and Early Mannerist Florence

Journal peer review process

Paper to Editor Editor to Experts in

subject area (Peers)

Peers consider for validity,Significance and originality.

Paper is accepted for publication,Returned with suggestions for improvements,

Or rejected.(resubmitted elsewhere)

Double Blind Review

• Both authors & referees anonymous• Survey shows it is preferred• Support highest in humanities and

amongst female authors (reduces bias)• Causes some problems with reviewers

ie comparing with earlier works• At odds with the open sharing of

information and transparency?

Open Review

• Some researchers now post pre-publication versions on web to invite feedback before formal submission

• Increasingly reader’s comments and criticisms, particularly for open access journals, add a positive element to the process

Why Peer Review?

• Quality control for scholarly information• Weeds out fraud• Lessens workload of Editor• Promotes originality and academic rigour

– Mechanism for improvement of manuscripts

• Adds a ‘human judgement’ element to the academic process

Social TextDuke University Press

Criticisms

• Slows up the research process• Time consuming for reviewers• For some manuscripts or proposals it may

be difficult to find experts qualified to review

• Bias of reviewer• Arbitrary, secret and subjective!• Researchers can be frustrated by process,

valuable time spent on ‘lateral’ research

Nature’s Peer Review Trial

• Open Peer Review• 1 June-30 September 2006• Authors of new papers submitted

invited to have papers hosted on internet for public comment

• Papers simultaneously subjected to standard peer review

Nature’s Open Review

• 71 out of 1,369 authors agreed to display papers for open comment (5%)

• Healthy online traffic, but did not convert into significant amount of useful comment

• Some authors reluctant for fear of ideas being ‘scooped’

• Nature to continue to explore open review but not to implement system

Peer Review in Humanities

• Applying bibliometric indicators in Arts can be problematical

• Very broad range of subjects• Some esoteric and specialised• Interdisciplinary nature• Performing arts difficult to peer review

Conclusion

• Peer review is a flawed, much criticised system of insuring academic rigour in published journal articles

• There is no better alternative system on offer

• System improved by open and or double blind review, and by training for peer reviewers