Reviewing it Right!

NACCQ 2007NACCQ 2007

Research Workshop Research Workshop Reviewing it Right! Reviewing it Right!

Dr Clare AtkinsSchool of Business & Computer TechnologyNelson Marlborough Institute of Technology

[email protected]

Background

NACCQ conference 2004Panel session – Demystifying the reviewing processThanks to all panel members

NACCQ Research Workshop 2004Reviewing Conference and Journal Papers Thanks to Tony Clear (AUT) and Carmel McNaught (CUHK)

Editor – New Zealand Journal of Applied Computing and ITEditor – NMIT Occasional Paper SeriesEditorial Board – Journal of Database ManagementReviewer for various international conferences.

Overview

What is reviewing?

How to do a useful review

How to get accepted as a reviewer

What is reviewing?What is reviewing?

Everyone has an opinion on what reviewing should be…or what makes a good reviewer….! So let’s start by determining the playing field!

How many of you have had work reviewed?

How many of you have had work ‘blind’ reviewed?

How many of you have reviewed fora journal?a conference?an international journal or

conference?a book?

(Activity)


Purpose to recommend whether to provide a public platform for the

‘submitter(s)’ to put forward their work. (to stop people getting egg on their faces!)

Issues for the Editor maintain (or enhance) the credibility and standing of the

public platform (conference/journal etc….) protect credibility of the presenter will it attract interest from others (will they subscribe to the

journal, come to the conference, link to the website)

So reviewers are the people who assist the editor to make the decision

The Editorial Board are those people who confirm the editor’s decision (not

reviewers)


Why is a reviewed paper considered more ‘prestigious’ than a non-reviewed paper?

Objective evaluation Judged to be of value by your peers

What does an editor want from a reviewer?

A clear recommendation! Reasoned arguments! Something useful to say to the author – particularly if reject

What does an author want from a reviewer?

Praise!! Constructive criticism New ideas for content or structure To be taken seriously. Reasons for recommendation……


What does it take to be a ‘good’ reviewer?

Ability to remain as objective as possible Reasonable content knowledge Good understanding of structure Good understanding of what the journal or

conference requires Good understanding of the audience Good understanding of the preferred style Ability to be kind but critical at the same time. Patience! Ability to know when you don’t know enough…. Ability to summarise your comments and

thoughts

How to write a useful reviewHow to write a useful review

What does it take to write a useful review?

There are a number of things that you might need before you start to write a review….

Working with a partner, brainstorm some of the things you might need or do before beginning to write your review……

Now

Write them as a list and prioritise them……


What does it take write a useful review?

Things I might need before I start….

A decent cup of coffee! Previous copy of the journal or call for papers

of conference or call for chapters for book Copy of paper (watch out for lost appendices!) List of questions/criteria from editor or Checklist of things you think are important Sometimes it can be useful to have some

reference material e.g. • Greenhalgh, T. (1997) How to read a paper. BMJ

Publishing Group, London At least an hour of uninterrupted time……


Questions to ask yourself

What is this paper about? Why was this paper written? What type of research was done? Was the type of research appropriate to the

question/hypothesis? Is the research method well described? Is the topic (or findings) interesting, original,

innovative? Is the paper well structured? Does the paper do what it promises at the

beginning?(Aside)

What do reviewers get asked?What do reviewers get asked?

Usually editor will provide questions for the specific reviewing process – if not ask for them!

Each journal/conference will have its own set.

You will almost always be asked whether a paper should be

• accepted as is,

• accepted with major revisions,

• accepted with minor revisions,

• rejected

You have to make that call and be able to justify it

What do reviewers get asked?What do reviewers get asked?

Does this paper represent a completed piece of work?

Is the research methodology sound ?

Is previous relevant research acknowledged/utilised?

Are the findings interesting/original?

Are the findings clearly based on an analysis of the results?

Is the structure and the language of the paper appropriate?

Is it appropriate to the conference/journal

Is it of interest/value to the community

Are there modifications to be made before publication?

Any comments to feedback to the author?

Any comments to feedback to the editor?

How to start ……How to start ……

Read abstract, introduction and conclusionGet a feel for the paper, the general area, the type of

research, the writing style. Remember many readers will read no more than that.

Scrutinise the referencesDo the references look reasonable, i.e. up-to-date, good

coverage, any obvious holes, on-line? Read from beginning to end

Don’t be critical just read! Try to get into author’s head! Try to enjoy

Have another cup of coffee!Seriously! Take some short time out to reflect on what

you have read.

Now you are underway ……Now you are underway ……

Read from beginning to end – againLook for things you missed first time throughWhy did you miss them? How do they affect your overall

judgement? Write a short summary of what the paper is

aboutAlthough this is for your eyes only at this stage it will help

you be clear about the topic that is covered Write a short summary of what the author did

Again this is only for you initially but it will help you focus on what the author did.

Check for consistency between the two summariesThis will tell you a lot about the quality of the paper

Now you are refining your thoughts…..Now you are refining your thoughts…..

By this stage you have probably formed your opinion!but

It is now useful to check the “5 Rs”** Relevance,

To audience, to community, to journal/conference Rigour,

Does the research have methodological rigour Readability,

Use of jargon, good English, good structure Requirements,

Does it meet with the editorial requirements – Length etc.. ‘Riginality

Does it have something new to say?

**Thanks to participants of research workshop in Hamilton for these!

Now write them up …..Now write them up …..

Write summary of what paper is about for editor Use your original summaries to prepare a short paragraph

Answer any questions that editor gave you You’ve done this already – just write it up

Write comments to editor You can be pretty free with what you say here Highlight areas where you are not sure or that are

outside your area of expertise Declare any bias here! Don’t try to be clever just be clear.

Now write them up …..Now write them up …..

Write comments to author This is your chance to help the author improve their

paper Be kind but firm! Don’t let self-doubt creep in Always try to begin with something positive Always try to indicate that you have understood the

major points Point out the revisions that are required (if any) If you have recommended a reject explain why and

provide hope!

Make final judgement on approve/reject

How to get accepted as a reviewer …..How to get accepted as a reviewer …..

Often all you need to do is ask! Pick a journal or conference that you would like to

publish in• Have a good look through past papers• Email editor and offer your services

Make contact with someone in your discipline area who already reviews and offer…..

Find out if there are email lists relating to your area, subscribe and keep an eye out for calls for reviewers

Watch out for conferences and again approach the programme chair or proceedings editor and offer…..

Tell me! Just occasionally you may be asked to do a trial review…

Final thoughtsFinal thoughts

Reviewing is only scary the first time you do it! After that it’s just stressful!

Reviewing is a really good way of improving your own ‘hit rate’ for publication

Reviewing makes you much more open to suggestions from your own reviewers!

Reviewing helps you to build networks of people researching in similar areas

Reviewing keeps you in touch with interesting work in your areas of interest

Reviewing builds your self-confidence and knowledge

GO! DO!


Working with the person next to you come up with some answers to these questions….

What is the purpose of reviewing?

Why is a reviewed paper considered more ‘prestigious’ than a non-reviewed paper?

What does an editor want from a reviewer?

What does an author want from a reviewer?

What does it take to be a ‘good’ reviewer?

In 5 words what has been your experience of • being reviewed?• being a reviewer?

Back

Structure of a Research Article

Beginning – What I’m going to talk about

Middle - Talking about it

End – What I talked about!

Yes anyone can do it!!

http://www.thesimpsons.com/

I do not know what it means to be an American because I am from Mars.

On Mars we don't have any houses or teachers. All you do on Mars is ride around in Space Bubbles like in Radioactive Man. You never have to go home on Mars because your Space Bubble is like your house. It's got a TV and a big refrigerator which makes whatever food you feel like eating. The other thing about Space Bubbles which isn't in the comic books is that you can do tricks like flipping it upside down or on its side while you're flying at a gazillion miles an hour. Also, there is no school on Mars.

That is what it means to be an American. Back

http://animatedtv.about.com/gi/dynamic/offsite.htm?site=http%3A%2F%2Fhometown.aol.com%2Fgene168%2Fessay.html

Education

Reviewing it Right!