IT 499: Seminar CourseWeek 4

Faculty: Dr. Afshan Jafri

6 March 2012

2

Outline

1. What is research?

2. How to prepare yourself for IT research?

3. How to identify and define a good IT research problem?

- Research Area

- Research Question / Topic

4. How to solve it?

- Research methods

- Research phases

5. How to write and publish an IT paper?

6. Research Ethics

Why write and publish?• Reasons for writing down

– to remember, because once something is forgotten, it cannot be reproduced correctly without having written notice

– to understand, as writing about a subject can only be accomplished by approaching the subject in a structured way, which itself leads to better understanding

– to gain perspective, as writing includes looking at something from different point of views

• Reasons for publishing research results

– some results are worth reporting

– progress scientific thought

– want your work to reach a broad audience

– improve your chance of promotion

Types of Journal Manuscript

• Original Research (Research Article, Article, Original Article):

– Most common type of journal manuscript

– Its format is suitable for many different fields and different types of studies

– The format includes full Introduction, Methods, Results, and Discussion sections

• Rapid Communications (letter):

– Communicates findings that stimulate further research in the field

– Usually published soon after submission to the journal

– Useful for scientists with results that are time sensitive (highly competitive or quickly-changing disciplines)

– Has strict length limits, so some experimental details may not be published until the authors write a full Original Research manuscript

Types of Journal Manuscript (2)

• Review Articles:

– Provides a a comprehensive summary of research on a certain topic

– Gives perspective on the state of the field and where it is heading

– Often written by leaders in a particular discipline after invitation from the editors of a journal

– Reviews are often widely read (for example, by researchers looking for a full introduction to a field) and highly cited

– Reviews commonly cite approximately 100 primary research articles

• Case Studies:

– Reports specific instances of interesting phenomena

– Make other researchers aware of the possibility that a specific phenomenon might occur

– This type of study is often used in medicine to report the occurrence of previously unknown or emerging pathologies

How to choose a target journal?Factors to consider are:

• The topics the journal publishes

– If your research is applied, target a journal that publishes applied science

– If it is clinical, you should target a clinical journal

– If it is basic research, you should target a journal that publishes basic research

• The journal’s target audience

– If you think researchers in other fields will be interested in your study, a journal that covers a broad range of topics may be best

– If only researchers in your field are likely to want to read your study, then a field-specific journal would be best

• The types of articles the journal publishes

– If you want to publish a Review Article, find out whether the journal publishes these

– If you wish to present a case study or a theorem, ensure that the journal you are targeting actually publishes the type of manuscript you wish to write

How to choose a target journal? (2)• Length restrictions

– Does the journal limit the number of words in the articles it publishes?

– Can your manuscript meet its requirements?

• The reputation of the journal

– A journal’s impact factor is only one measure of its reputation, but not always the most important

– Consider the prestige of the authors that publish in the journal

– Size of the journal’s readership

– Objectively consider how important your research is and what level of journal it is best suited for?

Preparing and Writing for Publication

Davis, M.: Scientific Papers and Presentations. Academic Press, San Diego (1997)

Paper Organization

Title

• Effective title should:

– identify the main issue of the paper

– Highlight the importance of the research

– begin with the subject of the paper

– are accurate, unambiguous, specific, and complete

– do not contain abbreviations (unless they are well known by the target audience, such as WWW or CPU), and

– attract readers

– be concise

Abstract• The Abstract is:

– A summary of the content of the journal manuscript

– A time-saving shortcut for busy researchers

– A guide to the most important parts of your manuscript’s written content

– A standalone section

• Write a concise abstract. An abstract should tell:

– Motivation: Why do we care about the problem and the results?

– Problem statement: What problem is the paper trying to solve and what is the scope of the work?

– Approach: What was done to solve the problem?

– Results: What is the answer to the problem?

– Conclusions: What implications does the answer imply? Why are these findings useful and important?

Keywords

• Keywords are a tool to help indexers and search engines find relevant papers. Keywords must be chosen carefully. They should:

– Represent the content of your manuscript

– Be specific to your field or sub-field

IntroductionIntroduction should guide the reader to current state-of-the-art allow the reader to understand the rest of the paper without referring to previous publications on the topic.

• How can I capture readers interest?

• What is my question really asking?

• Why is this question important?

• How did I collect information and data?

• What method of analysis will I use?

• How will the rest of the paper be organized?

• How can I introduce the next paragraph?

Introduction (2)• Establish a territory:

– bring out the importance of the subject and/or

– make general statements about the subject and/or

– present an overview on current research on the subject

• Establish a niche:

– oppose an existing assumption or

– reveal a research gap or

– formulate a research question or problem or

– continue a tradition

• Occupy the niche:

– sketch the intent of the own work and/or

– outline important characteristics of the own work

– outline important results

– give a brief outlook on the structure of the paper

BODY

• The body of a paper answers two questions, namely how was the research question addressed:

– Materials and Methods (answer the research question or problem identified)

– Results (what was found )

• Comprises of several subsections and the actual structure, organization, and content depends heavily on the type of paper.

• Check the ‘Instructions for Authors’ for your target journal to see how manuscripts should present the Materials and Methods. Also, as another guide, look at sample reports published online by the journal.

Body (2)The body of the

• Empirical paper:

– describes the material and data used for the study

– methodologies applied to answer the research questions

– results obtained

– describes the experiments in a way that makes it possible to reproduce it

• Case study paper:

– describes the application of existing methods, theory or tools

– crucial is the value of the reflections abstracted from the experience and their relevance to other designers or to researchers working on related methods, theories or tools.

Body (3)The body of the

• Methodology papers:

– describes a novel method which may be intended for use in research or practical settings (or both)

– Should be clear about the intended audience

• Theory papers:

– describes principles, concepts or models on which work in the field (empirical, experience, methodology) is based

– positions the ideas within a broad context of related frameworks and theories

– supports originality or soundness of the analysis

– Provides relevance of the theoretical content to practice and/or research in the field

RESULTS

• State what you found

• Do not interpret the results or discuss their implications

• Use subheadings to separate the results of different experiments

• Results should be presented in a logical order (order of importance)

• Use the past tense to describe your results; however, refer to figures and tables in the present tense

• Do not duplicate data among figures, tables, and text

• A common mistake is to re-state much of the data from a table in the text of the manuscript

• Use the text to summarize what the reader will find in the table, or mention one or two of the most important data points

• It is usually much easier to read data in a table than in the text

Discussion• Discuss your conclusions in order of most to least important.

• Compare your results with those from other studies: Are they consistent? If not, discuss possible reasons for the difference.

• Mention any inconclusive results and explain them as best you can. You may suggest additional experiments needed to clarify your results.

• Briefly describe the limitations of your study to show reviewers and readers that you have considered your experiment’s weaknesses.

• Discuss what your results may mean for researchers in the same field as you, researchers in other fields, and the general public. How could your findings be applied?

• State how your results extend the findings of previous studies.

• If your findings are preliminary, suggest future studies that need to be carried out.

• At the end of your Discussion and Conclusions sections, state your main conclusions once again.

References

References serve many purposes in a scientific manuscript.

• Establish where ideas came from

• Give evidence for claims (Justify the claims)

• Connect readers to other research

• Provide a context for your work

• Show that there is interest in this field of research

Referencing SystemsThe most commonly used referencing systems are:

• Name and Year System: References are cited by their respective authors and the year of publication, e.g., \Chuck and Norris (2003) define .....“ This system is very convenient for authors, as the citation does not have to be changed when adding or removing references from the list. The fact that sentences become hard to read when subsequently citing many references in one single parenthesis this way is one negative aspect for readers.

• Alphabet-Number System: This system lists the references in alphabetical order and cites them by their respective number in parentheses or (square) brackets, e.g., \As reported in [4], ...." This system is relatively convenient for readers, as it does not break the flow of words while reading a sentence with many citations. On the other hand, the author has to keep an eye on the references cited in the text as their numbers may change when the reference list is updated.

• Citation Order System: This system is similar to the alphabet-number system with one major difference: the reference list is not sorted alphabetically, but in the order of appearance (citation by number) in the text.

Reference Managers• Help you keep track of publications that you have reviewed

• Make it easy to organize and format citations

• Assist with managing libraries containing citations

• Organize manuscripts by subject

• Search your library using keywords

• Help you remember to appropriately cite manuscripts that you have read

• Some commonly uses reference managers are:

– BibTex

– EndNote

– LaTex

– Mendeley

– RefWorks

Formatting your manuscript• Before writing a complete draft of your manuscript, select an initial target journal.

• Read the formatting requirements for the journal on its website, THEN write your draft.

• Review all guidelines and ensure that your manuscript meets them: – Obeyed all word and character limits (title, running title, abstract, manuscript text)?

– Included all required sections?

– Met language requirements (US or UK English)?

– Supplied all requested contact information?

– Inserted figures in the correct location (in text, end of manuscript, separate files)?

– Correctly formatted references?

– Used the correct file format for your images (.jpg, .png, .pdf, .ppt)?

– Stated ANY conflicts of interest?

– Obtained consent from ALL authors?

Why is good writing important?

• Good writing clearly communicates your research

• Good writing is simple writing

• High-quality, simple writing:

– Increases the chances of acceptance for publication

– Increases the impact of a manuscript in the research community

– Accelerates understanding and acceptance of the research

– Increases the faith of readers in the quality of the research

Publication Types

Type of Publications

• Conference paper

• Workshop paper

• Journal article

• Technical report

• Masters or PhD thesis

• Monograph (research report)

• (Unpublished working paper)

Conference Paper• Call for papers (about 1 year before meeting)

• Submission (due 4-8 months before meeting)

• page limit (say 10 pages)

• details often omitted (eg proofs, design technicalities)

• Reviews by Program Committee

• check reasonableness, significance, originality, readability

• selection based mainly on interest to the community

• Final version for proceedings (due 3-4 months before meeting)

• revise by author in light of reviews

• but not checked again (except for a few top systems conferences)

• Oral presentation at meeting

• Annual or bi-annual conferences

Selection Process• When ranking conferences, you should look at the following factors:

– rate, review process, program committee, who the publisher of the proceedings is, and which database is indexing the published proceedings.

• Identify the top conferences in your area

• Affiliation (ACM, IEEE, etc..)

• Acceptance rate and review process

• Typically 3 reviewers

• Acceptance rate – varies

– Some 10-15%, others 50%

• Some review “blind” (author details not shown to reviewers), others do not

– Example – a reviewer’s form

Workshop Paper

• A workshop is typically a smaller meeting than a conference

• Sometimes workshop papers are just like conference papers

• Other workshops are more preliminary

• can publish a position paper (draft of an idea without evidence, or proposal for future work)

• less rigorously reviewed, the goal is mainly to allow the community to meet

Journal Article• Typically longer than a conference paper

• Submitted

• often based on a conference paper with additions, corrections, improvements

• usually an account of a contribution, but sometimes a survey that integrates a field

• Refereed by

– at least 3 reviewers, experts in the field

– they spend months on the paper checking details, etc.

• Decisions: accepted, accepted with minor revisions, major revisions and resubmission, rejected

– Revisions, refereed again

– Accepted, published after several months (journal issues have limited capacity)

• Time from submission to publication varies, typically 1-1.5 years but may be 3-4 years

Standard of Journals

• Many journals in each area with different standards

• Typically IEEE Transactions and ACM Communications are some of the top-ranked journals

• Not all IEEE Trans. and ACM Comm. are top journals

Technical Report

• Issued by the author’s department, with a number and date

• May be based on a conference paper

• Include all the boring details that are omitted from conference due to space limits

• Used to establish priority

• eg produce TR before submitting work to others

MSc and PhD Thesis

• Very extensive account

• Shows much of the research process

• Extensive survey of the literature

• Complete evaluation of the author’s work

• Establishes that the author is ready to become independent researcher in the community

• Typically checked by 2 or 3 readers

Monograph

• A collection of selected papers from a conference or workshop

– A bit more checking than for the conference/workshop

• An author can offer a coherent and unified account of a whole research topic

– often combines their own results with other people’s

– Revisits several papers using unified notation, better exposition, better literature review, etc.

– Publisher may get reviewers but their focus is “will it sell” not “is it correct”

REVIEW PROCESS

Conference Review Process

• Hard to generalize the review process for conferences, but most reputable conferences operate according to these basic rules:

• Paper is submitted to the technical program chair(s). Many current conferences require electronic submission, in either PostScript or PDF formats, occasionally in Word.

• Technical program chair (TPC) assigns the paper to one or more technical program committee members. TPC member identity is kept secret.

• TPC member usually provides a review, but may also be asked to find between one and three reviewers who are not members of the TPC.

• Any good conference will strive to provide at least three reviews, however, since conferences operate under tight deadlines and not all reviewers deliver as promised, it is not uncommon that you receive only two reviews.

Journal Publications• A journal paper reports on a finished piece of research or some significant achievement or

discovery in a certain scientific field. Unlike at conferences it is uncommon that international journals accept reports on research in progress at an early stage.

• Roles Involved. The main roles involved in the journal editing and publishing process and their responsibilities are:

• Referee

– Each journal has an editorial board that includes a number of referees (also known as reviewers) who are responsible for reviewing and evaluating submitted papers

– Each referee independently advises the editor whether to accept or to reject the paper. This is usually done using a peer review form provided by the editor

– However, final decisions are made by the editor

• Editor (Associate Editor)

– Make the final decision whether to accept or to reject a submitted paper

– Comments of the referees just serve as suggestions

– Editor usually relies on the advice of his editorial board

Journal Publications (2)• Managing Editor (Editor-in-Chief)

– Post-acceptance issues are taken care of by managing editors

• Publisher

– Publishers print accepted papers in (periodical) journal issues.

– Most journals appear quarterly, but there are also journals which appear monthly or bi-monthly.

– After the publisher receives the final version of an accepted paper, it is prepared for printing.

– A preview of the typeset paper to be published is then sent to authors, who check the so-called \page proofs" for any errors that survived the editing and typesetting stages. T

– This process is called proof reading; after the author is finished with proof reading the paper is finally ready to go to print.

Process of publishing a paper in a journal

What do journal editors want?

• Journal editors evaluate submissions and consider peer reviewers’ advice and make the final decision about what gets published.

• Journal editors want to publish manuscripts that

– Describes research that advances the field

– Is carefully prepared and formatted

– Uses clear and concise language

– Follows ethical standards

Citation Counts• Publication quality is just one aspect of research quality

• Impact is one aspect of publication quality

• Number of citations is one aspect of impact

• Citation counts rely on databases such as ISI, CiteSeer, ACM Digital Library, Scopus, and Google Scholar

• They have limitations

• An issue of concern to computer scientists is the tendency to use publication databases that do not adequately cover CS, such as Thomson Scientific’s ISIWeb of Science

ISI Citation Database• Very popular, established in 1960, contains >40million records, contains

– Arts and Humanities Citation Index (A&HCI)

– Science Citation Index (SCI)

– Social Sciences Citation Index (SSCI)

– Coverage is over 80% for physiscs or chemistry

– 38% for CS

• However

– it doesn’t index a large number of journals

– It does not index many top conferences

– ignores open-access journals

– ISI does not index many top conferences (for instance, The International Conference on Software Engineering (ICSE), the top conference in the field) but indexes SIGPLAN Notices, an unrefereed publication

• Should not make cross disciplinary comparisons based on number of citations.

Journal’s Impact Factor

• Journal impact factors

– Used to determine the importance of a journal

– E.g. journal impact factor for 2007 =# citations in 2007 to articles published in the journal in 2005-6= -------------------------------------------------------------------------------# articles published in the journal in 2005-6

• Check CS journal impact factors on ISI Web of Knowledge! (electronic journals -> J->Journal Citation Report)

Important CS journals and conferences

Here are examples of important journals and conferences, for different CS domains:

• Database: IEEE Trans on Knowledge and Data Engineering, ACM Trans on Database Systems, Int’l Conf on VLDB.

• Software Engineering: IEEE Trans on Software Engineering, ACM Trans on Software Eng. And Methodology, IEEE Int’l Conf on Software Engineering.

• Computer Networks: IEEE/ACM Trans on Networking, IEEE INFOCOM, ACM Mobicom.

• Parallel/Distributed Systems: IEEE Trans on Parallel and Distributed Systems, ACM Trans on Computer Systems, ICDCS, IPDPS.

• Neural Networks: IEEE Trans. on Neural Networks, Neural Computation, NIPS, IJCNN, ICANN, IWANN, ESANN.

RESEARCH ETHICS

Ethical principles• Professional ethics

– ACM Code of Conduct

– ACS Code of Conduct

• ACM

http://www.acm.org/about/code-of-ethics/

• IEEE

http://www.ieee.org/portal/pages/iportals/aboutus/ethics/code.html

• IEEE’s Plagiarism FAQ:

– http://www.ieee.org/web/publications/rights/plagiarism_FAQ.html

• ACM’s Plagiarism Policy:

– http://www.acm.org/publications/policies/plagiarism_policy

Publication Ethics• Ethical standards for publication exist to ensure high-quality scientific publications,

public trust in scientific findings, and that people receive credit for their ideas. It is important to avoid:

• Data fabrication and falsification:

– Data fabrication means the researcher did not actually do the study, but made up data.

– Data falsification means the researcher did the experiment, but then changed some of the data.

– Both of these practices make people distrust scientists. If the public is mistrustful of science then it will be less willing to provide funding support.

• Plagiarism:

– Taking the ideas and work of others without giving them credit is unfair and dishonest.

– Copying even one sentence from someone else’s manuscript, or even one of your own that has previously been published, without proper citation is considered plagiarism—use your own words instead.

Publication Ethics (2)• Multiple submissions:

– It is unethical to submit the same manuscript to more than one journal at the same time.

– Doing this wastes the time of editors and peer reviewers, and can damage the reputation of journals if published in more than one.

• Redundant publications (or ‘salami’ publications):

– This means publishing many very similar manuscripts based on the same experiment.

– It can make readers less likely to pay attention to your manuscripts.

• Improper author contribution or attribution:

– All listed authors must have made a significant scientific contribution to the research in the manuscript and approved all its claims.

– Don’t forget to list everyone who made a significant scientific contribution, including students and laboratory technicians.

– The International Committee of Medical Journal Editors has detailed guidelines on authorship that are useful for scientists in all fields

Credits• Basics of Research Paper Writing and Publishing?, Michael Derntl RWTH Aachen University,

Information Systems & Databases

• INFO4990: Information Technology Research Methods; Mary Lou Maher; Adapted from a lecture by A. Fekete; August 2005

• Research Ethics; Lecture by Alan Fekete, based in part on slides by Judy Kay

• R. Andonie, I. Dzitac; How to Write a Good Paper in Computer Science and How Will It Be

• Measured by ISI Web of Knowledge; Int. J. of Computers, Communications & Control, ISSN 1841-9836, E-ISSN 1841-9844; Vol. V (2010), No. 4, pp. 432-446