The use and abuse of The use and abuse of citationscitations

Robert West & Kerstin SteniusRobert West & Kerstin Stenius

www.isaje.netwww.isaje.netwww.parint.orgwww.parint.org

OutlineOutline

What are citations for?What are citations for? What is good citation practice?What is good citation practice? What are the common abuses?What are the common abuses? Why does it matter?Why does it matter? ConclusionsConclusions

What are citations for?What are citations for?

Referential citationReferential citation– to refer to a piece of research output to refer to a piece of research output

that contributes to the fieldthat contributes to the field Critical citationCritical citation

– to refer to research to draw to refer to research to draw attention to its flawsattention to its flaws

Citation functionsCitation functions

Verification functionVerification function– to allow the reader to examine the cited to allow the reader to examine the cited

work to check the veracity of a statement work to check the veracity of a statement that it is being made in the citing paperthat it is being made in the citing paper

Acknowledgement functionAcknowledgement function– to give credit to the cited authorsto give credit to the cited authors

Documentation functionDocumentation function– to document how a political debate or to document how a political debate or

historical process or a specific concept has historical process or a specific concept has developed and been defineddeveloped and been defined

Good citation practiceGood citation practice

1.1. Every substantive statement that relies on Every substantive statement that relies on another source should use a citationanother source should use a citation

2.2. Citations should not be used in support of Citations should not be used in support of obvious common-sense statementsobvious common-sense statements

3.3. The choice of cited sources should be based The choice of cited sources should be based on scientific considerations not convenience on scientific considerations not convenience or other biasesor other biases

4.4. Cited sources should be checked to make Cited sources should be checked to make sure they are being accurately reportedsure they are being accurately reported

5.5. Where there are conflicting findings this Where there are conflicting findings this must be reflected in the citationsmust be reflected in the citations

Bad citation practice 1Bad citation practice 1

Selectively citing sources that one happens to Selectively citing sources that one happens to be aware of: be aware of: e.g. papers from one’s research e.g. papers from one’s research groupgroup

Selectively citing oneself or one’s friends or Selectively citing oneself or one’s friends or work from your own country: work from your own country: e.g. a tendency e.g. a tendency for US researchers to cite only US papersfor US researchers to cite only US papers

Selectively citing sources that support one’s Selectively citing sources that support one’s opinions: opinions: e.g. failing to acknowledge that e.g. failing to acknowledge that evidence is conflictingevidence is conflicting

Citing sources that cannot be accessed: Citing sources that cannot be accessed: e.g. e.g. unpublished conference presentationsunpublished conference presentations

Bad citation practice 2Bad citation practice 2

Citing sources of evidence that have not been Citing sources of evidence that have not been evaluated by peer review and that do not give evaluated by peer review and that do not give enough detail to evaluate properly, e.g. enough detail to evaluate properly, e.g. conference abstractsconference abstracts

Citing without reading e.g. relying on secondary Citing without reading e.g. relying on secondary sourcessources

Citing sources in support of statements when in Citing sources in support of statements when in fact they do not do so, e.g. misrepresenting the fact they do not do so, e.g. misrepresenting the conclusions of the cited sourceconclusions of the cited source

Over-using citations to increase the apparent Over-using citations to increase the apparent erudition of one’s writing, e.g. very long lists of erudition of one’s writing, e.g. very long lists of citations for a single point or using a citation in citations for a single point or using a citation in support of some common sense statement)support of some common sense statement)

Getting citedGetting cited

Construct a title that includes Construct a title that includes important key wordsimportant key words

Let your colleagues know about Let your colleagues know about your paperyour paper

Choose journals that are read by Choose journals that are read by people in your fieldpeople in your field

Choose journals that are indexed Choose journals that are indexed in databases used by people in in databases used by people in your fieldyour field

Citation counts as Citation counts as quality markersquality markers Number of citations is used as a marker Number of citations is used as a marker

of quality and reputationof quality and reputation This is misleading becauseThis is misleading because

– bad citation practices fail to give due credit to bad citation practices fail to give due credit to researchers, particularly those outside the USresearchers, particularly those outside the US

– research in innovative areas will receive few research in innovative areas will receive few citationscitations

– critical citations also get countedcritical citations also get counted– it does not reflect the importance of the it does not reflect the importance of the

research for policy or clinical practiceresearch for policy or clinical practice– reviews are typically cited more than reviews are typically cited more than

empirical articlesempirical articles

Impact factorsImpact factors

Journals are judged by their journal Journals are judged by their journal impact factor (JIF)impact factor (JIF)

The JIF for 2004 is the number of The JIF for 2004 is the number of citations in 2004 to articles in 2002 and citations in 2004 to articles in 2002 and 2003 divided by the number of articles in 2003 divided by the number of articles in 2002 and 20032002 and 2003

Although, within a research area JIF Although, within a research area JIF correlates to some degree with correlates to some degree with reputation of journals, across areas it is reputation of journals, across areas it is very misleading because of differences in very misleading because of differences in publication lag, amount of research publication lag, amount of research activity and citation practicesactivity and citation practices

ConclusionsConclusions

Citations provide the vehicle for Citations provide the vehicle for transmission of information in sciencetransmission of information in science

Bad citation practice undermines science Bad citation practice undermines science and at its worst is little better than faking and at its worst is little better than faking resultsresults

Bad citation practice also distorts science Bad citation practice also distorts science by establishing reputations that are not by establishing reputations that are not deserved and failing to credit researchers deserved and failing to credit researchers who deserve itwho deserve it

Citation counts are a poor index of quality Citation counts are a poor index of quality but we appear to be stuck with thembut we appear to be stuck with them

Summary of things to Summary of things to look forlook for1.1. Is there a pro-US bias in the citation list?Is there a pro-US bias in the citation list?2.2. Is there evidence of self-serving citation bias?Is there evidence of self-serving citation bias?3.3. Are substantive statements being made that should be Are substantive statements being made that should be

supported by citations but are not?supported by citations but are not?4.4. Are citations being made to statements that do not need Are citations being made to statements that do not need

them?them?5.5. Are there long lists of citations for points that do not need Are there long lists of citations for points that do not need

them?them?6.6. Are citations being made to derivative studies rather than Are citations being made to derivative studies rather than

the pivotal studies that made the discovery?the pivotal studies that made the discovery?7.7. Are there citations to inaccessible sources?Are there citations to inaccessible sources?8.8. Are there citations to empirical work where the work has Are there citations to empirical work where the work has

not been and cannot be properly evaluated?not been and cannot be properly evaluated?9.9. Are sources being cited selectively to support a viewpoint?Are sources being cited selectively to support a viewpoint?10.10. Are sources being cited accurately?Are sources being cited accurately?