What is Impact?How do you measure it?
Wouter Gerritsma
Programme
Publishing
Journals
Citations
Impact
Journal impact
Publishing strategy
What is impact?
I have published in Nature
My report has been used by the government
I've been cited 22 times
My article has downloaded 2421 times
I was invited to the late night news
I filed a patent over my invention
Impact of a single publication
Citation enhanced A&I databases
Web of Science
Scopus
Google Scholar (http://scholar.google.com)
PsychInfo, SciFinder (A&I databases in Digital Library)
ArXiv (Physics)
Spires (high energy physics)
Citeseer (ICT)
Other OA Initiatives
Web of Science
Search:
●Articles are found based on Authors, Addresses, etc.
●For each article Times cited is presented
Cited reference search:
●Searches in the reference lists of records
●Not all of your articles are found. Non-cited articles are missing
How do we compare numbers
Scientist Z. Math has a publication from 2001 with 17 citations
Scientist M. Biology has a publication from 2009 with 24 citations
Baselines for Mathematics
Baselines for Molecular Biology
0
100
200
300
400
0 2 4 6 8 10 12
Years after publication
Cu
mu
lati
ve
no
. c
ita
tio
ns Baseline
top 10%
top 1%
Bibliometric indicators: An example
Kroes-Nijboer, A; Venema, P; Bouman, J; van der Linden, E (2009) The Critical Aggregation Concentration of beta-Lactoglobulin-Based Fibril Formation. Food Biophysics 4(2):59-63.
●Citations from WoS: 10
Journal: Food Biophysics
●Categorised by ESI in Agricultural Sciences
Baseline data for Agricultural Science.
●Article from 2009 in Agricultural Sciences:
●On average: 4.25 citations; top 10%: 11 citations; top1%: 26 citations
Relative Impact: 10/4.25 = 2.35 Values Oct. 2012
Essential Science Indicators
Analytical database, covering 10 years + current year building
Comparisons between Countries, Institutes, Scientists and Journals
Hot papers / Highly cited papers
Research fronts
Baselines
Steps in a citation analysis
1. Look up the citation data (Web of Science)
2. Matching Journal(s) with appropriate research fields (Essential Science Indicators)
3. Collect baseline data (Essential Science Indicators)
4. Calculate the relative impact
Interpretation of RI for small groups
With 10-50 publications per year
RI ≤ 0.8 : below world average impact
0.8 < RI ≤ 1.2 : world average impact
1.2 < RI ≤ 2.0 : above world average impact
2.0 < RI ≤ 3.0 : very good average impact
RI > 3.0 : excellent average impact
We do it also for groups
H-index
Balance between productivity and citedness
To rule out the effect of one or two highly cited papers
Applicable to authors, journals, research groups, compounds, subjects etc.
But there are some serious doubts about robustness
Waltman, L. & N. J. van Eck (2011). The inconsistency of the h-index. Journal of the American Society for Information Science and Technology 63(2):406-415 http://dx.doi.org/10.1002/asi.21678
h-index
Omnipresent h-index
Journal Performance Indicators
Journal performance indicators are based on citations to articles
Journal Citation Reports (JCR)
●a.o. standard Journal Impact Factors and 5-year Impact Factors
Scopus Journal Analyzer (SJA)
●a.o. SCImago Journal Rank (SJR) and Source Normalized Impact per Paper (SNIP)
●Also available on http://journalmetrics.com/
Journal Citation Reports (JCR)
Reports three measures
Impact factor
Immediacy Index
Cited half life
Adapted from: Amin, M and Mabe, M. (2000) Impact factors: use and
abuse. Perspectives in Publishing, No. 1, 6 pp.
http://www.elsevier.com/framework_editors/pdfs/Perspectives1.pdf
IF in 2010 for Agricultural Systems
Selecting journals on the basis of IF
Word of warning
●Our opinion: Be careful when using Journal Impact factors to judge the performance of a group or individual scientist
●Used for NWO grant applications and Tenure track at Wageningen UR
50% of articles generate 90% of all cites
Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ 314(7079): 497-502. http://bmj.bmjjournals.com/cgi/content/full/314/7079/497
Journals from Agricultural Economics & Policy: Quartile Scores
JCR Science Edition 2009
Q1
Q2
Q3
Q4
Journal quality and article impact 2003-2009, for Wageningen UR
Journal Quartile Pubs RI T10(%T10) T1(%T1)
Q 1 7170 2.26 2444(34%) 505(7%)
Q 2 2919 1.26 578 (20%) 61 (2%)
Q 3 1303 0.93 143 (11%) 10 (1%)
Q 4 587 0.66 30 (5%) 6 (1%)
Aggregate 11917 1.79 3195(27%) 582(5%)
Source: Wageningen Yield, Feb. 2012
Journal selection and impact universities globally
Changing publication behaviour @WUR
Sorting on IF possible in WUR catalogue
Sorting on IF possible in WUR catalogue
Alternatives for Impact Factor
All based on Scopus
●Scimago Journal Rank (SJR) http://www.scimagojr.com/
●Source Normalized Impact per Paper (SNIP) http://www.journalmetrics.com
Alternatives for JIF: SJR
Alternatives for JIF: SNIP
Altmetrics
Quickly developing
●ScienceCard
●Total-Impact
●Readermeter
●Microsoft Academic Search
●etc.
Wouters, P. & R. Costas (2012). Users, narcissism and control. Utrecht, NL: SURFfoundation. http://www.surffoundation.nl/en/publicaties/Pages/Users_narcissism_control.aspx.
Importance of social media for scientists
Nearly 300 article views in the 1st week
1400+ downloads after 3 months
Thank you!
On the Web:
@wowter
wowter.net
www.slideshare.net/wowter
Recommended
