Measuring Twitter activity of arXiv e-prints and published papers

Stefanie Haustein, Timothy D. Bowman, Benoît Macaluso, Cassidy R. Sugimoto & Vincent Larivière

Canada Research Chair on the Transformations of Scholarly Communication École de bibliothéconomie et des sciences de l’information

@stefhaustein

Introduction •  increase of Twitter use

•  230 million active users, 500 million tweets per day •  39% increase of users from 09/2012 to 09/20131

•  16% of US, 3% of world population in 20131

•  19% of US internet users 01/20142

•  uptake by researchers •  1 in 40 university faculty member in US and UK

have Twitter account (Priem & Costello, 2010)

•  9% of researchers use Twitter for work (Rowlands et al., 2011)

•  15% of German university faculty members, 70% of which at least occasionally in professional context (Pscheida et al., 2013)

1 Twitter statistics calculated based on data from: http://www.sec.gov/Archives/edgar/data/1418091/000119312513400028/d564001ds1a.htm and http://www.census.gov/population/international/data/ 2 Pew Research Center’s Internet Project surveys, 2010-2014 http://www.pewresearch.org/fact-tank/2014/06/11/can-twitter-survive-in-a-facebook-world-the-key-is-being-different/

Introduction •  social media activity around scholarly articles grows

5% to 10% per month (Adie & Roe, 2013)

•  scholarly documents on Twitter

1.6% of WoS papers with DOIs 2005-2011 (Zahedi, Costas & Wouters, 2014) 13.3% of WoS papers with DOIs 07-12/2011 (Costas, Zahedi & Wouters, 2014) 20.4% of PubMed/WoS 2012 (Haustein et al., 2014) 21.5% of WoS papers with DOIs 2012 (Costas, Haustein & Larivière, in prep.)

•  tweeting peaks shortly after publication •  of the published version in the journal of record (Eysenbach, 2011)

•  of the e-print (Shuai, Pepe & Bollen, 2012)

Research questions Twitter impact of e-prints and published versions

Research questions Twitter impact of e-prints and published versions

•  How many times are e-prints and published versions tweeted?

•  When does the Twitter activity occur? •  Do Twitter audiences differ between the two versions? •  To what extent are both activities picked up by Altmetric.com?

!  Should both kinds of activities been taken into account or are they equivalent?

Methods: direct match arXiv & Altmetric.com

Results: direct match arXiv & Altmetric.com

Methods: indirect match through WoS

exact&and&fuzzy&•  DOI&•  0tles&•  author&names&•  abstracts&•  0tle&length&

Results: indirect match through WoS

5.8%&documents&&w/&>1&Altmetric&id&

+1,507&papers&w/&more&tweets&

+4,600&TwiKer&counts&

+716&tweeted&papers&

Results: indirect match through WoS

Preliminary findings •  presence of DOIs in arXiv metadata influences ability to

match Twitter activity of e-print and published paper !  enriching arXiv metadata with DOIs eliminates

potential biases •  some improvement through indirect match, but actual

differences not as large as expected •  +3.2% tweeted documents, +7.7% tweets •  +7.6% papers with increase in Twitter activity !  Altmetric.com finds most Twitter activity of papers with

arXiv e-print !  most tweets refer to arXiv id, not DOI

Preliminary findings •  differences between arXiv primary categories

•  high Twitter rate in Quantitative Biology (8.4) and Computer Science (5.0)

•  high Twitter coverage in HEP Experiment (81.1%), HEP Theory (72.7%), HEP Phenomenology (69.6%), HEP Lattice (69.2%)

•  particular high coverage (44.9%) if compared to other studies

!  high presence of automated Twitter accounts! bots and cyborgs triggered by arXiv feeds: @hep_th @hep_ph @hep_ex @hep_lat

Outlook: automated tweets •  search for “arXiv” in Twitter user name and description:

•  many more automated accounts possible: •  journals •  publishers !  not equally distributed

!  distribution instead of impact

account'type& number'(%)'of'accounts& tweets& mean'

followers& mean'following& %'of'50,068'

tweets& mean'Truthy'BotOrNot'score&

arXiv&feed&(bot)& 43&(84.3%)& 87,389& 34.9& 0.6& 8.8%& 33%&topic&feed&(bot)& 4&(7.8%)& 10,040& 527.0& 491.5& 0.1%& 40%&selec0ve& 4&(7.8%)& 3,081& 361.8& 50.5& 1.0%& 46%&'& 51'(100%)& 100,510& 99.1& 43.0& 9.9%& 33%&

•  societies / associations •  institutions

•  authors

Outlook: automated tweets

25% 100%

27%

28% •  distinguishing type of tweet based on content e.g., similarity with article title (%)

engagement distribution

100%

86%

Outlook: next steps •  detecting bots and cyborgs

•  How much of Twitter activity in scholarly communication do they account for?

•  How are their tweets distributed? •  distinguishing between distribution and engagement

•  answering our original research questions

Stefanie Haustein

Thank you for your attention! Questions?

stefanie.haustein@umontreal.ca @stefhaustein

Thanks to Euan Adie and for access to their Twitter data!

References Adie, E. & Roe, W. (2013). Altmetric: Enriching Scholarly Content with Article-level Discussion and Metrics. Learned Publishing, 26(1), 11-17.

Costas, R., Zahedi, Z. & Wouters, P. (2014). Do altmetrics correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Sciences and Technology. arxiv: 1401.4321

Eysenbach, G. (2011) Can tweets predict citations? Metrics of social impact based on twitter and correlation with traditional metrics of scientific impact. Journal of Medical Internet Research, 13, e123. doi: 10.2196/jmir.2012

Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (2014b). Tweeting Biomedicine: An Analysis of Tweets and Citations in the Biomedical Literature. Journal of the Association for Information Sciences and Technology, 65(4), 656-669. doi: 10.1002/asi.23101

Priem, J., & Costello, K. L. (2010). How and why scholars cite on Twitter. Proceedings of the 73th Annual Meeting of the American Society for Information Science and Technology, Pittsburgh, USA.

Pscheida, D., Albrecht, S., Herbst, Minet, C. & Köhler, T. (2013). Nutzung von Social Media und onlinebasierten Anwendungen in der Wissenschaft. Erste Ergebnisse des Science 2.0-Survey 2013 des Leibniz-Forschungsverbunds „Science 2.0“ available from: http://www.qucosa.de/fileadmin/data/qucosa/documents/13296/Science20_Datenreport_2013_PDF_A.pdf

Rowlands, I., Nicholas, D., Russell, B., Canty, N., & Watkinson, A. (2011). Social media use in the research workflow. Learned Publishing, 24, 183–195.

Shuai, X., Pepe, A., & Bollen, J. (2012). How the scientific community reacts to newly submitted preprints: article downloads, Twitter mentions, and citations. PLOS ONE, 7(11), e47523. doi:10.1371/journal.pone.0047523

Zahedi, Z., Costas, R. & Wouters, P. (2014). How well developed are altmetrics? cross-disciplinary analysis of the presence of 'alternative metrics' in scientific publications. Scientometrics. doi: 10.1007/s11192-014-1264-0