Timothy D. Bowman, Ph.D. Candidate | 19th Nordic Workshop on Bibliometrics and Research Policy
CRC.EBSI.UMONTREAL.CA
WHY INVESTIGATE SCHOLARLY ACTIVITY IN SOCIAL MEDIA ?
- New technology allows for reassessment
and reevaluation of academia (Baldwin,
1998)
- Social media use provides insight into
customs and traditions (Greenhow, 2009)
- Social media use unveiling once
invisible backstage activity (Priem, 2014)
CRC.EBSI.UMONTREAL.CA
HAVE WE MOVED “BEYOND BIBLIOMETRICS”?
- We’ve moved beyond simply measuring citations
(Cronin & Sugimoto, 2014).
- New tools and data allow for new kinds of metrics
measuring wide array of indicators (Cronin, 2014)
- Electronic publishing magnifies a scholar’s
awareness of own performance (Wouters, 2014)
- Evaluation of bibliometric indicators by novices
allows for wide use of various ad hoc indicators
(Gingras, 2014)
CRC.EBSI.UMONTREAL.CA
WHY CONSIDER “ALTMETRICS” OR “INFLUMETRICS” OR
SIMPLY “SOCIAL MEDIA METRICS”?
- “Altmetrics” is the measure of scholarly communication
and dissemination within social media contexts (Priem &
Hemminger, 2010; Priem, Taraborelli, Groth & Neylon,
2010)
- Perhaps a better term is Influmetrics (Rousseau & Ye,
2013) or simply “social media metrics”?
- Social media indicators may measure immediate
assessment of academic impact and social impact
(Thelwall, Haustein, Larivière & Sugimoto, 2013)
- “Products,” not “publications” (Piwowar, 2013)
CRC.EBSI.UMONTREAL.CA
DO SCHOLARS USE TWITTER?
- 92% of Semantic Web scholars had Twitter account and
rated it as favorite for spreading scientific information
(Letierce, Passant, Decker, & Breslin, 2010)
- Total of 367 scholars reported increasing acceptance for
blogs and microblogs for consumption and
dissemination of scientific information (Gruzd, Goertzen,
& Mai, 2012)
- Scholars’ tweets tend to share information about (a)
professional discussions, (b) network with others, (c)
offer help / request help, (d) call attention to other social
media involvement, and (e) personal discussions, and (f)
impression management (Veletsianos, 2012)
CRC.EBSI.UMONTREAL.CA
DO SCHOLARS USE TWITTER? (CONT.)
- 43% scholars at 2012 STI Conference using
Twitter; it was used privately and professionally,
to distribute professional information, and to
improve visibility (Haustein et al., 2013)
- 80% DH scholars ranked Twitter as relevant for
consumption and 73% for dissemination of DH
information (Bowman et al., 2013)
- Differences by discipline found regarding the
way scholars used Twitter (Holmberg &
Thelwall, 2014)
CRC.EBSI.UMONTREAL.CA
RESEARCH QUESTIONS
1. What differences exist between the
tweeting behavior of scholars in the
natural and social sciences?
2. What kind of relationships exists
between tweeting and publication
behavior?
3. How does Twitter affordance use differ
across disciplines?
CRC.EBSI.UMONTREAL.CA
WHAT DATA IS IN THIS SAMPLE?
- 16,862 Associate, Assistant, and Full professors from webpages
at 62 AAU-member universities
- The faculty belonged to either Physics, Biology, Chemistry,
Computer Science, Philosophy, English, Sociology, or
Anthropology departments.
- 60 of the 62 universities rank in the top 125 according to 2014
CWTS Leiden Ranking
- Survey sent January and February 2014 with a response rate of
8.5% (1,910 responses)
- Of these responders, 32% (615) reported having at least one
Twitter account
- 289,934 tweets of 585,879 from 445 accounts (391 scholars)
were collected.
CRC.EBSI.UMONTREAL.CA
HOW WAS THE DATA COLLECTED?
- Twitter API, Local WoS Database, Manual
cleaning of authors
- Twitter:
- tweets, # of tweets, followers, friends, retweets,
created date
- affordances: @mention, #hashtag, URLs, media,
symbols, retweets
- WoS
- publications, citation averages
ALL 1,910 SURVEY RESPONDENTS :: HAVE TWITTER ACCOUNTS?
42.65%
36.42%38.89%
40.82%
24.96%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Less than 1Year
1 to 3 Years 4 to 6 Years 6 to 9 Years 10 Years ofMore
I'm not
38.10%
45.09%
38.27%34.31% 35.75%
29.68%26.58%
19.81%16.34%
5.26%2.38%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
26 to 30years
31 to 35years
36 to 40years
41 to 45years
46 to 50years
51 to 55years
56 to 60years
61 to 65years
66 to 70years
71 to 75years
Over 75years
by AGE
by ACADEMIC AGE
33.33%
29.11%
40.38%
25.00%
29.11%
50.00%
28.00%
0%
10%
20%
30%
40%
50%
60%
AmericanIndian /Native
American
Asian Black /African
American
Hispanic /Latino
White /Caucasian
PacificIslander
Other
by ETHNICITY
28.10% 27.52%
37.46% 36.90%
20.71%
50.00%
28.99%
23.64%
0%
10%
20%
30%
40%
50%
60%by DISCIPLINE
ONLY 391 SCHOLARS WITH TWITTER ACCOUNTS :: MEAN OF TWEETS PER DAY
1.06
0.53
1.96
1.41
0.670.52
0.73
1.18
by DEPARTMENT by GENDER1.14
0.80
1.02
Other Female Male
N=232
SD=2.3
N=122
SD=2.1
N=3
0.89
1.11
1.39
0.670.85
I'm Not 10 Yearsor More
7 to 9Years
4 to 6Years
1 to 3Years
Lessthan 1Year
by ACADEMIC AGE
N=2N=207
SD=2.4
N=53
SD=2.2
N=35
SD=2.6
N=39
SD=0.9
N=21
SD=1.1
0.92
0.98
1.03
Professor AssociateProfessor
AssistantProfessor
by PROFESSIONAL TITLE
N=116
SD=2.1
N=116
SD=1.7
N=156
SD=2.9
0
50
100
150
200
250
300
350
400
450
500
0 2 4 6 8
Physics
BY DISCIPLINE :: RELATIONSHIP OF MEAN TWEETS PER DAY TO # OF ARTICLES
0
2
4
6
8
10
12
0 5 10 15
Anthropology
0
5
10
15
20
25
0 5 10 15 20 25
Philosophy
0
10
20
30
40
50
60
0 5 10 15 20
Sociology
Art
icle
s
N=40 N=30
N=66
Tweets per Day
N=19
R² = 0.0133
0
1
2
3
4
5
6
0 5 10 15 20
English
BY DISCIPLINE :: RELATIONSHIP OF MEAN TWEETS PER DAY TO ARTICLES (CONT.)
R² = 0.0291
0
5
10
15
20
25
30
35
40
45
0 2 4 6 8 10
Biology
R² = 0.0118
0
5
10
15
20
25
30
0 1 2 3 4
Chemistry
R² = 0.0026
0
10
20
30
40
50
60
70
0 5 10 15 20
Computer Science
Art
icle
s
N=82N=73
N=20N=40
Tweets per Day
SCHOLARLY IMPACT? :: MEAN TWEETS PER DAY BY MEAN OF IAC AVERAGE
0
0.5
1
1.5
2
2.5
0 5 10 15
Anthropology
0
5
10
15
20
25
0 5 10 15 20
English
0
1
2
3
4
5
6
7
0 5 10 15 20 25
Philosophy
0
1
2
3
4
5
6
7
8
0 5 10 15 20
Sociology
N=30 N=19
N=82N=66
SCHOLARLY IMPACT? :: MEAN TWEETS PER DAY BY SUM OF IAC AVERAGE
R² = 3E-05
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0 2 4 6 8 10
Biology
R² = 0.0126
0
1
2
3
4
5
6
7
8
0 1 2 3 4
Chemistry
R² = 0.0089
0
2
4
6
8
10
12
14
16
18
0 2 4 6 8
Physics
R² = 1E-05
0
2
4
6
8
10
12
14
16
0 5 10 15 20
Computer Science
N=61N=73
N=20N=40
MEAN PERCENTAGE OF TWEETS WITH AFFORDANCE PER PERSON BY DEPARTMENT
AND MEAN OF REWTEETS BY COLLECTED TWEETS
6.41%
7.44%
0%
1%
2%
3%
4%
5%
6%
7%
8%
HASHTAGS
Anthropology
Biology
Chemistry
Computer Science
English
Philosophy
Physics
Sociology
16.28%
20.06%
0%
5%
10%
15%
20%
25%
MENTIONS
Anthropology
Biology
Chemistry
Computer Science
English
Philosophy
Physics
Sociology
0.72%
1.16%
0.25%
1.12%
0.53%
1.69%
0.77%
1.09%
0%
1%
2%
URLs
Anthropology
Biology
Chemistry
Computer Science
English
Philosophy
Physics
Sociology353
3291
0
500
1,000
1,500
2,000
2,500
3,000
3,500
RETWEETSAnthropology
Biology
Chemistry
Computer Science
English
Philosophy
Physics
Sociology
CRC.EBSI.UMONTREAL.CA
SUMMARY
• As expected, the data reflected differences of those who reported
having Twitter accounts based on academic age and actual age.
• Of the 391 scholars (445 Twitter accounts) that were collected, the
data did reflect differences in mean tweets per day based on gender,
discipline, and academic age and title
• Finally, it was found that the data reflected no strong relationships
between mean tweets per day and publication output or
• There was no real relationship between average citations and mean
tweets per day (scholarly impact)
• The data did reflect small differences in affordance use by discipline,
especially the differences in retweets but theses differences in
retweets are not an accurate representation of the retweets by the
scholar
CRC.EBSI.UMONTREAL.CA
ONGOING WORK
• Further analysis of retweets needed attempting to focus solely on
retweets made by the scholars themselves
• Using linguistic tools, the text of the 289,934 tweets will be used to
compare terms used in tweets with article titles at the level of the
scholar and discipline
• A social network analysis will be completed reflecting the mentions
used in tweets at the scholarly and discipline levels
• A closer examination of the actual affordances (unique hashtags,
unique URLs, unique mentions) used
• A categorization of tweets as either personal or professional by
Turkers
• A general discussion on what these social media metrics are actually
measuring including any correlations between social media use and
publication activity
CRC.EBSI.UMONTREAL.CA
THANK YOU
This work was partially funded by a grant
by The Alfred P. Sloan foundation
and a Canada Research Chair grant
DO YOU HAVE ANY QUESTIONS?
REFERENCESBaldwin, R. G. (1998). Technology’s Impact on Faculty Life and Work.
New Directions for Teaching and Learning, (76), 7–21.
doi:10.1002/tl.7601
Bowman, T. D., Demarest, B., Weingart, S. B., Simpson, G. L.,
Lariviere, V., Thelwall, M., & Sugimoto, C. R. (2013). Mapping DH
through heterogeneous communicative practices. In Digital
Humanities 2013. Lincoln, NE.
Cronin, B. (2014). Scholars and scripts, spoors and scores. In B.
Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing
multidimensional indicators of scholarly impact (pp. 3-22). Cambridge,
Mass.: MIT Press.
Cronin, B. & Sugimoto, C.R. (2014). Preface. In B. Cronin & C. R.
Sugimoto (Eds.), Beyond bibliometrics: Harnessing multidimensional
indicators of scholarly impact (pp. vii). Cambridge, Mass.: MIT Press.
Gingras, Y. (2014). Criteria for evaluating indicators. In B. Cronin & C.
R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing
multidimensional indicators of scholarly impact (pp. 109-126).
Cambridge, Mass.: MIT Press.
Greenhow, C. (2009). Social scholarship: applying social networking
technologies to research practices. Knowledge Quest, 37(4), 42–47.
Retrieved from
http://aasl.metapress.com/index/r282223126950757.pdf
Gruzd, A., Goertzen, M., & Mai, P. (2012). Survey results highlights:
Trends in scholarly communication and knowledge dissemination (p.
10). Retrieved from http://socialmedialab.ca/?p=4308
Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., &
Terliesner, J. (2013). Coverage and adoption of altmetrics sources in
the bibliometric community. arXiv, 1–12. Digital Libraries. Retrieved
from http://arxiv.org/abs/1304.7300
Holmberg, K., & Thelwall, M. (2014). Disciplinary differences in Twitter
scholarly communication. Scientometrics. doi:10.1007/s11192-014-
1229-3
Understanding how Twitter is used to spread scientific messages. In
Web Science Conference. Raleigh, NC.
Moran, M., Seaman, J., & Tinti-Kane, H. (2011). Teaching, learning,
and sharing: How today’s higher education faculty use social media.
Piwowar, H. (2013). Altmetrics: Value all research products. Nature,
493(159). doi:10.1038/493159a
Priem J., & Hemminger B.M. (2010) Scientometrics 2.0: Toward new
metrics of scholarly impact on the social web. First Monday 15.
Available:
http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2
874/257. Accessed 2011 December 7.
Priem, J., Taraborelli, D., Groth, P., Neylon, C. Alt-metrics: a
manifesto. 2010. Available from http://altmetrics.org/manifesto/
Priem, J. (2014). Altmetrics. In B. Cronin & C. R. Sugimoto (Eds.),
Beyond bibliometrics: Harnessing multidimensional indicators of
scholarly impact (pp. 263–288). Cambridge, Mass.: MIT Press.
Rousseau, R., & Ye, F. (2013). A multi-metric approach for research
evaluation. Chinese Science Bulletin, 58(3290), 1–7.
doi:10.1007/s11434-013-5939-3
Thelwall M., Haustein S., Larivière V., Sugimoto, C.R. (2013) Do
Altmetrics Work? Twitter and Ten Other Social Web Services. PLoS
ONE 8(5): e64841. doi:10.1371/journal.pone.0064841
Veletsianos, G. (2012). Higher education scholars’ participation and
practices on Twitter. Journal of Computer Assisted Learning, 28(4),
336–349. doi:10.1111/j.1365-2729.2011.00449.x
Wouters, P. (2014). The citation: From culture to infrastructure. In B.
Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing
multidimensional indicators of scholarly impact (pp. 47–66).
Cambridge, Mass.: MIT Press.
0.0% 1.0% 2.0% 3.0% 4.0% 5.0% 6.0%
Boston University
Brandeis University
Brown University
California Institute of Technology
Carnegie Mellon University
Case Western Reserve University
Columbia University
Cornell
Duke University
Emory University
Georgia Institute of Technology
Harvard
Indiana University
Iowa State
Johns Hopkins
McGill
Michigan State University (1964)
MIT
New York University
Northwestern
Princeton University (1900)
Purdue University (1958)
Rice University (1985)
Rutgers, The State University of New Jersey (1989)
Stanford University (1900)
Stony Brook University-State University of New York (2001)
Texas A&M University
The Ohio State University (1916)
The Pennsylvania State University (1958)
The University of Chicago
The University of Iowa (1909)
The University of Kansas
The University of North Carolina at Chapel Hill
The University of Texas at Austin
The University of Wisconsin-Madison
Tulane University
University at Buffalo, The State University of New York (1989)
University of Arizona
University of California, Berkeley
University of California, Davis
University of California, Irvine
University of California, Los Angeles
University of California, San Diego
University of California, Santa Barbara
University of Colorado Boulder
University of Florida
University of Illinois at Urbana-Champaign (1908)
University of Maryland
University of Michigan
University of Minnesota
University of Missouri-Columbia
University of Oregon
University of Pennsylvania
University of Pittsburgh
University of Rochester
University of Southern California
University of Toronto
University of Virginia
University of Washington
Vanderbilt University
Washington University in St. Louis
Yale University
University of Maryland (3.32%)
University of Wisconsin-Madison (4.85%)
Indiana University (4.08%)
APPENDIX: UNIVERSITY DISTRIBUTION
CRC.EBSI.UMONTREAL.CA
APPENDIX: 62 AAU-MEMBER UNIVERSITIES
Boston University, Brandeis University, Brown University, California Institute of Technology, Carnegie Mellon University, Case Western Reserve University, Columbia University, Cornell, Duke University, Emory University, Georgia Institute of Technology, Harvard, Indiana University, Iowa State, Johns Hopkins, McGill, Michigan State University, MIT, New York University, Northwestern, Princeton University, Purdue University, Rice University, Rutgers, The State University of New Jersey, Stanford University, Stony Brook University-State University of New York, Texas A&M University, The Ohio State University, The Pennsylvania State University, The University of Chicago, Tulane University, University at Buffalo, The State University of New York, University of Arizona, University of California, Berkeley, University of
California, Davis, University of California, Irvine, University of California, Los Angeles, University of California, San Diego, and University of California, Santa Barbara ,The University of Iowa, The University of Kansas, The University of North Carolina at Chapel Hill, The University of Texas at Austin, The University of Wisconsin-Madison, University of Colorado Boulder, University of Florida, University of Illinois at Urbana-Champaign, University of Maryland, University of Michigan, University of Minnesota, University of Missouri-Columbia, University of Oregon, University of Pennsylvania, University of Pittsburgh, University of Rochester, University of Southern California, University of Toronto, University of Virginia, University of Washington, Vanderbilt University, Washington University in St. Louis, Yale University