Embed Size (px)
Citation preview
Is there anybody out there?Searching the social space for signs of
intelligence
Mike TaylorResearch Specialist
http://orcid.org/[email protected]
For much of the the last century, our only measurement of the impact of scholarly research has been through the counting and analysis of citation: one authoring researcher acknowledging the contribution of another authoring researcher. Significant, certainly, given these caveats, but in a wider social context citation analysis begins to look like an edge case. Can the measurement of sharing on social networks provide a wider view of how research is consumed in society, or is it all chatter and noise – and how do we detect the conversations of true significance?
The world according to citation
• “I am writing an article and wish to cite another article”
The world according to alternative metrics
• “I’m writing an article and might cite this”• “You should read this article if you’re
interested in #thistopic”• “My PI wrote this paper”• “My daughter wrote this and I’m so proud”• “This article has a titivating title. Anyway, it
made me laugh”• “These scientists are going to cure cancer”
The phenomenally rich world of alternative metrics
• Social activity indicators: Twitter, Facebook, Delicious, Pintrest, Google+
• Scholarly activity indicators: Mendeley, Citeulike, Zotero
• Scholarly articles: blogs, reviews• Mass media: news papers, TV• Re-usage indicators: data, code, graphics
An example from 2013
• Huge potential for social impact• Press campaign: front page story
on much of the UK press• Great publisher support from
Nature• 1000s of tweets• But what’s missing?
The phenomenally poor world of alternative metrics
Current alternative metrics don’t count or model:• Poorly referenced mass media• Stories about stories• The flow of the story• Social media about stories, replies, re-tweets• Influence on professional bodies• Representation to Government, Government
policy
Not only are alternative metrics bigger than citations, they’re also different
• Public vs private• Anonymous vs attributable• Persistent vs fleeting• Positive vs negative (counts and sentiment)• Real time vs slower• But article driven, formal links
The different characteristics of alternative metrics
• Citation: one class of activity, with many sub-classes
• Alternative metrics: several types of activity, with many classes and countless sub-classes (all vying with each other)
The power of intelligent conversation
• Elevator pitch > monograph• A word in the ear of a President
versus
• Engaging with millions• Patient-power• Lobbying interests
The chatter of (how shall we say this?) less than intelligent
conversation
• Not all communication is equal• Not all communication is between equals• Noise is not meritocratic• But is Twitter just meaningless noise?
The myth of social networks
• Often assumed to be trivial, with a focus on titillating articles
• An analysis of 13.5k papers revealed striking differences:
• Top 0.5% of social activity – strong emphasis on policy, funding, areas where science and government overlap (stem cells, CERN, etc)
• Top 0.5% of scholarly activity – primary research
The academic networks are building
• Orcid / ODIN / THOR• Data DOIs• RDA data citation• Data metrics• Usage APIs / data• Open data, open articles
Mapping academic influence is becoming easier
• Heading towards a paradigm shift in mapping academic influence
• Academics probably won’t create negative links
• This is a matter-of-fact network, flat, a statement of “what is”
• Insufficient to understand social impact
Science in society
• Open science, citizen science, open access, open data, cloud infrastructure, open source code, virtualization
• Social networks, easy access to scholars• Too hard => too easy? • Explosion in communication and access
A partial view
• Moving towards a more complete scholarly network
• Data exists to get an idea of how research is being consumed in society
• Too much missing to extrapolate• Almost entirely devoid of political context
Correlations
• Not an even picture, there are threads of correlations – blogs – tweets – mass media
• We can’t make simple conclusions• We don’t have enough data to make complex
conclusions
Bigger data
• Deeper: Talking about people, departments, companies, movements
• More sensitive: Going from “being spoken about” to “what is being said”
• Wider: “who is speaking”, “to whom are they speaking”
• Further: “他們在中國說了什麼？”
The role of sociologists and economists
• Social potential, professional and academic perspectives
• Is $$$ a good reflection of impact?• People like to think of the “return on
investment” model, but it’s not that easy, and the conclusions may be uncomfortable seen in isolation
Is a social impact index computable?
• Social impact index = f(social capacity), f(social accessibility), f(social reach)
• If has no capacity for effecting social change, if is incomprehensible, if no-one is aware of it…
• We need the data and the maths to identify the intelligent versus the influential
