Digging for data: opportunities and challenges in an open research landscape_McLeish

Ben McLeishProduct Specialist

ben@altmetric.com@benmcleish

Digging for data: Opportunities and Challenges in an Open Research Landscape

An alternative,

more immediate measure of attention

From non-traditional

sources

To provide a larger context

What are altmetrics?

Not a replacement but a complement

Policy documents, blogs, mainstream news, social media

Providing a multi-faceted picture of engagement

Altmetric is a data science company that tracks attention to research outputs, delivering article level metrics via visually engaging, intuitive interfaces.

And we have been digging for data for a while.

Who are we?

What we look for

Research-Outputs

DOIs General

PubMed ID Medical/Health

arXiv ID Physics, Maths & Computer Science

ADS ID Astrophysis Data System

SSRN ID Social Sciences

RePEC ID Economics

Handles General

Where are we looking for data?

NEWS OUTLETS• Over 1,300 sites• Manually

curated list• Text mining• Global coverage

SOCIAL MEDIA AND BLOGS

Twitter, Facebook,

Google+, Sina Weibo

Public posts only8,000+ Blogs

REFERENCE MANAGERS

• Mendeley etc• Demographics

OTHERS…

YoutubeRedditF1000Q&A

Policy Documents…

POST PUBLICATIONPEER REVIEW

SITES

PublonsPubpeer

Policy documents as an essential source• ASHA Practice Policy • AWMF - Association of Scientific Medical Societies • European Food Safety Authority (EFSA) • Food and Agriculture Organization • GOV.UK - Policy papers, Research & Analysis• Intergovernmental Panel on Climate Change (IPCC)• International Committee of the Red Cross (ICRC)• International Monetary Fund • (IMF) – (Tracking working papers)• Medicins sans Frontieres (MSF) • Mental Health Foundation (UK)• NICE Evidence • UNESCO • World Health Organization (WHO)• Australian Policy Online • The World Bank• International Fund for Agricultural Development• Scottish Intercollegiate Guideline Network

And more being added each week…

44K

online mentions of scholarly articles every day.

1 mention every 2 seconds!

50K unique articles are shared each week.

>3.5M

articles with tracked attention data.

The conversation is moving online..

Source: Altmetric internal data, March 2015

We collect it and put it all together.

The Donut: Colours change dynamically with attention

High traction across all sources

Strong news coverage

Strong Facebook And G+/Twitter traction

Impact at a Glance

Benchmarking Insights at Article Level

Insights at Institutional Level

“Tell me what’s being said about my research right now.”

Do you even know how to find out?

Is your institution or your work making the news?

Has your institutional Has your output influ-enced government policy?

How is the public engaging with your research?

Do you even know how to find out?

Is anyone getting it wrong? Are your marketing and communications offices

aware?

Copyright Randall Munrow, XKCD, http://xkcd.com/386/

What we do with this data• Aggregates output level metrics into

dashboards for:– Individual researchers– Departments or units within an

institution/Research Groups– subject areas– The entire institution– Customized groups that include data

external to the institution

What we do with this data• Streams mentions in real time• Sorts outputs by numerous filters:– Journal, publisher, funder, unique

identifier, PubMed query and more• Sets up alerts• Emails and exports reports• Facilitates benchmarking and

comparative analysis of groups

Here’s how Altmetric for Institutions can help you answer them….

Remember those questions?

What is happening to your research right now online?

Summary report tells you at a glance breaking mentions out by source…

Track attention over time

What is happening to your research right now online?

Summary report tells you at a glance breaking mentions out by source…

Track attention over time

Is your institution or your work making the news?

Has your institutional output influenced government policy?

Direct Link to Fulltext

How is the public engaging with your research?

Does the PR or Comms Department need to manage any online discussions?

Copyright Randall Munrow, XKCD, http://xkcd.com/386/Tweet, Retweet, Reply – Right in the Interface!

Funders want to see a “broader impact…”

“The primary aim of SEP assessments is to reveal and confirm the quality and the relevance of the research to society and to improve these where necessary.”https://www.knaw.nl/nl/actueel/publicaties/standard-evaluation-protocol-2015-2021

Funding pools drying up: Show me the (broader) impact!

Grant funders looking for proof of “broader impacts” often defined as “an effect, change, or benefit to the economy, society, culture, public policies, health, the environment, etc.”

Research Excellence Framework, http://www.ref.ac.uk/panels/assessmentcriteriaandleveldefinitions/

Broaden dissemination to enhance scientific and technological understanding, for example, by presenting results of research and education projects in formats useful to students, scientists and engineers, members of Congress, teachers, and the general public.

http://www.nsf.gov/pubs/2007/nsf07046/nsf07046.jsp