Making use of (alt)metrics Phill Jones, PhD Head of Publisher Outreach Digital Science...

making use of (alt)metrics

Phill Jones, PhDHead of Publisher Outreach

Digital Sciencep.jones@digital-science.com

@phillbjones@digitalsci

@altmetrics

like all great movements…

it started with a manifesto…www.altmetrics.org

• Peer-review and citation metrics have been the gold standard for evaluating research quality

• Due to increased volume of content, those metrics no longer have adequate resolution on their own

• The Impact Factor is misapplied as a proxy measure of research quality – Article level metrics are needed for fair evaluation

• Citations are slow to accrue - researchers need a rapid measure of the importance of an article

The Evaluation Gap

institutions, librarians and scientometricians

http://www.hefce.ac.uk/pubs/rereports/Year/2015/metrictide/Title,104463,en.html

http://www.leidenmanifesto.org/

Grant funders looking for proof of “broader impacts” often defined as “an effect, change, or benefit to the economy, society, culture, public policies, health, the environment, etc.”

Research Excellence Framework, http://www.ref.ac.uk/panels/assessmentcriteriaandleveldefinitions/Broaden dissemination to enhance scientific and

technological understanding, for example, by presenting results of research and education projects in formats useful to students, scientists and engineers, members of Congress, teachers, and the general public.

http://www.nsf.gov/pubs/2007/nsf07046/nsf07046.jsp

funders are broadening their definition of “impact”

what can publishers do with altmetrics today?

• Professional society founded in 1915

• ~10k members

• Six peer reviewed publications

• ‘We struggle with the perception that we’re a little old-fashioned’ – Sue Silver

• Need to ‘modernize’

• Launch of Frontiers in Ecology and the Environment

Citations (Impact Factor)

Usage (Google

Analytics)altmetrics IMPACT!

Combining all the available metrics enables ESA to monitor and maximize all types impact and relevance

There’s no waiting 2-3 years, like you have to do with the Impact Factor

-Dr Sue Silver, ESA

• ESA began using social media marketing and refined it using Altmetric data

• Build and strengthen the community

• Fast feedback on promotional efforts

• Early signals on which articles are important

Method

• Put Altmetric badges on 5 of 39 Journals

• Asked for feedback from membership

Result

• “…It appeared that there was more potential interest from authors than readers” – Will Russell

• Authors used the data to assess which subjects, abstracts and even titles got the most attention from fellow chemists

Conclusion

• Buzz and interest from authors made Altmetric an established service to members

Hypothesis

• “We thought that the Altmetric Badges were going to present an alternative way for a reader to assess the impact of an article.”

• Pilot of Altmetric badges in May 2013• 10,086 click-throughs • Average time on page 3m12s• 47,000+ articles mentioned• 3,649 tweets• Estimated 5,000 news stories • 200 blog posts• 32 F1000 recommendations

Were the article metrics supplied on this paper useful to you?

I am more likely to submit a paper to a journal that supports

article metrics

How are readers using metrics?• To gauge the overall popularly of the article

• 87% of respondents strongly agreed or agreed

• To discover and network with researchers who are interested in the same area of their work • 77% strongly agreed or agreed

• To understands a paper’s influence on the scientific community • 66% strongly agreed or agreed

• To determine what journal to submit their next paper to • 60% of respondents strongly agreed or agreed

• To determine areas of research to explore • Only 37% of respondents strongly agreed or agreed

Questions:

• How do we expand our presence in the ereader market?

• How do we prove to our readers that we understand and appreciate them?

• How can we repurpose legacy content?

Answer:

• A new anthology product

• Editorial selection informed by altmetric data

Fascinating collection of

articles.

Awesome.

The collections hang together better than

compilations often do.

An essential addition to

one’s library.

Attractive and sensibly designed.

• Sales 2-4x higher than kindle versions of regular editions

• Authors pleased with increased exposure

• Editors like to highlight seminal pieces

• All built with repurposed legacy content!

http://www.altmetric.com/blog/publishers-making-altmetrics-work-for-you/

further reading on how publishers use altmetrics