| 1

Policy Impact of Knowledge and Knowledge Organisations

Qualitative & quantitative approaches to measuring impact:

existing methods, new approaches & challenges

Stephane Berghmans, DVM PhD

VP Academic & Research Relations EU

20 June 2017, Brussels

| 2

Research Metrics Throughout the Research Process

2

http://plumanalytics.com/learn/about-metrics/

www.snowballmetrics.com

| 3

Two Golden Rules for using research metrics to give a balanced, multi-dimensional view

Always use both qualitative and quantitative input into your

decisions

Always use more than one research metric as the quantitative input

Using multiple metrics drives desirable changes in behaviour

There are many different ways of being excellent

A research metric’s strengths can complement the weaknesses of others

Combining both approaches will get you closer to the whole story

Valuable intelligence is available from the points where these approaches differ in their

message

This is about benefitting from the strengths of both approaches, not about replacing one

with the other

| 4

• Founded to Tell the Story of

Research

• Developed comprehensive

metrics

• Timeline:

2012: Company founded

2014: Merged with EBSCO

2017: Joined Elsevier

An example with Plum Analytics

• 150+ worldwide customers,

including: • Texas A&M

• US Air Force Strategic Funding Office

• University of Western Australia

• Nanyang Technical Institute

• University of Helsinki

• University of Rome Tor Vergata

• University of Cologne

• Chalmers University of Technology

| 5

Types of Research Output

http://plumanalytics.com/learn/about-artifacts/

| 6

USAGE

(clicks, downloads, views,

library holdings, video plays)

CAPTURES

(bookmarks, code forks, favorites,

readers, watchers)

MENTIONS

(blog posts, comments, reviews,

Wikipedia links)

SOCIAL MEDIA

(+1s, likes, shares, tweets)

CITATIONS

(citation indexes, patents, clinical, policy)

Categorizing Metrics for Analysis

Do we need a doi for

policy publications?

| 7

ACI

Amazon

Airiti

bepress

bit.ly

CABI

CrossRef

Delicious

Dryad

dSpace

DynaMed Plus

EBSCO

ePrints

Facebook

figshare

Github

Goodreads

Google+

Mendeley

NICE (UK)

OJS Journals

PLOS

PubMed

PubMed Central

Reddit

RePEc

SciElo

Scopus

SlideShare

SourceForge

SSRN

Stack Exchange

Twitter

USPTO

Vimeo

Wikipedia

WorldCat (OCLC)

YouTube

DMP

Sources of Metrics

Joint Research Center

| 8

PlumX Dashboard

• Ability to group output. Examples include:

- Lab

- Department

- Researcher

- School

• Gather metrics on all research outputs

• Analytical Reports

• Productivity Reports

| 9

Research Artifact page

| 10

Research Artifact page

| 11

Summary

Not just Metrics

Both expert opinion and metrics are needed to fully describe impact

Metrics

A basket of metrics is a “menu” that contains diverse metrics for all entities

Policy Traceability

How can policy publications be made more traceable?

We are open for collaboration…

| 12

www.elsevier.com/research-intelligence

Thank you!