The Empty Chair at the Metrics TableDiscussing the absence of educational impact metrics and a framework for their creation

Lauren Ashby: Associate Editor, SAGE PublicationsMathias Astell: Marketing Manager, Nature Publishing Group

Why are we here?

Currently no metrics which focus on the educational impact of scholarly outputs

We propose the creation of a quantitative metric for educational impact

Questions

1.Who are the audiences of scholarly outputs?

2.What potential measures of educational impact currently exist?

3.What would an educational metric look like, and how might it be

used?

Who are the audiences of scholarly outputs?

Research focus in representing audiences

The Rest of the Picture

Practitioners

Students (School, Further and Higher Education)

Teachers

The journal as a universityJournal Textbook Monograph Reader Handbook ThesisWeb

ResourceSlides Videos

Researcher X X X X X X X

Practitioner X X X X X X

Student X X X X X X X X X

Teacher X X X X X X

What potential measures of educational impact currently exist?

Online Syllabi and Reading Lists

Homa, N. et al. (2013) “An Analysis of Learning Objectives and Content Coverage in Introductory Psychology Syllabi”, Teaching of Psychology, 40(3).

“...evidence from this research indicates that online syllabus citations are useful sources of evidence about educational value...” Kousha, K. & Thelwall, M.(2008) "Assessing the impact of disciplinary research on teaching: An automatic analysis of online syllabuses." Journal of the American Society for Information Science and Technology 59(13): p. 2067.

Usage

Publisher Usage Statistics

University Library Holdings Usage Data

Other sources

Counting of URLs

Journal clubs

Public and university library holdings

Cites in academic presentations

Reference management system

Dissertation and Thesis Citation Measurement

What would an educational metric look like, and how might it be used?

Blending approaches

Data gathered in previous research studies chiefly focused on benefit

to researchers - recast techniques to help expressing educational

impact

Open, not weighted and informational

Quantitative for use in qualitative assessments.

this information has to be used collectively in order to make a useful assessment of

the different ways in which a scholarly output is being used.

The benefits:

Credit for the creation of more educationally focussed outputs

Global awareness of usage of scholarly outputs in education

Support creating syllabi - and the post graduate teaching

community

Students accessing core information more easily

Supporting library collection development

In conclusion

Our thinking aims to devise a quantitative measure of educational impact: a count of when and how scholarly outputs are used in teaching and learning to better represent and promote these valuable resources.