Upload
hazel-hall
View
202
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Hazel Hall's invited keynote paper presented at the 6th International Evidence Based Library and Information Practice (EBLIP6) conference, Manchester, 30 June 2011. A journal article that draws on the some of the content of this presentation is also available at http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/12138/9373
Citation preview
Project output versus influence in practice
Impact as a dimension of research quality
6th International Conference on Evidence based Librarianship and Information PracticeKeynote presentation by Professor Hazel Hall, Director, Centre for Social Informatics, 30th June 2011
http://lisresearch.orghttp://lisresearch.org
If you are tweeting this session, please refer to me as @hazelh. Thank you!
If you are tweeting this session, please refer to me as @hazelh. Thank you!
LIS impact takes a number of forms Research impact measurement is tricky Research impact measurement matters Where impact is measured as a dimension of
research quality, a number of strategies can be deployed to extend project outcome reach
Key points
LIS impact takes a number of forms Research impact measurement is tricky Research impact measurement matters Where impact is measured as a dimension of
research quality, a number of strategies can be deployed to extend project outcome reach
Key points
Impact themes in LIS literature1. Evaluation of services – general use (debated)Impact themes in LIS literature1. Evaluation of services – general use (debated)
Impact themes in LIS literature2. Evaluation of service impact on end user behavioursImpact themes in LIS literature2. Evaluation of service impact on end user behaviours
Impact themes in LIS literature3. Evaluation of services as related to specific outcomes
Impact themes in LIS literature3. Evaluation of services as related to specific outcomes
Impact themes in LIS literature4. How to evaluate servicesImpact themes in LIS literature4. How to evaluate services
Impact themes in LIS literature5. BibliometricsImpact themes in LIS literature5. Bibliometrics
Impact themes in LIS literature5. Bibliometrics – publication impactsImpact themes in LIS literature5. Bibliometrics – publication impacts
Impact themes in LIS literature6. Bibliometrics research evaluationWe can stand on the shoulders of giants because librarians and information scientists have maintained a curated record
Impact themes in LIS literature6. Bibliometrics research evaluationWe can stand on the shoulders of giants because librarians and information scientists have maintained a curated record
Impact themes in LIS literature7. LIS research output LIS practiceImpact themes in LIS literature7. LIS research output LIS practice
Impact themes in LIS literature7. LIS research output LIS practiceImpact themes in LIS literature7. LIS research output LIS practice
Impact themes in LIS literature8. LIS practice LIS research (Brookes, 1976)
Impact themes in LIS literature8. LIS practice LIS research (Brookes, 1976)
Impact routes vary
Cumulative and indirect effect of exposure to research output
Direct policy mandate that derives from a particular study
Impact routes vary
Cumulative and indirect effect of exposure to research output
Direct policy mandate that derives from a particular study
LIS impact takes a number of forms Research impact measurement is tricky Research impact matters Where impact is measured as a dimension of
research quality, a number of strategies can be deployed to extend project outcome reach
Key points
Measurement challenge 1Attribution, and cause and effect, are not always accurate nor obvious
Measurement challenge 1Attribution, and cause and effect, are not always accurate nor obvious
Measurement challenge 1Attribution, and cause and effect, are not always accurate nor obvious
Measurement challenge 1Attribution, and cause and effect, are not always accurate nor obvious
Greer GarsonBest actress 1942 for role in Mrs Miniver Acceptance speech 5 minutes 30 seconds
Greer GarsonBest actress 1942 for role in Mrs Miniver Acceptance speech 5 minutes 30 seconds
Measurement challenge 2Sources of mediated training material are not always appreciated by practitioners
Measurement challenge 2Sources of mediated training material are not always appreciated by practitioners
Measurement challenge 3End-users are not normally aware of research effort to support services design and delivery
Measurement challenge 3End-users are not normally aware of research effort to support services design and delivery
Measurement challenge 4Certain types of research output attract more attention than others
Measurement challenge 4Certain types of research output attract more attention than others
Measurement challenge 5The impact of some is overwhelming (and disproportionate)
Measurement challenge 5The impact of some is overwhelming (and disproportionate)
LIS impact takes a number of forms Research impact measurement is tricky Research impact matters Where impact is measured as a dimension of
research quality, a number of strategies can be deployed to extend project outcome reach
Key points
Practitioner interest in research outputIncreases value of research undertakenNarrows research-practice gap benefits health of the profession
Practitioner interest in research outputIncreases value of research undertakenNarrows research-practice gap benefits health of the profession
Local contexts influence attitudes towards impact measurementREF2014 in the UK
Local contexts influence attitudes towards impact measurementREF2014 in the UK
LIS impact takes a number of forms Research impact measurement is tricky Research impact matters Where impact is measured as a dimension of
research quality, a number of strategies can be deployed to extend project outcome reach
Key points
NB presented here are strong messages that have emerged from RiLIES project empirical data analysed to date
http://lisresearch.org/rilies-project/
NB presented here are strong messages that have emerged from RiLIES project empirical data analysed to date
http://lisresearch.org/rilies-project/
Literature review; poll; focus groups; case studies; surveyLiterature review; poll; focus groups; case studies; survey
Research ideas for impact in practice
Focus on research areas of direct relevance to practitioners, both in terms of subject and context - time, place, sector
(Perth and London focus groups)
Research ideas for impact in practice
Focus on research areas of direct relevance to practitioners, both in terms of subject and context - time, place, sector
(Perth and London focus groups)
Research approach for impact in practice
Deploy action research approach
Mix in development and engagement, e.g. capacity building as hook, workshops throughout project
Encourage community creation around project for sustainability
Involve high profile, prestigious partners
(Evalued, Evaluating the impact of clinical librarian services, Open to all case studies)
Research approach for impact in practice
Deploy action research approach
Mix in development and engagement, e.g. capacity building as hook, workshops throughout project
Encourage community creation around project for sustainability
Involve high profile, prestigious partners
(Evalued, Evaluating the impact of clinical librarian services, Open to all case studies)
Project scale and diversity for impact in practice
Data collection needs to be on a scale for project recommendations to be applicable elsewhere
Ensure that projects are outward-looking to safeguard against accusations of bias
(Perth focus group)
Project scale and diversity for impact in practice
Data collection needs to be on a scale for project recommendations to be applicable elsewhere
Ensure that projects are outward-looking to safeguard against accusations of bias
(Perth focus group)
Face-to-face dissemination for impact in practice
Exploit face-to-face dissemination as the practitioners’ favoured route
Target dissemination according to career stage preferences
Invest project budget in training activities for project legacy
Launch events effective for policy maker target audiences
(Poll, focus groups, case studies)
Face-to-face dissemination for impact in practice
Exploit face-to-face dissemination as the practitioners’ favoured route
Target dissemination according to career stage preferences
Invest project budget in training activities for project legacy
Launch events effective for policy maker target audiences
(Poll, focus groups, case studies)
Accessible textual sources for impact in practice
Tailor a variety of reports according to intended audiences: tools they can access, career stage, sector…
Make reports accessible: easy to locate e.g. open access; understandable, e.g. possible to act on project outcomes
Work with profile dissemination partners (e.g. HEFCE, JISC, OCLC, Pew, RIN, SCONUL) to enhance authority of work and increase its reach
Ensure work is picked up by digest/alert services and bloggers
Accessible textual sources for impact in practice
Tailor a variety of reports according to intended audiences: tools they can access, career stage, sector…
Make reports accessible: easy to locate e.g. open access; understandable, e.g. possible to act on project outcomes
Work with profile dissemination partners (e.g. HEFCE, JISC, OCLC, Pew, RIN, SCONUL) to enhance authority of work and increase its reach
Ensure work is picked up by digest/alert services and bloggers
Conference registration openConference registration open
Training day announcementTraining day announcement
Publication of research reportPublication of research report
Study day announcementStudy day announcement
Study day announcementStudy day announcement
Conference registration to closeConference registration to close
Conference registration openConference registration open
Publication of blog postPublication of blog post
http://www.twitter.com/LISResearch provides an alert serviceshttp://www.twitter.com/LISResearch provides an alert services
Exploit network hubs5 key individuals in UK emerge from RiLIES project…
Exploit network hubs5 key individuals in UK emerge from RiLIES project…
5 UK research connectors 1. Rhona Arthur @RhonaEA5 UK research connectors 1. Rhona Arthur @RhonaEA
5 UK research connectors 2. Andrew Booth @scharrlock5 UK research connectors 2. Andrew Booth @scharrlock
5 UK research connectors 3. Phil Bradley @Philbradley5 UK research connectors 3. Phil Bradley @Philbradley
5 UK research connectors 4. Alison Brettle @BrettleAli5 UK research connectors 4. Alison Brettle @BrettleAli
5 UK research connectors 5. Hazel Hall @hazelh5 UK research connectors 5. Hazel Hall @hazelh
Face-to-face dissemination for impact in practice
What happens when formal opportunities for face-to-face dissemination are restricted:
training/travel budgets cut?
reduced professional memberships?
… and when research infrastructures are fractured? This has been addressed in UK in the formation of the Library and Information Science Research Coalition
Face-to-face dissemination for impact in practice
What happens when formal opportunities for face-to-face dissemination are restricted:
training/travel budgets cut?
reduced professional memberships?
… and when research infrastructures are fractured? This has been addressed in UK in the formation of the Library and Information Science Research Coalition
Researcher priorityPresent research output in an accessible way
Service manager priorityFacilitate research output consumption, e.g. reverse mentoring, job descriptions, reporting back from conferences
Course/accreditation priorityApplication of research in practice
Researcher priorityPresent research output in an accessible way
Service manager priorityFacilitate research output consumption, e.g. reverse mentoring, job descriptions, reporting back from conferences
Course/accreditation priorityApplication of research in practice
A greater role for social media?
Currently divides the practitioner community, e.g. blogs very popular with academic librarians but inaccessible to some others; Twitter essential to some, irrelevant to others
A greater role for social media?
Currently divides the practitioner community, e.g. blogs very popular with academic librarians but inaccessible to some others; Twitter essential to some, irrelevant to others
http://www.bite.ca/bitedaily/2010/07/social-media-monopoly-board-game/
A greater role for self-organising networks as communities of practice?
(with technical infrastructure provided by social media)
A greater role for self-organising networks as communities of practice?
(with technical infrastructure provided by social media)
Research funding to support researcher-practitioner links
http://www.lisresearch.org/dream-project
Research funding to support researcher-practitioner links
http://www.lisresearch.org/dream-project
DREaM project launch conferenceTuesday 19th July, London
Places still available
DREaM project launch conferenceTuesday 19th July, London
Places still available
LIS impact takes a number of forms Research impact measurement is tricky Research impact measurement matters Where impact is measured as a dimension of
research quality, a number of strategies can be deployed to extend project outcome reach Preliminary findings of RiLIES project dominated
by influence of dissemination strategy
Key points
Bringing it back to Salford: anyone recognise these lads?Bringing it back to Salford: anyone recognise these lads?
Bringing it back to Salford: anyone recognise these lads?Bringing it back to Salford: anyone recognise these lads?
What difference does it make? (1984)
What difference does it make? (1984)
Playlist 20 March 1984
Hand in glove; Still ill; Heaven knows I’m miserable now; This charming man; Girl afraid; Pretty girls make graves; This night has opened my eyes; Barbarism begins at home; What difference does it make? I don’t owe you anything; Handsome devil; You’ve got everything now; These things take time
Playlist 20 March 1984
Hand in glove; Still ill; Heaven knows I’m miserable now; This charming man; Girl afraid; Pretty girls make graves; This night has opened my eyes; Barbarism begins at home; What difference does it make? I don’t owe you anything; Handsome devil; You’ve got everything now; These things take time
Playlist 20 March 1984
Hand in glove; Still ill; Heaven knows I’m miserable now; This charming man; Girl afraid; Pretty girls make graves; This night has opened my eyes; Barbarism begins at home; What difference does it make? I don’t owe you anything; Handsome devil; You’ve got everything now; These things take time
Playlist 20 March 1984
Hand in glove; Still ill; Heaven knows I’m miserable now; This charming man; Girl afraid; Pretty girls make graves; This night has opened my eyes; Barbarism begins at home; What difference does it make? I don’t owe you anything; Handsome devil; You’ve got everything now; These things take time
Playlist 20 March 1984
Hand in glove; Still ill; Heaven knows I’m miserable now; This charming man; Girl afraid; Pretty girls make graves; This night has opened my eyes; Barbarism begins at home; What difference does it make? I don’t owe you anything; Handsome devil; You’ve got everything now; These things take time
Playlist 20 March 1984
Hand in glove; Still ill; Heaven knows I’m miserable now; This charming man; Girl afraid; Pretty girls make graves; This night has opened my eyes; Barbarism begins at home; What difference does it make? I don’t owe you anything; Handsome devil; You’ve got everything now; These things take time
Photo credits
Image: 'I Can Gather All the News I+Need+on+the+Weather+Report' http://www.flickr.com/photos/51035555243@N01/2671536366
Image: 'These Books are Sweet' http://www.flickr.com/photos/47823583@N03/4474869956
Image: ‘Radar Graph – Library Usage v Final Grades' http://www.flickr.com/photos/davepattern/4172110833/in/set-72157622847182901/
Image: '50lb weights' http://www.flickr.com/photos/62327186@N00/2185667691
Image: 'Article-level metrics at PLoS' http://www.flickr.com/photos/14829735@N00/3929622080
Image: '365: day 119' http://www.flickr.com/photos/33563858@N00/2451800342
Image: 'The locker misery' http://www.flickr.com/photos/33685308@N00/513100970
Image: '90" engine valves' http://www.flickr.com/photos/61132483@N00/2975215
Image: 'Oscar' http://www.flickr.com/photos/45756179@N03/5068766020
Image: 'ed vaizey' http://www.flickr.com/photos/25414047@N00/4065621328
Image: 'Greer Garson & Robert Young' http://www.flickr.com/photos/44186100@N00/2490143154
Image: ‘Waiting for the library to open’http://www.flickr.com/photos/davepattern/1424246518/in/set-72157602122007409
Image: 'BC007' http://www.flickr.com/photos/51548378@N05/4745027706
Image: 'Money to burn' http://www.flickr.com/photos/8363329@N05/811315055
Image: 'Warhol's Light Bulbs' http://www.flickr.com/photos/66814335@N00/3036254720
Image: 'Cheers' http://www.flickr.com/photos/71477195@N00/204879104
Image: 'P1010121' http://www.flickr.com/photos/67614763@N00/4304374413
Image: 'My social Network on Flickr, Facebook, Twitter+and+MyblogLog' http://www.flickr.com/photos/49503019876@N01/1824234195
Smiths at Salford Lads Club (inside cover of The Queen is dead)by Stephen Wright
Brookes, B.C. (1976). A new paradigm for information science. The Information Scientist, 10(3), 103-12.
Peter Cruickshank, Ella Taylor-Smith, Jenny Gebel (RiLIES project team)
Stephanie Kenna (LIS Research Coalition) All those who have contributed data to date
Reference & acknowledgements
Professor Hazel [email protected] @hazelh