Upload
elaine-beasley
View
215
Download
2
Tags:
Embed Size (px)
Citation preview
2
General Issues
RAE: Research Assessment Exercise Research quality over period 2001–07
to be assessed a longer period than usual
Criteria and working methods published – lesson, read the exam question!
A new grouping of sub-panels into main panel areas main panel 36
Funding to be affected from 2009-10
3
Make up of the panel Panel members plus 3 – sub panels
Accounting and Finance Economics and Econometrics Information Management
Employment Relations Entrepreneurship Innovation Leadership Management Education and Management Learning Management Science Organisational Psychology Public Management and Administration Service Operations Small Business Strategic Management and Technology Technology Management
4
Additional specialist advice Corporate Social Responsibility International Business Tourism The more mathematic aspects of Operational
Research
The panel recognised that outputs couldn't always be characterised as falling neatly into disciplines and that in considering how best they might be distributed and assessed we were sensitive to other categorisations such as theme, sector or function
5
Main
pan
el I: C
hair
David
Otl
ey
34: Economics & econometrics (David Greenaway, Nottingham)
35: Accounting & finance (Andy Stark, MBS)
36: Business & management (Mike Pidd, Lancaster)
37: Library and information mgt(John Feather, Loughborough)
Ray Paul
David BlackabyJane
Broadbent
Structure of main panel I
6
Some history
Previous proper RAEs in 1992, 1996, 2001 Applied a single rating to the whole of a
submission
2001: each output rated as International, National or Sub-national quality 5* implies
>50% output of International standard Very little output rated below National level
Grade applies to all staff submitted, however
7
Main differences from 2001
£0.00
£5.00
£10.00
£15.00
£20.00
£25.00
£30.00
5* 5 4 3 2 1
2001 RAE rating
£000s/head
R monies 2001 Same amount for all staff in
department Leads to cliff edge funding
R monies post-2008 No cliff edge Profile funding
Main panel I Accounting & finance Economics & econometrics Business & management
Studies Library & information
management Staffing rules: no overlap period
8Dual funding system for research
Research income
HEFCE R moniesBased on RAE
Pays for research time &
infrastructure
Other research income, including research councilsPays for research
projects & programmes
9
RAE 2008 quality profile Quality level: 4*, 3*, 2*, 1*, unclassified
Unclassified = zero stars Percentage of research activity in each
category Based on FTE staff submitted No requirement to state % submitted
To apply to research output, the research environment indicators of esteem & impact
10RAE 2008 quality criteria for outputs
Its originality, significance & rigour
As a possible point of reference* in field or sub-field
4* World leading Primary
3* International standard of excellence
Major
2* International quality Contributes
1* National quality Limited contribution
u/c Below national
* To knowledge, theory, policy or practice
11
Judgements of quality Originality
e.g. innovation or distinctiveness of the methodological approach
Data sets used Research questions posed Underlying hypotheses or theoretical framework
Significance Insight and scope of coverage of the work Impact on the discipline in the UK or internationally Extent to which has opened up new areas of research Current or potential impact on policy and practice
Rigour Contextualisation of the work Strength ,appropriateness and intellectual coherence Extent to which the research outcomes are supported
12
Elements of assessment
Researchenvironment
(20%)
Esteem & impact
indicators (10%)
Research outputs: 4 per person (70%)
Quality profile
Weighted and aggregated across each submission
e.g.Research incomePhD studentsStaff development
13RAE 2008: calculation of R monies
Quality level 4* 3* 2* 1* u/c
% research activity
20
25
30
15
10
e.g. Univ of North Midlands enters 50 FTE staff
!!!staff!not papers are stars
and stars for income/FTE :Where
.15.30.25.20 1234
iR
RRRRIncome
i
We do not know what the R values will be
14
RAE 2008 important dates 2008
17th December profiles given to University Vice Chancellors for their institutions
18th December Profiles published in the press for all institutions
2009 4th January institutional feedback given to
UoAs and output, environment and esteem indicators released
March Full submissions published for all institutions which will include papers submitted
15
Research outputs For established staff: 4 per person expected
Unless work is exceptional Or time out from research (apply pro rata rule) Part-timers (apply pro rata rule) Multi-authored work: avoid joint submission from
same department unless work is exceptional Different for early career researchers
Should flag up (possibly) Exceptional work (e.g. potential 4* but not in top-
ranked outlet) Early career researchers People who’ve had time out or part-time (equal
opps) Category C ??
16
The numbers
90 submissions 97 3300 category A FTE staff (3500 on
headcount) 3000 700 category B 50 category C 12,600 outputs 10,000
70% outputs 20% environment 10% esteem
17
Managing the process Of the 12,600 about 10% were cross referred to i
34, 14% to i 35 and 4% to i 37 All outputs were read in detail Each panel member selected 4* outputs and
these were discussed to ensure a common process of calibration
Some submissions were clearly not for Business and Management
For environment and esteem sub panel members were asked to read submissions and profiles were decide by the whole panel
A user member was a member of the panel and where submissions proved evidence of a research environment that was geared to policy and practice appropriately high marks were awarded
18
Research Income In 2001 (a 5 year period) total research income
was £200m In 2008 (a 7 year period) total income was £360m
OST/OSI funding £90m
In 2001 the no of research associates was 500+ In 2008 the number of research associates was 470
In 2001 PhD students numbered 2,600 In 2008 PhD students numbered 3,450, 4.87 per
research active FTE – so room for growth! Some high numbers (e.g. 12) were associated with low quality profiles
19
Issues for HRD Issues relating to ‘field journals’ and their
quality in respect to building a knowledge base Other sub disciplines in a similar position
Hospitality and tourism Entrepreneurship and small business
Issues relating to judging impact in relation to policy and practice
Theory vs Empirical vs Applied Issues relating to Journal ‘quality’ impact
factors and citations Listings of journal quality Published impact factor data
Location of journals Quantitative verses Qualitative
20
Final Thoughts – does HRD need to ‘up its game’? Something about the development of theory in
the field – so what is being added to what? We need to be better connected to the knowledge base
Something about research design. Currently a great deal relies on the use of cases – how well do we theorise from the cases and can we be more innovative
Something about the originality of method – how can we be more rigorous in the methods we use and be able to make claims for policy and practice
Something about being picked up in citations and how we write, e.g. Jeff’s - ‘what a load of bollock’ paper: a story of the hairdresser and his suit’
21
Early career researchers
Central RAE definition “Entered the academic profession on
employment terms that qualified them for submission to RAE 2008 as Category A staff on or after 1st August 2003.”
Submission requirements: Business & Mgt Appointed 1/8/03 to 31/7/05: normally 2
outputs Appointed 1/8/05 or later: normally 1 output Working papers may be submitted The denominator will be adjusted so there is no
point submitting more than this