RESEARCH AND THEORY

Towards a framework forresearch career development

An evaluation of the UK’s Vitae ResearcherDevelopment Framework

Robert Bray and Stuart BoonCentre for Academic Practice & Learning Enhancement (CAPLE),

University of Strathclyde, Glasgow, UK

Abstract

Purpose – The purpose of this paper is to evaluate the use of a framework and planner for researcherdevelopment introduced into the United Kingdom in 2010 by Vitae: an organisation whose purpose isto support the development of UK researchers.

Design/methodology/approach – Qualitative and quantitative data from two cohortsparticipating in an accredited researcher development course designed and delivered by the Centrefor Academic Practice and Learning Enhancement at the University of Strathclyde were analysed.

Findings – Participants reported that the Vitae Researcher Development Framework (RDF) personaldevelopment planner (PDP) was useful in facilitating career development. Most found it relatively easyto use once initial perceptions of the tool as being overly detailed and complex were overcome.In addition, some technical problems with the software were identified. There was great variation inthe manner in which the RDF was used (for instance in the number of descriptors selected). Althoughuse was highly individualised, the full range of descriptors was used between the course participants.

Practical implications – The results suggest that the RDF PD planner has great potential insupporting researcher development, provided certain specified conditions are met – in particular theneed to ensure individualisation, support, and researcher ownership of the outcomes. Furtherevaluation is necessary.

Originality/value – This is the first report on the RDF PDP being used in a researcher developmentcourse.

Keywords United Kingdom, Researcher development, Career development, Professional development,Early career researchers, Research staff, Vitae, Training

Paper type Research paper

IntroductionWithin higher education (HE), early career researchers particularly, and research staffmore generally, have, until recently, been an often neglected and marginalised group(Hakala, 2009) with little structured support provided for their career development(Akerlind, 2005; Lee et al., 2010). This lack of recognition and support – and theconcomitant negative consequences for both individuals and institutions – are only nowreceiving the proper attention they deserve. In the last decade a number of landmarkinitiatives have highlighted the need for comprehensive changes in policy and practiceregarding developmental opportunities and support for career researchers. Significantamong these are the launch of the European “Charter for Researchers and Code ofConduct for their Recruitment” in 2005 and, in the UK, Sir Gareth Roberts ’ (2001)

The current issue and full text archive of this journal is available at

www.emeraldinsight.com/2048-8696.htm

Evaluation ofthe Vitae RDF

99

International Journal for ResearcherDevelopment

Vol. 2 No. 2, 2011pp. 99-116

q Emerald Group Publishing Limited2048-8696

DOI 10.1108/17597511111212709

“SET for Success” report and the re-launch of “The Concordat to Support the CareerDevelopment of Researchers” (2008). Within the UK, policy initiatives are now focusedon the Vitae programme, inaugurated in 2008 and funded by Research Councils UK(RCUK). Vitae’s remit is to promote and support researcher development in the UK.It incorporates the work and responsibilities of the UK GRAD programme forpostgraduate researchers and the UK Higher Education Researcher Developmentnetwork for research staff (Vitae, 2010a; UK Concordat to Support the CareerDevelopment of Researchers, 2005; UK GRAD Programme, 2001).

These various initiatives now provide a structural framework for on-goingdevelopment of researcher career development within UK higher education institutions(HEIs), supporting a range of researchers from postgraduate students to early careerresearchers (e.g. post-doctoral, etc.) and staff researchers (e.g. individuals usuallyemployed on fixed term contracts to undertake specific research) through to traditionalacademics. However, the implementation of such initiatives requires the development ofspecific and suitable methods of support, as the means and techniques used for careerdevelopment in other sectors may not prove appropriate for university researchers.

McMahon et al. (2004) point out that traditional career counselling has relied on “test andtell” approaches: the use of quantitative tests, derived from occupational psychology andcarried out by experts, to identify appropriate career paths. More recent approaches rejectthis methodology in favour of qualitative, discursive approaches that usefully emphasisethe subject’s own understanding of their circumstances and opportunities (MacMahon andPatton, 2006; Maree, 2010). However, few such practical techniques have been developed todate. The Researcher Development Framework (RDF), developed by Vitae in collaborationwith the UK HE sector and other stakeholders, represents a productive midpoint betweenthese two differing approaches in that it requires self-assessment of perceived attributesagainst an objective and quasi-quantitative scale without the use of psychologicalconstructs. It is, therefore, essentially an empirical, heuristic technique designed to be ofpractical relevance at any level of research career development.

The RDF was developed to provide a technique for the recording and analysis ofresearcher attributes. Its initial development was characterised by extensive research andconsultation. Vitae initiated the process in 2009 with the intention of designing a model ofprofessional development and support that built on the Joint Skills Statement (2001),a product of the UK GRAD programme in collaboration with RCUK and the HE sector.The development process included an extensive literature review and interviews withover a hundred researchers (Vitae, 2010a). A draft of the RDF was made available forconsultation in 2009 and was followed by the full launch in 2010 (Lee et al., 2010). At thetime of the launch, Vitae unveiled the RDF career and professional development (CPD)tool: an interactive, Excel-based version of the original paper-based RDF, intended forpersonal use. This prototype was renamed the RDF professional development planner(RDF PDP) in 2011, and we refer to it henceforth as the RDF planner.

The RDF was designed for “planning, promoting and supporting the personal,professional and career development of researchers”, enabling researchers to assesstheir “knowledge, skills, behaviours and personal qualities” against explicit standards,in order to encourage them “to aspire to excellence” (Vitae, 2010a). The RDF providesa matrix of descriptors – research attributes – at up to five different phases of expertise.A total of 63 descriptors are organised within four domains and 12 sub-domains(Figure 1).

IJRD2,2

100

Individual descriptors represent attributes and capacities of successful researchers,encompassing:

[. . .] the knowledge, intellectual abilities, techniques and professional standards to doresearch, as well as the personal qualities, knowledge and skills to work with others andensure the wider impact of research (Vitae, 2010b).

For example, within the sub-domain of A2 (cognitive abilities), the five descriptorshighlight the need for capacity in analysing, synthesising, critical thinking, evaluatingand problem solving, respectively. For each of the 63 descriptors, the RDF furtherdescribes between three and five levels of ability, allowing the researcher to align heror his current stage of development to the chosen descriptor.

Since descriptor phases describe different levels of expertise with a particular skill orattribute, the researcher must identify the phase that best matches her/his currentexpertise. For example, phase one of the collegiality descriptor in the sub-domain D1

Figure 1.The four RDF domains

and 12 sub-domainsSource: Vitae (2010b)

Evaluation ofthe Vitae RDF

101

(working with others) is exemplified by the baseline traits “Shows considerationto others” and “Listens, gives and receives feedback and responds perceptively to others”(Vitae, 2010b). Phases four and five of the same descriptor highlight themore sophisticated traits of acting as an “exemplar for collegial behaviourin department/institution” who “cascades knowledge” and “solicits and attends tofeedback from colleagues at all levels” (Vitae, 2010b). Thus, a clear trajectory is providedfor researchers interested in improving their collegiality between phase one and phasesfour and five of the descriptor.

An individual researcher’s use of the RDF planner is intended to involve fourdiscrete stages of activity:

(1) Descriptor selection: the user identifies which of the 63 descriptors s/he isinterested in examining. There are no requirements for how many descriptorsshould be selected.

(2) Phase selection: the user identifies which of the five phases best represents“where s/he is now” and which end-state phase represents “where s/he wouldlike to be”. Choices are recorded and the user provides evidence of her or hiscurrent phase. The process is intended to be both reflective and iterative: theuser may return to and amend her/his earlier entries at any time; descriptorsmay be added or deleted.

(3) Planning: for each descriptor selected, the user is asked to answer and/orprovide the following:. How will you complete your target phase?. How will you measure it?. A target date.

(4) Reporting: the completed descriptors may be saved and printed as a report.

Table I, presents an example of how one researcher in our sample completed the D2descriptor relating to publications and with specific plans for future outcomes andactivities. Columns 2 and 3 present the precise wording provided within the RDF,content presented in italics in columns 4-6 represents the user’s responses and her/hisselected target date.

During 2010 the Centre for Academic Practice and Learning Enhancement,the professional development unit for the University of Strathclyde, developeda new pathway through its Postgraduate Certificate/Diploma in AdvancedAcademic Studies (PG Cert AAS) programme specifically for early career researchersand more-established research staff. This new researcher development pathwaycomplements a long established and successful programme for academic teaching staffaccredited by the Higher Education Academy (HEA, 2008) – an organisation funded byUK HE funding bodies to provide “national leadership in developing and disseminatingevidence-informed practice about enhancing the student learning experience”. Withinthe researcher development pathway a new core module entitled “Building a successfulresearch career” was designed and first ran in September 2010 and then for a second timein January 2011.

A total of 33 participants split over two cohorts (the September 2010 and January2011 cohorts) have, at the time of writing, completed the module. Tables II and IIIpresent breakdowns of the module participants by role and affiliation. As can be seen,

IJRD2,2

102

D2

(com

mu

nic

atio

nan

dd

isse

min

atio

n)

Cu

rren

tp

has

eT

arg

etp

has

eE

vid

ence

tosu

pp

ort

you

rcu

rren

tp

has

e

How

wil

ly

ouac

hie

ve

you

rob

ject

ives

?H

oww

ill

you

mea

sure

it?

Tar

get

dat

e

Pu

bli

cati

onR

egu

larl

yp

ub

lish

esan

dis

inv

olv

edin

edit

ing

/may

be

edit

orof

nat

ion

alp

ub

lica

tion

.A

ims

for

the

mos

tp

rest

igio

us

pu

bli

cati

onin

acad

emic

and

non

-aca

dem

icou

tlet

s.A

ctiv

ely

seek

sco

llab

orat

ive

and

/or

inte

rdis

cip

lin

ary

par

tner

s;is

lead

auth

oron

co-a

uth

ored

outp

uts

.S

up

por

tsan

den

able

sle

ssex

per

ien

ced

rese

arch

ers

top

ub

lish

.W

illi

ng

lyp

eer

rev

iew

sp

ub

lica

tion

s

Ch

oose

sto

acti

vel

yp

ub

lish

ina

var

iety

ofou

tlet

s,so

met

imes

soli

cite

dco

ntr

ibu

tion

s;is

inv

olv

edin

edit

ing

/is

edit

orof

inte

rnat

ion

aljo

urn

alor

oth

erfo

rmof

dis

sem

inat

ion

.T

arg

ets

rig

ht

jou

rnal

s/ou

tlet

sto

gai

nan

“ex

ten

siv

etr

ack

reco

rdof

hig

hq

ual

ity

pu

bli

shed

rese

arch

”

Multiplepublications,

multipleforums,

including

collaborations

Improveresearch

output.Make

more

timeforpaper

writing

Specifically

timetable

paperwriting

time

January

2011

Table I.An example of one user’s

completion of the RDFPDP in relation to

descriptor D2 of the RDF

Evaluation ofthe Vitae RDF

103

15 participants held traditional academic positions (e.g. lecturer, senior lecturer, etc.)that included both teaching and research responsibilities, while a further14 participants held research-only roles. Three participants held teaching-only posts,but were interested in taking on a research role.

These data and our experience in delivering the module combine to suggest that thecourse and its content are appropriate for, and appeal to, a wider participantconstituency than one comprising research staff only (as was originally the intent).Moreover, two participants in academic support roles from the University ofStrathclyde’s professional services – again, interested in taking on a research role – alsotook the module (Table III). Data relating to career stage were not collected, although theposts/roles held by participants suggest a split between those in early career positions(e.g. research assistant/associate) and those who were more established and/or withmore experience (e.g. academic and research and teaching fellows).

From design to delivery, use of Vitae’s RDF was integrated into this new module.The RDF planner is incorporated as a central component in the first of threecoursework assignments, where participants undertake a personal self-evaluation anddesign an action plan (abbreviated as the PEAP). Module participants made use of theRDF planner as a structured approach to this evaluation and their subsequent plans.(The detailed instructions provided to participants are provided in an appendix to thispaper.) The RDF planner is also used extensively in the second courseworkassignment, which requires participants to use the objectives identified in their firstassignments to each design a substantial, developmental project. Each participant isrequired to provide a detailed rationale for her or his chosen activity, emphasizingpersonal outcomes and how those outcomes should be revisited and amended oncompletion of the project. The purpose of the assessment exercises and the nature ofthe structured approach taken are explained in the first classroom-based session,which also includes a detailed introduction to the RDF and RDF planner. Participantsalso receive individual support on demand, are allocated to supportive, collaborative

Faculty affiliation Number

Business 3Engineering 18Humanities and social science 6Science 4(Non-faculty) professional services 2

Table III.Breakdown of “building asuccessful researchcareer” moduleparticipants (bothcohorts) by universityfaculty affiliation

Position held in the university Number

Academic support 2Academic 15Research assistant 2Research associate 2Research fellow 9Teaching associate 2Teaching fellow 1

Table II.Breakdown of “building asuccessful researchcareer” moduleparticipants (bothcohorts) by role

IJRD2,2

104

working groups of three or four peers, and are encouraged to reflect upon and discussdifferent means of developing chosen attributes and/or capacities.

Further key features of the implementation and use of the RDF and the RDF plannerwithin the “Building a successful research career” module are that:

. it is a compulsory, assessed element within an accredited course which is takenvoluntarily as part of Strathclyde University’s PGCert AAS;

. there is no prescribed way of completing the RDF: participants are encouraged touse it in any fashion that suits their personal and professional circumstances;and

. use of the RDF was necessarily time-limited, being confined to a single semester.

Our experience over the past 18 months has shown that the use of the RDF opens up newand exciting possibilities for researcher development in HEIs. The novelty of the RDF –and specifically of the RDF planner – has meant that is has not yet been subjectto systematic, independent evaluation of its use in the field. This paper seeks to initiatethat process, focusing on three primary areas of evaluation: utility, accessibility, andusage. By “utility”, we mean the extent of the planner’s usefulness and relevance to thecareer development of the participants; by “accessibility”, we mean the degree of ease inusing the planner, and the extent and nature of any particular difficulties experienced;and by “usage”, we mean the overall pattern of use in terms of the number of descriptorsexamined by participants, and the specific domains and sub-domains that were selected.

Additionally, Vitae itself has identified a number of specific concerns about theapplication of the RDF (Vitae, 2010a) in a range of areas during the initial consultation,namely:

. The RDF as a tool for personal development: should the framework be linked toappraisal and performance review?

. The generic framework: how does the language and content work for allresearchers?

. The tools and resources: how will an online tool for researchers work? How canthe RDF best be presented to a range of stakeholders?

We consider each of these questions in our discussion below.

MethodAs Walsh et al. (2010) point out, programme evaluation may suffer from over-relianceon a single evaluation methodology. Accordingly, a number of different approaches toevaluation of the RDF with the participants were employed, capturing bothquantitative and qualitative data. For both cohorts qualitative data were obtained viaquestionnaire, module evaluation forms, feedback provided in classroom discussionsand comments provided in an online forum. Quantitative data were taken directly fromthe submitted RDF plans and used to interrogate how the planner was used and, inparticular, which descriptors were selected.

The questionnaire was originally designed to gather responses to two broad,evaluative questions: first, how usable was the RDF planner as a tool for researcherdevelopment (e.g. how effective was it in helping participants to considering ways todevelop research skills and capacities, etc.)?; and second, how personally meaningful

Evaluation ofthe Vitae RDF

105

were the outcomes derived from engaging with the RDF (e.g. was the process relevant,inspirational, and/or successful in producing meaningful change, etc.)? A response rateof 58 per cent was obtained from the first cohort (n ¼ 19) and 62 per cent from the second(n ¼ 14).

With the second cohort, two additional questions were added to focus on specificdifficulties regarding the usability of the planner expressed by the first cohort, and toascertain whether participants felt that the RDF descriptors accurately reflected thedevelopmental needs of researchers. Module evaluation forms for all the participants(n ¼ 33) were collected and comments taken from them were collated with feedbackfrom discussions in class and online. These qualitative data were then subjected tomodified content analysis in relation to:

. overall usefulness and relevance of the RDF; and

. how the RDF planner could best be used.

ResultsWe present our results in relation to each of the issues identified above.

1. Utility: how useful and relevant was the planner to the career development of theparticipants?This question was addressed using qualitative data from questionnaires, moduleevaluation forms, feedback provided in classroom discussions and comments providedin an online forum. Additionally, our analysis incorporated representative quotationsfrom participants’ comments on evaluation forms and the online forum. The majority ofcomments (30 out of 33) generated by the questionnaires indicated that participants hadfound the RDF planner to be useful. The following are indicative: “the PEAP was themost important part of the entire module”[1], “a useful tool and worth completing”. Onlya small minority of comments (3 out of 33) were negative, for example: “the PEAP wascompleted in hope that it would give insights – not particularly beneficial [. . .]”, and “thePEAP is more like paperwork and too ‘rational’.”

The last comment suggests that individual personality differences might besignificant in how useful participants find the planner. Some, for example, reported aself-conscious predilection for this type of activity, for example: “[I] looked forward tousing RDF as I enjoy opportunities to reflect on myself [. . .]”. One comment reflectedevident recognition that some participants might have quite different preferences andnot find this sort of reflection to be either easy or interesting:

My experience with this kind of self-assessment tool is that the success and usefulness of such atool depend a lot on the attitude and seriousness of the person. I think it worked quite well withus since we decided to “play the game” and filled it properly. I can imagine that other people(probably “old school” folks) would not be doing it so seriously. Therefore, rolling out themechanism across the university might not bring as much as we did obtain [. . .]

Another participant suggested that such reflection raises some difficultepistemological questions:

The PEAP process [. . .] was and is a good reflective tool. Regarding underlying weaknessesI agree that this is difficult as the reflective process may be a distorted mirror (adjusted bycertain psychological self-perceptions which may or may not be an objective truth).

IJRD2,2

106

Some participants discussed the different ways in which the RDF planner could beused, and explicitly commented on the way in which it could identify needs andfacilitate planning:

[We] had to think about our strengths and weaknesses, achievements, decide on our ownlearning and to plan our own personal or professional development.

By focussing on where I want to be in 12-18 months’ time, it helped me understand what I neededto achieve and how this could be done. I think many others could benefit from understandingmore fully what areas of their research areas could be developed and strengthened.

From such comments it seems reasonable to infer that module participants recognisedthat the planner had two main functions: first, it helped to identify areas thatneeded development in order to further their research careers; and, second, it initiatedstructured plans for how such development should be implemented. It was alsorecognised that such career development includes a wide number of work-related tasks,not only those concerned with research:

Although the module was mainly about research, I didn’t feel the PEAP was limited to that asit helped me understand how I had to develop supervision skills and how my research mightbenefit from my teaching, and vice versa.

One participant also commented on how the planner was useful in preparing for theuniversity’s accountability and development review (ADR) process and similar annualstaffing review processes:

I [. . .] would suggest that the PEAP be provided as a tool for people to use to help defineobjectives and targets for the current ADR.

However, there was considerable concern that the planner should remain confidentialand not be open to use by others, such as managers: “best not to review with yourmanager/boss.”

This sentiment was one that echoed across many classroom and online forumdiscussions.

This analysis of overall usefulness and relevance found, in summary, that mostparticipants found it relevant to their own needs and provided a means for them to analyseareas that required development as well as helping them plan such development.

2. Accessibility: how easy was the planner to use and were any particular difficultiesexperienced?To address this question we again used qualitative data and collected representativequotations from comments from evaluation forms and online forums.

The majority of questionnaire responses indicated few problems with theRDF planner. Typical comments suggested that it was, for example: “[. . .] prettystraightforward to use”. However, half of the responses also indicated that it had seemedinitially daunting, for instance: “first impressions of the tool are not very good”. Yet all33 of the respondents went on to indicate that the explanations of how the plannerworked (provided through a PowerPoint presentation in the classroom, guidance notesin the class handbook, and in the planner itself) were sufficient to overcome these initialdoubts about accessibility:

Evaluation ofthe Vitae RDF

107

I have just completed the PEAP. Was a bit confused by some things but read through all theinfo in the folder and looked at the presentation PowerPoint’s and it all became clear again!

There were also some technical criticisms, particularly those concerningcross-platform usability, for instance:

I have two minor comments: first, it doesn’t work in the latest version of Office for Mac (2011),so I had to find a Windows machine to run it on, and they are scarce in my research group!Second, I think it would work better as a web-based application. Excel isn’t an applicationdevelopment environment, so any attempts to use it as such generally result in clunky apps(like the RDF tool).

One of the issues most widely discussed by participants in the online forums was howmany descriptors should be completed (the actual number of descriptors selected, andhow they were distributed among the sub-domains, is examined below). No expectationwas specified in the induction session and participants were repeatedly reminded thatthe choice of descriptors was entirely their own; nonetheless, there was some anxietyregarding a “right” way of using the RDF planner. At the same time, moduleparticipants made it clear that reducing the choice of descriptors would resultin insufficient room for an adequate exploration of one’s career needs. At the other endof the spectrum, they felt that choosing too many descriptors would not only betime-consuming but also would necessarily mean that there would be less time for thenecessary depth of explanation in each. In both cohorts some participants initiallyselected more descriptors than they subsequently found time to deal with adequately:

The PEAP did take longer than I thought. Part of it was attributed to the learning curveassociated with using a new tool, but most of it was because I completed 18 descriptors,targeting the PEAP at the next 12-14 months.

Others took strategic decisions and reduced the number of descriptors examined:

It took me a while to do my PEAP, mainly because I think I can improve on most things –I had to work to get it down to a manageable set of descriptors. In the end I focused on what ismost important to me right now.

The planner’s facility for iteration and revision was clearly important in this respect:

It’s quite straightforward to use, but as expected I think I chose too many of the “descriptors”in too many “domains”. They are all relevant, but it is a lot of work to think of the supportingevidence [. . .]. I might go back and take out some of the descriptors.

This last comment raises the issue of evidence. The planner requires “Evidence tosupport your current phase” (Vitae, 2010b) and for some of the more abstract descriptors(such as “perseverance” and “integrity” in Domain B1), recording such evidence clearlyprovides a challenge to some:

I’m already struggling [. . .] providing evidence to support my current phase: I imagine this isfairly common with users of the PEAP tool, but I’m finding it difficult to provide evidence forelements which are difficult to quantify.

One participant found it difficult selecting a single level to reflect her/his current phasein some descriptors:

You should spell out that you can be at different levels at the same time.

IJRD2,2

108

Overall, the problems encountered in using the planner were relatively minor and did notprevent or hinder its completion (in most cases in considerable detail) before a shortdeadline.

3. Usage: what was the overall pattern of use in terms of (a) the number of descriptorsexamined by participants and (b) the specific domains and sub-domains that wereselected?Quantitative data collected from the participants’ submitted personal evaluations andaction plans were used to analyse their usage of the RDF planner as a tool forresearcher development. In total, 31 of the 33 participants submitted completed RDFaction plans. Of the remaining two, one had submitted only a summary that identifiedthe key development needs and action planning points, while the other had used anearlier version of the RDF. Submissions were evaluated in terms of:

. the number of descriptors examined by each participant; and

. the frequency with which descriptors from specific domains and sub-domainswere examined.

3(a) The number of descriptors examined by participants. In terms of the total numberof descriptors examined by each participant, there was a quite remarkable degree ofvariation, with the minimum number being 3 and the maximum 52. However, of the31 participants only four had examined more than 20 descriptors, although this didinclude one participant’s exploring 52 of the RDF’s 63 descriptors. The mean valuenumbers of descriptors completed was 13.1, but as the frequency distribution was sohighly skewed the median value of nine descriptors examined is a more useful statistic.The modal value was 7 and the standard deviation was very high at 11.9. Figure 2,shows the frequency distribution.

The degree of variation suggests that participants used the planner in very differentways, adapting it to their particular and individual circumstances. It should be notedthat all of the submissions were assessed against clear criteria (in addition, fourrequired amendment and resubmission). Thus, even those who chose to work withvery high or very low numbers of descriptors met the requirements for engagement,reflection and the provision of evidence.

Figure 2.Frequency distribution

of participants’descriptor choice

Number of descriptors used

Number of descriptors used

Nu

mb

er o

f p

arti

cip

ants

14

12

10

8

6

4

2

05-9 10-14 15-19 20+0-4

Evaluation ofthe Vitae RDF

109

3(b) Pattern of use in terms of the domains, sub-domains that were selected. The courseteam was initially concerned about whether participants would favour some domainsand sub-domains and neglect others; for instance, would there be a tendency forparticipants to select only sub-domains related directly to the research task (such asA1: “knowledge base”) and avoid those more nebulous sub-domains that dealt with“soft-skills”, such as A3 “creativity” or D1 “working with others”? In order to addressthis question, each participant’s responses were analysed to identify the numbers ofdescriptors selected in each sub-domain. Figure 3, shows a summary of this analysis.

As can be seen, there is a reasonable degree of spread of selected descriptors amongall the domains. The most used sub-domain (“B3 career development skills”, with77 per cent of participants selecting at least one descriptor) was clearly much morepopular than the least used (“C1 professional conduct”, with 26 per cent). However, mostsub-domains received a moderate degree of attention and there was no clustering aroundsome or clear avoidance of others. Moreover, a detailed analysis of each participant’sresponses indicated that very few had concentrated on a narrow range of sub-domains.These findings suggest that the RDF was being used in a holistic fashion, reflecting thefull range of attributes that characterise successful research careers.

To summarise the results, both qualitative and quantitative, these findings suggestthat most of the participants taking the “Building a Successful Research Career” modulefound the RDF planner useful. There was a minority of dissenting voices and this maywell reflect variations in learning styles and preference regarding personal reflection.This suggests that the planner is best used in a voluntary fashion and that any degree ofcompulsion is likely to be met with resistance by at least a significant minority.The importance of individual choice is also reflected in the degree of variation in thenumber of descriptors used by each participant: by emphasising the open-ended natureof the activity, the process enabled participants to use the planner in highlyindividualised ways. This is clearly most appropriate when dealing with phenomena asinherently individualised and personal as research career development.

Figure 3.Percentage of participantsusing each RDFsub-domain

Percentage of participants using sub-domain

A1 Knowledge base

0 20 40 60 80 100

A2 Congnitive abilities

A3 Creativity

B1 personal qualities

B2 Self-management

B3 Career Development Skills

C1 Professional conduct

C2 Research management

C3 Finance, funding, resources

D1 Working with others

D2 Dissemination

D3 Engagement and impact

IJRD2,2

110

For most participants the RDF planner was suitably easy to use. However, initialimpressions – in some case, misconceptions – of size, complexity and an overlyrestrictive structure need to be overcome by clear guidance and ready support. Takentogether, this guidance and support enables participants to move quickly beyond theirinitial doubts and to engage meaningfully with the detailed content offered in theplanner. At this stage the emphasis on individual choice in the number of descriptorsselected seemed particularly important in facilitating engagement.

Participants found the planner useful for identifying career needs and for planningdevelopment activities to meet them. Many also felt that it contributed substantially totheir preparation for staff review. However, there was a clear preference that ownershipof the planner should remain with the researcher and that data should not be sharedwith managers. Suggestions that the planner should be embedded within institutionalweb pages met with considerable concern regarding the guarantee of confidentiality.While there was widespread agreement that the RDF could (and should) be used muchmore widely within the university, there was also a perception that there weresome dangers inherent in using the planner in a top-down, overly corporate fashion.This corroborates concerns uncovered during Vitae’s consultation process, and raisedby many within the HE sector, that the RDF should be differentiated from humanresource functions and thus not be used for job descriptions, person specifications, orappraisal systems (Vitae, 2010a).

The use of the RDF within the University of Strathclyde programme was alsoanalysed using the Rugby Team Impact Framework (RTIF): an evaluation modeldeveloped specifically for researcher development activities in HE (Bromley et al.,2008) [2]. The RTIF model uses a “pathway” approach, within which the impact ofdevelopment activities can occur at five points or levels: foundations (the development ofinfrastructure); reaction (responses of participants); learning (change in attitudes,knowledge and/or skills); behaviour and outcomes. Applying this model to ourevaluation of the RDF planner, it is clear that there has been significant impact in termsof reaction (i.e. positive attitudes towards using the planner) and learning (e.g. theassessment of the work submitted by participants showed that learning outcomes wereachieved). There was also substantial evidence of change in behaviour, especially fromreflective activities that showed some participants had used the understanding of theircareer development needs to develop new research priorities, for instance, by identifyingkey research problems.

DiscussionThe RDF has been developed within the context of a rapidly evolving conceptualframework for career development interventions. As outlined in the introduction to thispaper, there has been a move away from traditional, positivist approaches that focused onthe need to match individuals to appropriate jobs based on their objective characteristics.Such perspectives, developed in an era when careers were linear, stable and predictable,seem less applicable in the modern occupational context where careers are more dynamic,turbulent and hard to predict (Metz and Guichard, 2009). Accordingly, establishedpositivist theories have been challenged by more person-centred approaches that are oftenlabelled post-modern (Diemer and Gore, 2009; Maree, 2010). These frequently adopta social constructivist approach, de-emphasising objective measures of individualcharacteristics and replacing them with a concern for individual aspirations. Typical of

Evaluation ofthe Vitae RDF

111

such person-centred approaches is the Systems Theory Framework of career development(McMahon et al., 2004), which proposes that clients are guided to create their own careernarratives, wherein meaning is derived from the individual’s own perspective without anynecessity for an external frame of reference.

The implications of this fundamental reorientation for the techniques used indevelopment interventions has been the abandonment by some practitioners ofnomothetic techniques, where inferences and predictions are made from individual inputsby comparison with group measures, usually using quantitative data. They have begun tobe replaced by idiographic methods, where comparisons are made within the individualframe of reference without consideration of external benchmarks or norms (Diemer andGore, 2009). Unsurprisingly, given the quite opposed epistemological and ontologicalassumptions of the traditional and post-modern career theories, there has been muchdisagreement at both theoretical and practical levels. There has been a continuing currentof opinion arguing that its fundamental premises are unsubstantiated; for example, Metzand Guichard (2009) document the level of disagreement within the vocational psychologyfield, in which a significant segment argues that the nature of work has not changed sofundamentally as to justify such radical changes in career theory or practice. From thisperspective there is a continuing need for nomothetic techniques that provide objectivemeasures of individual characteristics that can be compared statistically to grouped data.

Similar issues have arisen in the development of competency-based models.Bromley et al. (2007), who developed a self-assessment method of the skills to bedeveloped by research students, raised the question of how verifiable such methodsare, given what is known of the problems that individuals experience in evaluatingtheir own capability or performance. However, these authors’ technique was developedprimarily to evaluate researcher development programme interventions; it can beargued that methods designed for such a purpose, such as Alpay and Walsh’s (2008)skills perception inventory, require a greater level of external verification andobjectivity (to ensure accountability in the evaluative process), compared to methodsprimarily designed for personal and career development.

The RDF planner lies at a mid-point between these two opposed positions oftraditional, positivist methods and post-modern, phenomenological approaches. On theone hand, it does adopt a social constructivist perspective on the task itself: users areasked to employ their own judgements. Although evidence is required, it is notobjectively verified. Similarly, the output of the planner is an individual profile,without comparison to norm groups. On the other hand, the content of the planner isderived from empirical data, gathered through rigorous and carefully documentedresearch, on the nature of researcher careers. Moreover, this is expressed in aquasi-quantitative fashion, in that five distinct career phases are identified, withinwhich users are asked to place their current level of achievement. The planner cantherefore be viewed as an integrative and synthesising method, recognising theinterdependence of the subjective and the objective aspects of career development, sothat career aspirations can conform to the reality of occupational structures.

If researcher development can be defined as the process by which a person’scapacity to carry out research is enhanced with some degree of permanence (Evans,2011), the question arises from these debates: whose responsibility is suchenhancement? Does it lie primarily with the individual, or with his or her employingorganisation? (see also Evans, 2009, for a discussion of this issue). Is researcher

IJRD2,2

112

development a process that researchers undertake themselves, or that is done to them?This in turn raises question of ownership and control over the development process.While the practice of researcher development has made great strides, such theoreticalconsiderations have been largely neglected. The discussion of the findings from thisstudy should therefore be located within the context of broader and more fundamentalquestions concerning the purpose and direction of researcher development itself.

As discussed in the introduction, Vitae (2010a) reported that it had identified anumber of specific concerns about the application of the RDF. Below, we apply thefindings from our evaluation of the use of the RDF at Strathclyde University to offersome tentative responses to Vitae’s concerns:

. Vitae has posed the question: should the RDF be linked to appraisal andperformance review? Our findings, reflecting the overwhelming preference ofthose researchers participating in the “Building a successful research career”module at Strathclyde, revealed a consensus that the RDF should not be a formalelement of appraisal and performance review, but that it should be madeavailable to researchers in their preparation for such reviews.

. In response to Vitae’s concerns about the generic framework, prompting thequestion: how does the language and content work for all researchers? Our findingssuggest that, taken as a whole, the RDF does work well for most researchers. A smallnumber – whose learning style does not predispose them towards structuredreflection – will not easily engage, but most others will find enough of the contentrelevant to their individual situations to make the planner relevant and useful.

. Addressing Vitae’s questions about the planner and resources: how will anonline planner for researchers work? and how can the RDF best be presented to arange of stakeholders? Our findings suggest that the RDF will work best if someusability aspects are addressed, as the Excel prototype (used by the participantsin this study) provides some sub-optimal features. An updated version of theRDF PDP, which is easier to navigate, is now available. It is understood thatVitae is currently developing an online version of the RDF planner, which aimsto further address these issues.

Bearing in mind the limitations of our study (its specific use of the planner within oneuniversity, with a limited number of participants), it nevertheless reveals that the RDFplanner can be used successfully to contribute to the enhancement of researcher careerdevelopment in HEIs. Indeed, it can provide a meaningful structure to an otherwisecomplex and potential chaotic activity. That said, our findings suggest that effective useof the planner – and, therefore, successful research career development within HEIs –relies on adherence to three key principles: individualisation, support and researcherownership. More specifically, our use of the term “individualisation” conveys our beliefthat the researcher should be free to use the RDF in whichever way best reflects her or hisown circumstances and needs. There should be considerable flexibility in when and howit is used and as few constraints as possible on how many descriptors are selected or howthe “evidence” and “targets” sections are completed. By “support”, we mean that clearguidance should be provided when the planner is first introduced, in order to facilitateusers moving rapidly beyond “first impressions” which create an impression that its sizeand complexity make the planner appear daunting and unhelpful as a developmentaltool. Further documentary (online or offline) guidance and individual support should be

Evaluation ofthe Vitae RDF

113

available on demand throughout the user’s engagement with the planner. Finally, withour use of the term “researcher ownership” we intend to convey our recommendationthat the outcomes of the researcher’s development plans should remain the exclusiveproperty of the individual researcher and no requirement or structure should be put inplace that circumvents this singular ownership without his or her express permission.

Taken in concert with these three principles, we suggest – on the basis of ourevaluation study – that Vitae’s RDF and RDF planner provide a powerful means ofachieving constructive and meaningful researcher development in HE today.

Notes

1. The RDF planner was used within the context of the first class assignment, also known asthe PEAP activity, and many respondents referred to it as such.

2. The “Rugby Team” is a working group with representatives from a cross-section of highereducation bodies and other stakeholders in the UK. The Rugby Team Impact Framework(RTIF) is an evaluation model for training and development activity of early careerresearchers in higher education.

References

Akerlind, G. (2005), “Postdoctoral researchers: roles, functions and career prospects”, HigherEducation Research and Development, Vol. 24 No. 1, pp. 21-40.

Alpay, E. and Walsh, E.A. (2008), “A skills perception inventory for evaluating postgraduatetransferable skills development”, Assessment and Evaluation in Higher Education, Vol. 33No. 6, pp. 581-98.

Bromley, A.P., Boran, J.R. and Myddelton, W.A. (2007), “Investigating the baseline skills ofresearch students using a competency-based self-assessment method”, Active Learning inHigher Education, Vol. 8 No. 2, pp. 117-37.

Bromley, T., Metcalfe, J. and Park, C. (2008), The Rugby Team Impact Framework (RTIF),available at: www.vitae.ac.uk/CMS/files/1.Rugby%20Impact%20Framework_33.pdf(accessed 6 October 2011).

Diemer, M.A. and Gore, P.A. Jr (2009), “Culture and assessment: nomothetic and idiographicconsiderations”, The Career Development Quarterly, Vol. 57 No. 4, pp. 342-7.

Evans, L. (2009), “S/he who pays the piper calls the tune? Professionalism, developmentalism andthe paucity of in-service education within the research profession”, ProfessionalDevelopment in Education, Vol. 35 No. 2, pp. 289-312.

Evans, L. (2011), “The scholarship of researcher development: mapping the terrain and pushingback boundaries”, International Journal for Researcher Development, Vol. 2 No. 2, pp. 75-98.

Hakala, J. (2009), “The future of the academic calling? Junior researchers in the entrepreneurialuniversity”, Higher Education, Vol. 57 No. 2, pp. 173-90.

HEA (2008), “Academy 2008-13 strategic plan”, Higher Education Academy, HEA, available at:www.heacademy.ac.uk/resources/detail/aboutus/2008-13strategicplan (accessed8 September 2011).

Lee, L., Gowers, I., Ellis, L. and Bellantuonoa, I. (2010), “Well rounded postdoctoral researcherswith initiative, who are not always ‘tied to the bench’ are more successful academically”,International Journal for Researcher Development, Vol. 1 No. 4, pp. 269-89.

McMahon, M. and Patton, W. (Eds) (2006), Career Counselling: Constructivist Approaches,Routledge, London.

IJRD2,2

114

McMahon, M., Patton, W. and Watson, M. (2004), “Creating career stories through reflection:an application of the systems theory framework of career development”, AustralianJournal of Career Development, Vol. 13 No. 3, pp. 13-17.

Maree, J.G. (2010), “Brief overview of the advancement of postmodern approaches to careercounselling”, Journal of Psychology in Africa, Vol. 20 No. 3, pp. 361-8.

Metz, A.J. and Guichard, J. (2009), “Vocational psychology and new challenges”, The CareerDevelopment Quarterly, Vol. 57 No. 4, pp. 310-18.

Roberts, G. (2001), “SET for success”, available at: http://webarchive.nationalarchives.gov.uk/;www.hm-treasury.gov.uk/set_for_success.htm (accessed 6 October 2011).

UK Concordat to Support the Career Development of Researchers (2005), “The UK Concordat toSupport the Career Development of Researchers”, available at: www.researchconcordat.ac.uk/ (accessed 6 October 2011).

UK GRAD Programme (2001), Joint Statement of the UK Research Councils’ TrainingRequirements for Research Students, available at: www.vitae.ac.uk/cms/files/RCUK-Joint-Skills-Statement-2001.pdf (accessed 8 September 2011).

Vitae (2010a), Researcher Development Framework: Summary of the Analysis of ConsultationResponses, available at: www.vitae.ac.uk/CMS/files/upload/Vitae-RDF-consultation-analysis-may-2010.pdf (accessed 6 October 2011).

Vitae (2010b), Researcher Development Framework, available at: www.vitae.ac.uk/CMS/files/upload/Vitae-Researcher-Development-Framework.pdf (accessed6October2011).

Walsh, E., Seldon, P., Hargreaves, C., Alpay, E. and Morley, B. (2010),“Evaluation of a programme of transferable skills development within the PhD: viewsof late stage students”, International Journal for Researcher Development, Vol. 1 No. 3,pp. 223-47.

Further reading

European Commission (2005), European Charter for Researchers and Code of Conduit for theirRecruitment, European Commission, available at: http://ec.europa.eu/eracareers/pdf/am509774CEE_EN_E4.pdf (accessed 6 October 2011).

van Vianen, A.E.M., De Pater, I.E. and Preenen, P.T.Y. (2009), “Adaptable careers: maximizingless and exploring more”, The Career Development Quarterly, Vol. 57 No. 4, pp. 298-309.

Appendix. Instructions on using the RDF within the first assignment (from the classhandbook)Building a successful research careerAssignment 1: PEAP. This assignment is based on the Vitae RDF. Vitae have developed adownloadable Continuing Professional Development (CPD) tool using the RDF (http://vitae.ac.uk/policy-practice/1393-291211/RDF-CPD-tool.html).

You will work through this tool, carrying out the following:. Identifying which of the descriptors you want to work on (you can choose as many or a

few as you wish: however, the fewer you choose the greater the amount of detailedinformation who will need to supply).

. For each of these descriptors, identify your current level of working (“current phase”) andthe level to which you aspire at the moment (“target phase”).

. Supply evidence for your current level of attainment.

Evaluation ofthe Vitae RDF

115

. Identify what actions you can take to move towards your target phase (this will form thebasis of your Action Plan), together with target dates and how you will assess yourprogress towards these targets.

. Any other relevant comments.

Note that there is no prescribed best way to use this tool: it is up to you to explore it and decide onhow you can use it most effectively.

Session 2 will present interactive and video-based case studies that will facilitate personalevaluation.

Assessment criteria for the PEAP. You will be expected to:. have completed the RDF CPD tool (you are not expected to comment on all the

descriptors);. have provided appropriate evidence for attainment in your current phase;. identify appropriate means to achieve to target objectives; and. show feasible means by which progress towards objectives can be measured.

Corresponding authorRobert Bray can be contacted at: Robert.bray@strath.ac.uk

IJRD2,2

116

To purchase reprints of this article please e-mail: reprints@emeraldinsight.comOr visit our web site for further details: www.emeraldinsight.com/reprints