49
Vistas The ARMA 2014 Conference Proceedings: Learning from the past; preparing for the future Edition: 01

ERIC - developing an impact capture system

Embed Size (px)

Citation preview

VistasThe ARMA 2014 Conference Proceedings:

Learning from the past; preparing for the future

Edition: 01

2

CONTENTS

Foreword, 3

David Coombe

ERIC: Developing an impact capture system 8

Julie Bayley, Julia Ryall

Shifting to a research culture 14

Susan Blum

Yes, I am a research manager, but I don’t cost bids 19

Sue Starbuck

Internal communication strategies for stimulating and embedding a successful research culture 23

Jenny Rivers, Rachel Curwen, Ruth Sandland

Impact – whose responsibility is it? 27

Elizabeth Garcha, Eliot Marston

Diversification of income streams: bridging the academia/industry gap 32

Andrew Stevenson, Stephanie Maloney

Support structures for public engagement with research 37

Kenneth Skeldon, Lucy Leiper

3

Foreword

This year’s annual conference ‘Learning from the past; preparing for the future’ might equally have

been entitled ‘Speaking truth to power’, owing to hard-hitting speeches by two speakers with high-

level experience of research-informed policy-making from the policy-making angle and the research

angle, respectively. Andrew Miller MP, Chair of House of Commons Select Committee on Science

and Technology critiqued the use (and abuse) of scientific evidence and called for greater openness

to evidence-based policy making, but also for more willingness on the part of universities to involve

themselves in providing and communicating that evidence; Professor Dame Judith Rees, co-Director

of the high-impact Grantham Research Institute on Climate Change and the Environment, President

of the Royal Geographical Society and former Director of the London School of Economics, exposed

the under-funding of science policy and the cross-subsidisation of research by student tuition fees,

alongside the irrationality of building the nation’s science capacity on the vagaries of undergraduate

student demand. Her words were heralded by at least one delegate as likely to ‘transform the sector’;

while the impact is yet to be seen, the speech certainly caused waves among senior funders and finally

brought out into the open an awkward truth. If anyone wants to dispute it – and policy-makers will –

they need look only at the evidence.

The requirement to reorient research

But back to the future (and to past lessons): if Judith gave us an exposé of Government science policy,

Andrew offered a critique of the other side to the uneasy relationship between Government and science

in terms of the use of scientific evidence in areas of policy-making. He criticised the propensity toward

‘policy-based evidence-making’ exemplified by John Howard’s complaint that “policy makers are being

bullied by zealots” (though Judith and her climate change colleagues have experience of bullying in

the opposite direction!) and mainstream media offers of a false ‘balance’ in the face of overwhelming

evidence. But he also sounded an optimistic note, pointing to the significant success of his committee

in promoting the use of science and the increasing number of chief science advisors even at the highest

levels of Government.

Nor did Judith blame all the sector’s ills on science policy. Pressures to publish (or perish) and to

publish in ‘top’ journals, the skewing of the jobs market and internal promotions, and the undervaluing

of innovative and interdisciplinary research cannot be blamed on the Research Excellence Framework

(REF), she said, for the issues are to be found across national boundaries. The blame lies squarely at

the feet of academics themselves, who are making these decisions and forming these judgements. As

we prepare for the future, the call therefore to all research managers and administrators is to look

inside the institution to ensure that the best research is rewarded; and to look outside, to communicate

4

findings to those who can use it – while not forgetting to keep up the pressure for full funding of

research.

As ever, the conference offered a large number of sessions of enormous variety, and provided all

the support needed to look both inward and externally, from the minutia of equipment sharing to

business development strategy, and all points in between: policy and strategies, new and old (and

better) practices, funding opportunities, and many more structured networking opportunities and

international perspectives than ever before.

Industry collaboration can achieve wide economic benefit

The University of Lincoln has been notable over recent years for the high prominence it has achieved

in engaging with its local community and with industry, so the conference was pleased to learn from

those involved how the University went about its strategy, its purposes and its progress. Lincoln’s

success is cited by David Sweeney and others as an exemplar of the role that a university can play at

the heart of its community, acting as an engine of regional economic redevelopment and growth, not

only directly through the student economy, but also through skills training for the regional workforce

and the many wider benefits of scientific development.

Whereas other universities have closed their chemistry and science departments, Lincoln has

received funding from the Higher Education Funding Council of England (HEFCE) to reinvigorate

its Engineering, Chemistry and Mathematics provisions. In their piece (Diversification of income

streams: bridging the academia/industry gap), Andrew Stevenson, Director of Research & Enterprise,

and Dr Stephanie Maloney, Head of Research at the University, provided insights into the strategy and

provide a model example of the holistic institution, where a university development born of a regional

policy priority is rooted in the community, grounded on teaching (and sustained by teaching income)

but ultimately succeeds as a university only by developing its research profile in close cooperation with

industry, which itself feeds back into the student experience.

As a case study in developing a research strategy almost from scratch, Lincoln is worth learning

from. While its research grew initially out of its teaching and its teaching staff, its experiences also

demonstrate the importance of fundamental strategies linked to the recruitment of research staff and

students. Its focus on ‘meaningful research’ raises questions about whether a planned approach to

research development (as opposed to the more laissez faire bottom-up approach) inevitably leads to a

focus on more ‘impactful’ research, or whether the fact that solely research borne out of teaching and

engagement with industry leads in this direction. The interest of Whitehall in Lincoln stems from its

demonstrable and laudable regional impact, which is evident from the data that Andrew and Stephanie

presented. A question for the future is whether a similar case might be made for developing a research

5

profile further from the market (the economy or society) with measures that can demonstrate the

value of such research just as convincingly. I do hope so.

Successful research culture: all for one and one for all

The University of Lincoln example demonstrates the possibility of creating a research culture almost

from scratch. By contrast, in their presentation (Internal communication strategies for stimulating

and embedding a successful research culture), Dr Jenny Rivers, Research & Knowledge Exchange

Manager, University of Liverpool, Dr Rachel Curwen, Research Development Associate, University of

York, and Ruth Sandland, UK Research Funding Manager, Anglia Ruskin University, chose to focus on

the different but equally challenging aspects of effecting cultural change within their own institutions

(all very distinctive, and representative of the common challenges facing us all).

The role of the research manager and administrator is now well defined, thanks in part to ARMA’s work,

but not very well understood. The number of sessions across the conference dealing with themes picked

up in this piece (not to say anything of my own experiences and, I know, that of others), demonstrates

just what closer cooperation and collaboration between academic and administrator can achieve,

but also how great the barriers are to realising this. If anything, this suggests that relationships are

deteriorating rather than improving, and that even between academics the potential for collaboration

is lost within a culture that rewards individual success (even if that success comes in collaboration

with others), where success is judged almost solely on research. The experience of Lincoln here is

instructive of the value of a holistic approach to higher learning.

The question posed is how to identify the drivers of success, and more importantly how to ensure that

all staff (academics and administrators) work together when so often success is rewarded individually.

If community is so important for the academy, how do we nurture and sustain it? While Rivers, Curwen

and Sandland offered an insightful contrast to Lincoln, in that a bottom-up approach is advocated,

they also reinforced the message from Lincoln that ultimately an institution’s research culture relies on

the fundamentals of staff policies.

Perhaps their reflection that will immediately resonate with all readers, whatever their institution, is

that ‘often we find that the view of our institution from ‘the outside’ is substantially stronger than the

view of the institution from our own staff ’’. Strange, that in an age when our roles so often include

professional communications skills, the view within the institution would benefit from an external

perspective.

6

Returning research to the real world

That the role of the research manager and administrator has been professionalised in recent years

must have something to do with the greater accountability of the academy, both formally (through

more exacting contract terms, as well as research and teaching assessment) and informally (the results

of the shift in the importance of the university to the industrial policy of the government of the day).

This comes together most visibly in knowledge exchange, encouraged by government keen to get

academics out of their ivory towers, and now assessed in the REF.

There has been interesting work in recent years on the idea of the ‘third space’ professional, most

notably by Celia Whitchurch. If the impact agenda represents the point at which the university has

gone full circle (after all, universities were initially created as training places for vocations, and much

of their early research in latter centuries focused on industrial and social benefits), the blended

professional reflects a return to the original idea of the administrator emerging from the academic.

As Dr Kenneth Skeldon and Dr Lucy Leiper of the University of Aberdeen showed in their piece

(Support structures for public engagement with research), this is nowhere more evident than in the

practice of knowledge exchange. The skill sets required of academics who wish to engage outside the

academy are significantly different from those of academic staff content to stay within it. In all cases,

the opportunity costs of engagement are high and therefore provide the ideal situation in which the

comparative advantage of the professional administrator comes into its own. Technical and personal

skills are not enough, however, and Skeldon and Leiper were able to demonstrate how to bridge the

gap between these and the substantive academic subject matter through a novel placements scheme.

As with the other presentations, the common thread here is the importance of the right incentives and

rewards that will enable the culture to change.

Substantiating research: the impact factor

It is not enough however, to change research cultures and thereby achieve research success. Externally,

the stringencies of the REF and broader accountabilities require evidence to be produced, and,

internally, if we are to drive change, we also have to be able to measure or at least ‘see’ the change. ARMA

members will therefore be looking closely at the work presented by Julie Bayley (Impact Officer and

Senior Researcher) and Julia Ryall (Programmes Director, Post Award) at the University of Coventry

on capturing impacts (ERIC: Developing an Impact Capture System). The technical requirements of

any such system – not just the IT technicalities, but the identification of the impact proxies and, more

significantly still, the challenge of creating the culture in which these are shared by the individual with

the community – are not to be underestimated.

7

If one required any evidence of just how much the role of the research manager or administrator has

changed, one need look no further than the piece from Lizzie Garcha at York and Eliot Marston at

Birmingham (Impact – Whose Responsibility Is It?), where the generalist is now exposed as the odd

one out! But if the similarities between our institutions are greater than the differences, Sue Starbuck at

Surrey has demonstrated just how important those differences can be (Yes, I am a Research Manager,

but I don’t cost bids). We can all learn from one another, but we must always understand contextual

disparities. Anyone trying to benchmark their office will know just how difficult comparing like with

like can be.

But to return to the similarities: anyone reading the piece by Susan Blum, Director of Research

Services at the University of Saskatchewan (Shifting to a Research Culture) will recognise immediately

how universal are the challenges we all face, and how universal the solutions. On another side of the

world, we find that, just as at Lincoln and at all other universities represented here, the importance

of knowing one’s intrinsic strengths in research goes hand-in-hand with the support provided by the

university and in particular, by the research manager and administrator.

David Coombe, Director of Conference Planning, ARMA and Director of the

Research Division of the London School of Economics and Political Science, UK

8

ERIC: Developing an impact capture system

To facilitate demonstration of the specific impacts of research, Coventry University has developed

a systematic approach in which research impact is built into the project lifecycle and the outcomes

are collected and monitored as a matter of course. The development process has highlighted the

need to engage and upskill staff across the institution

Julie Bayley, Impact Officer and Senior Researcher ([email protected]), Coventry University, UK

Julia Ryall, Programmes Director, Post Award ([email protected]), Coventry University, UK

The 2014 REF, and its predecessor, the Research Assessment Exercise (RAE), have been integral to

the university funding allocation process. Prior to REF 2014, assessments were based on research

excellence as judged by reviews of academic outputs against collaboratively-agreed ‘star ratings’. For

the 2014 exercise, however, REF 2014 heralded a new era of ‘impact’, with 20 per cent of the score

newly determined by the ‘effect on, change or benefit to the economy, society, culture, public policy or

services, health, the environment or quality of life, beyond academia’ (Research Excellence Framework,

2011). The resulting labour-intensive, retrospective activity has since impelled organisations to begin

integrating impact into their research infrastructure.

With the urgency of the 2014 submission now passed, there is now an opportunity to explore impact

in more depth as well as to support planning for REF 2020. Sector-wide activities include consultations

on metrics (such as those of HEFCE, 2014) and alternative measures (Altmetrics, 2011), growing

networks (including the ARMA Special Interest Group), and multiple conferences and review events.

Higher Education Institutions (HEIs) are clearly seeking to ‘get impact right,’ especially with the

potential that impact weighting will increase for REF 2020 (Witty, 2013). In parallel, the augmentation

of impact in funding bids (such as for Horizon 2020) necessitates institutional solutions to impact

management from the outset of a project.

In this climate, universities are now seeking systematised and embedded solutions for planning the

achievement of impact more proactively and capturing resulting impact data. This entails a range of

challenges for organisational support, academic engagement and information management systems,

and demands considerable institutional effort. Coventry University has developed an impact capture

system in parallel with broader engagement strategies to embed impact across the organisation.

Prototype development

Coventry University’s approach to impact has evolved over time, supported initially by Joint Information

Systems Committee funding to develop a prototype research impact capture system. Like many other

HEIs, Coventry had developed a strategic approach to applied research but had not captured impact

9

in a systematic way. The Embedding Research Impact at Coventry (ERIC) project aimed to develop a

technical solution in alignment with the university’s existing information management systems, and

was undertaken by a team of business development, information management and academic partners

(Everall and Hilton, 2012 and Bayley, 2014).

Data from an early academic and stakeholder needs assessment highlighted a range of conceptual

issues (for example, the need for increased understanding of impact), practical concerns (such as the

sourcing of evidence) and technical requirements (such as clear workflows for data entry using existing

systems). Next, a table of indicative system content was produced, based on REF guidelines, health

research case studies and discussions with academics. This matrix disaggregated impact areas, types,

metrics and sources of evidence, and the information management team subsequently translated this

into a prototype system.

The alpha version was piloted with academics who trialled ERIC whilst verbalising their experience

following the ‘think aloud’ usability testing methodology (Charters, 2003). The feedback from this,

along with broader interview questions, led to a suite of refinements and edits to create a beta version

of ERIC.

Upon seeing the potential of ERIC, the University Vice Chancellor commissioned institution-wide

implementation. The project development team was reconvened, with the academic partner as leader,

to extend ERIC beyond its health-related pilot to cover the full breadth of disciplines across Coventry

University. To accelerate this process, an external database of impact markers was purchased from an

impact reporting consultancy named Vertigo Ventures, and this was then modified for usage in ERIC.

In parallel, the development team conducted additional academic consultations and also reviewed

external guidance to align ERIC with emerging good practice.

The ERIC system

As ERIC was built to integrate with Coventry University’s existing bespoke in-house research

information management system, the system enables academics to plan impact from project inception

and capture the effects during (and beyond) the project lifecycle. Users can add details of multiple

planned impacts, including those which occur unexpectedly, or remove those which do not transpire.

The system itself is split into three processes: planning the impact, determining evidence of achievement

and then entering the evidence once accomplished.

Each individual impact is added via a series of four logic-based drop-down boxes (with optional

accompanying narrative description), articulating impacts from the broad area down to the indicative

metrics, as shown in Figure 1.

10

Figure 1: ERIC Impact entry screen

For each impact, users then select how the impact will be evidenced and a date by when this will likely

be available, as shown in Figure 2.

Figure 2: ERIC evidence selection screen

Evidence is then uploaded into an institutional repository, either directly into ERIC or via an automated

email at the planned date. Users can also defer the date of entry of evidence, edit the impact/evidence

or remove the impact altogether, as shown in Figure 3.

11

Figure 3: ERIC evidence upload screen

The system is designed to grow through shared use, enabling additional content and revisions to

terminology: benefits include the development of an ‘institutional memory’, a reduced reporting

burden overall and support for marketing and funding activities.

Embedding impact into the project lifecycle

A technical solution alone is unlikely to be sufficient to address an HEI’s impact needs. The scale and

significance of impact demands an ongoing, collaborative institutional approach to ensure that impact

is integrated into the project lifecycle from the outset (Hitchins and Bayley, 2014). Such a strategy

helps to optimise resources, allocate staff and harness service support to avoid the academic working

in isolation.

To achieve this most effectively, research support staff need to develop a broader understanding of the

complexities of impact in parallel with the academic community. The resulting coordinated approach

bridges the academic/research-support divide and weaves impact into the matrix of research activity.

At Coventry University, a crucial activity has been to work both vertically (engaging all levels of

the university) and horizontally across teams (for example, the Open Access strategy group, Data

Management group, Ethics and Governance, Pre/Post Award teams and the Library). This has both

established the relationships and protected the space (physical and time) needed to begin ingraining

impact within core institutional practice. An essential third strand of activity has been the engagement

of academic ‘champions’, senior staff who act as gatekeepers for faculties, overcome scepticism and

help set foundations for a growing impact culture and system integration. Ultimately, this conjoined

approach is shifting institutional focus away from impact as a by-product of cited academic outputs…

(see Figure 4)

12

Figure 4: Historic approach to academic impact

... to an integrated and planned approach with multiple pathways, underpinned by an impact-positive

culture (see Figure 5). Such an embedded and coordinated approach is integral to the full adoption of

any technical solution.

Figure 5: New approach to academic impact

Challenges of impact capture

The process of developing and embedding ERIC at CU has not been simple, with a range of barriers

identified during needs assessment. Undoubtedly for some, REF 2014 has left a negative legacy, either

from the effort of inclusion or the disappointment of exclusion. Consequent academic exhaustion,

suspicion and ennui create a challenging REF 2020 landscape for academics and research support staff

alike. This is compounded by limited understanding of impact paths, metrics and tracking, alongside

concerns about the role of impact data in evaluating the value of research.

13

More complex challenges include difficulty in standardising impact across disciplines and attributing

the ownership of impact to individuals (which is so crucial to academic progression), particularly in

collaborative efforts. The (in)tangibility of impact, and the feasibility of setting metrics for it continues

to attract debate across the sector and to make systematised approaches less easy to implement

(discussion of information management demands in Fedorciow and Bayley 2014). Thus, while the

definition of impact is largely straightforward, its application is far less so.

In summary, the Coventry University experience has highlighted a range of challenges and solutions

for implementing an institution-wide impact capture system. A technical solution can vastly enhance

the ‘impact readiness’ of an institution, but it is essential that this be anchored within a broader set of

complementary measures to upskill and engage staff. Academic-led approaches, with sufficient top-

down support, can overcome concerns, suspicions and resistance which otherwise can decelerate or

block impact strategies.

Realistically, each HEI context will differ, but Coventry University’s experience suggests that systematic

approaches to impact are best facilitated by continuous dialogue about the evolving needs of staff. Just

as the inclusion of impact in REF 2014 demanded a shift in the academic mindset, increased funder

emphasis similarly demands methodological refocus; overt consideration of impact must begin at

project inception, translated across disciplines and joined across the academic/support divide.

Coventry University’s approach will continue to evolve with increased coordination across teams,

increased incorporation of behaviour change techniques, and co-development of allied drivers such as

Open Access. More broadly, all HEIs will undoubtedly need to keep abreast of emerging best practice,

to best support high quality impact attainment as well as optimisation of future REF submissions.

14

Shifting to a research culture

Over just 14 years, the University of Saskatchewan has transitioned into a highly regarded research-

intensive institution through a change management programme that altered mindsets as well as its

organisational structures, capabilities and facilities

Susan Blum, Director of Research Services ([email protected]),

Office of the Vice-President Research, University of Saskatchewan, Canada

Some universities have a historical culture of research woven into every thread of their daily lives – ‘it’s

just what we do’. Others have had to work at shifting the mentality of faculty members from research

being something they ‘might’ do, to something that they ‘must’ do. The HEI has to employ many

strategies to achieve such a cultural shift, while at the same time implementing approaches to assist

faculty in achieving research success.

The University of Saskatchewan (U of S) in Saskatoon, Canada, was established in 1907 as an

agricultural college. It is now a member of the prestigious U15 group of Canada’s top 15 research-

intensive universities. Over the last 14 years, the U of S has evolved to a position where there are 75 staff

and five offices under the Office of the Vice-President Research (OVPR) which support research. The

strategies employed to achieve this position have all been beneficial in terms of shifting the university

towards a culture of research.

The university’s evolution has not been static, but a continual process. At the institutional level,

organised research structures needed to be put in place, research areas of focus had to be identified,

and plans needed to be developed so that hiring practices and limited resources could be targeted

towards research. At the operational level, effective strategies which directly assist faculty members

had to be applied, such as facilitating the decentralisation of research support, establishing internal

review programmes for key funding opportunities and assisting bids for internal funding, and the

implementation of a mentorship programme.

Institutionalising a research culture

The U of S currently has around 21,000 students and is uniquely diverse, with more than 2,300

international students and 2,000 First Nations students. There are 17 colleges and schools, 58 degree

and certificate programmes, and 100 research/support centres. The average annual research revenue

between 2007 and 2012 was CAD $179 million, and for 2013 it was CAD $196 million.

The U of S is now home to leading research facilities such as the Vaccine and Infectious Disease

Organisation-International Vaccine Centre, the Canadian Light Source (Canada’s national synchrotron)

15

and the largest academic toxicology research training centre in Canada. Signature research areas are

Aboriginal Peoples: Engagement and Scholarship; Agriculture: Food and Bioproducts for a Sustainable

Future; Energy and Mineral Resources: Technology and Public Policy for a Sustainable Environment;

One Health: Solutions at the Animal-Human-Environment Interface; Synchrotron Sciences:

Innovation in Health, Environment and Advanced Technologies; and Water Security: Stewardship of

the World’s Freshwater Resources.

Leadership capabilities

The development of a research culture requires leadership in research skill and management practice;

both are critical for a fruitful research environment (Bland and Ruffin, 1992).

In 2000, there was one office consisting of nine staff which supported research administration in

addition to the OVPR. The university recognised that research intensity in the health area was lacking.

As a result, a senior faculty member was seconded to provide the leadership required and a PhD-

trained staff member was hired to assist and manage the process of development. Staff expertise in the

area of grants, contracts, and ethics began to be developed, in a move away from a Jack-of-all-trades

approach.

The right leadership is instrumental in shifting towards a research culture and achieving a healthy

research environment (Zajkowski and Dakin, 1997); and having clearly-identified areas of research

emphasis is a key characteristic of productive research environments (Bland and Ruffin, 1992). A new

Vice-President Research (VPR) was appointed in 2003 and created the position of Associate Vice-

President Research (AVPR) in 2004. The AVPR was put to work to compile a planning document

entitled Extending Horizons: University of Saskatchewan Research, Scholarly and Artistic Landscape,

which identified five areas of institutional strength and promise.

The Associate Dean Research (ADR) position was created and standardised across campus. The

VPR chaired monthly meetings with the ADRs, AVPR, and Director of Research Services (RS) to

improve communication and collaboration. Accountabilities for the ADR positions included strategic

positioning of the college for research success, comprising the college research plan, promotion of

the research agenda, leadership on university-level initiatives, and facilitating research collaborations;

creating a positive, effective research culture and environment by ensuring compliance of projects,

championing the role of research during faculty recruitment, and developing innovative research-

intensive training opportunities for students; and leading communication and promotion with internal,

external and international bodies, as well as fostering partnerships and collaborative opportunities.

16

Research facilities and support

Establishing research facilities is a central component of a research culture (Polk, 1989). The creation

and governance of research centres became more formalised, with policy development as well as

clearly-defined expectations of centre directors. The directors were also pulled into the fold of regular

communication through the creation of monthly meetings with the VPR, AVPR and Director RS.

Faculty leadership was developed to support all areas of research in addition to health. Senior faculty

members with success in obtaining research funding were appointed to lead in the natural science and

engineering, and social sciences and humanities areas.

Between 2003 and 2004, staff numbers within RS grew to support research administration. This

resulted in the creation of two new offices: research ethics (RE) and industry liaison (IL).

From 2005 to 2007, the main focus shifted to building critical mass and expertise. The University

President invested time and was very influential in attracting top talent. For example, the U of S was

able to attract the world’s most cited author in the combined fields of Ecology and Environmental

Sciences. During this period, operations also began at Canadian Light Source.

Academic structure

Between 2007 and 2008, three new schools were established based on centred expertise, areas of

excellence and attracting top talent in terms of both students and faculty: the School of Environment

and Sustainability; the School of Public Health; and the Johnson-Shoyama Graduate School of Public

Policy.

Having direction, niche and strategy are crucial for the development and success of an active research

culture (Hill, 1993).

In 2008, senior leadership transitioned to ensure that the right people were in place to promote the

University’s focus on increasing research intensity. This change included a new VPR, AVPR, and

Directors of RS, RE and IL. Once the individuals were in place, the VPR put forward a process to

identify ‘signature areas,’ those where the university excelled and was internationally recognised. These

signature areas then determined the focus of resource development and allocation for the upcoming

years.

Administrative and support structures

A stable administrative support system is crucial (Lubin, 1992). It is important to acknowledge that

research management and administration is a professional practice and an academic field in its own

17

right, enveloping an exceptionally broad range of talents and expertise (Krebs, 1992). With new

leadership, in 2009 units were reorganised to ensure client-focused services for researchers. This

consisted of the creation of teams with experienced and skilled staff to support researchers within

specified colleges and schools.

The role of facilitation should establish a partnership with the researcher and research staff and be of

service to the researcher (Hill, 1993). The facilitation role is thus a key function that supports faculty

within a research administrative structure (Lawrence, 1991).

Over 2010 and 2011 a research facilitation model was implemented to support researchers and improve

communication and support between faculty, colleges and RS. Skilled Masters and PhD qualified staff

were hired and embedded in the colleges to provide faculty with high-level support for finding funding

opportunities, grant development and facilitating research groups. These positions co-reported to the

Director RS and the ADRs and were equally funded by the OVPR and each college.

Megel et al. (1988) noted that the highest producers of research were strongly motivated by internal

peer support of research team members. Faculty leadership positions were further enhanced with the

allocation of ownership of the implementation of an internal review programme for obtaining federal

funding and health research funding from other sources. This was an internal process that included

a deadline set eight months prior to the deadlines of the funding agencies. Applications were peer-

reviewed with a faculty-specific focus, with the committees mimicking federal review processes.

As a result of wide internal and external consultation on what is required to ensure research success,

it was recognised that new faculty members needed particular support. A Research Mentorship

Programme was initiated in 2012 where every new faculty member on campus is matched with a

research mentorship team to provide personalised support and guidance in developing a five year

research plan; they are also assigned to half-yearly development workshops.

College research strategies were also developed, at the request of the VPR. All colleges were required

to develop a research plan that identified research strengths as compared with their national peers,

define goals and how they would achieve them, and help to establish multidisciplinary collaborations.

The university provided funding for the implementation of an online research management enterprise

system to transform the way support was provided to both faculty and administration. The system

supports the full research lifecycle, meeting the needs of faculty and the university for timely and

accurate research administration, management, reporting and compliance.

18

Recognition of success

The establishment of a research culture takes time, careful planning, resources and the right

environment, which, when combined, hold the key to the development of research capability (Pratt et

al., 1999).

The U of S continues to seek better ways of supporting faculty, meeting compliance requirements

and furthering the development of new signature areas. Natural selection factors are inherent parts

of this process, as increased compliance requirements, declining faculty research funding success and

heightening restrictions on university budgets make for a very competitive environment.

In 2011, the U of S was added to the list of Canada’s top 15 research universities and recognised for

its research excellence. With the signature areas already identified, efforts were focused on obtaining

resources to develop institutes in those areas. These efforts secured CAD $140 million from funders,

provincial government and industry for the development of the Global Institute for Water Security,

the Global Institute for Food Security, the Sylvia Fedoruk Canadian Centre for Nuclear Innovation

and the International Minerals Innovation Institute. The university is also currently developing a new

School in the area of synchrotron sciences.

19

Yes, I am a research manager, but I don’t cost bids

Outside the structured environment of research support, there are others who fulfil essential

roles. The value of these non-traditional roles to institutions and the challenges of such roles at an

individual and institutional level require further attention from the research support profession

Sue Starbuck, Faculty of Arts & Human Sciences Information Officer

([email protected]), University of Surrey, UK

As universities have recognised the importance of effective and efficient support for research and

research support has become highly professionalised, many institutions have developed clearly-defined

roles within a well-structured central research support department. This is reflected in the growth and

expansion of organisations such as ARMA, who provide not only support and training for the research

support community, but increasingly offer them an officially-recognised voice in consultations and

high-level policy discussions.

Research support covers many and varied activities. Traditional or standard research support can take

a number of forms depending on the size, structure and working practices of the university. While

all research support staff work across the university organisation, in some institutions they operate

from a single office, whereas in others they are dispersed across faculties or departments. In general,

research support staff are linked via a hierarchical structure led by a Director with senior status within

the university.

However, there are many research support professionals who do not sit within the traditional

framework. Some have roles with a wide remit, possibly working in small institutions, or at Faculty or

School level in larger ones. Alternatively, they may have an unusual combination of responsibilities or

may be specialists who occupy central university roles outside the central research support function.

Such individuals can be said to fulfil non-traditional research support roles.

One of the difficulties in assessing the extent of non-traditional research support is that the recognition

of research support as a profession is relatively new, and is constantly evolving in response to the

shifting dynamics of universities and the research environment.

Defining research support roles

So, how to define traditional versus non-traditional research support? One means is to list areas of

work and tasks. Carter and Langley (2009: 32) identify 10 core elements in the management and

administration of research. These cover the development of funding opportunities, bid support, project

support, ‘production, management and use and promotion of the outputs of research’, postgraduate

20

research, data, research governance, research assessment, research strategy and policy, and systems

and administration. Many embody a variety of sub-areas, so there is considerable diversity. Whether

they are all covered within a formalised research support structure is the issue.

When asked to define traditional research support, participants at an ARMA 2014 workshop found

considerable difficulty in reaching a consensus, using terms like ‘fluidity,’ ‘changing perspectives’ and

‘blurred boundaries’ to articulate the problem. What one university may see as part of their traditional

structure may be alien to another, as there are so many variations in culture, size, organisation and

structure. This leads to the notion that using the centralisation of the research office as a defining

factor for traditional research support may obscure the issue of definition.

An alternative is to consider individual roles rather than organisation structure. It could then be

argued that the development of research support has led to specialisations because in order to be

effective, a research support structure needs to contain specialists in such areas as the production

of research grant applications, or in the costing of proposals. One proposition is that a traditional

research support role is that of a specialist; and a non-traditional role is that of a generalist or, more

accurately, a research support specialist who does not specialise in any one particular area of research

support.

While there is clearly a need for areas of specialisation, it is often difficult for central offices to relate

closely to academic staff on the ground and there can be a lack of flexibility within a large support

structure in reacting to changing circumstances. Thus many specialists involved in research support

may not sit within the research support office: for example, postgraduate research is often dealt with by

a Graduate School, systems of all kinds are likely to be within the province of IT, visibility of research

may be led by Marketing and there is a strong move amongst the information community to increase

the involvement of university libraries in supporting research.

In some cases, particularly in small institutions, individual roles clearly rely more on the capabilities

of the person inhabiting them than on the structure and definition of the role. The headline tasks in

many non-traditional roles are not unique, but their combination can be. In my own case, I would

argue that my research support background is a valuable addition to the Doctoral Training Centre/

Partnership management in which I am involved and that my involvement enables me to integrate this

aspect more fully into the research agenda of my Faculty.

21

The value of non-traditional roles

However we describe them – individual or non-traditional – a surprising number of these roles

exist, so what value do they add? Often, the role is embedded in the academic community, so the

incumbent has better local knowledge and experience and therefore a better understanding of the

politics and drivers affecting individuals, schools and research groups. Given that they are not part

of a larger, centralised structure, they may have more flexibility and so be able to react more quickly

to change; they may also have a more holistic view of ‘research’ and a personal, vested interest in all

stages of the research cycle. If they are part of a faculty or small unit, they will recognise that individual

responsibility does not stop once funding is received. Although not specialists in any particular area of

research support, such individuals may also have built up micro skills in a particular technical area or

have in-depth subject knowledge. If more of a generalist, they may have acquired the ability to juggle

a wide range of different tasks at different levels and have developed, from necessity, the creativity and

diplomacy to manage conflicting drivers and priorities. Such benefits have been generally recognised

and central units can (and do, where the institutional culture allows) go some way towards achieving

closer links to academic staff by physically placing central research support office staff within faculties

or departments.

It can also be argued that those in non-traditional roles appear to have a less administrative and a more

academic attitude, which facilitates a more balanced relationship with academic staff; an inherent

danger of centralised support is that staff become more remote and so find it difficult to build up the

relationships which are an important part of research support.

Challenges of non-traditional support

There are challenges in having a non-traditional role. Individuals are potentially isolated in terms of

career structure and job progression possibilities. Where their remit is unusual, it is often difficult to

find relevant training and, without access to central resources, there is often a restricted budget to

pay for that training. Where there are both traditional and non-traditional roles within a university,

there is also the danger of duplication of work, which can only be managed by close liaison and the

development of good relationships on both sides.

Another challenge presents itself around the issues of control and good governance. Should non-

traditional roles really sit outside the control of the Director of Research Support? Most Directors would

say no, citing the interest of good governance. But there is also considerable value to independence,

and with it greater flexibility to react to a changing environment or mould the implementation of the

role to particular local needs or working practices.

22

Bringing everything together into a central structure is not the answer. The independence and

flexibility of non-traditional roles adds a necessary dimension to research support, and centralisation

is of course impossible where these roles sit in small organisations where there is no central research

support structure. As a profession, we need to fully recognise the less structured support role. We

need to discuss how best to harness the extensive skills non-traditional support personnel offer and

at the same time how to provide them with the support and training essential for good governance.

23

Internal communication strategies for stimulating and embedding a successful research culture

A successful external institutional research record depends to a great extent on internal adoption

and engagement, which can be fostered through communications that underscore the desirability

of a culture of research excellence

Jenny Rivers, Research & KE Manager ([email protected]), University of Liverpool, UK

Rachel Curwen, Research Development Associate ([email protected]), University of York, UK

Ruth Sandland, UK Research Funding Manager (ruth.sandland anglia.ac.uk), Anglia Ruskin

University, UK

This discussion is based on our reflections after running the workshop, ‘Internal communication

strategies for identifying, stimulating and embedding a successful research culture: how can we engage

our researchers and enable them to engage with each other?’ at the 2014 ARMA Annual conference.

In that session, we addressed the challenges, barriers and possible solutions to effective, inclusive

internal communication, to stimulate and embed a successful research culture that is powerful in

simultaneously supporting independent research, developing a research community and driving

institutional research performance.

Holistic research involvement

Research managers and administrators provide an ever-increasing level of support to the academic

community. Recognition of research support staff as professionals with specific expertise that adds

value to the research process progressively breaks down barriers between academic and professional

staff, resulting in genuine partnerships working towards shared objectives.

There is a diverse range of information to disseminate to both academic and support staff colleagues,

including a vision of an environment where research is supported, opportunities are captured and

communicated and success is celebrated and shared.

Defining internal communication as broadly as possible seeks to dispel a ‘them and us’ attitude, to

encourage research support and academic colleagues to consider all of their engagement with other

individuals and their variety of roles on a day to day basis. This translates across the complex variations

in institutional structure in HE (devolved or centralised) and includes challenges within units/teams

and working across departments, schools or faculties.

24

Merging perspectives on research culture

When HEIs seek to answer the question, ‘what does a successful research culture look like?,’ the

answer is invariably framed in terms of published outputs, impact, grant income, public engagement,

postgraduate researcher numbers, external collaborators and a host of similar measures: the familiar

terms of university objectives and the indicators quoted in strategic plans. We think ‘institutionally’

about how to evaluate success, how we can prioritise activities to achieve certain targets and how we

can ensure that our HEI’s performance and that of its staff is as good as it can possibly be. Internal

communication of these expectations plays a vital role, but the extent to which they are embedded,

rather than imposed, finding a balance between ‘top-down’ and ‘bottom-up,’ or something in between,

is not always clear.

If we asked researchers the same question, would we expect the same response? They might think

instead in terms of feeling valued, being well supported, receiving appropriate training to achieve

personal ambitions or realising appropriate balances between research and teaching and between

autonomy and a sense of research community. Particularly in terms of institutional identity, should

we assume that all researchers are at their institution because they believe that it is the best place for

them to pursue their research, being where they can make the most substantial contribution to their

field, in terms of their values? Should it follow that researchers, through virtue of employment at

their institution, though supported to pursue research independently, are expected to engage in, and

contribute to, wider collaborative activity? In terms of successful research culture, researchers who

find engaging with colleagues beneficial to their research career, whether within or across disciplines,

might think in terms of knowing about relevant opportunities, or belonging to a vibrant community

of interesting and interested colleagues where new ideas are stimulated and tested.

Recognising and understanding researchers’ different motivations – for example that many are more

likely to engage with their discipline before their institution – should prompt us to think about how

we can stimulate an appropriate environment, and generate incentives to achieve a successful research

culture for all. In addition, successful partnership between researcher and research support professional,

sharing their vision and objectives, can make a substantial contribution in the development and

embedding of a research culture. An appropriately-supported and widely-disseminated strategy for

internal communication to extend the reach and sustainability of this approach could have a significant

impact on subsequent growth in research performance.

Balancing internal and external communication

There is often more emphasis and provision of resources devoted to raising the profile of research

beyond the institution, to attract collaborators, students and funding. While this is extremely valuable,

25

often we find that the view of our institution from ‘the outside’ is substantially stronger than the

view of the institution from our own staff. Given that partnerships are often built on the strength

of individual, personal relationships, in terms of the sustainability of our efforts to get the message

out there, could this be stronger if we redressed the balance and used internal communication to

support more effective external communication? This triggers the most fundamental question: Could

we concentrate more on developing a successful research culture that ‘speaks for itself ’?

Implementing strategies which help embed a successful internal research culture, of which all

researchers feel part, can drive institutional research performance and so raise its external profile. This

can be particularly powerful in terms of increased participation and the willingness of more staff to

act as ambassadors across discipline areas, and to directly influence the direction of growth as opposed

to resisting ‘the centre’.

A shared responsibility may also encourage more researchers to champion activities such as

involvement in supporting colleagues through peer reviewing their grant applications and outputs, or

joining committees and groups. Such an approach might enable us to achieve more collectively while

helping independent researchers to achieve individual objectives. This does not imply however that

all researchers should actively be part of everything all of the time: space for independent research

remains paramount for innovation. But a balance could be identified by developing a holistic strategy

to generate a successful research culture in the first place.

Are innovative communication tools needed?

There is a wealth of examples of best practice across the sector, but we continue to be faced with the

difficult question of an overall approach that is consistent: how can we use tools to motivate employees

of an institution to be genuinely part of and involved in that institution? How can we create a research

environment that is sufficiently exciting and interesting for researchers to want to share information,

to work collaboratively and of which they are proud?

In the search for tools and practices, printed marketing materials, digital platforms and systems and

processes for capturing information for sharing on a regular basis – for example funding opportunities,

research policies, research outcomes and expertise – have been explored to different degrees across

the sector. Case studies including ‘raising the profile of research staff ’, ‘communicating a new policy’,

‘finding the right person to help’, ‘presenting funding opportunities’, ‘creating or raising the profile of

your services or those of your team’ and ‘making face-to-face networking work’, have been presented

to explore the benefits of different methods of communication, networking and collaborative working,

seeking to empower research professionals to work in partnership with the research community. The

conclusions drawn from these studies indicate that challenges are universal, and innovation is required

26

in the approach and attitude to the internal drivers for research culture, rather than in the particular

tools available. All examples shared the need to identify and be responsive to target audiences with

differing motivations and to exploit available mechanisms flexibly, in order to realise the potential of

a comprehensive communication strategy.

In an evolving digital world, there are many potential platforms for communication, to include social

media and websites, as well as printed materials such as leaflets, guidance documents and newsletters.

To drive research performance, institutions have had to find creative uses of all communication tools.

In particular, a research blog is a comparatively resource-effective way of disseminating a wealth of

information in bite-size format. With a broad remit, it enables the sharing of information, including

research outcomes and success stories, but can also cover diverse topics, offering advice and information

on such items as writing a good application, open access or the REF.

Whether innovative means are necessary is largely irrelevant: the focus should be on the target

audience, the objective(s), which tool is the most suitable and the resources available. A simple, brief

newsletter, circulated by email, serves as a way to introduce people, ideas and activities, but also to

celebrate success by sharing news of recent grants and awards. A local example reported that after

circulating a first couple of newsletters, a number of regular emails identifying omissions and asking

for inclusion in future circulation were received in response, demonstrating that the document had

been opened and that researchers were keen to share their achievements. The distribution list of those

beyond the original target audience expanded, by direct request from individuals and other areas of

the University. As a result, there was an increased awareness of research outcomes and success stories,

directly informing the development of more robust systems and processes for capturing and sharing

information.

Whatever the means of facilitating communication, the direct relevance of internal to external

communication is abundantly clear and can be achieved simultaneously, raising the profile of research

to both academic and non-academic audiences inside and outside of our institutions.

27

Impact – whose responsibility is it?

Impact is not just an academic concept: it drives every kind of organisation’s objectives and

successes. The viewpoints and experiences of external partners are thus fundamental to realising

the transformative potential of research activities

Elizabeth Garcha, Research Impact Strategy and Policy Officer ([email protected]),

University of York, UK

Eliot Marston, Bupa Translational Research Manager ([email protected]), University of

Birmingham, UK

“The stakes are high for society and the research community and so everyone involved in the research

lifecycle, from research idea to societal benefit, should consider their role, how they conduct it, and

how we can work together to improve and ensure the impact of research.” (Professor Dame Sally C

Davies FRS FMedSci, Chief Medical Officer for England, June 2014)

The concept of ‘impact’ continues to cause consternation across UK universities, from researchers and

administrators to the funders themselves, but one thing is absolutely clear – there is no escaping it for

the foreseeable future. In particular, research funder expectations encouraging active impact planning

and evaluation over several years of Research Councils UK (RCUK) ‘Pathways to Impact’ on research

grant applications, and an eventful REF 2014 exercise which assessed the impact of university research

for the first time, demonstrated a potential cornucopia of definitions. However, university research

support staff should all be reasonably comfortable with the straightforward definition that an ‘impact’

can be loosely classed as a benefit to society (whether health, wealth, policy or creative contribution

outside of academia) that can be evidenced and clearly linked to research.

Few involved in research and its support would argue with the importance of output delivery from

publicly-funded research, bearing in mind that not all research will (or should) necessarily lead directly

to finished outputs, particularly over short timescales. However, to effectively capture the sometimes

nebulous final impact as a ‘product’, we need to understand the journey that research undertakes to

get there. It is no mean feat, and involves a bewildering array of collaborators, users and processes,

over many (perhaps most!) of which we have little or no control. But having an understanding – and

an ability to contribute throughout the lifecycle of impact conception and delivery beyond the original

research – must lie at the heart of our strategies to support and deliver the best possible outcomes from

the research undertaken in our institutions.

As UK universities move from reactively dealing with research impact in response to external

pressures, to a more proactive and supportive approach that drives enhanced research outcomes,

28

some outstanding challenges and opportunities are raised in terms of institutional ownership of

the processes. The question is how research administrators and managers may best support their

institutions in embedding impact within research projects, particularly in light of challenges around

funder and partner expectations, strategy and evaluation.

Understanding ultimate impact

We often seem to forget that public and private organisations have their own specific impacts to deliver,

and that these are the core reason for their existence: the more effectively they deliver these impacts,

the more successful they are as organisations. We (as individual HEIs and the wider sector) need to

move to viewing such organisations as partners and stakeholders in the delivery of the best possible

impacts in their own spheres, for which they will have much more extensive expertise and clarity of

vision than we, rather than as vehicles onto which we can simply push research.

We therefore need to understand how external groups view research within their roles and experiences,

and in what ways they need, want or understand it. A range of levels of leadership and involvements

may pertain, from the personal to peer groups, locally, nationally and internationally. In addition,

their personal drivers and responsibilities might not immediately match their abilities – financial or

otherwise – to deliver outputs. Knowing how they evaluate their own success – as well as who else

evaluates it, and what influences these parties have on their decision-making and plans for longer-

term sustainability – means that we will be better able to serve their needs and work with them to

deliver the impacts they require.

We should also seek to understand how these groups are currently accessing research, and at what

kinds of organisational levels this happens. It may well be that they do not access research regularly

because standard mechanisms do not deliver what they need, or that the value of research has never

been made clear to them. Knowing this, we can work on better utilising these mechanisms, improving

them and/or creating new mechanisms to make sure that more high quality research can reach them

in ways that are most useful.

Our task is thus to develop impact strategies within our organisations which embrace external

perspectives of impact rather than the heavily academic-driven strategies on we have so far relied.

However, this presents challenges in terms of expectations and ownership.

Expectations and ownership of benefit

The responsibility for achieving impact as an HEI is inherently limited. Universities and individuals

can create the environment and provide the resources required, but ultimately, the achievement of any

impact is controlled by the beneficiary. Institutions can nevertheless set expectations and agree the

29

‘ownership’ of knowledge exchange processes or other products, to help develop an understanding of

what is achievable.

Funder expectations around impact essentially come down to money from an institutional viewpoint.

While it should not be the primary motivating factor for an HEI, and it may not motivate individual

academics and researchers to engage in and undertake impact-related activities particularly well,

the institution generally views the implicit meaning of impact as an increase in income, either via

grant success or via REF success. There need to be broader university-led reasons to achieve impact,

such as benefits for our local communities, exposure and value for our research and researchers, or

benefits for our research partners and users. Understanding our partners, their priorities and their

motivations, will bring universities a richer perspective that will inspire us towards both specific and

broader-impact goals equally. We therefore need to take the time to ask our partners and collaborators

what impact they wish to achieve through their work, and why.

Strategic management of external relationships

Before developing an impact strategy, institutions need a clear sense of which people are willing and

able to contribute to their impact agenda, both researchers and administrative staff, and how much

they are expected to contribute. Additionally, HEIs should consider who will have responsibility for

oversight and day-to-day management of their major strategic relationships with external organisations.

When external relationships have grown organically over time, this is not a simple question, but the

information can indicate what is viable, while yet ambitious, for an institution to achieve.

The ownership of relationships with partner organisations is a developing area. Many institutions

have encouraged a wide range of relationships with a wide range of organisations, particularly with

larger organisations. There are often multiple links, through a number of individual relationships that

have developed organically or serendipitously. As these grow, the need to manage these relationships

centrally within an institution, especially if they are with large corporate organisations, becomes a

priority.

It could be argued that universities should control all relationships that it considers strategically

important, in order to coordinate them and create the best potential for impact. However, this is

likely to be disempowering for the individuals who originally formed the relationships, and

may ultimately be ineffective. On the other hand, leaving all relationships with organisations

completely devolved can lead to frustration on all sides if the people talking to one another

do not have access to relevant information about current activities and interactions.

Institutions need to develop a portfolio of relationships, and if they move towards managing some of

these more centrally, they must ensure that they can simultaneously provide optimal support for both

30

the partner organisation and the researchers themselves. Creating these more managed relationships

can be the first steps towards a truly forward-looking impact strategy.

Impact planning and strategy

From an institutional perspective, there are several key points to consider when exploring impact

planning and strategy. Firstly, there is not always consensus on the nature of impact. Our preferred

perspective is to think of ‘engagement’ rather than ‘impact’; as university people, we need to embrace

our roles as partners in processes that lead to the eventual delivery of impact by an external community

of organisations.

Secondly, for most institutions it is impossible to have a single top-level strategy for delivering impact

effectively which completely covers the possibilities inherent in the complex organisation of research

across academic departments. Leadership is required at various levels, from a single project to the

wider group, department, faculty and institution. Being specific and bespoke at an individual level

may reap rewards in terms of focus and being able to track outputs effectively, but it also loses many

of the benefits of operating at scale in terms of accessing available internal resources and external

partners. At each level, it is important that there is a shared understanding of impact (or engagement),

and clear lines of responsibility, monitoring and evaluation concerning progress and outcomes that

dovetail with the wider institutional support framework.

Finally, it is also crucial to ensure a good understanding of the external environment, in terms of

the specific target stakeholders, collaborators and audiences. One of the greatest misconceptions in

research engagement is that there is a ‘general public’ who are eagerly awaiting research outcomes:

in reality, this is a hugely complex, overlapping assortment of specific groups with different interests,

values and understandings of research. This is also true of other sectors, which are rarely ‘one size fits

all’. Targeting everyone equally almost invariably means that you are not effectively reaching anyone.

The value of recording and assessment

Now that evaluation and recording has come to the fore, record keeping is essential – though it

is, understandably, not the most exciting part of participating in engagement or impact activities.

Importantly, keeping good records and consistent evaluations for monitoring progress against strategy

and understanding what is being achieved within a project, also helps to identify any negative impacts

and when to stop an activity. Institutions have to be willing to recognise, share and learn from negative

impact.

If universities had a better understanding of the aims of their research, more highly focused evaluation

methods would be possible. While our partner organisations and communities will have their own

31

aims, many will have their own evaluation techniques from which we universities could learn:

ultimately, the questions our partners need to ask should be included in shared evaluations, reflecting

joint interest and responsibility.

Good evaluation and record keeping does not need to be onerous; and the burden should be in

proportion to the activity. It is not the case that we should collect all information possible –time and

financial pressures are such that we necessarily focus on the more important pieces of information.

We should take steps to ensure than we do not move to a system where records and evaluations lead to

instrumentalised relationships with partners, as was the case in REF 2014, when many partners were

hard-pressed to help supply the large quantities of information asked of them.

In research partnerships, sharing the process and the outcomes, and, most importantly of all, taking

action based upon them for evaluating project purposes and setting out the future, is key. For example,

if an impact strategy is never achieved, either the strategy or the means of implementation needs some

level of change. Carrying on regardless undermines the value and need for evaluation.

32

Diversification of income streams: bridging the academia/industry gap

The University of Lincoln has demonstrated remarkable success in research with real world impact

by accessing new funding streams through strategic relationships with industry

Andrew Stevenson, Director of Research and Enterprise ([email protected]),

University of Lincoln, UK

Stephanie Maloney, Head of Research ([email protected]), University of Lincoln, UK

In a challenging funding landscape of reduced budgets and increased competition, and with the REF

dominating current policy initiatives, we have seen particular emphasis on inter-disciplinarity, impact

and dissemination of research. Diversification of our income streams is paramount and collaboration

is key to addressing this challenge in unlocking new funding streams such as European Union (EU)

programmes and that of Innovate UK (formerly the Technology Strategy Board (TSB)). As universities,

we are now looking at how we best engage with employers/beneficiaries of our research in working

with industry not only as a funding source, but also as a route for ensuring the employability of our

graduates.

The University of Lincoln’s interdisciplinary approach to collaborative research in responding to

industry needs has been cited as an example of excellence and in Sir Andrew Witty’s final review of

Universities and Growth (2013).

The University of Lincoln: leveraging innovation

The University of Lincoln was established in 2001 and its rise to a top 50 university has been rapid. It

has established a record of high student satisfaction, producing graduates well prepared for work and a

growing portfolio of excellent research. This progress has been built on innovation, and by challenging

norms to sustain our vision and a future in which we are creating purposeful, knowledgeable and

confident graduates to serve our local, national and international communities. We are recognised

internationally as having a distinctive reputation for the way in which we partner with both employers

and students, and have created a university campus that is wholly an innovation park of the future,

with industry and academia diffused throughout.

A key impetus has been the recruitment, retention, training and development of high calibre

researchers and postgraduate research students, not only to enhance our capacity and capability for

cutting-edge research and our reputation, but also to ensure that our students will be educated in an

outstanding research-informed environment. With many funding streams shifting towards fellowship

models that support people throughout their careers, attracting high calibre early career researchers

has allowed us to secure funding for existing areas of strength while bridging gaps between key

33

areas, crucial to building a sustainable research culture and ensuring the future strength of Lincoln

in an increasingly competitive funding landscape. The University’s size has enabled us to be both

proactive (deliver opportunity) and responsive (build on success), by investing strategically in our

research infrastructure and engaging our stakeholders throughout the research lifecycle, from design

to delivery, to maximise impact.

Meaningful research

The concept of research ‘impact’ is a key driver nationally, which is unsurprising given that the majority

of funding comes from public money. Good impact comes from good communication, in the design,

delivery and dissemination of research, and we must embrace this.

In bringing together cross-departmental teams to define the grand challenges where we can deliver

real solutions of benefit to industry, health and wellbeing and also wider social outcomes, we can

truly anchor ourselves with local, national and international reach. Bringing people together and

encouraging interaction between disciplines allows us to leverage areas of academic strength and

foster new collaborations/structures, so maximising opportunities to secure competitive research

funds such as the RCUK multi-council Lifelong Health and Wellbeing stream.

How do we create impact, evidence it and embed it in practice? The University of Lincoln has been a

key driver of economic growth in Lincolnshire. The University is worth more than £250 million to the

local economy (doubling local economic growth rates) and has created 5,000 new jobs. Addressing

a Higher Education (HE) ‘cold spot,’ we have seen rapid growth in student numbers: from 2,500 in

2001 to 13,500 in 2014 (from a population of about 100,000). The investment of more than £140

million in the stunning Brayford Pool campus has transformed a city centre site, revitalised the area

and attracted further investment from the retail, leisure and property sectors. Our award-winning

business incubation centre, Sparkhouse, has supported more than 100 new and growing businesses to

establish themselves in the region, creating over 200 new jobs.

Growth and diversification

However, these are challenging times for HE. The introduction of higher student fee rates means

that value for money is at the forefront of potential students’ minds. A tough jobs market means

that employability is paramount for our graduates. Research income is harder to secure in times of

austerity, with research competition being greater than ever and the REF dominating current policy

initiatives.

The REF has been hugely important over 2014, and whilst we all eagerly await the results at the end of

the year, it is critical to use lessons we have learned from our submissions in terms of general culture

34

around longer-term academic planning, particularly around publishing practice and impact planning.

In the current funding landscape, we recognise that there is increased competition for diminished

national funds. To address this, collaboration is key: we need to further establish and sustain strategic

research partnerships with other HEIs, and also with local, national and international enterprises and

organisations, to raise our profile and increase visibility.

European funding (approximately €80 billion of Horizon 2020 funding for 2014 to 2020) is one of

the largest opportunities, but requires international partnerships. Industry-linked collaborations can

unlock new funding streams (for example, the TSB collaboration with RCUK fund of £440 million for

2013/14). For RCUK and charity funding, there is increasing need to show the value of past and future

investments in terms of cross-HEI collaborative working and sharing.

Taking on board the recommendations of the Wilson Review (2013), as universities we must be

accessible to local, national and international businesses, particularly SMEs seeking to innovate, and

we must also be clear in the mutual benefits and breadth of interventions. In augmenting our industrial

collaborations, we need to maximise the impact of our high class research and relationships where

small investments can leverage much larger funding pots, leading to improved graduate employability

and potential secondment opportunities for our early career researchers.

While we maximise opportunities for collaborative bidding with our partners, stakeholder engagement

remains paramount in influencing funding and policy. Raising awareness of our institutions with

funders, potential partners (both HEIs and industry) and the general public will assist us to become

a destination of choice for students and collaborators and will help to convince industry and funding

body reviewers that we are a good place in which to invest.

Multi-level engagement with employers and beneficiaries

As an innovative institution, we are leading the way in treating our industrial partners not only as a

funding source, but working with them as a route for preparing graduates for real-world employment

and as the potential beneficiaries of our research.

Our National Centre for Food Manufacturing is a state-of-the-art training factory sponsored and

updated entirely by industry. Our recent success in securing £7 million Catalyst funding from HEFCE

to create new schools of Chemistry and Mathematics repositions our STEM (Science, Technology,

Engineering and Mathematics) provision; and the development of the Lincoln Science and Innovation

Park in partnership with the retailers Lincolnshire Co-operative will evolve to be an innovative

environment for research, teaching, learning and employer engagement.

35

Whilst embedding expertise of academics and graduates in national and local companies through

Knowledge Transfer Partnerships (KTPs), contract research and consultancy, we are continuing to

develop our work-based learning for industry and the armed forces, increasing workforce skills and

savings for UK taxpayers. Building on the great achievements we have delivered through Sparkhouse

in nurturing and developing young businesses and supporting student entrepreneurship, securing £1

million Regional Growth funding to support businesses in Lincolnshire, our partnerships with the

Lincolnshire Echo newspaper and the Lincolnshire Co-operative will create new jobs and grow our

local economy. And with our Local Enterprise Partnership (LEP), we are continually seeking new

ways to engage while keeping standards high, whether through joint design and implementation of

provision, shared facilities, joint ventures or flexible approaches to delivery and accreditation.

Working with Siemens

Siemens is a leading global engineering and technology services company. The company

has been active in the UK for over 70 years, and currently employs more than 13,760 people

(including about 5,000 in the manufacturing sector). Revenues last year were £3.36 billion.

Within Lincoln, the Siemens Industrial Power and Energy business is the largest employer. It

is located on an old city centre industrial site and at one point was reviewing its position and

considering pulling out of Lincoln. At the time, the University of Lincoln had a good working

relationship with Siemens, but no practical engagement.

In 2009, through a £37.5 million collaboration, we co-created the first new School of

Engineering in the UK for more than 20 years. In doing so, we retained the city’s largest private

sector employer and our shared facilities now constitute an innovative hub of higher learning.

These shared facilities encompass academic spaces, including lecture theatres, seminar rooms,

teaching and project laboratories, alongside industrial engine and gas turbine testing facilities:

they enable student engagement in the workplace while studying, in preparation for rewarding

and successful careers in the future.

Benefits to industry

We are one of five UK institutions to have principal partner status with Siemens. Siemens have

relocated their customer and staff training school to our purpose-built premises. Product training

and development takes place in and alongside the academic Engineering department, facilitating

high levels of employer engagement and HE/industry knowledge exchange. The benefits to Siemens

include upskilling of their work force to address a strategic skills gap, prestige for their training courses,

36

reductions in operating costs, access to world-class research, reduction in the length of time taken

to train graduates and improved staff retention. For us, co-design and delivery of our Engineering

curriculum maximises our graduates’ employment prospects.

The School has already engaged with over 400 engineering businesses and organisations and delivered

contract research for a wide range of organisations (such as retailers Marks and Spencer and the

Mitsubishi corporation). It directly contributed to the Bifrangi forging company’s decision to locate

to the UK and invest more than £40 million in the creation of a closed-die forging facility. The first

phase investment created over 40 new jobs; a second phase is in development to create a Research and

Development centre, with the potential for over 300 additional new jobs. As a consequence of a KTP

to improve the efficiency of turbochargers and their resilience to manufacturing tolerances, the School

has also had direct impact on Napier Turbochargers, for whom efficiency savings of over £50,000 and

additional revenues of more than £400,000 are now projected per year.

Our first cohort of students graduated in July 2013. 90 per cent were picked up by Siemens and 10

per cent by Rolls Royce. The funding we subsequently secured to open our new Schools of Chemistry

(2014) and Mathematics (2015) is attributable to the ways in which we work with industry to design

courses, deliver high quality collaborative research and produce graduates with the skills they want

and need.

Observations and recommendations

Our experience shows that broadening industrial partnership can drive both income diversification and

volume, in addition to supporting key strategic institutional objectives such as graduate employability,

high quality research outputs and demonstrable impacts. Genuine wide and deep relationships provide

greater opportunities for enhanced translation of research excellence through a range of different

models, one example being secondments, particularly of early career researchers.

Universities should thus consider the range of ways in which they can engage with industrial partners

and explore how they might deploy these as a holistic package that ensures the long-term sustainability

of each partnership, increasing its total value and impact.

37

Support structures for public engagement with research

The University of Aberdeen has adopted a dual approach, based on internal factors that influence

change, to provide professional support for embedding public engagement into researchers’

planning and practices

Kenneth Skeldon ([email protected]), University of Aberdeen, UK

Lucy Leiper ([email protected]), University of Aberdeen, UK

The term ‘engaged research’ is becoming ever more frequent in the HE sector, driven in large part by the

need to demonstrate the impact and benefit of research on wider society. Coupled with this, an array

of definitions has emerged in connection with public engagement (PE) and the players involved, many

of which are used interchangeably; reference is made to partners, collaborators, beneficiaries, publics,

stakeholders and external actors in the context of engaged research, often leading to confusion in the

academic community. Much of this stems from the wide spectrum of possible public interactions with

research, extending from co-creation of knowledge to straightforward dissemination of results.

At the University of Aberdeen, we have examined how best to support researchers in PE, for example

how the various ‘engaged parties’ and PE activities should be featured in proposals; the nature of

professional services required for the process; the challenges arising from structural, strategic and

even traditional boundaries of responsibility and the interventions required to address them.

Public engagement: a new academic landscape

The last few years have seen a significant shift in attitude towards PE in the research process, from

being perceived as a worthy add-on to an embedded ‘pillar’ of research impact and excellence. Several

drivers are responsible for this, including public accountability (RCUK Impact Framework, 2009;

RCUK Concordat 2010), the researcher professional development agenda (Roberts, 2002; RCUK

Concordat 2008; Hodge Review) and pathways to impact, as well as an emphasis on co-research

metrics in the REF 2014. The result is that universities are exploring different models of exploiting PE

to enhance academic activity.

This presents an interesting conundrum for universities. On the one hand, we might find the research

community ill-equipped to effectively embrace PE if left entirely undirected. On the other hand,

effective PE with research implies ownership of PE by the research community and its acceptance as a

fundamental aspect of the research process. A key question then is whether effort should be deployed to

support PE through professional services, or should support be devolved in some shape or form to the

academic community? At the University of Aberdeen, we have adopted a dual approach: a dedicated

central professional service team complemented by seconded research-active academic staff.

38

The Aberdeen Catalyst Model

In 2012, we were successful in our proposal to host an RCUK Catalyst for Public Engagement with

Research (RCUK Catalysts, 2011) and influence culture change in PE across the institution. Our

approach was to integrate researcher development and PE into a cohesive strategy which we term

Researcher Engagement and Development (Leiper and Skeldon, 2013). Our guiding principle is clear:

that our academic community and institution recognise PE as core business, an embedded aspect of

the research process and an integral part of career development.

We established a Public Engagement with Research Unit (PERU) within the Directorate of External

Relations, a small core team that operates in close partnership with other professional services and

the academic community. Additionally, a Public Engagement Coordinator from each of the academic

Colleges was seconded (on a 0.2 full-time-equivalent basis) to the central team, with a remit to establish

and extend roots into the academic community. Our Researcher Development Unit (RDU) is located

in our Directorate of Academic Affairs, meaning that a high degree of cross-directorate working is

required to follow our catalyst strategy.

Pillars of Change

Having identified various drivers at play in PE within the HE sector, it is tempting to simply implement

procedures to tackle those drivers. However it is important to consider the various factors, many of

which are local, that can influence changes in behaviour and effect positive change towards PE. These

can be quite diverse in nature, stemming from a mix of personal perspectives, institutional cause and

effect and external stimuli.

Within the University of Aberdeen, we identified a group of influencing factors (outlined in Figure

1) via consultation with the research community and our experience of the operational structures

and approaches of the institution. These factors are: Opportunity, Professional Development, Peer

Influence, Reward, Research Competitiveness and Institutional Profile. Having identified these ‘Pillars

of Change’, our guiding principle was to ensure that operational practice connected directly with one

or more of them on implementation. For each one, we have indicated influences and outcomes, which

in turn lead to operational approaches.

39

Figure 1: The six Pillars of internal change to which we have dedicated effort at Aberdeen.

Pillar one: opportunity

Easy access to PE opportunities and support is often cited when seeking the requirements of an

academic community. While it can be argued that this might defeat the purpose of a bottom-up,

embedded approach, it can also provide important first steps.

Cross-working with the University’s events team and external partners, we have created an annual

programme of research-led public events including community café discussion programmes (40

events a year), PechaKucha nights (four events per year), annual University May Festival leading

with academic led content (in which 40 events are staged over a weekend) and programmes for the

National Science and Engineering Week and the ESRC’s Festival of Social Science (eight events each).

Annually, this offers over 250 staff and students channels of interaction with 20,000 people. Access to

opportunities is communicated through a prominent website, www.abdn.ac.uk/engage, and monthly

e-zines.

Pillar two: professional Development

Low confidence and communications skills are frequently cited as barriers to becoming involved in

PE. Therefore, real opportunity exists to upskill the academic community via the route of training in

and practice of PE skills.

40

One of the hallmarks of our RCUK Catalyst project was a joint coordination model between our

PERU and our Researcher Development Unit, the latter being the internal team that is most closely

associated with this Pillar. When considering PE in the context of professional development, the

synergy between them falls out very naturally. We have jointly introduced incentives that include an

annual Principal’s Prize for Public Engagement with Research and Enabling Funds offering up to £500

for skills development and/or small research-led PE activities.

Pillar three: peer influence

The role of academics as agents of change, through peer influence and advocacy, cannot be

underestimated. To connect with this influence at the University of Aberdeen, we recruited research-

active staff to join our PERU, in a 20 per cent workload model so as not to compromise their research

activity.

We have three academic colleges (Life Science and Medicine, Physical Science, and Arts and Social

Science) and therefore now have three part-time college-based coordinators for PE strengthened by

close working with the central team. As well as building capacity, representing PE on college research

committees and generally developing roots in the academic community, interaction between the

coordinators is helping to foster cross-disciplinary activity around PE. This synergises well with

multidisciplinary research efforts in major areas such as drugs development, synthetic biology and

climate science.

We regularly publish PE case studies, locally and through RCUK publications (such as Inspiration to

Engage, 2013) which help to feed into peer influence. Internal players important to this Pillar include

Heads of College and Directors of Research.

Pillar four: reward

Although not a national driver, reward is an important Pillar of Change, due to the message it sends

to the academic community – namely, that high quality PE can have an impact on career progression.

At the University of Aberdeen, we have two main routes. Firstly, our Framework of Academic

Expectations sets out the core attributes expected of our academic staff and students and PE is now

embedded within this for all levels of staff. Secondly, we have worked hard to have PE as an explicitly-

cited factor in formal promotions criteria, applying also to negotiated remuneration for senior and

professorial staff. Evidence of the effectiveness of this is being gathered from the promotions list, with

many names recognisable as having championed PE through their work. The important internal team

in this area is our Human Resources department.

41

Pillar five: research competitiveness

The main driver in terms of the competitiveness of research derives from the priorities of major

funders and research councils. However, local influencing factors are linked to how PE is supported,

and how grants are monitored and internally peer reviewed.

At the University of Aberdeen, we have connected with our research and innovation colleagues to

encourage scrutiny where co-research activities are involved. By paying attention to PE elements,

submissions can be strengthened: for example, research councils often report an increase in embedded

PE, but also report a general lack of appropriate costing. In terms of the REF, we often situate PE in

the ‘impact’ agenda, while the support and training of staff is in the domain of ‘environment’. This may

be an oversimplification, but it helps to bring focus on the 35 per cent weighting in the REF given to

research environment and impact, and the role PE can play in those terms. The role of PE in doctoral

training and EU grants is also extremely important, and with multi-partner grants, such as for EU

projects, often circumventing internal peer review, it is vital that opportunities to embed PE are not

missed.

Pillar six: institutional profile

Institutions’ strategic plans are ultimately focused on enhancing profile and positioning in rankings.

PE can have a pervasive effect, because it can have an impact on staff and student experience, external

profile, teaching practice and research excellence.

With an increasing shift in the HE sector towards installing centralised PE support infrastructures, PE

is becoming an influencing argument for any single institution to maintain position and leadership. At

the University of Aberdeen, the support of senior management for integrating focused PE within the

overarching strategic plan has always been evident; however, we do find that appraisal of the evolving

landscape and succinct reporting of this to vice chancellors and vice principals is an ongoing necessity.

Outstanding issues

There are still some pertinent challenges in PE. For instance, the role PE will play in research funding is

still evolving and varies from one funder to another. The impact agenda is still evolving and weightings

in future REF exercises will be important. With many grants spanning traditional academic disciplines

and touching upon the PE domain (particularly with regard to EU Horizon 2020), the question arises

whether PE teams where centralised should attract funding themselves.

42

At the University of Aberdeen, we have adopted this approach, arguing that to be spearheading an

event, such as a European Researchers’ Night for Horizon 2020, is collateral that any researcher can

use to lend weight to their own EU grant dissemination plans. There is also the issue of monitoring:

for instance, how should we evaluate research application and fellowship success rates in connection

with PE strategy, when there is no comparison or control?

Another challenge is maintaining an overarching approach. There is a growing number of PE posts

in academia linked to specific research projects, where resource has been requested directly from

funders. While this is a welcome development, it does raise the question of how institutions should

make best use of such resource, so that PE activity does not become fragmented or skewed to specific

areas.

43

References and further reading ERIC: Developing an Impact Capture System

Altmetrics (2011), Altmetrics: a manifesto.

http://altmetrics.org/manifesto/

Bayley, J. (2014), How-to guide for building a university-administered impact management

tool for academics.

http://blogs.lse.ac.uk/impactofsocialsciences/2014/02/07/

eric-impact-management-tool-for-academics

Charters, E. (2003), The Use of Think-aloud Methods in Qualitative Research:

An Introduction to Think-aloud Methods. Brock Education. 12(2): 68-82.

Everall L. & Hilton, S. (2012), Case Study, Embedding Research Impact round 2: Coventry

University.

http://repository.jisc.ac.uk/5110/

Fedorciow, L. & Bayley, J. (2014), Strategies for the management and adoption of impact. capture

processes within research information management systems. Procedia Computer Science 33: 25–32

http://www.sciencedirect.com/science/article/pii/S1877050914007959

HEFCE (2014), Independent review of the role of metrics in research assessment.

http://www.hefce.ac.uk/whatwedo/rsrch/howfundr/metrics/

Hitchins, C. & Bayley, J. (2014), The role of the Research Funding Officer in building robust and

dynamic foundations for impact.

http://blogs.lse.ac.uk/impactofsocialsciences/2014/08/28/

research-funding-officers-foundations-for-impact/

Horizon 2020. What is Horizon 2020?

http://ec.europa.eu/programmes/horizon2020/en/what-horizon-2020

JISC funding opportunities.

http://www.jisc.ac.uk/fundingopportunities.aspx

Research Excellence Framework (2011), Assessment framework and guidance on submissions.

http://www.ref.ac.uk/media/ref/content/pub/assessmentframeworkandguidanceonsubmissions/

GOS%20including%20addendum.pdf

44

Witty, A. (2013), Encouraging a British Invention Revolution: Sir Andrew Witty’s Review of

Universities and Growth.

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/249720/

bis-13-1241-encouraging-a-british-inventionrevolution-andrew-witty-review-R1.pdf

Shifting to a research culture

Bland, C.P. & Ruffin, M.T. (1992), Characteristics of a productive research

environment: literature review. Academic Medicine 67(6): 385-397.

Hill, R.A. (1993), Establishing and Sustaining a Research Culture: A working paper.

Presented to the Australian and New Zealand Academy of Management Conference,

held at Deakin University, Geelong, Victoria, Australia, December 1993.

Krebs, R.E. (1992), A curriculum for Professional Research Administrators.

SRA-Journal of the Society of Research Administrators 24(1): 25-30.

Lawrence, S. (1991), What’s It All About – This Research Administration?

SRA-Journal of the Society of Research Administrators 23(1): 39-41.

Lubin, B. (1992), Market Memo: Clinical Research Depends on Integration

and Support. Health Care Strategic Management 10: 18-22.

Megel, M.E. Langston, N.F. & Cresswell, J.W. (1988), Scholarly productivity: a survey

of nursing faculty researchers. Journal of Professional Nursing 4(1): 45-54.

Polk, G.C. (1989), Research & Clinical Practice: Building a nursing research

culture. Journal of Psychosocial Nursing & Mental Health Services 27: 24-27.

Pratt, M. Margaritis, D. & Coy, D. (1999), Developing a Research Culture in a University

Faculty. Journal of Higher Education Policy and Management 21(1): 43-55.

Zajkowski, M.E. & Dakin, S.R. (1997), Leadership in the University Research Culture:

two Australian cases. Journal of Institutional Research in Australasia 6(2): 1-17.

Support structures for public engagement with research

RCUK Impact Framework (2009), Excellence with Impact Framework.

http://www.rcuk.ac.uk/documents/framework/framework.pdf

RCUK Concordat (2010), Concordat for Engaging the Public with Research

http://www.rcuk.ac.uk/per/Pages/Concordat.aspx

45

Roberts, G. (2002), SET for Success: ‘the Report of Sir Gareth Roberts’ Review’.

http://webarchive.nationalarchives.gov.uk/+/http://www.hm- treasury.gov.uk/ent_res_roberts.htm

RCUK Concordat (2008), Concordat to support the Career Development of Researchers. An

agreement between the funders and employers of researchers in the UK.

http://www.researchconcordat.ac.uk/documents/concordat.pdf

Hodge Review, Review of Progress in implementation of Sir Gareth Roberts’ recommendations,

regarding employability and career development of PhD students and research staff, Professor Alison

Hodge, MBE.

http://www.rcuk.ac.uk/RCUK-prod/assets/documents/skills/IndependentReviewHodge.pdf

REF2014 Research Excellence Framework.

http://www.ref.ac.uk/

RCUK Catalyst Call (2011).

http://www.rcuk.ac.uk/per/Pages/catalysts.aspx

Leiper, L. & Skeldon, K, UKCGE International Conference on Developments

in Doctoral Education & Training Proceedings 2013, 34-42.

Inspiration to Engage (RCUK Concordat).

http://www.rcuk.ac.uk/publications/reports/inspiration/

Horizon 2020 European Researchers’ Night.

http://ec.europa.eu/research/participants/portal/desktop/en/

opportunities/h2020/topics/53-msca-night-2014.html

Yes, I am a research manager, but I don’t cost bids

Carter, I. & Langley, D (2009), Overview of research management and administration, Perspectives:

Policy and Practice in Higher Education, Vol. 13(2): 31-32.

http://dx.doi.org/10.1080/13603100902805359

Langley, D. & Heinze, K. (2009), Restructuring research support offices

Perspectives: Policy and Practice in Higher Education, 13(2): 37-41.

http://dx.doi.org/10.1080/13603100902805409

RIN and RLUK, (2011), The Value of Libraries for Research and Researchers, London: RIN.

http://www.rluk.ac.uk/content/value-libraries-research-and-researchers

46

Taylor, J. (2006), Managing the Unmanageable: the Management of Research in Research- Intensive

Universities, Higher Education Management and Policy, 18(2): 9-33.

www.oecd.org/edu/imhe/42348780.pdf

47

48

49

VistasARMA(UK) Ltd,

Sheraton House,

Castle Park, Cambridge, CB3 0AX.

Tel: +44(0)1223 370052.