The Research Experience of Postgraduate Research … · The research experience of postgraduate research students at the University of Oxford Headlines • The overall experience

I n s t i t u t e f o r

t h e A d v a n c e m e n

o f U n i v e r s i t y

L e a r n i n g

TThhee RReesseeaarrcchh EExxppeerriieenncceeooff PPoossttggrraadduuaattee RReesseeaarrcchhSSttuuddeennttss aatt tthhee UUnniivveerrssiittyy

ooff OOxxffoorrdd

Keith Trigwel

Harriet Dunbar-Goddet

UNIVERSITY OF OXFORD

t

l

The research experience of postgraduate research students at the University of Oxford

Contents Page Headlines ........................................................................................................................ 2 Summary......................................................................................................................... 3 1. Introduction ................................................................................................................. 9 2. Previous studies of the experience of students in postgraduate research (PGR)

programmes .............................................................................................................. 10 2.1 Presage (student characteristics and the established research context) ............ 11 2.2 Research Process ............................................................................................... 14 2.3 Product ................................................................................................................ 17 2.4 A model of the research experience of postgraduate research students ............ 20

3. Students’ approach to research and how research approaches are related to research environment items and scales.................................................................... 21 3.1 An Approach to Research Scale ......................................................................... 21 3.2 Perceptions of research environment scales....................................................... 22 3.3 Relations between perceptions of research environment and approaches to research..................................................................................................................... 23

4. PGR students’ perceptions of their research environment........................................ 25 4.1 Research environment and skills scales scores.................................................. 25 4.2 Response to individual items............................................................................... 28 4.3 Student satisfaction with overall aspects of their research environment ............. 30

5. Relations between the research variables in the study............................................. 32 6. Perceived research environment and doctoral completion times.............................. 39 7. Students’ perceptions of their research environment and the RAE .......................... 40 8. A research experience questionnaire for Oxford....................................................... 41

8.1 Supervision, departmental infrastructure and departmental intellectual climate . 41 8.2 College support ................................................................................................... 42 8.3 Satisfaction items ................................................................................................ 43

9. PGR students’ experience of the Roberts skills agenda ........................................... 44 10. The teaching experience of PGR students ............................................................. 48 11. Other factors (funding, language …)....................................................................... 50 Annexe I: Frequencies and means of questionnaire item responses ........................... 51 Annexe II: Scales .......................................................................................................... 57 Annexe III: Departmental information............................................................................ 61 Annexe IV: Completion times (total terms by commencement year by division) .......... 62 Annexe V: Proposed OPREQ ....................................................................................... 63 Annexe VI: Study methodology..................................................................................... 66 Annexe VII: Study questionnaire................................................................................... 68 Annexe VIII: Qualitative study – PGR student Interview Schedule ............................... 72 Annexe IX: References ................................................................................................. 73 Keith Trigwell and Harriet Dunbar-Goddet Institute for the Advancement of University Learning University of Oxford October 2005 Version2.0

The research experience of postgraduate research students at the University of Oxford Headlines • The overall experience of research at Oxford, as described by most PGR students,

is positive (Section 4.3) with 74.8% of students agreeing that they are satisfied with the quality of their research experience, 9.8% disagreeing, and with a neutral response from 15.4% (Annexe I, Table A, Item 39);

• The quality of research supervision is described, in the research literature, as a major factor affecting the quality of students’ research. At Oxford, supervision is seen by students as an important factor, but not as important as their departmental intellectual climate (Section 4.1);

• The number of times students meet with their supervisor differs between the Sciences and the Arts, but both contain variation. When the frequency of meeting is highest (in both cultures) students describe a more supportive supervisory and intellectual climate, and higher levels of overall satisfaction (Section 5);

• When students experience a supportive departmental intellectual climate, they are more likely to describe satisfaction with the services and facilities, with supervision, and with their overall research experience (Section 5);

• Whether students’ supervision is single, co- or group, there is no difference in their overall satisfaction with the quality of their research experience, or satisfaction with the quality of supervision (Section 5);

• Students’ responses in describing their overall satisfaction with the quality of their research experience are consistent with divisional variation in completion times and departmental variation in RAE ratings (Sections 6 and 7);

• There is no relation between a students’ allocated college (or whether it is a graduate or combined college) and their overall satisfaction with the quality of their research experience (Sections 5 and 8);

• A proposed Oxford Postgraduate Research Experience Questionnaire containing 42 items in six areas found in this study to be related to the quality of the outcome of the students’ programme, is attached. It is recommended as a way of monitoring students’ perceptions and the effects of interventions aimed at changing students’ perceptions (Section 8);

• Students have varying conceptions of research, and that variation is related to how they individually experience their research environment (Section 3). Addressing students’ conceptions of research may be a path to enhancing their experience;

• Only 3.5% of doctoral students at Oxford disagree that developing a wider range of skills (such as communication and team-working) is a valuable part of their research programme (Annexe I, Table A) (Section 9);

• The development of research skills may be assisted by also addressing and improving students’ perceptions of their research environment, such as supervision, departmental intellectual climate and departmental infrastructure, as well as raising awareness of variation in students’ conceptions of and approaches to research (Section 9);

• About one half of the students who want to teach get an opportunity, and almost one half do not think it is clear how to find out how to do so (Section 10);

• Nearly 50% of students agree that they are concerned that their financial situation may affect the quality of their research. Students whose first language is not English also express concerns about the quality of their research (Section 11).

Institute for the Advancement of University Learning 2

The research experience of postgraduate research students at the University of Oxford Summary 1. The study conducted for this report had five aims:

• To explore how Oxford Postgraduate Research (PGR) students conceive of and approach their research and to see how those self-reported approaches relate to a range of research environment indicators;

• To use a stratified representative sample to assess students’ perceptions of their research environment;

• To provide information for the university on a possible questionnaire to regularly gauge the experience of PGR students (OPREQ);

• To investigate students’ response to the Roberts skills agenda; • To investigate aspects of teaching by PGR students.

A summary of the results obtained in the study for each of these aims is provided under the relevant question heading in the remainder of this summary. 2. The effective postgraduate research course context is described in detail in the Code of Practice for the Assurance of Academic Quality and Standards in Higher Education, Section 1: Postgraduate Research Programmes (The Quality Assurance Agency for Higher Education, 2004). The document includes sections on institutional arrangements (such as the availability of the code of practice); the presence of an environment where high quality research activity is occurring; clear and equitable admissions systems, effective supervision; progress, review, feedback and complaints and appeals mechanisms; opportunities for students to develop research and other skills; and clear, fair and accessible assessment criteria. 3. The Oxford Commission of Inquiry (1997a) found some evidence of students’ dissatisfaction with the research intellectual climate, with 32% of respondents experiencing their academic contact with Senior Members as poor, and 22% describing their academic contact with other graduates as poor. This dissatisfaction appeared to be unrelated to discipline or other factors. The Commission of Inquiry report also noted that students in the Arts, Humanities and Social Sciences are more likely to rate their academic conditions as poor than students within Science, Medicine and Education. 4. The quantitative study reported here used an adaptation of an existing instrument (Postgraduate Research Experience Questionnaire, GCCA & ACER, 2002) sent to a 1 in 5 stratified sample of PGR students, and returned by 626 of them (82%). This report contains a separate analysis for each of the two-cultures (Arts and Sciences). Department of Continuing Education students are included proportionally in the sample, but analysed separately only in Section 8 (A research experience questionnaire for Oxford) because of the low numbers involved. What are the perceptions Oxford students have of their research environment? 5. The results from this study show that the overall experience of research at Oxford is positive, with students agreeing in very high proportions that the library facilities support research, that supervision is available when needed and is of high quality, that a range of important skills are being developed, and that colleges provide an opportunity for social contact with other postgraduate students (Section 4.2). Nearly 75% of students agree that they are satisfied with the quality of their research experience, about 10% disagree, and a neutral response is received from 15% (Annexe I, Table A, questionnaire item 39). 6. The high mean scores in the areas described in point 5 above hide a significant body of disagreeing (dissatisfied) students. Around 11% of the sample (or approximately 450


The research experience of postgraduate research students at the University of Oxford PGR students in the University) disagree that they are satisfied with their overall experience of research supervision (Annexe I, Item 38). The number of students in the survey sample (of 626) who do not consider that they are a part of a supportive departmental intellectual research climate is over 120 (Annexe II) (or about 800 students University-wide). The majority of students surveyed do not agree that they are clear about how complaints procedures are handled, that colleges provide supportive subject area communities, and that appropriate financial support is available in Arts faculties/ departments (Annexe I, Items 36, 45, 28). 7. Students’ responses are consistent with completion times and departmental RAE ratings. Bowen & Rudenstine (1992) found that doctoral completion rates were better in the Sciences. They attribute this variation to disciplinary differences in the areas of supervision and the extent to which students work in groups or teams. This Oxford study confirms these results but also shows that where students perceive a more supportive departmental environment such as in the Science disciplines (in Mathematical and Physical Sciences and Medical Sciences), students complete their studies in a shorter time, whereas completion times are longer in divisions where the environment (mainly departmental infrastructure and intellectual climate) is perceived as being less supportive (Section 6). Finally, students in departments that are more highly RAE-rated say they are more satisfied with their supervision and with their overall research experience (Section 7). 8. A summary of the main conclusions from the analysis is presented by key area in points 9-16 below. This study also focused on how variation in these key areas is related to other aspects of the research context, as a means of ascertaining the value of monitoring these areas. These results are included in each key area analysis. 9. Supervision: Students’ perceptions of their research supervision suggest that it is a strength of the Oxford system. The mean response on a five-point agree-disagree scale to the item: Overall, I am satisfied with the quality of my research supervision, is 4.01, with 70 students (11.2%) disagreeing. Satisfaction is highest in the Humanities and Mathematical and Physical Sciences divisions. Students who scored highly on a ‘perceptions of the quality of supervision’ scale were more likely to say they are satisfied with the quality of their research experience. The Commission of Inquiry (1997a) concluded that the frequency of meetings between students and supervisors varies between the two-cultures (Arts and Sciences) and that ‘70% of students saw this as about right’. However, in this study, for the whole sample and within each of the two-cultures, more frequent supervisor meetings are associated with students’ perceptions of a more supportive research environment; and students who experience more supportive supervision are more likely to say they develop graduate skills and express higher satisfaction with their research experience. Students in the Sciences perceive a more supportive supervisory arrangement when they have only one supervisor. There is no difference between supervision types for Arts students (Section 5, Table 6a). Overall satisfaction with supervision is not related to gender, but Science students who spend more time on their research, and those in Arts and Sciences in the earlier years of their programme, are more satisfied with the quality of their supervision (Section 5). 10. Departmental intellectual climate: Nearly a quarter of students do not agree that their department provides them with a supportive intellectual climate. In this study, this area included items such as ‘A good seminar programme for postgraduate students is provided’; ‘The research ambience in the department stimulates my work’; and ‘I feel that this department provides a supportive working environment’. Students’ responses vary significantly by division with Medical Sciences and Mathematical and Physical Sciences being significantly more supportive than Humanities, Social Sciences and Life and Environmental Sciences. Recent literature suggests that doctoral programmes in


The research experience of postgraduate research students at the University of Oxford Arts and Social Science areas do not need to be as isolating as they are (Cadman, 2000; Golde, 2000; Elton & Pope, 1989; Deem and Brehony, 2000). Social isolation, according to Wright & Lodwick (1989), is a departmental issue and one that does require attention. In their longitudinal study of factors affecting first year progress in doctorates in all disciplines, they write that departments must do more to encourage participation and foster a sense of collegiality. Anderson (1996) goes further and suggests that students would do well to seek out highly collaborative programmes. Correlations between departmental intellectual climate and research outcome indicators are found for the Arts and the Sciences and the climate is considered more supportive when students meet more frequently with their supervisor(s). Departmental intellectual climate is not related to gender, and students towards the end of their studies find it less supportive (Section 5), as do students in RAE-rated 4 and 5 departments (Section 7). 11. Departmental infrastructure: The Sciences divisions are seen by students as offering more supportive departmental infrastructures, where issues such as access to equipment and technical support are included along with items on departmental administration, financial support and access to working space. Students who scored highly on the Departmental Infrastructure scale were more likely to say that they are satisfied with the quality of their research experience. Perceptions of more supportive departmental infrastructures are related to more frequent meetings with supervisor(s), to group research supervision in the Sciences (Section 5) and RAE-rated 5* departments (Section 7). There are no relations with gender or college (Section 5). 12. General infrastructure: Approximately 10% of students do not agree that they are satisfied with the university admission and enrolment process, nearly 20% do not agree that the university administration supports their research, and 30% disagree that complaints handling procedures are clear. These items are part of a loose collection of university-wide items that are related to an experience of high quality research. The other items on library provision, common room space and general conditions of study are answered much more positively. There is a strong correlation between students’ perceptions of the general infrastructure and their satisfaction with the quality of their research experience, but few other relations with other variables used in this study (Section 5). 13. Awareness of assessment requirements: How students are assessed is a major factor in the quality of their experience. Students were asked whether they agreed that the requirements of Transfer of Status and of the final assessment processes were clear. More than 15% disagreed in both cases. Students who are more aware of the assessment requirements are more likely to describe satisfaction with the quality of their supervision and their overall experience. These relations are similar in the Sciences and the Arts. Awareness of assessment requirements is not strongly related to gender, year of study, hours per week spent on research, frequency of supervisory meetings, type of research supervision (solo, co- or group) or college type (Section 5). 14. College: Seventy percent of students in the sample are from combined colleges. On a scale of eight items concerned with college support, there was no significant difference between the combined college group response and the graduate college group response. On only one of the eight individual items was there a difference, where students in the combined colleges were less likely to say that they feel they are a part of a community of scholars in their subject area than those in graduate colleges (means of 2.53 and 2.82 respectively). Students in the combined colleges were more likely to say that their college offers some financial support for their research activities than those in graduate colleges (means of 3.50 and 2.96 respectively). Students’ college type is not related to their perceptions of their research environment, but students feel


The research experience of postgraduate research students at the University of Oxford supported more when they live in the college. There is considerable variation between colleges in how students experience the support from their college (Sections 5 & 8.2). 15. Student Characteristics: Nearly 50% of students agree that they are concerned that their financial situation may affect the quality of their research. Students whose first language is not English also express concerns about their research quality (Section 11). In the whole study only one significant gender difference was observed, in which male students self-rated the quality of their research higher than females. However, ‘hours spent per week on research’ is related to many variables. When students say they spend less than 30 hours per week on their research, they are less likely to agree that their department offers a supportive infrastructure or intellectual climate, they agree that they have less development of research skills and they have an overall lower satisfaction with the quality of services and facilities, supervision and overall research experience. There are few differences between those who spend between 30 and 45 hours a week, and those who spend more than 45 hours a week on their research (Section 5). Students in the later years of their doctoral studies experience less supportive departmental infrastructure, less supportive departmental intellectual climates and less supportive supervisory contexts (Section 5). 16. Overall satisfaction: Overall satisfaction with the quality of their research experience (item 39) does not vary significantly by division, college or type of supervision, but it is higher with higher frequency of supervision meetings. Satisfaction with the quality of supervision is very high (university mean 4.01) and highest in the Humanities division. It is higher than the same students’ mean response to the item on overall satisfaction with the quality of the research experience, in all divisions except the Medical Sciences. Differences between the divisions in satisfaction with services and facilities are significant, but where they are lower, the satisfaction with the quality of supervision (and other factors) appears to compensate, as differences between divisions in overall satisfaction are not statistically significant. Science students’ satisfaction with their research experience is related to the amount of time they have spent on their research. Similar relations are not found in the Arts. Overall satisfaction is not related to gender or year of study. What is the proposed structure and content of an Oxford research experience questionnaire? 17. A proposed Oxford research experience questionnaire (OPREQ) is attached (Annexe V). It does not address all aspects of the students’ research experience, but it does include the students’ perceptions variables that, from this Oxford study, have been shown to be related to indicators of research outcomes (student satisfaction, skills acquisition and, indirectly, completion times and RAE ratings). For example, when students experience a supportive departmental intellectual climate, they are more likely to describe satisfaction with the services and facilities, with supervision, and with their overall research experience (Section 5). Previous cohorts of students from the same departments are more likely to have completed their studies in shorter times (Section 6). 18. The proposed questionnaire contains 42 items that do not clash with questions currently asked in most departmental questionnaires. It can be administered electronically, and the results can be made available on the Oxford website. Use of this questionnaire with current students would show students describing their supervisory arrangements as most supportive of their studies, their departmental intellectual climate as least supportive, with departmental infrastructure, college and general infrastructure being similarly and moderately supportive. There are differences between divisions in departmental intellectual climate, departmental infrastructure, and students’ awareness


The research experience of postgraduate research students at the University of Oxford of the assessment requirements, but as with the undergraduate Oxford Student Course Experience Questionnaire, the best use of the OPREQ is to monitor change in one context over time, rather than compare different contexts (Section 8). 19. Interventions aimed at changing the perceptions of students already accepted at Oxford, in those areas related to outcomes, are more likely to lead to an enhanced research experience than a focus on areas that are not related to outcomes. While perceived support from college varies considerably from college to college, this variation does not translate into students’ overall satisfaction with their research experience: relations between allocated college and overall satisfaction show no significant difference. For this reason, and because most colleges have their own feedback systems, it is suggested that the OPREQ contain only an overall college satisfaction item, rather than a College Support scale (Section 8). Do students with different views of what research is about experience a different research environment? 20. Most students see their research as being more about contributing to the ‘big picture’, integrating and discussing ideas within and outside the project, as being worthwhile, as something they have more control over, and they adopt a more integrated or holistic approach than other students (Section 3.1). When students experience their research in this more holistic way, they have a higher self-rating of the quality of their research, they feel that they are making more progress in their development of research skills, they are more satisfied with the quality of their supervision and with their overall research experience, and describe more supportive research environments in all areas except department infrastructure and awareness of assessment (Section 3.3). 21. The relations described above suggest that changing students’ perceptions of their research environment, particularly their departmental intellectual climate, general infrastructure support, supervision arrangements, and college environment, may lead to more holistic approaches to research. Holistic approaches to research, which are associated with higher quality outcomes, may also be enhanced through discussions aimed at making them more explicit. What are Oxford research students’ perceptions of the Roberts skills agenda? 22. Only 3.5% of doctoral students at Oxford disagree that developing a wider range of skills (such as communication and team-working) is a valuable part of their research programme (Section 9). 23. Students’ assessment of their acquisition of key skills is unrelated to their division, college and type and frequency of supervision meetings, but it is related to their approach to research, their perceptions of their supervision, departmental intellectual climate and departmental infrastructure support (Section 5). These relations suggest that the development of research skills may be assisted by also addressing and improving students’ perceptions of their research environment, such as supervision, departmental intellectual climate and departmental infrastructure, as well as raising awareness of variation in students’ conceptions of, and approaches to research. What are Oxford research students’ views about teaching undergraduate students during their research programme? 24. Approximately half (46%) of doctoral students report that they currently teach, or have recently taught, undergraduate students. The most common form of teaching is


The research experience of postgraduate research students at the University of Oxford tutoring, which is done by 194 (68%) of the graduates who teach. When graduate students say they plan to pursue an academic career, they are significantly more likely to say their research will benefit from teaching undergraduates during their research degrees (Section 10). 25. Of the 500 students (81%) considering an academic career, 87% agree that teaching is likely to benefit rather than hinder their research, with students who see their research approaches as more holistic also being more likely to see teaching as benefiting their research. 11.1% of the whole sample disagree that teaching is likely to benefit rather than hinder their research (Section 10).


The research experience of postgraduate research students at the University of Oxford 1. Introduction The study conducted for this report had five aims:

• To explore how Oxford Postgraduate Research (PGR) students conceive of and approach their research and to see how those self-reported approaches relate to a range of research environment indicators;

• To use a stratified representative sample to assess students’ perceptions of their research environment;

• To provide information for the university on a possible questionnaire to regularly gauge the experience of PGR students (OPREQ);

• To investigate students’ response to the Roberts skills agenda; • To investigate aspects of teaching by PGR students.

Each of these aims is addressed using results from a survey of PGR students, and reported in a different section following this introduction and an overview of the literature on the experience of postgraduate research students. Tabular summaries of the results from the survey are presented in Annexes I-IV. Annexe V contains the proposed Oxford Postgraduate Research Experience Questionnaire (OPREQ), which has been developed from the results of this study and is presented for discussion in the collegiate university. Descriptions of the study methodology, sample size and selection, and analytical processes used are included in Annexe VI. Annexe VII contains the study questionnaire and Annexe VIII information on the qualitative parts of the study. The project described in this report, does not aim to repeat the research done at Oxford and elsewhere on issues such as graduate admissions and enrolments (Atkinson, et al., 2004) or on completion rates. Nor does it aim to address issues related to the quality of individual supervision or student welfare, finances and personal issues. The previous research that has been reported in these areas is summarised in Section 2. The focus of this study extends that literature to look at the way aspects of the research environment established by the university and colleges are experienced by research students, with the aim of suggesting how those perceptions might be changed to improve the experience of research students. Previous studies of the doctoral experience, including the Oxford Commission of Inquiry, consistently show disciplinary differences on many indicators used. For example, the Oxford Commission of Inquiry (1997a) found that students within Science and Medicine had more day-to-day contact with their supervisors, and expected more discussion with their supervisors, than students within the Humanities and Social Sciences. Rather than repeat these known results, the analysis of the data from this study have been conducted separately for the ‘two-cultures’ (the Sciences and the Arts). The main reason for this approach is to explore the relations within similar disciplines. If within similar cultures, the students who experience less day-to-day contact with their supervisors are also the students who report lower attainment of research skills, or lower levels of satisfaction, then that effect is less likely to be discipline-based, and may be a subject justifying further investigation.


The research experience of postgraduate research students at the University of Oxford 2. Previous studies of the experience of students in postgraduate research (PGR) programmes The research experience of postgraduate research students has been the subject of extensive research. Two comprehensive texts, one from Britain (Becher, Henkel, & Kogan, 1994) and the other from the USA (Bowen & Rudenstine, 1992), present overviews on graduate education in their respective counties. Becher, et al. focus largely on policy issues, as they relate to the characteristics of students and departments. They note the importance of the debate on the purposes of the doctorate and how it is increasingly being seen as research training rather than new knowledge generation. Studies that inform their views are mainly qualitative. Bowen & Rudenstine present a more quantitative study, focusing on policy, trends and factors affecting outcomes. At Oxford, the Commission of Inquiry (1997a) carried out a survey of Graduate Students in Hilary Term 1996. The questionnaire concentrated mainly on the amount and type of teaching received, the use of facilities such as libraries and computers, and on financial aspects as well as questions about the amount of teaching available. Three issues were frequently commented on by the students: the collegiate system, the quality of research supervision and facilities, whilst course content and opportunities also provoked some comments. A common complaint was that both the University and the colleges ‘gave priority to undergraduate needs and research over graduate requirements, leaving graduates as second-class citizens.’ (Commission of Inquiry, 1997b, p.491). The most complete studies of the form described in this report have been carried out using the Postgraduate Research Experience Questionnaire (PREQ) which was developed in Australia (Australian Council for Education Research (ACER), 1999). All Australian postgraduates who have completed a research degree are asked to complete the PREQ (Graduate Careers Council of Australia (GCCA) & Australian Council for Educational Research (ACER), 2002). The questionnaire was developed through literature reviews of research students’ experience, current institutional evaluation research, good practice, and outcomes in relevant area; existing instruments; feedback from Higher Education staff; a special conference, and focus groups with research students (Marsh, Rowe & Martin, 2002). The PREQ contains six scales, five of which examine students’ experiences of the process of their research degrees (these scales examine students’ perceptions of the infrastructure of the environment in which they are studying, of the intellectual climate in which they are studying, of the goals and expectations of their courses, of supervision, and of their thesis examination), and one which examines the outcomes for students through their perceptions of the skills they have developed through their doctoral studies. An overall satisfaction with their doctoral studies item is also included. These scales have been found to be internally consistent and in a confirmatory factor analysis have been found to be a good fit with the data (Marsh et al., 2002). The student data elicited using the PREQ have also been found to have a similar structure across different broad areas of study (Arts/Humanities vs. Science/Engineering) (Ramsden, Conrad, Ginns & Prosser, 2003). In this review of this and other postgraduate research we address separately (a) the characteristics of students and the context as established by the university, (b) processes of research and (c) outcomes of research. The three areas correspond to Presage, Process and Product (3P) elements of the research experience.


The research experience of postgraduate research students at the University of Oxford The final section brings the three sections together in an adapted 3P model that shows the factors that this literature suggests will impact on postgraduate research students’ experience of the research environment at Oxford. It is this model that is used to inform the study that is described in Sections 3-5. 2.1 Presage (student characteristics and the established research context) Studies of presage factors found to be related to success on doctoral programmes reveal few useful insights. Wright and Cochrane (2000) included students’ age; mode of study (f/t or p/t); origin (home/EU/overseas student); sex; funding; class of first degree; university of first degree; and discipline, in their study of 3579 theses from the University of Birmingham. Only discipline (the success rate was better in the Sciences) was found to be a reliable predictor, but intrinsic student characteristics (such as motivation and conceptions of research) and institutional conditions were not included in the study. Some studies that do include intrinsic student characteristics and institutional conditions are presented in the following sections. 2.1.1 Student characteristics Reasons for undertaking a research degree Both Anderson and Swazey (1998) and Atkinson, et al. (2004) investigated the reasons given by students in undertaking their research degrees. The study by Atkinson, et al. looked specifically at Oxford University entry, and for 57% of students, the reasons for applying only to Oxford were for the reputations of the course or the supervisor. Anderson and Swazey found that the distributions of importance (very and somewhat) in students’ decisions to go to graduate school were as follows: Desire for knowledge (99%); Desire to do research (94%); Desire to benefit others through this work (82%); Desire to teach in higher education (71%); and then a big drop to 44% for career related reasons. Demographic information The Atkinson et al. (2004) pilot study of Oxford’s UK graduate entry student demographic characteristics suggests that new graduate students at Oxford are drawn more or less half from Oxbridge and half from other universities, with only a quarter coming from universities outside the Russell Group. The graduate students arriving from universities other than Oxford or Cambridge are less frequently funded by Oxford-specific scholarships, are more frequently members of graduate colleges, are over-represented in Arts subjects, have more frequently applied to universities other than Oxford and are more likely to belong to an older age group than the new graduates who come from Oxbridge. Students’ experience of research contexts are related to their cultural backgrounds. Deem & Brehony (2000) note that international and part-time students have the most difficulty accessing peer cultures and academic cultures, but international students are much more favourably disposed to research training cultures than others. Cadman (2000) also addresses the issues of divergent cultural backgrounds in postgraduate programmes which may assume unquestioningly that only students from cultures different to that of the award-offering University need to change their academic goals and practices, especially in relation to critical thinking. It is argued that the challenge is to learn on both sides. Suggestions for departmental changes from Deem & Brehony (2000), Elton & Pope (1989) and Cadman (2000) are discussed under experienced intellectual climate below.


The research experience of postgraduate research students at the University of Oxford Gender While there are large differences between the numbers of male and female students enrolling to study in the different disciplinary areas, there is little evidence of gender differences within the disciplinary areas. Wright & Cochrane (2000) found no significant differences between the sexes in success rates in doctoral studies at the University of Birmingham. Bowen & Rudenstine (1992) note that in the USA, gender is only a very small part of the explanation for lower completion rates in the Humanities and Social Sciences (p. 125). 2.1.2 The research context The effective postgraduate research context is described in detail in the Code of Practice for the Assurance of Academic Quality and Standards in Higher Education, Section 1: Postgraduate Research Programmes (The Quality Assurance Agency for Higher Education, 2004). The document includes sections on institutional arrangements (such as the availability of the code of practice); the presence of an environment where high quality research activity is occurring; clear and equitable admissions systems, effective supervision; progress, review, feedback and complaints and appeals mechanisms; opportunities for students to develop research and other skills; and clear, fair and accessible assessment criteria. In this section we focus on what we consider to be five key university presage areas: supervision, the infrastructure of the research environment, the intellectual research climate, discipline and mode of study, and assessment. Supervision The research supporting the Code of Practice has, as its main focus, the context of supervision. Holdaway, Debois, & Winchester (1995) summarise comments from nearly 700 experienced supervisors of Canadian masters and doctoral students in eight disciplines. The synthesis of the literature focused on creativity, mentoring, support; procedures to ensure progress; and disciplinary differences, which were combined with the research results to produce a list of good supervisory practice. Pole, Sprokkereff, Burgess, & Lakin (1997) also use this type of research to develop guidelines for supervisory practice. Their guidelines include issues such as maximum supervisory loads; relations between the supervisor’s research and the doctoral project; the importance of what the supervisor does, such as giving feedback, keeping the project manageable, and making revisions to the project where necessary. In the Australian context, Heath (2002) describes variation in supervisory provision in which 32% of 355 respondents say they have a single supervisor, with the rest saying they have one or more associate supervisors. Dissatisfaction with supervision was at about 6%. At Oxford, the Commission of Inquiry reported that ‘the quality of supervision was a major source of dissatisfaction for some respondents, although there were many who were very happy with their supervisor.’ (Commission of Inquiry, 1997b p.492). Some of the respondents to the consultation questionnaire made suggestions that included having formal guidelines about supervisors’ obligations that include a minimum time allocation and also a more formal system for appointing a second supervisor. Departmental infrastructure of the research environment Resources in departments, such as necessary equipment, technical support, library access and space are not usually reported as issues affecting success in doctoral programmes. However, Wright & Lodwick (1989) describe the problem of social isolation as being a departmental issue and one that does require attention. In their longitudinal study of factors affecting first year progress in doctorates in all disciplines,


The research experience of postgraduate research students at the University of Oxford they write that departments must do more to encourage participation and foster a sense of collegiality. They also found that problem areas included the identification of the research topic and project planning, writing, and learning about key research skills such as accessing information sources; research techniques, design and implementation; and organising and managing the thesis. At Oxford, the Commission of Inquiry also reported on the academic facilities such as library access, workspace and computing and they also flagged up that some students in the Humanities and Social Sciences felt they ‘suffered from a lack of academic support because they did not belong to a formally organised department’ which led to feelings of isolation both academically and socially (Commission of Inquiry, 1997b, p.494). Both Elton & Pope (1989) and Collinson (1998) see some of the difficulties experienced by students being related to unclear purposes of the PhD. The PhD is now seen in the UK as a period of training for future researchers as distinct from the generation of new knowledge, says Collinson. These are conceptions of research held by staff and ‘university departments’ and they influence the research ethos experienced by students. The training and the problems that go with this approach – generic skills, research design, data collection and the application of instruments, as described in the Roberts report (Roberts, 2002) are questioned. She sees research as essentially a craft, and though it is theoretically informed, it is essentially a practical endeavour. The new demands put even more pressure on students as some proposed outcomes may not be possible. Intellectual climate As noted above, suggestions for the improvement of the intellectual climate (Deem & Brehony, 2000) include the development of collegiality (also from Elton & Pope, 1989) to overcome power issues and the idea of collective responsibility (Cadman, 2000). Most writers claim that Social Sciences degrees do not need to be as isolating as they are. Integration by students into the research and social community were found by Golde (2000) to be crucial factors in doctoral attrition. His results include a reminder of the complexity of graduate student lives, and the conclusion that attrition is a shared responsibility of department and student. Three themes emerged: academic integration – relations with faculty; social integration – the student community; and telling others about leaving. It seems the decision to leave is not readily told to anyone, especially the department. Anderson (1996) investigates collaboration in Science and Social Science areas and concludes that there is much to recommend the collaborative department. Johnson, Lee, & Green (2000) use a gender-based analysis to question the idea of autonomy and the independent learner, particularly in the Humanities and Social Sciences, and what it does to the intellectual climate. They note that ‘traditional practices of PhD supervision …assume “autonomy” – in the form of the exemplary figure of the independent scholar – to be the desired outcome. Our analysis suggests that figure is a problematic one for the profoundly gendered character of the assumptions of reason and autonomy it invokes.’. (p.141) ‘Necessary to the sense of autonomy that is the endpoint of this pedagogy for the development of the rational, independent scholar, the student must experience themselves in control, as author of their intentions, as exercising free will and independence.’ (p.142) Disciplines and mode There are significant differences between the disciplines in doctoral completion rates, with Humanities and Social Sciences having the lower success rates and the Sciences the highest successes (Bowen & Rudenstine, 1992; Wright & Cochrane, 2000). The


The research experience of postgraduate research students at the University of Oxford main disciplinary differences appear in the areas of supervision and the extent to which students work in groups or teams. Studies with a disciplinary focus include Morton & Thornley (2001) in Mathematics, Pole et al. (1997) in Science, Johnson et al. (2000) and Collinson (1998) in the Social Sciences. Chiang (2003) looks at the concept of teamwork in Chemistry and Education. Issues associated with part-time research studies receive little attention, though Evans (2002) describes the contribution they make to the knowledge economy and argues for greater understanding of the impact and benefits for the public good. Taylor (2002) in a paper on trends in graduate education highlights increasing proportions of part-time students along with an expansion of international recruitment and changes in course delivery with the use of new technologies as the major change areas. Assessment While there is variation internationally in the modes of doctoral assessment, the processes across disciplines within nations, are quite uniform. Oxford, which has a process similar to most UK universities appoints two examiners, neither of which is the supervisor, to consider the thesis and abstract, to examine the student orally, to satisfy themselves whether the student has a good general knowledge of the field of learning, and to report to the faculty concerned on the scope, character and quality of the work submitted (University of Oxford, 2003, p.818). 2.2 Research Process 2.2.1 Motivation and Conceptions There is little research that directly examines students’ motivations and conceptions within the PGR learning environment, as opposed to students’ motivations and conceptions before they graduate. One area relating to students’ perceptions of the goals and expectations of their courses that has been explored in the wider literature is students’ conceptions of what is expected in reviewing the literature that relates to the area of their research. Bruce (1994) in a phenomenographic study of research students’ early experiences of the dissertation literature review constituted six qualitatively different ways in which students conceived of their literature reviews. These ranged from students seeing the literature review as a collection of items representing the literature of the subject to seeing the literature review as a written discussion of the literature that draws on previous research. Bruce (1994) argues that her findings suggest that some students early in their courses have impoverished ideas of what the literature is about. In a similar way in a US study, Caffarella & Barnett (2000) found that preparing critiques on other students’ work and, in turn, receiving critiques of their work from other students as well as their professors, was perceived by research students as highly influential in helping them to understand the process of scholarly writing. Both of these findings could be interpreted as relating to conceptions of the literature (and of research) or issues surrounding students’ perceptions of the goals and the expectations of their courses. Bruce's (1994) finding could be interpreted as suggesting that some students are unclear as to the purpose of their literature review early in their studies, whilst the Caffarella & Barnett (2000) study suggests a way in which research students might become clearer about what is expected in academic writing. 2.2.2 Perceptions of the research environment


The research experience of postgraduate research students at the University of Oxford The Student Course Experience Questionnaire (SCEQ), which provided many of the process variables for the ‘student perceptions of the learning environment’ element of the Oxford undergraduate learning experience research (Trigwell & Ashwin, 2003), has proved to be unsuitable for research students. This led to the development of the Postgraduate Research Experience Questionnaire (PREQ) in Australia (Australian Council for Education Research (ACER), 1999) described above. Four of the process scales areas from the PREQ (students’ perceptions of the infrastructure of the environment in which they are studying, of the intellectual climate in which they are studying, of supervision, and of their thesis examination), will be used to structure the rest of this section of the review. However, it will draw upon literature that goes beyond that used in the development of the PREQ. Students’ perceptions of the infrastructure of the environment in which they are studying Within the PREQ, students’ perceptions of the infrastructure in which they are studying refers to the extent to which students feel that they had access to an environment that would support their learning in terms of physical and financial resources. The PREQ study (GCCA & ACER, 2002) found that there were disciplinary differences in students’ perceptions of their infrastructure, with students in the Arts, Humanities and Social Sciences experiencing less access to resources than students in the Sciences. In the UK, Chiang (2003) found that Chemistry students perceived their infrastructure as more supportive than Education students. Within Oxford, the Commission of Inquiry (1997a) had similar findings with students in the Arts, Humanities and Social Sciences more likely to rate their academic conditions as poor than students within Sciences, Medicine and Education. In some ways these findings are unsurprising given the differences in the ways in which research degrees in different disciplines are structured. The important question, from the perspective of this review, is the impact this has on the quality of students’ overall experience. Due to the differences across disciplines, the effect of access to a supportive infrastructure is probably best examined within disciplines. Students’ perceptions of the intellectual climate in which they are studying Within the PREQ, students’ perceptions of the intellectual climate in which they are studying refers to the opportunities available for social contact with other students, the extent to which they felt they were integrated into the departmental community, the opportunities to become involved in a broader research culture, the extent to which students perceived that there was a stimulating research ambience and their perceptions of the quality of the seminar programme that was provided for them. Students in the Humanities and Social Sciences have been found to have significantly lower scores on this scale than students in the Sciences (GCCA & ACER, 2002). In the US Anderson & Swazey (1998) found that Sociology students were less likely than students in Chemistry, Civil Engineering and Microbiology to see their departments as community orientated. Similarly in the UK, Chiang (2003) found that Chemistry students were more likely to perceive the research environment as effective than Education students in terms of staff approachability, students-staff interaction and the research culture. Chiang (2003) argues that this is due to the ‘teamwork research training structure’ in Chemistry, in which research students and their supervisors work on the same projects, whereas Education has an ‘individualist research training structure’, where students and supervisors work on separate problems. Within Oxford, the Commission of Inquiry (1997a) found some evidence for students’ dissatisfaction with the intellectual climate, with 32% of respondents experiencing their academic contact with Senior Members as poor, and 22% describing their academic


The research experience of postgraduate research students at the University of Oxford contact with other graduates as poor. This dissatisfaction appeared to be unrelated to discipline or other factors. Students’ perceptions of the goals and expectations of their courses There appears to be very little literature on how students’ perceptions of the goals and expectations of their research degrees impact on their experience of studying. The PREQ items that make up this scale are related to students’ perceptions of how well they felt they understood the standard of work required in their studies and in their theses (GCCA & ACER, 2002). Students’ experiences of supervision The PREQ items on the supervision scale focus on students’ perceptions of the availability of supervision, their perceptions of the support, guidance and feedback provided by their supervisors, including guidance relating to the literature review. Use in Australia of the PREQ on students’ experiences of supervision shows students from Humanities and Social Sciences feeling more supported than students from Sciences and Engineering (GCCA & ACER, 2002). It is also clear from examining the wider literature in this area, that students in different disciplines have very different experiences of research supervision. However, the differences are not the same, or even in the same direction as those found using the PREQ. Broadly stated these differences are that students within the Natural Sciences, particularly in experimental disciplines1, tend to be part of a research group, in which they have a clear role and in which they work alongside other researchers on related topics. They tend to be more involved in group discussions and group publications. However, they tended to be assigned the problems that they were to research in the completion of their doctorates. Those within the Social Sciences tend to report feeling socially isolated, as there tend to be fewer of them and they were often not involved in conferences and colloquia. They also report being intellectually isolated as they are treated as individual scholars with responsibility for their own projects. For Social Science students, this can make the supervisory relationship even more central to their doctoral experience (Chiang, 2003; Delamont, Atkinson, & Parry, 2000; Pole et al., 1997). Disciplinary differences have also been picked up within Oxford. The Commission of Inquiry (1997b) found that students within Sciences and Medicine had more day-to-day contact with their supervisors and expected more discussion with their supervisors than students within the Humanities and Social Sciences. However, the amount of contact was seen as ‘about right’ by over 70% of the students within each discipline. Within the supervisory relationship, a tension has been highlighted between the extent to which students should be given autonomy in deciding their own work and the extent to which they should be guided by their supervisors (Delamont et al., 2000; Gurr, 2001). Such tensions are clearly related to the disciplinary differences outlined above and to the stage that the student has reached in their doctoral research. However, there is the potential for conflict if student and supervisor have different views of how this tension should be balanced (Wright & Lodwick, 1989; Dawson, 1996; Gurr, 2001; Haksever & Manisali, 2000; Lee, 1998). One area of particular conflict appears to be around the degree of support in learning how to conduct research that students expect to receive from their supervisors, with students’ satisfaction with the level of support they receive in this area being related to their satisfaction with the supervision generally (Haksever & Manisali, 2000; Heath, 2002).

1 It should be noted that in theoretical scientific fields students may experience isolation (for example, see Morton & Thornley (2001) on Mathematics doctoral students in New Zealand)


The research experience of postgraduate research students at the University of Oxford Students’ perceptions of their thesis examination The PREQ thesis examination scale looks at students’ perceptions of the fairness of, and satisfaction with the examination process as well as whether the examination was completed in reasonable time (GCCA & ACER, 2002). However, it should be noted that in Australia, theses are examined without viva examinations. This, as well as the focus of our research being on current research degree students, means that this scale is less relevant to our research and, indeed, when Ramsden et al. (2003) carried out a study using the PREQ with existing research students, they also dropped this scale. Despite this, students’ perceptions of the ways in which their work will be assessed is likely to impact on their approaches to studying. There is often a disjunction between students’ expectations and their actual experiences of their viva examinations and the variability of such examinations (Tinkler & Jackson, 2002). 2.2.3 Approaches to research There has been very little research into variation in the approaches to research adopted by PGR students. Parallels with the undergraduate learning experience would suggest that approaches to research are likely to be focused more on the students’ personal search for meaning than on attempts to satisfy an external assessor, such as a supervisor or examiner. The research by Brew (2001), Bruce (1994), Kiley & Mullins (2003), Meyer, Shanahan, & Laugksch (2003) and Pearson & Brew (2002) suggests that some students may take a more holistic and integrated approach to their research, seeing it as part of an integrated whole, whereas others are more likely to see their research as being about developing techniques and about what it is that external others, such as supervisors, see as being important. These hypothesised differences are tested in this study. 2.3 Product The ‘product’ variables are the outcomes of the research experience for the students. The two outcomes scales areas of the PREQ are the focus of this section of the review. However, as in the ‘process’ section above, it will draw upon literature that goes beyond that used in the development of the PREQ. The PREQ (GCCA & ACER, 2002) outcomes areas are the students’ perceptions of their skill development and their overall satisfaction with the research degree experience. Closely related to these positive outcomes are the reasons why students do not complete, or take a long time to complete, which can be seen as negative outcomes of the experience. The literature points to ‘quality of supervision’ as a key factor to satisfaction (Haksever & Manisali, 2000; Harman, 2002; Sayed, Kruss, & Badat, 1998), however other factors such as the research environment (Anderson & Swazey, 1998; Asmar, 1999; Golde, 2000; Harman, 2002) also contribute to students’ satisfaction. The possibility of finding a job post-PhD (Bazeley, 1999) and the learning of key skills are also outcomes that have received attention in the research literature (Caffarella & Barnett, 2000; Sayed et al., 1998). In this section we will look in turn at three of these elements of the research experience: completion time, key skills and satisfaction. 2.3.1 Completion Typically studies on the doctoral experience focus on students who complete their degree. Golde (2000) suggests that a lot of information could be gleaned from students who leave their doctoral programme. Two of the most often cited problems are a lack of methodological skills and isolation (Sayed et al., 1998), which shows that the doctoral experience is both academic and social. Academic reasons were central in the attrition


The research experience of postgraduate research students at the University of Oxford reported by Golde (2000) and Haksever & Manisali (2000). Haksever & Manisali (2000) report that one of the major reasons for non-completion is deficiencies in the supervision received and their results also showed that the biggest discrepancy between expected and provided supervision was in ‘direct research-related help’. This is unsurprising given that Seagram, Gould, & Pyke (1998) note that more frequent supervision is strongly associated with successful completion. Beyond the supervision aspect, other academic reasons combine to affect the time to completion or a decision to drop out, for example students have no or inadequate experience of research and thesis/academic writing, students may be enrolled part-time and have demanding professional careers or other priorities (Evans, 2002; Sayed et al., 1998). The personal/social aspect of the experience is also problematic with issues such as isolation, and for women in particular a ‘chilly climate’, family commitments, finance, and problems with personal relationships, affecting the students’ doctoral research and leading to withdrawal or non-completion in some cases (Sayed et al., 1998; Seagram et al., 1998). In Wright & Cochrane's (2000) study the only factor that emerged as a reliable predictor of successful submission was whether a student was researching a Science-based or an Arts and Humanities-based subject. In their study the most successful were students in a Science-based subject, funded by research council, 21-26 years old at entry and who had a 1st or upper second. They found no gender effect, no nationality effect, and no effect from continuing their doctoral studies at the same university. What is interesting is that these most successful students are the ones least likely to face the difficulties mentioned above (financial difficulties, family commitments, no experience of academic writing). 2.3.2 Key (transferable) skills The doctorate is now seen in the UK as a period of training for future researchers, and part of that training involves the development of a range of skills. The objective is to produce professionally trained researchers who are able to undertake any research project in their general discipline area having completed the skills training (Collinson, 1998). The literature describes the key skills students expect to acquire during the PhD experience as academic (such as methodological skills (Sayed et al., 1998), how to produce scholarly writing (Caffarella & Barnett, 2000) or more particularly literature reviews (Bruce, 1994)), or personal/social. The more personal skills such as growing confidence and autonomy are given as positive outcomes of the research experience (Sayed et al., 1998). There is also some mention of the skills that future non-academic employers would like postgraduates to work on such as team-work, project management and communication skills (Bazeley (1999) reports Clark (1996)). The PREQ (GCCA & ACER, 2002) scale relating to skill development in Australia asks about the students’ problem solving skills, written work, analytic skills, work planning and tackling unfamiliar problems. Students who completed the PREQ in 2000 reported a high level of satisfaction with their own development of research skills. An Australian study has reported that women felt that the opportunities for research skills development during candidature were given to men (Asmar, 1999) although it should be noted that according to Holdaway (1996) quoted by (Bazeley, 1999) the acquisition of knowledge and skills is only a secondary focus in postgraduate studies in Australia, as opposed to the American model. Sayed et al. (1998) suggest that in order to facilitate completion there is need for more structured and formal learning, teaching and training around research, for example through ‘research workshops’ that would: introduce students to the concept of research,


The research experience of postgraduate research students at the University of Oxford and cover issues related to knowledge production, developing a research proposal and project, conducting a literature review, understanding the relationship between theory and research, using research methods and techniques, using data sources, issues related to note-taking and writing, and revising and editing an academic text. A current government initiative in the UK, following the Roberts Report (Roberts, 2002) is that doctoral students develop certain ‘transferable skills’ during their doctoral programmes. The UK Research Councils have issued a Joint Statement of Skills Training Requirements (JSSTR) (Research Councils UK, 2001), that doctoral research students funded by the Research Councils would be expected to develop during their research training. The JSSTR describes seven broad skills areas: A: Research Skills and Techniques, B: Research Environment, C: Research Management, D: Personal Effectiveness, E: Communication Skills, F: Networking and Team-working, and G: Career Management. It is felt by the UK GRAD Programme (2004) that a Personal Development Planning (PDP) process is a way to ‘realise the ethos of the Roberts review’. Indeed the most recent Code of Practice for the Assurance of Academic Quality and Standards in Higher Education requires that institutions provide research students with appropriate opportunities for personal and professional development, which include the development of research and transferable skills. It suggests that ‘Planning for skills development and checking that necessary guidance and support has been provided should form part of the process of personal development planning’. It also suggests that ‘research students may find it useful to use the PDP tools provided by their institutions to record their personal progress and development, including reference to research and other skills’. (The Quality Assurance Agency for Higher Education, 2004). The PDP will also give students an opportunity to reflect on their skill development which might otherwise be rather implicit or embedded, and this should in turn lead to higher scores on a questionnaire like the PREQ. 2.3.3 Satisfaction The PREQ (GCCA & ACER, 2002) asks students about their overall satisfaction with the quality of their higher degree research experience. Harman (2002) reports relatively low satisfaction ratings given by PhD students to their overall course experience, particularly with supervision, and that females are more dissatisfied than males. Only 57% of students said their experience was satisfactory or very satisfactory, while 13% said unsatisfactory or very unsatisfactory. Harman goes on to explain that unsatisfactory or lowly-rated course experience may indicate that academic departments have been less than successful in offering rich PhD training and research experiences, while dissatisfaction with course experience is likely to have an adverse impact on career plans and interest in pursuing a research career. Also related to students’ satisfaction is whether or not they have found secure, appropriate employment (Bazeley, 1999), how they experienced the research environment (Anderson & Swazey, 1998; Asmar, 1999) and supervision (Sayed et al., 1998; Seagram et al., 1998). For example Asmar (1999) found that the departmental environment during the PhD had been more satisfying overall for men than for women. Bazeley (1999) reports that candidates with a strong career focus were among the most disgruntled and gives examples of the intense frustration among those who were unable to gain secure, appropriate employment after all the years of studying and training to be a researcher, whilst those who did a PhD out of interest in research or the topic spoke of positive outcomes of their study such as a gain in status and the development of networks.


The research experience of postgraduate research students at the University of Oxford It is also important to note that gender, whether a course is Science-based or Arts-based and whether the student is full-time or part-time is related to course satisfaction and/or completion. For example Smeby (2000) reports that female PhD students are more likely than males to drop out before completion, whilst Seagram et al. (1998) make mention of the ‘chilly climate’ experienced by women. 2.4 A model of the research experience of postgraduate research students The factors shown in Figure 1 are hypothesised to be related in ways that help to explain variation in outcomes of postgraduate students’ research at Oxford. The following sections contain the results of the study conducted to explore these factors and relations.

cational

Student characteristics: Gender, cultural background, previous eduexperience, conception of graduate studies

Figure 1: Adapted 3P (presage-process-product) model of student learning for

postgraduate research programmes

Motivation & conceptions: Motivation, self-efficacy, conception of learning, conception of research

Course context: Division, supervision, monitoring and feedback mechanisms, intellectual climate,departmental conceptions of the purpose of the research degree

Approach to research: Holistic/inte-grated or serialist/fra-gmented?

Research outcomes: Completion, key skills, satisfaction

A consistent message from the literature review described above is that there are quite different cultures in different disciplines. Variation between the disciplines is therefore to be expected. With respect to this study, the variation within a disciplinary area is of more interest, as it may offer insight into ways in which activities in that disciplinary area might be adjusted to achieve higher quality outcomes. In the rest of this report, the sample of 626 doctoral candidates is analysed (a) as a whole and (b) in two groups (two-cultures: Arts and Sciences) and in a few cases, by division.


, supervision, assessment

Perceptions of environment: Perceptions of research infrastructure, intellectual climate, course goals and expectations

The research experience of postgraduate research students at the University of Oxford 3. Students’ approach to research and how research approaches are related to research environment items and scales 3.1 An Approach to Research Scale The question we sought to address in this part of the study is: What is it that students who think and act more like researchers value more in their research environment? The following four items are part of a seven-item Approaches to Research scale used to describe thinking and acting like a researcher (Annexe II, Table C). To experience what it means, you may wish to record what you consider to be the preferred student response to these items and note the total of the sum of the numbers in your response. Item strongly disagree neutral agree strongly

disagree agree 1. It is important to me that my research is well integrated with existing knowledge and topics in the field ① ② ③ ④ ⑤ 2. Often I find myself wondering whether the work I am doing here is really worthwhile . . . . . . . . . . . . . ⑤ ④ ③ ② ① 3. I see my research as contributing in some way to ‘big picture’ issues. . . . . . . . . . . . . . . . . . . . . . . . . ① ② ③ ④ ⑤ 4. I usually try to discuss with others new ideas I have in my research. . . . . . . . . . . . . . . . . . . . . . . . ① ② ③ ④ ⑤ Total = _________ The response to these four items from 626 PGR students who returned questionnaires varied widely as shown in Figure 2. Their individual totals range from 6 to 20 with a mean of 15. The mean score is high, but it is the variation that is significant. A similar distribution is also found in each division, and for each of the Sciences and Arts (Annexe II, Figure A).

0

20

40

60

80

100

120

6 7 8 9 10 11 12 13 14 15 16 17 18 19 204-item total

Freq

uenc

y

Figure 2: Variation in 4-item total for students’ approaches to research (n=626)

An Approaches to Research scale was constituted from these four, and three other similar items (Annexe II, Table C). Some students see their research as being more about contributing to the ‘big picture’, integrating and discussing ideas within and outside the project, as worthwhile, and as something they have more control over. They have a more integrated or holistic approach than other students. Research by


The research experience of postgraduate research students at the University of Oxford Anderson and Swazey (1998) found that the reason 99% of students decided to go to graduate school was a desire for knowledge. The results found for this scale suggest that what is considered as knowledge, and how it is attained, varies widely within this Oxford group. Previous studies have shown that research students vary in their perceptions of their research environment (Ramsden, Conrad, Ginns and Prosser, 2003). This Approach to Research scale is used to address the question: Do students whose response is towards the right of Figure 2 have a different perception of their research environment than those more to the left? If they do, does this relation suggest a path to changing students’ perceptions? 3.2 Perceptions of research environment scales Using the information in the literature (Section 2.2.2) and especially the QAA Code of Practice (Quality Assurance Agency for Higher Education, 2004) and the Postgraduate Research Experience Questionnaire (PREQ) (GCCA & ACER, 2002), the questionnaire used in this study made use of 31 items, making up five scales of perceptions of the research environment. The perceptions scales are in the areas of: supervisory support, departmental intellectual environment, departmental infrastructure support, general infrastructure support and awareness of the assessment. A new scale on college support was also added based on pre-study interview data. Each scale is briefly described below. The items making up each scale, and the extent of the internal consistency of these items (scale reliability, alpha) is given in Annexe II, Table C. Supervision: The Supervision Scale contains 5 items that address issues such as availability of the supervisor, feedback, literature and topic area guidance and the students’ perceptions of the efforts made by supervisors to meet their needs. It has been designed to accommodate various forms of supervision, including sole and group supervision. Departmental Intellectual Climate: The ten items in this scale address the extent to which students experience support in areas such as opportunities in their department for social and academic engagement, working environments and a feeling of belonging to a departmental academic community. Departmental Infrastructure Support: This scale contains six items on students’ perceptions of their access to necessary equipment, technical support, working space, finance and the effectiveness of departmental administration. General Infrastructure Support: Six different broad topic areas are drawn together under this heading, such as library facilities, university administration, admissions and enrolments and complaints handling procedures. The extent of commonality between the items is low, but in this report the six items are combined to form a scale. Awareness of Assessment: This scale contains four items on the students’ perceptions of the clarity of the assessment processes in the programme as well as the perceived support available. College Support: Perceptions of the supportiveness of the students’ allocated college is a scale composed of 8 items on areas such as support from College Advisor, from college officers, availability of college computing facilities, and the opportunities for contact with other students and academic tutors.


The research experience of postgraduate research students at the University of Oxford 3.3 Relations between perceptions of research environment and approaches to research Table 1 contains correlation co-efficients of relations between Approach to Research scale scores and perception of research environment scores. It shows that the students who describe a more holistic approach to research (higher scores on Approach to Research scale) perceive a more supportive departmental intellectual climate, a more supportive general infrastructure, a supportive supervisory arrangement and a supportive college environment (correlation co-efficients (whole sample) of 0.27, 0.26, 0.30 and 0.26 respectively). A similar structure is found in both two-culture analyses (Table 1, Arts and Sciences). Table 1: Analysis of relations (correlations) between Approach to Research and perceptions of the research environment (for scale information, see Annexe II Table C).

Approach to Research Perceptions of environment Whole sample Arts Sciences

Departmental Intellectual Climate Scale .27 .30 .37

Departmental Infrastructure Scale .09 .12 .30

General Infrastructure Scale .26 .24 .32

Supervision Scale .30 .30 .37

College Support Scale .26 .21 .24

Awareness of Assessment Scale .09 .10 .16 N = 618-626 (whole) 315 (Arts) 298 (Sciences); numbers in bold (red) are statistically significant (p<.05) The Approaches to Research scale also shows strong relations with indicators of outcomes of research. Table 2 shows the correlations between students’ self-rating of the quality of their research (item 70), their development of a range of research skills, and with their descriptions of their satisfaction with three areas of the research context (Items 37, 38 and 39). The Skills Development scale (the mean of a student’s score on seven generic skills acquisition items) is described in more detail in Section 9 and Annexe II, Table C. Table 2: Analysis of relations (correlation co-efficients) between approach to research and outcomes of the research approach Outcomes of research Approach to Research Whole Arts Sciences

Self-rating (item 70) .46 .46 .46

Skills Development Scale .44 .44 .54

Satisfaction with services .15 .21 .28

Satisfaction with supervision .24 .22 .33

Satisfaction with research generally .41 .43 .50

N = 626 (whole) 315 (Arts) 298 (Sciences); numbers in bold are statistically significant (p<.05)


The research experience of postgraduate research students at the University of Oxford When students experience their research more holistically, they have a higher self-rating of the quality of their research, they feel that they are making more progress in their development of research skills, and they are more satisfied with the quality of their supervision and with their research experience overall. The results presented in this section have also been used to test the validity of the perceptions of the research environment scales and the outcome indicators. Students’ perceptions of more supportive environments are expected to correlate positively with more holistic approaches to research, and to higher quality outcomes and satisfaction. The cluster analyses (Annexe VI, Tables A-C) conducted using these variables indicate that these relations are found for the whole sample and the two-cultures (Arts and Sciences) populations. In summary, the results presented here suggest:

(a) that discussion with research students about their conceptions of, and approaches to research may lead to students adopting the more holistic approaches to research which are associated with higher quality outcomes;

(b) that changing students’ perceptions of their research environment, particularly in relation to departmental intellectual climate, general infrastructure support, supervision arrangements and college environment may lead to more holistic approaches to research.


The research experience of postgraduate research students at the University of Oxford 4. PGR students’ perceptions of their research environment 4.1 Research environment and skills scales scores The mean response scores from students (Table 3) on the research environment scales (described in Section 3.2) and the Skills Development scale (Annexe II, Table C) show students in highest agreement that they have acquired a range of generic research skills and lowest agreement that they have a supportive departmental intellectual climate and college support, though for all scales, the mean response is well above the scale midpoint (3.0). The highest mean score is in the non-perceptions area, Approach to Research. Table 3: Scale means and variation in agreement for research environment scales (for scale information, see Annexe II Table C). Scale Mean SD % Disagree* % Neutral* % Agree* Dept. Intellect. Climate 3.38 .69 24.0 6.5 69.5 College Support 3.52 .71 19.5 5.9 74.6 Dept. Infrastructure 3.54 .60 16.1 4.3 79.6 General Infrastructure 3.57 .51 9.1 8.0 82.9 Aware of Assessment 3.67 .79 14.7 9.0 76.3 Supervision 3.69 .79 15.7 5.3 79.0 Skills Development 3.80 .56 6.9 2.7 90.4 Approach to Research 3.84 .53 4.0 7.5 88.5

*% Agree: Scale scores >3.0; % Neutral: Scale score = 3.0; % Disagree: Scale scores <3.0 Two scales, Supervision and Awareness of Assessment, show higher standard deviations (.79) than other scales, suggesting greater spread (higher proportions of strongly agreeing and strongly disagreeing students). A breakdown by division, of scores on the three key perceptions of research environment scales, and for the whole university, are shown in Figure 3 (Annexe II contains more scale information). The responses are on a 1-5 scale from (1) strongly disagree to (5) strongly agree.

33.13.23.33.43.53.63.73.83.9

HUM LES MPS MED SS Univ

General InfrastructDept. InfrastructDept. Intell. Climate

Figure 3: Variation by division in perceptions of research environment indicators Figure 3 shows that General Infrastructure support is perceived to be similar across all divisions as would be expected given that these items are mostly about central services such as enrolments, libraries and administration. Students’ comments in this area include:



Facilities in the Bodleian need modernising – the absence of any common room facilities is a big problem; the computers are too old/slow/unreliable. I also note the paradox of an increasing reliance on online services (journals etc) that is accompanied by very high printing and copying costs. There is little point in investing in applications i.e. journals etc if the infrastructure (especially computer maintenance) is so unreliable. It is not acceptable to shift increasing costs onto users of a library when the facilities remain so poor. Much the same might be said of the book stack request system – how many research hours are lost as a result of broken conveyer belts, sending up the wrong books, short-staffing? The Bodleian remains an excellent collection of books, but lacks the relevant infrastructure to be a world-class library. My experience in the British Library, Library of Congress, University of California libraries, only highlight the deficiencies here. (Humanities, Fourth Year) My research experience at Oxford has been a very expensive undertaking. I am wholly unaware of where my fees go and what they actually pay for in the process. With the exception of the outstanding library collections, I feel that I have not received educational support proportionate to these fees. Rather, I feel that the university administration has been an incredible hindrance to my academic pursuits. (Social Sciences, Third Year)

Figure 3 also shows that Infrastructure support in the departments and a supportive departmental intellectual climate vary noticeably (and statistically significantly) across the divisions, being highest in Mathematical and Physical Sciences and Medical Sciences in the case of both scales. Two comments from students on these scales are included below.

My particular department provides a fantastic environment for research. Supervision is excellent without being intrusive. Lab and IT support is equally exemplary. I have had the opportunity and been encouraged to present my research at numerous national and international meetings. I cannot recommend my department highly enough. (Medical Sciences, Third Year) Our supervisors are not at the department; our lectures/seminars/teaching are not in the department; our library is not in the department – only the faculty admin, whom we only very rarely need to contact, and printing/photocopying facilities are actually there. Ideally, graduate student offices would be of immense value they would a) provide workspace, which is currently exceedingly limited. b) encourage interaction (academic as well as social) between post grads. c) improve the use of the department facilities. (Humanities, Third Year)

Students’ scores on their perceptions of supervisory support (from the Supervision scale), Skills Development and Awareness of Assessment, by division are shown in Figure 4 (and Annexe II). The responses are on a 1-5 scale from (1) strongly disagree to (5) strongly agree. Responses to the Supervision scale show high levels of agreement with a range of aspects of supervision, with the perceptions of Humanities students being highest in describing a supportive supervisory environment.



You will note that a great deal of my positive feedback regards my supervisor and in some respects, my college. These are great variables for many students and I consider myself extremely lucky. This also shows the extent to which a supervisor can make or break a student’s graduate experience. The extent to which I am informed of procedures has also been aided by my supervisor. (Life and Environmental Sciences, Fifth Year)

3

3.2

3.4

3.6

3.8

4


Skills developmentSupervisionAssessment

Figure 4: Variation by division in skills development, supervisory support and awareness

of assessment requirements

I saw my supervisor on average once a term for 3 years. In my 4th and last year, I saw him perhaps 24 times. I think research is affected in a fundamental way by one’s relationship with one’s supervisor, and perhaps more accurately, one’s expectations about how that relationship will unfold. I think it is vital to ensure that there is some framework within which student/supervisor relationships should operate. I think the absence of that framework took the pressure off in relation to the amount of research that is undertaken. In the first few years I did not chase my supervisor and he did not chase me. I therefore didn’t do nearly the volume of work in the first few years that I did in my last year. There would not have been this squeeze at the end if I knew that I could expect to have work in every month or two weeks (for example) from the very first term. I wish there had been some framework, which would have guided me in relation to what to expect in this respect. Such a framework, I have no doubt, would have facilitated my research no end. It is so important that students are not left to hang, and I feel that the lack of this framework is conducive to less (and less good) research. This might seem more paternalistic but my own experience leads me to believe that it would improve the rate and quality of research output. (Social Sciences, Fourth Year)

While students report that the quality of supervision is high in most cases, these two quotes illustrate the importance of supervision to students. Atkinson et al. (2004) found that 57% of Oxford doctoral students apply to come to Oxford because of the reputation of the course and/or the supervision. The results shown here indicate that these expectations are, in most cases, being met, though the 10% who do not find their supervision supportive suggests there are around 400 students University-wide who consider that they are not getting satisfactory supervisory support. Skills development and awareness of the assessment have high levels of agreement and do not differ significantly across the divisions. For some students, some requirements are not clear.



As a PRS student exempted from the Masters requirements, I’ve found the transfer of status process extremely confusing. Of the 3 students I know in the same position, we have all received different and been given different information. (Humanities, First Year)

In summary, these scale results show:

(a) most students describe their supervisory arrangements as most supportive of their studies, their departmental intellectual climate as least supportive, with departmental infrastructure, college and general infrastructure being similarly and moderately supportive;

(b) perceived general (university-wide) infrastructure support does not vary by division, but there are significant differences between divisions in departmental intellectual climate, departmental infrastructure, and students’ awareness of the assessment requirements; and

(c) students’ overall satisfaction varies moderately by division and is highest in the Physical and Medical Sciences (Annexe II, Table B).

4.2 Response to individual items The responses to the individual items used in the survey are shown in Annexe I, Table A. They show several noteworthy features.

• Almost 50% of students say the department is their usual working space, with another 40% saying it is either home or library or a combination, including department. Only 3% say it is their college alone, but another 6.4% say it is college in combination with another area (Annexe I, Table C). However, this is highly variant by discipline, as the columns on the right of Table C show. For the Sciences group 87% work in the department, with over 95% working in a combination of the department, home and library. For the Arts group, 36% normally work at home, and another 28% in library and home. The majority of students who work in college are from the Arts.

• Over two-thirds of students (68%) experience sole supervision (78% in the Arts

and 58% in the Sciences), with 27% having co-supervision, and 5% with group supervision (a total of 30 students with group supervision and all but 2 being in the Sciences).

• 43% meet with their supervisor less than four times a term, and this is true for

60% of respondents in the Arts. 55% of the Sciences group meets with their supervisor(s) 8 or more times a term.

• For the whole sample, 28% spend 30 or less hours a week on their research,

43% spend between 31 and 45 hours, and 29% spend more than 45 hours a week on their research. For the Arts, the percentages are 43, 39 and 18 respectively, while for the Sciences, they are 9, 48 and 40 respectively.

In terms of closed response item means (for questionnaire items 1-70) Annexe I, Table A shows positive responses to all but three items (when reverse-direction items are taken into account). Highly positive responses (means greater than 4.0) where high proportions of students strongly agree or agree, are found for 12 items:

1. The library facilities at Oxford support my research 13. Research supervision is available when I need it 14. My research has sharpened my analytical skills 30. As a result of my research, I have developed the ability to learn independently



38. Overall, I am satisfied with the quality of my research supervision 40. My college provides opportunities for social contact with other postgraduate students 49. I see my research as contributing in some way to ‘big picture’ issues 53. When I’m working on a research topic, I try to see in my own mind how all the ideas fit

together 54. Ideas that arise in my research often set me off on chains of thought of my own 58. It is important to me that my research is well integrated with existing knowledge and

topics in the field 64. I see university teaching as an important skill to acquire 69. I consider that developing a wider range of skills (such as communication, research

management, personal effectiveness, networking and team-working, career management) is a valuable part of my research programme (scale 1-6)

When the highly positive response items are analysed for each of the two-cultures, in the Arts all these items, except item 69, have means greater than 4.0. In the Sciences there are 10 items with means greater than 4.0, but only 7 of them (13, 14, 30, 40, 53, 64, 69) appear in the list above. The other highly rated items by students in the Sciences are:

2. I have access to a suitable working space 10. I have access to a common room or a similar type of meeting place 19. I have good access to computing facilities and services in my department

Science students are less satisfied with libraries and supervision, less likely to set off on their own chains of research thought and to see integrated knowledge as important. The overall impression to be gained from the responses to these items is very positive, with students agreeing in very high proportions that the library facilities support research, that supervision is available and of high quality, that a range of important skills are being developed and that colleges provide an opportunity for social contact with other postgraduate students. The items with the lowest means and higher proportions of disagreeing students are:

28. There is appropriate financial support for research activities in the department (2.99) 36. Complaints handling procedures are clear to me (2.90) 45. In my college I feel I am a part of a community of scholars in my subject area (2.62)

In the Arts the lowest means items include the three items above as well as:

8. I feel integrated into the research community in the department/faculty 11. As a result of my research, I have developed the ability to work collaboratively with

other researchers The Sciences share only items 36 and 45 with the Arts. Collaborative work within the Arts is not something most students agree with, and they share with scientists a lack of a feeling of there being a subject area community in their college. Both cultures experience lack of clarity in complaints handling procedures (Item 36). For a total of 26 items there are significant differences (p<.01) between the means for Arts and Science students. These items and means are shown in Table 4 (p. 31). In summary these results show: (a) that in four areas of the research context (students’ workspace, type of

supervision, frequency of meetings with supervisor, and time spent on research) there is wide variation within the whole population, and within the Arts and the Sciences. The implications of this variation are illustrated in Tables 5-8 in Section 5;


The research experience of postgraduate research students at the University of Oxford (b) that students agree in high proportions that the library facilities support research,

that supervision is available and of high quality, that a range of important skills are being developed and that colleges provide an opportunity for social contact with other postgraduate students;

(c) that colleges do not provide subject area communities, students are not clear about how complaints procedures are handled, and appropriate financial support is not available in the Arts departments/faculties;

(d) Arts students are more likely than Sciences students to agree that libraries support their research, that their research has sharpened their analytical skills, that they feel isolated in their department, that supervision provides helpful feedback on progress, that they feel integrated into their college community, that their research is affected by other duties, and that university teaching is an important skill to acquire;

(e) Arts students are less likely than Sciences students to agree that they have access to suitable working space, that they feel integrated into their department, that they can manage their career progression, that they have the ability to work collaboratively, that they have the needed technical support, that they have access to computing facilities, that their departmental research ambience stimulates their work, that their department provides appropriate financial support and a supportive work environment, that they are satisfied with the quality of the services and facilities, that they do not have sufficient control over their research, and that developing wider skills is valuable.

4.3 Student satisfaction with overall aspects of their research environment Student satisfaction with (a) services and facilities, (b) supervision, and (c) the overall research experience is shown by division and for the University-wide sample in Figure 5. The responses are on a 1-5 scale from (1) strongly disagree to (5) strongly agree (Items 37-39, Annexe I, Table A). University-wide means are 3.73, 4.01 and 3.85 respectively.

3

3.2

3.4

3.6

3.8

4

4.2


ServicesSupervisionOverall

Figure 5: Variation by division of three satisfaction items (37, 38, 39) Satisfaction with the quality of supervision is high and above the mean for overall satisfaction with the quality of the research experience in all but the Medical Sciences Division. However, 11.2% of students disagree that they are satisfied with the quality of their research supervision. This amounts to 70 students in this sample and approximately 450 students university-wide. The proportions of disagreeing students are similar across the divisions. Differences between the divisions in satisfaction with services and facilities are significant, being highest in MPS and Medical Sciences. Differences between divisions in overall satisfaction are small and not statistically significant but the trends are similar


The research experience of postgraduate research students at the University of Oxford to those found for the perceptions of the research environment (Figure 3) and this overall satisfaction score correlates with completion times (Section 6) and departmental RAE ratings (Section 7). Table 4: Items and means showing significant differences between Arts and Science

Mean Item Arts Science

1. The library facilities at Oxford support my research 2. I have access to a suitable working space 8. I feel integrated into the research community in the

department/faculty 9. My postgraduate research studies have helped to develop my

awareness of what I need to manage my own career progression

11 As a result of my research, I have developed the ability to work collaboratively with other researchers

12. I have good access to the technical support I need 14. My research has sharpened my analytical skills 19. I have good access to computing facilities and services in my

department 20. I tend to feel isolated within this department/faculty 22. My research supervision provides me with helpful feedback on

my progress 25. The research ambience in the department stimulates my work 28. There is appropriate financial support for research activities in

the department 29. I feel that this department provides a supportive working

environment 37. Overall, I am satisfied with the quality of the services and

facilities 41. I feel integrated into my college community 42. I have good access to computing facilities and services in my

college 44. Interaction with other postgraduate students is actively

encouraged in my college 53. When I’m working on a research topic, I try to see in my own

mind how all the ideas fit together 54. Ideas that arise in my research often set me off on chains of

thought of my own 55. I concentrate most of my research effort on technical and/or

descriptive processes 56. In my research I concentrate on just those areas that are of

direct relevance to my topic 59. I feel as if I do not have sufficient control over the direction of

my research 60. The quality of my research work is affected by the amount of

time I need to spend on aspects of my life other than research 63. Teaching undergraduates at Oxford during my graduate

studies is one of my aims 64. I see university teaching as an important skill to acquire 68. Sufficient support is available for Transfer of Status 69. I consider that developing a wider range of skills (such as

communication, research management, personal effectiveness, networking and teamworking, career management) is a valuable part of my research programme

4.30 3.64

2.90

3.39

2.88 3.36 4.27

3.48 2.97

3.94 3.00

2.56

3.29

3.59 3.61

3.74

3.97

4.30

4.19

2.58

2.64

2.30

3.64

3.65 4.31 3.29

3.87

3.92 4.34

3.54

3.60

3.77 3.82 4.09

4.07 2.38

3.54 3.39

3.49

3.70

3.92 3.29

3.46

3.76

4.12

3.98

3.28

2.98

2.57

3.19

3.41 4.02 3.59

4.16


The research experience of postgraduate research students at the University of Oxford 5. Relations between the research variables in the study While the results shown in Section 4.0 for the scales and the individual items show that there is variation in how students experience these aspects of their environment, the areas that might best be seen as the focus for change leading to improvements are those where this variation is related to higher quality research approaches and outcomes. This section contains a description of the relations between variables used in the study and the significance of these relations. Table 5 shows the relations between the six perceptions of the research environment indicators, and the five indicators of outcomes of research used in this study. The magnitude and direction of the same 30 relations are shown for each of the Arts and Sciences samples in Table 5a. Table 5: Relations (correlation co-efficients) between students’ perceptions of their research environment and the five research outcome variables Outcomes of

research Satisfaction with:

Perceptions of environment scales

Self rating Skills Services Supervision Overall Exp

Dept. Intellectual Climate .12 .49 .59 .36 .58

Departmental Infrastructure .06 .35 .68 .27 .46

General Infrastructure .17 .37 .51 .30 .45

Supervision .26 .35 .32 .83 .53

College Support .13 .19 .13 .09 .17

Awareness of Assessment .13 .24 .34 .30 .31 N = 615-626 (numbers in bold (red) are statistically significant) (p<.05) Correlation co-efficients range from –1.0 to +1.0. In studies of this type, numbers between –0.2 and +0.2 are zero to low correlations, and numbers above +0.5 or below –0.5 are high correlations. Table 5a: Relations (correlation co-efficients) between students’ perceptions of their research environment and the five research outcome variables for each of the two-cultures Outcomes of


Perceptions of environment Scales

Self rating Skills Services Supervision Overall Exp

Arts Sci Arts Sci Arts Sci Arts Sci Arts Sci

Dept. Intellectual Climate .09, .22 .46, .55 .59, .46 .37, .42 .60, .60

Departmental Infrastructure .02, .22 .32, .39 .70, .61 .29, .38 .48, .46

General Infrastructure .08, .29 .35, .45 .64, .36 .29, .31 .51, .39

Supervision .16, .37 .35, .42 .37, .35 .83, .83 .55, .55

College Support .05, .18 .21, .19 .18, .13 .06, .12 .19, .17

Awareness of Assessment .01, .27 .18, .33 .37, .27 .30, .33 .31, .31 N = 315 (Arts) 298 (Sciences); (numbers in bold (red) are statistically significant) (p<.05)


The research experience of postgraduate research students at the University of Oxford For both tables all scales except the college support scale show significant positive correlations with outcome indicators, suggesting that there is some validity in using these scales in a research experience inventory (see Section 8). Perceptions of departmental intellectual climate, departmental infrastructure, general infrastructure and supervision are all strongly related to satisfaction and skills development items and scales. Students’ awareness of assessment requirements are moderately related to these outcomes. The notable difference between Arts and Sciences (Table 5a) is that Science students’ rating of their own research performance is much more strongly related to their perceptions of their research environment, than is the case for Arts students. In all other areas, the Arts-Sciences differences are very small, though Science students have lower correlations between perceptions of general infrastructure support and their overall satisfaction and satisfaction with services and facilities. These results show that (a) doctoral candidates’ perceptions of their research environments are related to several measures of research outcome (for example, when students experience a supportive departmental intellectual climate, they are more likely to describe satisfaction with the services and facilities, with supervision, and with their overall research experience), and (b) the correlations found for the whole sample are also found within each of the two-cultures (even though the frequency of meetings between supervisors and students differs considerably between the Arts and Sciences, in both areas students who experience more supportive supervision are more likely to say they develop graduate transferable skills and express higher satisfaction with their research experience). The relations between aspects of the established research context and students’ perceptions of the research environment are shown in Table 6. The magnitude of the same 24 relations are shown for each of the Arts and Sciences samples in Table 6a. Table 6: Relations between aspects of the established research context and students’ perceptions of the research environment (effect size of relation) Established research context Perceptions of environment scales

Number of meetings with

supervisor

Type of Supervision (solo–group)

College (Grad or comb)

Type of residence

(in or out coll)Dept. Intellectual Climate large none small small

Departmental Infrastructure large large none none

General Infrastructure none none none small

Supervision large small none none

College Support none none none large

Awareness of Assessment small none none none N=618-626. Small effect size 0.2-0.35, medium 0.4-0.55, large >0.55 Effect size is used to describe the magnitude of relations in cases where correlation co-efficients cannot be used. When students have more frequent meetings with their supervisor in the Arts or Sciences, they are more likely to describe a supportive departmental intellectual climate, a supportive departmental infrastructure, and supportive supervision.


The research experience of postgraduate research students at the University of Oxford The type of supervision experienced by students is not related to variation in any of the research environment scales, except in the Sciences where students in sole supervisory contexts describe a more supportive supervision context, and students in group supervisory contexts describe a more supportive departmental infrastructure. Table 6a: Relations between aspects of the established research context and students perceptions of the research environment for Arts and Sciences (effect size of relation) Established research context Perceptions of environment scales

Number of meetings with supervisor

Type of Supervision (solo–group)

College (Grad or comb)

Type of residence

(in or out coll) Arts Sci Arts Sci Arts Sci Arts Sci

Dept. Intell. Climate medium medium none None none none none none

Departmental Infrastructure

small small none large none none none none

General Infrastructure none none none None none none none none

Supervision large large none large none none small none

College Support none none none None none none large large

Awareness of Assessment

none none none None none none none none

N = 315 (Arts) 298 (Sciences). Small effect size 0.2-0.35, medium 0.4-0.55, large >0.55 Whether students are in graduate or combined colleges has no significant effect on any of the research environment scales, but students living out of college describe a less supportive college environment. None of the established research context variables are significantly related to variation in perceptions of general infrastructure support. These results show: (a) more frequent supervisor meetings are associated with students’ perceptions of a

more supportive research environment; (b) students’ college type is not related to their perceptions of their research

environment, but students feel more supported when they live in the college; and (c) students in the Sciences perceive a more supportive supervisory arrangement

when they have only one supervisor, but a more supportive departmental infrastructure when in group supervision.

The relations between student characteristics and their perceptions of the research environment are shown in Table 7. The magnitude of the same 18 relations are shown for each of the Arts and Sciences samples in Table 7a. There is no difference by gender in how the environment is perceived by the students and neither are perceptions of the general infrastructure, college or issues to do with assessment related to year of study or hours spent on research per week. This is consistent with the results from earlier research described in Section 2.1.1 There are differences, for the whole group (Table 7), where perceptions of departmental intellectual climate, departmental infrastructure, and supervision are found to be related to the students’ year of study. There is a downward trend by year of study for all three of these scales with students in the first year of doctoral study being


The research experience of postgraduate research students at the University of Oxford more likely to agree and those who have been studying for four or more years being less likely to agree that these environments are supportive. For example, the students in their first year of study are more likely to agree that their supervision is supportive than those in their fourth or more year of study (a mean of 3.84 compared to a mean of 3.42, which is a medium effect size). Table 7. Relations between student characteristics and their perceptions of the research environment Student characteristics Perceptions of environment scales

Gender Year of study

Hours per week on research

Dept. Intellectual Climate none large medium

Departmental Infrastructure none medium medium

General Infrastructure none none none

Supervision none medium none

College none none none

Assessment none none none

N = 606-626. Small effect size 0.2-0.35, medium 0.4-0.55, large >0.55. Table 7a. Relations between student characteristics and their perceptions of the research environment for Arts and Sciences Student characteristics

Gender Year of study


Perceptions of environment scales

Arts Sci Arts Sci Arts Sci

Dept. Intellectual Climate none None small medium medium medium

Departmental Infrastructure none None none none none small

General Infrastructure none None none none none none

Supervision none None none none none none

College none None none none none none

Assessment none None none none none none N = 315 (Arts) 298 (Sciences). Small effect size 0.2-0.35, medium 0.4-0.55, large >0.55. These effects are not as strong for each of the Arts and Sciences groups (Table 7a). Perceptions of departmental intellectual climate and departmental infrastructure are also related to hours per week students say they spend on research. Students who spend 30 or less hours a week agree less that they experience the supportive departmental intellectual climate than those who spend more than 30 hours (a mean of 3.13 compared with means above 3.44). Similarly, the students who spend 30 or less hours a week agree less that they experience a supportive departmental infrastructure than those who spend more than 30 hours (a mean of 3.36 compared to means of over 3.60). No relations are found with the other scales. There are no Arts/Science differences, with the only statistically significant relation being a medium effect size


The research experience of postgraduate research students at the University of Oxford relation between a supportive departmental intellectual climate and more hours per week spent on research for both groups. These results show: (a) that students’ perceptions of their research environment are not gender related; (b) that students in the later years of their doctoral studies experience a less

supportive departmental infrastructure, less supportive departmental intellectual climates and less supportive supervisory contexts; and

(c) that students who spend the most time on their research activities are more likely to describe a more supportive departmental infrastructure and intellectual climate, but not a more supportive supervisory context.

The relations between the established research context and outcomes are shown in Table 8. Table 8: Relations (effect size) between aspects of the established research context and the five research outcome variables Outcomes of


University context Self rating Skills Services Supervision Overall Exp

Division small none small none none

No. of meetings with supervisor none none small small small

Type of supervision none none small* none none

Allocated college none none none none none

Type of residence none none none none none N=618-626. Small effect size 0.2-0.35, medium 0.4-0.55, large >0.55. * variation due to divisional effect (Sciences students with more co- and group supervision respond that they are more satisfied) In the divisions, students in Social Sciences rate the quality of their research work slightly higher than students in the other divisions where students give roughly the same rating. The only other differences between divisions on the outcome indicators is a small effect-size difference in students’ satisfaction with the quality of their services and facilities, with students in the Mathematical and Physical Sciences/Medical Sciences being most satisfied. The number of times a term students meet with their supervisor(s) is not related to their self-rating or to the extent to which they feel they acquire research skills, except in Mathematical and Physical Sciences where less frequent meetings are associated with significantly lower self ratings. Frequency of meetings is related to all three aspects of satisfaction. Satisfaction with the quality of services is lower when students meet less frequently with their supervisor. This is the case in all divisions except Mathematical and Physical Sciences. Satisfaction with the quality of supervision, and overall satisfaction with the research experience is lower when students meet less frequently with their supervisor, for all divisions, but this effect is strongest in the Mathematical and Physical Sciences and Medical Sciences divisions. The type of supervision experienced by students is not related to their outcome indicators. A higher satisfaction with services and facilities is related to group supervision, but almost all this type of supervision is found in Mathematical and


The research experience of postgraduate research students at the University of Oxford Physical Sciences and Medical Sciences, where satisfaction is highest with services and facilities. This result is significant. It suggests that whether the supervision is single, co- or group, there is no difference in overall satisfaction with the research experience or satisfaction with the quality of supervision. Neither the students’ allocated college type (combined or graduate), nor whether they reside in or outside the college, are related to any of the outcome indicators. Comparing Tables 5 and 8 (even focusing on the amount of bold (red) type) it can be seen that attempts to improve outcomes are more likely to be successful through action taken on students’ perceptions of the research environment, such as departmental intellectual climate, than changes to the context, such as type of supervision (where there are no significant discipline specific correlations). Across all divisions students express uniformity in their rating of their own performance. They also express a high degree of satisfaction with their supervision and the overall quality of their research experience, though this varies more by department. These results show: (a) that overall satisfaction with the research experience does not vary significantly by

division, college or type of supervision, but it is higher with higher frequency of supervision meetings; and

(b) that students’ assessment of their acquisition of key skills is unrelated to their division, college and type and frequency of supervision meetings.

The relations between student characteristics and research outcomes are shown in Table 9. The magnitude of the same 15 relations are shown for each of the Arts and Sciences samples in Table 9a. Of all five outcomes and satisfaction items and scales, only self-rating shows variation with gender for the group as a whole. This is consistent with the results from earlier research described in Section 2.1.1. Male students rate themselves more highly than female students on the self-rating scale of their research performance (a mean of 3.48 compared to a mean of 3.33). The difference is significant at p<0.01 and the effect size is small. Separate analyses for Science and Arts show that male Arts students are more satisfied with their supervision. Table 9: Relations (effect size) between student characteristics and the five research outcome variables Outcomes of research Satisfaction with: Student characteristics Self rating Skills Services Supervision Overall

Exper. Gender small none none None none

Year of study none none none large medium

Hours per week on research none medium small small medium N = 604-626. Small effect size 0.2-0.35, medium 0.4-0.55, large >0.55 With year of study, satisfaction with research supervision and overall research experience show significant differences for the whole group. Students in their first year of doctoral study, compared to those who have been studying for more than four years, were more satisfied with the quality of their research supervision (a mean of 4.19 compared to a mean of 3.63). Students in their first year of doctoral study, compared to


The research experience of postgraduate research students at the University of Oxford those who have been studying for more than four years, are also more satisfied with the quality of their research experience (a mean of 3.99 compared to a mean of 3.59). Similar, but lesser associations are observed by year of study for each of the two-cultures groups. Table 9a: Relations (effect size) between student characteristics and the five research outcome variables in Arts and Sciences Outcomes of research Satisfaction with: Student character-istics

Self rating Skills Services Supervision Overall Experience

Arts Sci Arts Sci Arts Sci Arts Sci Arts Sci

Gender small small none none none none small none none none

Year of study none none none none none none medium small medium none


none medium none medium none none none medium medium medium

N = 315 (Arts) 298 (Sciences). Small effect size 0.2-0.35, medium 0.4-0.55, large >0.55 The amount of time students spend on research per week is related to the students’ skills development scores as well as to all three overall ratings of satisfaction. The students who spend 30 or less hours a week on their research are significantly less in agreement about their skills attainment, and less satisfied with the quality of services and facilities, the quality of supervision and the quality of their overall research experience. Taking each of those items and scales in turn the students who spend 30 or less hours a week score less high on agreement of skills development than those who spend either between 31-45 hours or more than 45 hours (a mean of 3.65 compared to means of 3.84 and 3.90 for the other two groups). The differences are significant at p=0.000. They are less satisfied with the quality of the services and facilities than those who spend either between 31-45 hours or more than 45 hours (a mean of 3.54 compared to means of 3.81 for the other two groups). They are also less satisfied with the quality of their research supervision than those who spend 31-45 hours (a mean of 3.90 compared to a mean of 4.13). The difference is significant at p= 0.028. Finally, they are less satisfied with the quality of their research experience than those who spend either 31-45 hours or more than 45 hours (a mean of 3.56 compared to means of 3.97 and 3.98). The differences are significant at p=0.000. Time spent on research is a more important factor in the Sciences than in the Arts, and explains most of the relations found in the group as a whole. These results show that: (a) Science students’ satisfaction with their overall research experience and with

supervision is related to the amount of time they have spent on their research. Similar relations are found in the Arts for overall satisfaction only.

(b) Students’ satisfaction with research experience and skills acquisition are not significantly related to gender or to year of study in either the Sciences or the Arts, though there is a small relation between gender and supervision in the Arts, where male students are more satisfied with their supervision.


The research experience of postgraduate research students at the University of Oxford 6. Perceived research environment and doctoral completion times

The time taken for doctoral students to complete their studies is an outcome indicator related to the students’ perceived quality of the research environment. None of the students in this study had completed their degrees at the time of writing this report, however, completion times of the cohorts who began in Michaelmas term 1997 through Trinity term 2001 (a total of 3203 students) have been calculated (including lapsed students and those whose work did not satisfy the examiners) as a guide to current divisional differences. Figure 6 below shows how average completion time (years = terms/3) by division is inversely related to three sets of research environment scale scores for the same division. A similar pattern is found with students’ overall satisfaction with the quality of their research experience (item 39).

2.7

3.2

3.7

4.2

4.7


Supervision (score)Dept Infrastr. (score)Dept Climate (score)Completion (years)

Figure 6: Relations between research environment scale scores and completion times by

division

Wright and Cochrane (2000) note that only the students’ discipline (not age; mode of study (f/t or p/t); origin (home/EU/overseas student); sex; funding; class of first degree; or university of first degree) was found to be a reliable predictor of completion time, in their study of 3579 theses from the University of Birmingham. As with the University of Oxford study, and with that of Bowen & Rudenstine (1992), they found that the success rate was better in the Sciences. The main disciplinary differences attributed to this variation appear in the areas of supervision and the extent to which students work in groups or teams. The message in Figure 6 is that where students perceive a more supportive departmental research environment such as in the Sciences disciplines (in Mathematical and Physical Sciences and Medical Sciences), they complete their studies in a shorter time, whereas completion times are longer in divisions where the environment (mainly departmental infrastructure and intellectual climate) are perceived as being less supportive.


The research experience of postgraduate research students at the University of Oxford 7. Students’ perceptions of their research environment and the RAE The 2001 Research Assessment Exercise (RAE) suggested that there was variation between Oxford departments in research. This section contains an analysis of the question: ‘do students in departments with high RAE ratings experience a different research environment (as measured by the indicators used in this study) to students in departments with lower RAE ratings?’ The mean scores on the three overall satisfaction items (37-39) and four perceptions of research environment scales (Departmental Infrastructure and Climate, General Infrastructure, and College) for students in 5* 2001 RAE-rated departments (320) were compared with the mean score of students in departments rated 4 and 5 by the RAE (187). The items/scales where there are differences between the two RAE-rated contexts are shown in Table 11. All the differences are statistically significant (at p<.01) except students’ overall satisfaction with the quality of their supervision. Table 11: Average score (and statistical significance, p) of students’ response to satisfaction items (1-5 scale) and Departmental Infrastructure and Intellectual Climate scale scores in departments with different 2001 RAE ratings.

Department RAE Item / scale 4 or 5 5*

p

Overall, I am satisfied with the quality of my research supervision Overall, I am satisfied with the quality of my research experience Departmental infrastructure (scale score) Departmental intellectual climate (scale score)

3.92 3.74 3.38 3.23

4.09 3.96 3.60 3.47

0.071 0.006 0.000 0.000

Table 11 shows that students in 5* RAE departments have higher average overall satisfaction scores than students from departments rated 4 and 5 by the RAE. The difference is statistically significant only in the case of students’ overall satisfaction with their research experience. The same analysis conducted with Departmental Infrastructure and Departmental Intellectual Climate scale scores (bottom two rows of Table 11) shows statistically significant differences between students’ responses in departments with different RAE ratings. Students in 5* RAE departments have higher average overall scale scores for departmental infrastructure support and for a supportive departmental intellectual climate than students from departments rated 4 and 5 by the RAE. The significance of these results is in confirming the capacity of these scales and items to measure variation in research contexts.


The research experience of postgraduate research students at the University of Oxford 8. A research experience questionnaire for Oxford A proposed Oxford Postgraduate Research Experience Questionnaire (OPREQ) is presented in Annexe V. It contains 42 items, with 6 scales. It is suggested that the questionnaire be sent electronically, every two years, to students in the final year of their research programme, with the results being distributed through a website similar to the OSCEQ (http://ceq.oucs.ox.ac.uk/). The research supporting the selection of these items and scales is presented in an earlier section (5) of this report. All the research environment perceptions scales (Supervision, Departmental Intellectual Climate, Departmental Infrastructure, Assessment and General Infrastructure) tested in this study, with the exception of General Infrastructure, have shown significant variation by division, and that variation is systematically related to the five research outcome indicators, to the Approaches to Research scale, and as shown in Sections 6 and 7, to programme completion times and RAE ratings. These five perceptions scales have been included as part of the proposed OPREQ. The General Infrastructure items include perceptions of a variety of components that are the responsibility of the university, such as libraries and enrolment processes. They do not form a set to which individual students respond with any consistency (as reflected in a scale alpha reliability of only 0.59), but they are retained in the OPREQ, as individual items to provide information. The full set of items also includes one overall satisfaction item and a Transferable Skills Development scale with items linked to the Joint Statement of Skills Training Requirements (JSSTR) (Research Councils UK, 2001). The College Support scale tested in this study showed significant variation between colleges, but there is very little relation between students’ assigned college and the research process or outcome variables. The recommended OPREQ therefore contains only an overall satisfaction with college item. Finally, three items on aspects of the research environment that may effect time spent on research (awareness of teaching opportunities, finances and other responsibilities) are included to monitor these areas. This inventory is expected to show meaningful differences between divisions, as shown for three of the perceptions scales in Figure 7, but the best use of it will be in monitoring changes over time brought about through the introduction of planned improvements. The following sections summarise the research behind the proposed OPREQ items and scales. 8.1 Supervision, departmental infrastructure and departmental intellectual climate Scores on Supervision, Departmental Infrastructure and Departmental Intellectual Climate scales, by division and Continuing Education for the whole university, are shown in Figure 7. The items used to make up these scales are shown in Annexe II, Table C. Students’ agreement with the extent of their supervisory support is high and relatively uniform, with statistically significant differences only between the extremes of the range. Infrastructure support in the departments and a supportive departmental intellectual climate vary noticeably (and statistically significantly) across the divisions, being highest in Mathematical and Physical Sciences and Medical Sciences in the case of both scales.



3

3.2

3.4

3.6

3.8

4

HUM LES MPS MED SS CE Univ

SupervisionDept InfrastructDept Climate

Figure 7: Variation in Supervision, Departmental Infrastructure and Departmental

Intellectual Climate scales by division 8.2 College support The following eight items were used to construct a scale to gauge College Support.

College Support (response range 1-5) (alpha=0.81) 40. My college provides opportunities for social contact with other postgraduate students 41. I feel integrated into my college community 42. I have good access to computing facilities and services in my college 44. Interaction with other postgraduate students is actively encouraged in my college 45. In my college I feel I am a part of a community of scholars in my subject area 46. Support from college officers (e.g. welfare, accommodation) is available when I need it. 47. I have access to appropriate general academic support (e.g Tutor for Graduates) in my college. 48. I have appropriate support from my College Advisor Each student’s mean response to all eight items was used to compute a College Support scale score, and the average scores for the 27 colleges where a response from over 10 students was received are shown in Figure 8. The figure shows that perceived support from college varies considerably across the colleges with scores on the five-point scale ranging from under 3.0 to over 4.0, though the majority are around 3.2-3.6. This is a statistically significant variation, but as noted above, this variation does not translate into students’ overall satisfaction with their research experience: relations between allocated college and overall satisfaction show no significant difference. For this reason, and because most colleges already collect a range of information from graduate students, it is suggested that the OPREQ contain only an overall college satisfaction item, rather than a College Support scale.

2.62.8

33.23.43.63.8

44.2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

College support score (1-5)

Figure 8: Variation in College Support scale by college (for 27 colleges where n>10)


The research experience of postgraduate research students at the University of Oxford Information on students’ responses to each of the eight items in this scale is given in Annexe I, Table A. 8.3 Satisfaction items Three overall satisfaction items were used in the study to investigate satisfaction with the quality of services and facilities, supervision, and overall experience. In order to keep the inventory short, only the total research experience item is recommended for use in the OPREQ: Overall, I am satisfied with the quality of my research experience. The results for this item by division and Continuing Education are shown in Table 12. Table 12: Mean scores (and standard deviations) for overall satisfaction item (Item 39) by division and Continuing Education Division n mean sd HUM LES MPS MED SS CE

145 88

170 100 107 13

3.85 3.70 3.95 3.94 3.79 3.54

1.01 .89 .72 .98 .96 .97

The 42 items suggested for use in the OPREQ do not clash with questions currently asked in departmental questionnaires, nor do they address all aspects of the students’ research experience, but they do include all the areas found in this study to be related to the quality of the outcome of the students’ programme (as measured in this study). Interventions aimed at changing students’ perceptions of these areas are more likely to lead to enhanced research experience than a focus on areas that are not related to outcomes.


The research experience of postgraduate research students at the University of Oxford 9. PGR students’ experience of the Roberts skills agenda Only 3.5% of doctoral students at Oxford disagree that developing a wider range of skills (such as communication and team-working) is a valuable part of their research programme (Annexe I, Table A). This is one third of the number of students who are dissatisfied with the quality of their supervision. The students’ self-report of the extent to which they consider they have attained the range of skills is high. Table 13 shows the responses to the seven skills items used in this study. Table 13: Skills development items and student responses

Questionnaire item n m sd %SD %D %N %A %SA

4. I have developed an awareness of the wider research community in my field

626 3.85 .843 1.1 6.5 17.4 55.6 19.3

7. My postgraduate research studies have helped me to develop a range of communication skills

623 3.65 .892 1.4 9.0 27.4 47.0 15.1

9. My postgraduate research studies have helped to develop my awareness of what I need to manage my own career progression

621 3.47 .956 2.9 13.5 28.3 44.3 11.0

11. As a result of my research, I have developed the ability to work collaboratively with other researchers

624 3.29 1.047 4.6 19.6 29.5 35.3 11.1

14. My research has sharpened my analytical skills

626 4.19 .699 0.2 1.4 11.5 53.2 33.7

17. As a result of my postgraduate research studies, I feel confident about managing a research project

617 3.67 .915 1.8 9.1 25.4 47.3 16.4

30. As a result of my research, I have developed the ability to learn independently

622 4.10 .808 0.8 3.5 12.7 51.0 32.0

m=mean, sd=standard deviation, SD=Strongly disagree, D=disagree, N=neutral, A=agree, SA=strongly agree These seven items are used to explore students’ responses to a part of each of the Joint Statement of Skills Training Requirements (Research Councils UK, 2001) skills areas as in Table 14. Table 14: Questionnaire item numbers and Joint Statement of Skills Training Requirements (JSSTR) (Research Councils UK, 2001) skills areas

Item number

JSSTR skills area Code

4 7 9

11 14 17 30

Research Environment Communication Skills Career Management Networking and Team-working Research Skills and Techniques Research Management Personal Effectiveness

B E G F A C D

In two areas (Item 11: attainment of the ability to work collaboratively with other researchers, and Item 9: developing an awareness of what is needed to manage my own career progression) the percentage of those who disagree is around 20%. Working


The research experience of postgraduate research students at the University of Oxford collaboratively with other researchers is very different between the Sciences and the Arts, with only 9% of Science students disagreeing compared with 37% of Arts students. In the other five areas, the disagreeing percentage is lower and nearer to 10% or less. The percentages of students who agree that they have attained these skills ranges from around 50% to 80%. Students in the different years of their doctoral studies do not describe differently the extent to which they have attained these skills. This may be due in part to the recent introduction of programmes such as UKGrad. Of the other established research context and student characteristic variables (type of supervision, frequency of meetings, college, gender and time spent on research), only variation in the amount of time students say they spent on research is related to variation in skills attainment, with students who describe spending 30 or less hours per week on their research describing less agreement with skills attainment than those who spend more than 30 hours per week on research. How students perceive their research environment is also related to their experience of skills acquisition. As shown in Table 5 (Section 5.0) the Skills Development scale scores correlate moderately strongly with perceptions of the Departmental Intellectual Climate (r=0.49), Departmental Infrastructure (0.35), General Infrastructure (0.37) and Supervision (0.35).

I have been helped considerably by the Faculty’s Graduate Research Skills sessions, which have generally been appropriately designed and well run. My only concern is that these sessions would ideally be supplemented by similar conversations with my supervisor. At the moment I tend to assume that my supervisor does not really consider it to be his job to talk to me about study skills, methods and strategies etc, as opposed to talking about literature and ideas. I would find it enormously useful if supervisors were actively encouraged to broach more practical subjects of conversation with their students. (Humanities First Year)

Students also see the acquisition of a wider range of skills (as in the Joint Statement of Skills Training Requirements list (Research Councils UK, 2001) Table 14) as being a valuable part of their research (Item 69 mean = 4.01), with 4.8% saying it did not apply to them, but only 3.5% of the rest disagreeing that these skills constitute a valuable part of their research programme.

Re: Q69 [I consider that developing a wider range of skills is a valuable part of my research programme] These would be useful if they were offered to graduate students! There seems to be a complete lack of training in professional/ teaching/transferable skills in Oxford. I find it staggering, and yet the university seems so complacent about the quality of its programmes. I noted with disbelief that in your [IAUL] recent illuminating publication two or three tiny pilot schemes are reported as huge successes. As per usual, Oxford is years behind the times. (Life and Environmental Sciences Third Year)

The degree to which these skills are seen as important by students does not vary by students’ perceptions of departmental intellectual climate, but does vary significantly by their conceptions of what is research, and how they go about it (Approaches to research). When analysed using each individual skills area (seven items) rather than the scale, a similar result is found. An example is shown in Figure 9 for the relation between skills


The research experience of postgraduate research students at the University of Oxford items attainment scores and perceived supportiveness of the departmental intellectual climate.

2.72.93.13.33.53.73.94.14.3

skills

Low Medium High

Departmental Intellectual Climate

Net- & teamworkingCareer managementCommunicationResearch environmentResearch managementPersonal effectivenessResearch skills

Figure 9: Variation in scores on individual skills items by intellectual climate

Figure 9 shows that the students who perceive the lowest departmental intellectual climate support describe the lowest agreement with the achievement of all seven skills. The students who perceive the highest departmental intellectual climate support describe the highest agreement with the achievement of all seven skills. Figure 10 shows the relations between the way students conceive of and approach their research (Approach to Research), and the extent of their agreement with skills attainment items.

33.23.43.63.8

44.24.4

skills

Low Medium HighApproach to Research scale score

Net- & teamworking

Career management

Communication

Research environment

Research management

Personal effectiveness

Research skills

Figure 10: Variation in scores on individual skills items by approach to research score

A similar pattern to that shown in Figure 9 is found in Figure 10. The students who describe the least holistic approach to research describe the lowest agreement with the achievement of all seven skills, and the students who describe a more holistic research approach also describe the highest agreement with the achievement of all seven skills. The correlations described above suggest that the development of research skills may be assisted by also addressing and improving students’ perceptions of their research environment, such as supervision, departmental intellectual climate and departmental infrastructure, as well as raising awareness of variation in students’ conceptions of and approaches to research.


The research experience of postgraduate research students at the University of Oxford Another way to assist with the development of skills, as seen from the literature (Section 2.3.2), is to give PGR students an opportunity to reflect on their skill development. This could be carried out through the process of personal development planning, or as the Humanities student quoted above seems to be suggesting, through discussions with supervisor(s). This reflection could be complementary to the discussions about approaches to research already suggested in Section 3 and would enable students to see the research experience in a more holistic way. Students’ views of the opportunities they have to develop teaching skills, and the extent to which they are achieved, are addressed in the next section.


The research experience of postgraduate research students at the University of Oxford 10. The teaching experience of PGR students Anderson & Swazey (1998) found that 71% of students expressed a desire to teach in higher education as an important reason for deciding to go to graduate school. Over 80% of Oxford PGR students say they either possibly or definitely intend to pursue an academic career. In exploring the undergraduate learning experience at Oxford (Trigwell and Ashwin, 2003), observed that ‘Students who had over 50% of their tutorials with graduate students perceived that their teaching was good, but less good, when compared with the response from students who had no tutorials with graduate students. For this same group the goals of their courses were less clear, their workload and the way they were assessed were perceived to be less appropriate, they experienced less collegiality and perceived that they had less encouragement to develop their own academic interests. They also had lower scores on a Conceptions of Learning scale, were less motivated, and had lower levels of self-efficacy. Finally, they felt that they had less support in improving skills in both oral and written communication and they reported not feeling as confident about tackling unfamiliar problems.’ (p. 7) The commonality of the directions of these relations point to the ability of the undergraduate students to assess differences between teaching of less and more experienced teachers. This effect of the teaching by graduate students suggests that support should be given to the development of graduate students’ teaching skills, but other factors should also be considered. Harland & Plangger (2004) found that teaching helped research students in developing their personal identity as well as broadening their subject knowledge and providing an opportunity for them to practice the language of their disciplines. Whilst students experienced some downsides to teaching, such as feelings that they had less autonomy and less support compared to their research, these findings do suggest that opportunities to teach can be an important part of research students’ experience. In Oxford, the Commission of Inquiry (1997a) found that opportunities to teach appeared to be unequally distributed, with students who had first degrees from Oxford nearly twice as likely to be involved in teaching compared to students from other UK universities and from overseas. Some reasons for this difference include tutors’ greater knowledge and trust of students that they have previously taught and the students’ knowledge of teaching at Oxford. The results from most of the teaching oriented items in this survey (type of teaching conducted and items 61-64 and 69, Table 15 and Annexe I, Table B) show that the context described in the Commission of Inquiry (1997a) continues to be the position in Oxford. Research students see the acquisition of teaching skills as important, with 83% of respondents agreeing that university teaching is an important skill to acquire (item 64, mean = 4.16). (This includes the 88 students (14%) who do not see themselves pursuing an academic career.) However, only 55.3% of the respondents (346) agree that teaching undergraduates at Oxford during their graduate studies is one of their aims (item 63). A total of 287 indicate that they have taught at Oxford in the last year, and 196 of them are among the 346 who expressed a desire to teach. Of the 346 who aim to teach, 152 (44.6%) disagree that it is clear to them how to find out about opportunities to teach at Oxford. The 287 (45.8%) of respondents reporting that they had done any teaching in the last year describe tutorial teaching as the most common form of teaching (done by 67% of this group) (Annexe I, Table D). Laboratory teaching is conducted by 27.7% with the next most common teaching form being classes or seminars (20.7%).


The research experience of postgraduate research students at the University of Oxford Since only 88 students (14%) stated that they were not interested in pursuing an academic career, many students are currently not able to get the experience that may help them achieve their desired careers.

In fact, since ‘doctor’ of course means ‘teacher’, I consider it scandalous that so little emphasis is put on teaching as a central part of graduate work – it is much more important than research and as such needs to be recognised. (Humanities Fourth Year)

66.5% of all students agree that teaching is likely to benefit rather than hinder their research. This percentage rises to 87% once students with no ambition for an academic career are excluded. Table 15 explores the relations between aspects of teaching, three research perceptions scales, Approach to Research and overall satisfaction. Table 15: Relations (correlation co-efficients) between teaching items (61-64) and research environment scales and outcomes Item Super-

vision Intellect Climate

Depart. Infrastr.

Research approach

Overall Satisfact.

61. It is clear to me how to find out about opportunities for me to teach undergraduates at Oxford

.09

.11

.15

.06

.11

62. I consider that teaching under-graduate students in my field is likely to benefit rather than hinder my research

.04

.06

-.01

.13

.07

63. Teaching undergraduates at Oxford during my graduate studies is one of my aims

.03

.07

-.05

.10

.02

64. I see university teaching as an important skill to acquire

.07 .07 -.04 .13 .02

N=614-621; Bold (red) numbers are significant at p<.01 All the correlations in the table are small. There are few relations between teaching and research. For the whole sample, research students who see teaching as benefiting rather than hindering their research are more likely to report a more holistic approach to research (r = 0.13; p = 0.002) (i.e. they see their research ideas as being part of a bigger picture and something over which they have greater control). This view was not uniformly held:

I feel that teaching undergrads affected my research too. I took on teaching jobs to supplement my income but in hindsight realise that it did take more time than expected. (Social Sciences, Fourth Year)

How clear students are about how to find out about opportunities to teach does vary with perceptions of Departmental Intellectual Climate (r = 0.11; p = 0.006), but not by approach to research. Where students experience a supportive intellectual environment, they are more likely to say they are clear about how to find out about teaching opportunities. Conversely, considering that teaching is likely to benefit research does not vary with perceptions of departmental intellectual climate. In interviews conducted in preparation for this study, PGR students described their perceptions of the context in which they teach. Many of the areas described below mirror the results described above.

• Teaching is enjoyable and is seen to be valuable experience;



• There are tensions between research and teaching with more pressure coming from research;

• More opportunities for teaching development are desired; • There is a general lack of feedback on performance; • Being closer to students helps them to be more effective in their research; • There is an opaqueness in knowing how to get to teach; • The preparation time needed and pay offered is acceptable; and • More of a teaching social network is needed.

These results show that only about one half of the students who want to teach get an opportunity, and that nearly one half of those who want to teach do not think it is clear how to find out how to do so. 11. Other factors (funding, language …) Several other factors (funding, confusion caused by the Oxford system, and language problems) emerged from the analysis of the students’ comments and their response to related questionnaire items. Half of the students surveyed agreed that they were concerned that their financial situation might affect the quality of their research work (Item 51, Annexe I, Table A). Atkinson, et al. (2004) found that the UK graduate students arriving from universities other than Oxford or Cambridge are less frequently funded by Oxford-specific scholarships.

My financial situation in the first 2 years obliged me to work to pay my studies. This has taken significant time and effort that could have been applied directly to my research project. (Life and Environmental Sciences, Second Year) Funding – I am effectively doing a full-time degree with part-time temporary research contracts. This is the reality for many PG students who don’t get research council funding, but the university and dept. are totally unsupportive and I feel like I am doing something wrong despite having hit all progress targets on time so far. (Social Sciences, Second Year)

Nearly 60% say that the quality of their research is affected by other aspects of their life that compete with their research time (Item 60, Annexe, I, Table A). International students who were confused by the Oxford system, described their situation as follows:

The system in Oxford is very much different and somewhat complicated for people from outside and it would be always good if either departments or colleges could explain this unique system clearly at the very beginning of the graduate course (especially for the overseas students!) (Humanities, Third Year)

Items related to these areas are being retained on the proposed OPREQ (Section 8).


The research experience of postgraduate research students at the University of Oxford Annexe I: Frequencies and means of questionnaire item responses Table A: Research related items

N Humanities

%

Life and Environmental

Sciences

%

Maths and Physical Sciences

%

Medical Sciences

%

Social Sciences

%

Continuing Education

%

Division

626 23.3 14.2 27.2 16.0 17.3

2.1

N

1st %

2nd %

3rd %

4th %

More than 4 %

Year of Study

621 24.2 24.2 25.8 18.5

7.4

N Female %

Male %

Gender

626 46.0 54.0

N

Single %

Co-supervision %

Group %

Type of supervision

626 68.2 27.0 4.8

N

0-3 %

4-7 %

8 or more %

Meeting times per term with supervisor

624 43.3 27.4 29.2

N College Accommodation %

Non-college Accommodation

%

Current Residence

620

43.7 56.3

N

30 or less %

31-45 %

More than 45 %

Hours per week spent on research

611 28.2 43.0 28.8



Item N Mean S.D. StronglyDisagree

%

Disagree

%

Neutral

%

Agree

%

StronglyAgree

% 1. The library facilities at Oxford support my research 626 4.12 .903 1.4 5.4 10.7 45.0 37.4

2. I have access to a suitable working space 626 3.98 1.012 3.0 7.7 10.9 45.0 33.4

3. The department/faculty provides opportunities for social contact with other postgraduate students

626 3.41 1.061 5.0 15.2 27.6 38.2 14.1

4. I have developed an awareness of the wider research community in my field 626 3.85 .843 1.1 6.5 17.4 55.6 19.3

5. My supervisor(s) make(s) a real effort to understand difficulties I face 624 3.91 1.029 2.2 8.8 17.9 37.8 33.2

6. I am able to organise good access to necessary equipment 621 3.75 .798 0.8 4.5 28.8 50.2 15.6

7. My postgraduate research studies have helped me to develop a range of communication skills 623 3.65 .892 1.4 9.0 27.4 47.0 15.1

8. I feel integrated into the research community in the department/faculty 626 3.19 1.097 6.1 23.5 26.7 33.1 10.7

9. My postgraduate research studies have helped to develop my awareness of what I need to manage my own career progression

621 3.47 .956 2.9 13.5 28.3 44.3 11.0

10. I have access to a common room or a similar type of meeting place 626 4.00 .853 1.4 5.3 12.0 54.6 26.7

11. As a result of my research, I have developed the ability to work collaboratively with other researchers

624 3.29 1.047 4.6 19.6 29.5 35.3 11.1

12. I have good access to the technical support I need 622 3.57 .936 2.1 11.1 28.5 44.2 14.1

13. Research supervision is available when I need it 626 4.04 .948 1.6 7.2 11.7 44.4 35.1

14. My research has sharpened my analytical skills

626 4.19 .699 0.2 1.4 11.5 53.2 33.7

15. The department/faculty provides opportunities for me to become involved in the broader research culture

620 3.45 .947 2.3 15.5 27.9 44.2 10.2

16. I feel that other postgraduate students in my department/faculty are supportive 626 3.70 .860 1.1 7.3 27.5 48.2 15.8

17. As a result of my postgraduate research studies, I feel confident about managing a research project

617 3.67 .915 1.8 9.1 25.4 47.3 16.4

18. I am given good guidance in topic selection and refinement 625 3.54 1.001 3.0 13.6 24.5 44.0 14.9

19. I have good access to computing facilities and services in my department 623 3.77 1.007 3.9 7.1 20.5 45.4 23.1

20. I tend to feel isolated within this department/faculty 617 2.69 1.143 14.7 34.5 23.8 20.4 6.5

21. In my research I consider it is more important to find new ways of thinking than to develop research skills

619 3.12 .835 1.1 20.7 48.9 23.7 5.5

22. My research supervision provides me with helpful feedback on my progress 626 3.73 .998 2.2 11.8 17.7 46.8 21.4

23. Interaction with other postgraduate students is actively encouraged in my department 623 3.17 1.029 6.3 18.8 35.3 31.3 8.3




%

Disagree

%

Neutral

%

Agree

%

StronglyAgree

% 24. A good seminar programme for postgraduate students is provided 626 3.47 1.061 5.1 14.7 22.4 44.1 13.7

25. The research ambience in the department stimulates my work 623 3.18 .943 4.2 18.3 39.5 31.8 6.3

26. I have received good guidance in my literature search 624 3.23 1.001 4.3 19.1 34.9 32.5 9.1

27. I manage to find conditions for studying which allow me to get on with my work easily 623 3.62 .894 2.2 9.5 24.4 52.0 11.9

28. There is appropriate financial support for research activities in the department 622 2.99 1.221 15.0 19.9 25.9 29.3 10.0

29. I feel that this department provides a supportive working environment 622 3.47 .882 2.3 10.8 33.4 44.7 8.8

30. As a result of my research, I have developed the ability to learn independently 622 4.10 .808 0.8 3.5 12.7 51.0 32.0

31. I feel respected as a fellow researcher within my department/faculty 623 3.43 .929 3.0 11.7 34.2 41.1 10.0

32. The department administration is effective in supporting my research 624 3.41 .996 5.8 9.9 32.5 41.5 10.3

33. The university administration is effective in supporting my research 622 3.11 .900 5.3 13.7 51.1 24.1 5.8

34. I was satisfied with the admission and enrolment processes 625 3.65 .874 2.7 7.4 23.4 55.4 11.2

35. The department provides clear, comprehensive and up-to-date ‘Notes of Guidance’

623 3.32 .962 4.5 14.4 32.9 40.8 7.4

36. Complaints handling procedures are clear to me 625 2.90 .906 7.0 23.0 44.5 23.4 2.1

37. Overall, I am satisfied with the quality of the services and facilities 624 3.73 .893 1.6 10.6 15.4 57.9 14.6

38. Overall, I am satisfied with the quality of my research supervision 625 4.01 1.021 2.6 8.6 10.1 42.4 36.3

39. Overall, I am satisfied with the quality of my research experience 623 3.85 .909 1.6 8.2 15.4 52.8 22.0

40. My college provides opportunities for social contact with other postgraduate students 626 4.27 .776 0.6 2.4 9.3 45.2 42.5

41. I feel integrated into my college community 626 3.47 1.292 9.1 16.0 21.2 26.0 27.6

42. I have good access to computing facilities and services in my college 625 3.61 1.063 3.8 11.7 25.8 37.4 21.3

43. My college offers some financial support for my research activities 624 3.34 1.211 10.4 15.4 19.6 39.3 15.4

44. Interaction with other postgraduate students is actively encouraged in my college 625 3.87 1.006 2.1 7.7 22.4 37.0 30.9

45. In my college I feel I am a part of a community of scholars in my subject area 625 1.206 18.2 35.5 21.6 15.7 9.0

46. Support from college officers (e.g. welfare, accommodation) is available when I need it 622 3.66 .926 3.1 6.6 27.7 46.9 15.8

47. I have access to appropriate general academic support (e.g. Tutor for Graduates) in my college

625 3.50 .997 5.4 9.6 25.8 48.0 11.2

48. I have appropriate support from my College Advisor 624 3.16 1.236 14.3 12.7 30.6 28.2 14.3

2.62




%

Disagree

%

Neutral

%

Agree

%

StronglyAgree

% 49. I see my research as contributing in some way to ‘big picture’ issues 625 4.02 .841 1.6 4.0 12.6 54.1 27.7

50. I usually try to discuss with others new ideas I have in my research 625 3.81 .862 1.1 8.5 16.5 56.5 17.4

51. I am concerned that my financial situation might affect the quality of my research work 625 3.27 1.293 9.9 23.0 18.1 28.2 20.8

52. As I develop my own research-informed opinion I find myself challenging the opinion of others including that of my supervisor(s)

624 3.73 .858 1.1 7.9 23.4 51.9 15.7

53. When I’m working on a research topic, I try to see in my own mind how all the ideas fit together 624 4.21 .651 0.3 0.8 8.7 58.5 31.7

54. Ideas that arise in my research often set me off on chains of thought of my own 623 4.09 .765 0.5 2.7 14.0 52.8 30.0

55. I concentrate most of my research effort on technical and/or descriptive processes 622 2.92 .967 5.0 32.5 31.2 28.0 3.4

56. In my research I concentrate on just those areas that are of direct relevance to my topic 625 2.83 1.028 6.2 40.2 22.6 26.9 4.2

57. Often I find myself wondering whether the work I am doing here is really worthwhile 626 2.82 1.255 17.1 28.0 20.9 24.0 10.1

58. It is important to me that my research is well integrated with existing knowledge and topics in the field

624 3.99 .767 0.2 5.0 13.9 57.4 23.6

59. I feel as if I do not have sufficient control over the direction of my research 624 2.42 .970 14.9 45.4 25.0 12.2 2.6

60. The quality of my research work is affected by the amount of time I need to spend on aspects of my life other than research

620 3.44 1.083 3.1 21.9 18.2 41.6 15.2

70. Indicate, with a cross on the horizontal line, your rating of the quality of your own research work

618 3.41 .687

n m sd NA %

SD %

D %

N %

A %

SA %

65. The requirements of the final assessments (thesis and oral examination) are clear to me

621

3.66

1.06 1.1 2.1 13.7 14.3 50.2 18.5

66. Transfer of Status (from PRS to MLitt/DPhil) processes are fair

622

3.58

1.25 6.3 1.8 5.3 18.6 50.5 17.5

67. The requirements of Transfer of Status are clear

622

3.43

1.25 4.2 3.7 14.3 15.6 47.4 14.8

68. Sufficient support is available for Transfer of Status

616

3.42

1.23 5.2 1.8 12.0 22.4 44.0 14.6

69. I consider that developing a wider range of skills (such as communication, research management, personal effectiveness, networking and team-working, career management) is a valuable part of my research programme

620

4.01

1.20 4.8 0.6 2.9 11.0 41.6 39.0

m=mean, sd=standard deviation, NA=not applicable; SD=strongly disagree; D=disagree; N=neutral; A=agree; SA= strongly agree


The research experience of postgraduate research students at the University of Oxford Table B: Aspects of Teaching by Graduate Students

N yes %

no %

Teaching, or recently taught undergraduate students

626 45.4 54.6

N yes %

possible %

no %

don’t know %

Plan to pursue an academic career

620 43.9 36.6 13.9 5.6

Item n m sd SD

% D %

N %

A %

SA %

61. It is clear to me how to find out about opportunities for me to teach undergraduates at Oxford

621 2.85 1.231 15.1 30.9 15.9 30.0 8.1

62. I consider that teaching undergraduate students in my field is likely to benefit rather than hinder my research

621 3.76 .950 1.3 9.8 22.4 44.1 22.4

63. Teaching undergraduates at Oxford during my graduate studies is one of my aims 617 3.52 1.127 4.5 15.9 24.3 33.9 21.4

64. I see university teaching as an important skill to acquire 621 4.16 .854 0.8 4.2 12.4 43.6 39.0

m=mean, sd=standard deviation, SD= strongly disagree; D=disagree; N=neutral; A=agree; SA= strongly agree


The research experience of postgraduate research students at the University of Oxford Table C: Usual research work space Location N (whole) % Arts % Sci % Department 308 49.4 14.1 87.2 Home 124 19.9 35.8 2.7 Home & Library 58 9.3 18.2 0.0 Home & Department 45 7.2 7.3 6.4 Library 29 4.7 9.9 0.0 College 19 3.0 5.4 0.3 Home & College 8 1.3 2.2 0.3 Home & Department & Library 7 1.1 1.9 0.3 College & Department 7 1.1 1.3 1.0 College & Library 1 .2 0.3 0.0 Home & College & Library 2 .3 0.6 0.0 Data sources 2 .3 0.6 0.0 Home & College & Department 1 .2 0.3 0.0 Library & Department 4 .6 0.6 0.7 Home & College & Department & Library 2 .3 0.6 0.0 Department & Home & Other 1 .2 0.3 0.0 Department & Other 2 .3 0.0 0.7 Home & Work 2 .3 0.0 0.3 Other 1 .2 0.3 0.0 Total 623 100.0

Missing 3 Total 626

Table D: Type of teaching Frequency Percent Tutorials 122 42.5 Laboratory Demonstrating 51 17.8 Tutorials & Classes 27 9.4 Classes/Seminars 25 8.7 Tutorials & Demonstrating 21 7.3 Tutorials & Lectures 10 3.5 Lectures 7 2.4 Demonstrating & Classes 1 .3 Tutorials & Demonstrating & Classes 2 .7 Undergraduate projects 3 1.0 Clinical teaching 1 .3 Tutorials & Demonstrating & Lectures 1 .3 Tutorials & Demonstrating & Project supervision 1 .3 Other 2 .7 Lectures & Classes & Tutorials 3 1.0 Demonstrating & UG projects 2 .7 Tutorials & Revision sessions 2 .7 Tutorials & Seminars & Lectures 1 .3 Tutorials & Seminars 1 .3 Tutorials & Clinical Teaching 1 .3 Field trips 1 .3 Demonstrating & Tutorials & Field trips 1 .3 Tutorials & UG project 1 .3 Total 287 100.0

Not teaching 339 Total 626


The research experience of postgraduate research students at the University of Oxford Annexe II: Scales Table A: Mean scale scores and distributions (university-wide) of response for each scale in ascending mean order (approximately 626 respondents) Scale Mean SD % Disagree* % Neutral* % Agree* Dept. Intellect. Climate 3.38 .69 24.0 6.5 69.5 College Support 3.52 .71 19.5 5.9 74.6 Dept. Infrastructure 3.54 .60 16.1 4.3 79.6 General Infrastructure 3.57 .51 9.1 8.0 82.9 Aware of Assessment 3.67 .79 14.7 9.0 76.3 Supervision 3.69 .79 15.7 5.3 79.0 Skills Development 3.80 .56 6.9 2.7 90.4 Approach to Research 3.84 .53 4.0 7.5 88.5

%Agree: Scale scores >3.0; %Neutral: Scale score = 3.0; %Disagree: Scale scores <3.0 Table B: Scale means (and standard deviations) by division Scale HUM LES MPS MED SS Dept. Intellect. Climate 3.28 (.73) 3.16 (.71) 3.59 (.55) 3.57 (.70) 3.21 (.68) College Support 3.68 (.66) 3.39 (.75) 3.52 (.70) 3.33 (.68) 3.54 (.71) Dept. Infrastructure 3.26 (.62) 3.42 (.62) 3.79 (.48) 3.82 (.55) 3.39 (.54) General Infrastructure 3.59 (.52) 3.47 (.55) 3.61 (.46) 3.55 (.44) 3.54 (.56) Aware of Assessment 3.64 (.83) 3.58 (.84) 3.81 (.72) 3.67 (.72) 3.54 (.86) Supervision 3.88 (.78) 3.60 (.83) 3.68 (.73) 3.61 (.85) 3.62 (.78) Skills Development 3.76 (.53) 3.76 (.55) 3.83 (.57) 3.93 (.53) 3.76 (.55) Approach to Research 3.89 (.53) 3.93 (.52) 3.72 (.50) 3.81 (.56) 3.89 (.52)


The research experience of postgraduate research students at the University of Oxford Table C: Scale and group items and reliabilities (Cronbach alpha) GENERAL ITEMS Supervision (response range 1-5) (alpha=0.86) (Arts 0.85; Sci 0.86) 5. My supervisor(s) make(s) a real effort to understand difficulties I face 13. Research supervision is available when I need it 18. I am given good guidance in topic selection and refinement 22. My research supervision provides me with helpful feedback on my progress 26. I have received good guidance in my literature search Skill Development (response range 1-5) (alpha=0.76) (Arts 0.76; Sci 0.79) 4. I have developed an awareness of the wider research community in my field 7. My postgraduate research studies have helped me to develop a range of communication

skills 9. My postgraduate research studies have helped to develop my awareness of what I need to

manage my own career progression 11. As a result of my research, I have developed the ability to work collaboratively with other

researchers 14. My research has sharpened my analytical skills 17. As a result of my postgraduate research studies, I feel confident about managing a research

project 30. As a result of my research, I have developed the ability to learn independently Departmental Intellectual Climate (response range 1-5) (alpha=0.88) (Arts 0.88; Sci 0.87) 3. The department/faculty provides opportunities for social contact with other postgraduate

students 8. I feel integrated into the research community in the department/faculty 15. The department/faculty provides opportunities for me to become involved in the broader

research culture 16. I feel that other postgraduate students in my department/faculty are supportive 20. I tend to feel isolated within this department/faculty (reversed) 23. Interaction with other postgraduate students is actively encouraged in my department 24. A good seminar programme for postgraduate students is provided 25. The research ambience in the department stimulates my work 29. I feel that this department provides a supportive working environment 31. I feel respected as a fellow researcher within my department/faculty General Infrastructure (response range 1-5) (alpha=0.59) (Arts 0.62; Sci 0.53) 1. The library facilities at Oxford support my research 10. I have access to a common room or a similar type of meeting place 27. I manage to find conditions for studying which allow me to get on with my work easily 33. The university administration is effective in supporting my research 34. I was satisfied with the admission and enrolment processes 36. Complaints handling procedures are clear to me Departmental Infrastructure (response range 1-5) (alpha=0.72) (Arts 0.68; Sci 0.67) 2. I have access to a suitable working space 6. I am able to organise good access to necessary equipment 12. I have good access to the technical support I need 19. I have good access to computing facilities and services in my department 28. There is appropriate financial support for research activities in the department 32. The department administration is effective in supporting my research Overall Satisfaction (response range 1-5) 37. Overall, I am satisfied with the quality of the services and facilities 38. Overall, I am satisfied with the quality of my research supervision 39. Overall, I am satisfied with the quality of my research experience


The research experience of postgraduate research students at the University of Oxford VALIDATION AND EXTRA ITEMS College Support (response range 1-5) (alpha=0.81) (Arts 0.81; Sci 0.82) 40. My college provides opportunities for social contact with other postgraduate students 41. I feel integrated into my college community 42. I have good access to computing facilities and services in my college 44. Interaction with other postgraduate students is actively encouraged in my college 45. In my college I feel I am a part of a community of scholars in my subject area 46. Support from college officers (e.g. welfare, accommodation) is available when I need it. 47. I have access to appropriate general academic support (e.g Tutor for Graduates) in my

college 48. I have appropriate support from my College Advisor Approach to Research (response range 1-5) (alpha=0.68) (Arts 0.67; Sci 0.70) 49. I see my research as contributing in some way to ‘big picture’ issues 50. I usually try to discuss with others new ideas I have in my research 53. When I’m working on a research topic, I try to see in my own mind how all the ideas fit

together 54. Ideas that arise in my research often set me off on chains of thought of my own 57. Often I find myself wondering whether the work I am doing here is really worthwhile (R) 58. It is important to me that my research is well integrated with existing knowledge and topics in

the field 59. I feel as if I do not have sufficient control over the direction of my research (R) Awareness of Assessment (response range 1-6) (alpha=0.84) (Arts 0.83; Sci 0.84) 65.The requirements of the final assessments (thesis and oral examination) are clear to me 66. Transfer of Status (from PRS to MLitt/DPhil) processes are fair 67. The requirements of Transfer of Status are clear 68. Sufficient support is available for Transfer of Status Items not in scales 21. In my research I consider it is more important to find new ways of thinking than to develop

research skills 35. The department provides clear, comprehensive and up-to-date ‘Notes of Guidance’ 43. My college offers some financial support for my research activities 51. I am concerned that my financial situation might affect the quality of my research work 52. As I develop my own research-informed opinion I find myself challenging the opinion of others

including that of my supervisor(s) 55. I concentrate most of my research effort on technical and/or descriptive processes (R) 56. In my research I concentrate on just those areas that are of direct relevance to my topic (R) 60. The quality of my research work is affected by the amount of time I need to spend on aspects

of my life other than research 61. It is clear to me how to find out about opportunities for me to teach undergraduates at Oxford 62. I consider that teaching undergraduate students in my field is likely to benefit rather than

hinder my research 63. Teaching undergraduates at Oxford during my graduate studies is one of my aims 64. I see university teaching as an important skill to acquire 69. I consider that developing a wider range of skills (such as communication, research

management, personal effectiveness, networking and teamworking, career management) is a valuable part of my research programme (scale 1-6)

70. Indicate, with a cross on the horizontal line, your rating of the quality of your own research work (poor/fair/good/very good/outstanding)


The research experience of postgraduate research students at the University of Oxford Figure A: Research Approach (RESINTSA) score distributions for Arts (top) and Sciences

(bottom)

RESINTSA

CULTURES: 1.00 arts

RESINTSA

20.0019.00

18.0017.00

16.0015.00

14.0013.00

12.0011.00

10.009.00

7.00

Freq

uenc

y

60

50

40

30

20

10

0

RESINTSA

CULTURES: 2.00 science

RESINTSA

20.0019.00

18.0017.00

16.0015.00

14.0013.00

12.0011.00

10.009.00

8.006.00

Freq

uenc

y

60

50

40

30

20

10

0


The research experience of postgraduate research students at the University of Oxford Annexe III: Departmental information Information for departments on scale scores and overall satisfaction have been circulated to Directors for Graduate Studies.


The research experience of postgraduate research students at the University of Oxford Annexe IV: Completion times (total terms by commencement year by division) How long it takes doctoral students to complete their research degrees (i.e. the number of terms taken to submit their theses) is a research outcomes indicator. As the questionnaire for this study was administered to current (2005) research students, none of the students in the sample had completed their degrees. It was therefore decided to look at the characteristics of cohorts of research students for the academic years 1997-98 to 2000-01, as students who had started in these years would be more likely to have submitted their theses. Using data supplied by the Student Records Office at the University of Oxford for 3203 students, an average number of terms to submission of thesis was calculated for each division as follows:

1997 1998 1999 2000 HUM LES MPS MS SS

13.90 13.17 11.92 12.19 14.78

14.41 13.60 11.98 12.11 14.17

14.18 13.39 11.87 11.96 12.50

13.22 12.49 12.14 11.74 13.07

These completion times, by division, for 4 cohorts, are shown graphically below:

11

11.5

12

12.5

13

13.5

14

14.5

15

1997 1998 1999 2000

HumLESMPSMedScSocSc


The research experience of postgraduate research students at the University of Oxford Annexe V: Proposed OPREQ The information in this Annexe is included as an example of a PREQ for Oxford.

To PGR Students in final year

Postgraduate Research Experience Questionnaire I am writing to ask you to participate in a survey. The aim of the survey is to gather information to further inform the decisions made about the research environment in the Collegiate University. The questionnaire will take about ten minutes to complete and will be of use to future Oxford students. The questionnaire has been designed for use across the University. If your research programme is in an area of the University that does not have departments, please read any items with that wording as referring to your faculty or sub-faculty. The statistical information based on your returned questionnaire will be made available to your faculty or department for consideration as an additional form of feedback and used by the University to monitor the experience of research students. The information supplied will be processed in accordance with the Data Protection Act 1998. The forms will be destroyed at the end of the process and only the anonymized data will remain for statistical analysis.

Pro-Vice-Chancellor (Education)

Please return completed forms in the enclosed envelope by FRIDAY xxth MAY 2006

DO NOT FOLD THIS QUESTIONNAIRE



Institute for the Advancement of University Learning

Beside each statement please the box that most accurately reflects your research experience

FOR OFFICE USE ONLY

1. The library facilities at Oxford support my research 2. I have access to a suitable working space

Strongly Disagree Disagree Neutral Agree

Strongly Agree

3. The department/faculty provides opportunities for social contact with other postgraduate students

4. I have developed an awareness of the wider research community in my field 5. My supervisor(s) make(s) a real effort to understand difficulties I face 6. I am able to organise good access to necessary equipment 7. My postgraduate research studies have helped me to develop a range of

communication skills 8. I have had the opportunities I desired to learn how to teach 9. My postgraduate research studies have helped to develop my awareness of what

I need to manage my own career progression 10. I have access to a common room or a similar type of meeting place 11. As a result of my research, I have developed the ability to work collaboratively

with other researchers 12. I have good access to the technical support I need 13. Research supervision is available when I need it 14. My research has sharpened my analytical skills 15. The department/faculty provides opportunities for me to become involved in the

broader research culture 16. I feel that other postgraduate students in my department/faculty are supportive 17. As a result of my postgraduate research studies, I feel confident about managing

a research project 18. I am given good guidance in topic selection and refinement 19. I have good access to computing facilities and services in my department 20. I tend to feel isolated within this department/faculty 21.The requirements of the final assessments (thesis and oral examination) are

clear to me 22. My research supervision provides me with helpful feedback on my progress 23. Interaction with other postgraduate students is actively encouraged in my

department 24. A good seminar programme for postgraduate students is provided 25. The research ambience in the department stimulates my work 26. I have received good guidance in my literature search 27. I manage to find conditions for studying which allow me to get on with my work

easily 28. There is appropriate financial support for research activities in the department 29. I feel that this department provides a supportive working environment 30. As a result of my research, I have developed the ability to learn independently 31. I feel respected as a fellow researcher within my department/faculty 32. The department administration is effective in supporting my research 33. The university administration is effective in supporting my research 34. I was satisfied with the admission and enrolment processes 35. I am concerned that my financial situation might affect the quality of my research

work 36. Complaints handling procedures are clear to me 37. The requirements of Transfer of Status are clear 38. Sufficient support is available for Transfer of Status 39. The quality of my research work is affected by the amount of time I need to

spend on aspects of my life other than research 40. It is clear to me how to find out about opportunities for me to teach

undergraduates at Oxford 41. Overall, I am satisfied with the quality of the support from my college 42. Overall, I am satisfied with the quality of my research experience

64


OPREQ Scales and Items Supervision (response range 1-5) (alpha=0.86) 5. My supervisor(s) make(s) a real effort to understand difficulties I face 13. Research supervision is available when I need it 18. I am given good guidance in topic selection and refinement 22. My research supervision provides me with helpful feedback on my progress 26. I have received good guidance in my literature search Skill Development (response range 1-5) (alpha=0.76) 4. I have developed an awareness of the wider research community in my field 7. My postgraduate research studies have helped me to develop a range of communication skills 8. I have had the opportunities I desired to learn how to teach 9. My postgraduate research studies have helped to develop my awareness of what I need to manage my own

career progression 11. As a result of my research, I have developed the ability to work collaboratively with other researchers 14. My research has sharpened my analytical skills 17. As a result of my postgraduate research studies, I feel confident about managing a research project 30. As a result of my research, I have developed the ability to learn independently Departmental Intellectual Climate (response range 1-5) (alpha=0.86) 3. The department/faculty provides opportunities for social contact with other postgraduate students 15. The department/faculty provides opportunities for me to become involved in the broader research culture 16. I feel that other postgraduate students in my department/faculty are supportive 20. I tend to feel isolated within this department/faculty (reversed) 23. Interaction with other postgraduate students is actively encouraged in my department 24. A good seminar programme for postgraduate students is provided 25. The research ambience in the department stimulates my work 29. I feel that this department provides a supportive working environment 31. I feel respected as a fellow researcher within my department/faculty General Infrastructure items (response range 1-5) 1. The library facilities at Oxford support my research 10. I have access to a common room or a similar type of meeting place 27. I manage to find conditions for studying which allow me to get on with my work easily 33. The university administration is effective in supporting my research 34. I was satisfied with the admission and enrolment processes 36. Complaints handling procedures are clear to me Departmental Infrastructure (response range 1-5) (alpha=0.72) 2. I have access to a suitable working space 6. I am able to organise good access to necessary equipment 12. I have good access to the technical support I need 19. I have good access to computing facilities and services in my department 28. There is appropriate financial support for research activities in the department 32. The department administration is effective in supporting my research Overall Satisfaction (response range 1-5) 41. Overall, I am satisfied with the quality of the support from my college 42. Overall, I am satisfied with the quality of my research experience Awareness of Assessment (response range 1-6) (alpha=0.77) 21.The requirements of the final assessments (thesis and oral examination) are clear to me 37. The requirements of Transfer of Status are clear 38. Sufficient support is available for Transfer of Status Items not in scales 35. I am concerned that my financial situation might affect the quality of my research work 39. The quality of my research work is affected by the amount of time I need to spend on aspects of my life other than research 40. It is clear to me how to find out about opportunities for me to teach undergraduates at Oxford


The research experience of postgraduate research students at the University of Oxford Annexe VI: Study methodology The study was conducted in two phases. The first focused on testing existing instruments (such as the Postgraduate Research Experience Questionnaire (GCCA & ACER, (2002)) and on identifying areas of research interest. This phase was largely qualitative, and involved consultation and interviews with students and staff of the university. Interviews were conducted with 33 students (see Annexe VIII for the interview schedule) who were also asked to test some of the proposed questionnaire items. A questionnaire (Annexe VII) was developed from the material gathered in this first phase and from a university-wide consultation process. In week 4 of Trinity Term 2005, it was mailed to a sample of graduate students across the University of Oxford. A total of 4096 postgraduate students were in the student database (Students Records Office at the University of Oxford) in May 2005. In preparing the sample, the following were excluded:

Students who had suspended their course; Students studying for a MSc, MSt or MLitt, due to the small numbers involved; Students with Recognised Student Status.

That left 3855 postgraduate students in the population from which a sample of 1 in 5, as had been done by the Commission of Inquiry (Commission of Inquiry, 1997b p.400) was selected. The sample was stratified by Department, Status (DPhil or Probationary Research Student (PRS)), College and Gender. This gave a sample size of 756. Email was used to contact students who had not returned their questionnaire within two weeks of receiving it. A total of 626 students (82.8%) returned the questionnaire. The proportions of the constituencies in the response set closely match those in the full sample. Some demographic details of respondents are shown in Annexe I. The data were analysed using Statistical Package for Social Scientists (SPSS). The capacity of the scales and items to measure variation in research environments was tested by comparing the results with two external indicators: completion times (Section 6) and RAE ratings (Section 7). The validity of the relational approach to the research adopted in this study has been tested using cluster analyses. For the whole sample, a hierarchical cluster analysis (Ward’s method) was calculated using all seven scales to determine cluster occupancy, which maximises the differences between clusters. The pattern of variables in each cluster then indicates the relations, for that group of students, between the variables. The cluster analysis was followed by between-group contrasts using cluster membership to form the groups. Mean (and standard deviation) cluster scale z-scores for Departmental Intellectual Climate, Departmental Infrastructure, General Infrastructure, Supervision Approach to Research scales and Overall Satisfaction and Skills Development Scales by cluster for the whole sample and the two-cultures are shown below in Tables A-C).


The research experience of postgraduate research students at the University of Oxford Table A: Cluster analysis of perceptions of research environment, approach to research and outcomes variables (whole sample, n=623) (z-scores and s.d.) Cluster 1 Cluster 2 Variables N = 212 N = 411 Departmental Intellectual Climate Departmental Infrastructure General Infrastructure Supervision Approach to Research Overall satisfaction Skills Development

.72 (.72)

.76 (.67)

.76 (.71)

.69 (.62)

.68 (.80)

.74 (.58)

.71 (.68)

-.37 (.92) -.40 (.91) -.40 (.90) -.36 (.97)

-.35 (.91)

-.38 (.96) -.37 (.94)

Table B: Cluster analysis of perceptions of research environment, approach to research and outcomes variables (Arts, n=305) (z-scores and s.d.) Cluster 1 Cluster 2 Variables N = 153 N = 152 Departmental Intellectual Climate Departmental Infrastructure General Infrastructure Supervision Approach to Research Overall satisfaction Skills Development

.47 (.81)

.46 (.83)

.63 (.70)

.49 (.70)

.56 (.78)

.60 (.57)

.50 (.76)

-.47 (.95) -.47 (.94) -.63 (.86)

-.49 (1.02)

-.56 (.89)

-.62 (.94) -.51 (.95)

Table C: Cluster analysis of perceptions of research environment, approach to research and outcomes variables (Sciences, n=293) (z-scores and s.d.) Cluster 1 Cluster 2 Variables N = 152 N = 141 Departmental Intellectual Climate Departmental Infrastructure General Infrastructure Supervision Approach to Research Overall satisfaction Skills Development

.55 (.73)

.59 (.66)

.48 (.78)

.61 (.62)

.41 (.88)

.56 (.61)

.46 (.76)

-.59 (.92) -.65 (.90) -.53 (.95) -.67 (.90)

-.44 (.94)

-.57 (.96)

-.50 (1.00)


The research experience of postgraduate research students at the University of Oxford Annexe VII: Study questionnaire








The research experience of postgraduate research students at the University of Oxford Annexe VIII: Qualitative study – PGR student Interview Schedule Interviews were conducted with 33 PGR students in preparation for the quantitative study. Each interview lasted for about an hour. It was audio-taped and transcribed. It had several parts, two of which were part of this study and focused on teaching and the research environment. (a) Teaching background and experience (15 min)

1. What teaching are you doing now at Oxford? (tutorials, classes, labs …) 2. What sort of things do you do in a typical tutorial/class/lab … 3. Why do you do it that way? 4. How do you think students feel about your teaching compared with that of

tutors? 5. Why do you teach? 6. What support/training have you had? What more do you need? 7. Is your pay fair for what you do? 8. Is there anything else you would like to add about teaching

(b) Questionnaire development (30 min)

1. Please briefly describe what the research you are doing is about? 2. Describe a typical week? 3. How do you go about this? Why do you do it that way?

(Deep-surface; parts-whole; multistructural-relational) 4. How would you describe the environment in which you work? (supportive,

lonely, hierarchical, power-laden, excluding …) 5. Why are you doing research? (confidence, motivation, importance, interest) 6. What would you say has been the value that Oxford has added, if any? 7. What do you see as the benefits of being in your college? (feelings of

place/hierarchy in college) 8. Oxford is considering expanding (increasing) graduate student numbers –

what do you see as being the benefits/disadvantages/issues for you? 9. What aspects of your experience at Oxford would you most like to change? 10. What aspects of your experience at Oxford would you most like to retain? 11. What do you think a survey of research students at Oxford should include

that has not already been discussed today?

At the end of this section, each interviewee was asked to test a part (6 items) of the proposed questionnaire.


The research experience of postgraduate research students at the University of Oxford Annexe IX: References Anderson, M. S. (1996). Collaboration, the Doctoral Experience, and the Departmental

Environment. The Review of Higher Education, 19(3), 305-326. Anderson, M. S., & Swazey, J. P. (1998). Reflections on the Graduate Experience: An

overview. New Directions for Higher Education, 101. Asmar, C. (1999). Is there a gendered agenda in academia? The research experience

of female and male PhD graduates in Australian universities. Higher Education, 38, 225-273.

Atkinson, T., Francis, M., Heath, A., Zimdars, A., & McLean, M. (2004). Pilot Study: The demographic profile of UK Graduate Students admitted to the University of Oxford. Oxford: University of Oxford.

Australian Council for Education Research (ACER). (1999). Evaluation and validation of the trial Postgraduate Research Experience Questionnaires. Canberra: DETYA.

Bazeley, P. (1999). Continuing research by PhD graduates. Higher Education Quarterly, 53(4), 333-352.

Becher, T., Henkel, M., & Kogan, M. (1994). Graduate Education in Britain. London: Jessica Kingsley.

Bowen, W., & Rudenstine, N. (1992). In pursuit of the PhD. Princeton: Princeton University Press.

Brew, A. (2001). Conceptions of Research: a phenomenographic study. Studies in Higher Education, 26(3), 271-285.

Bruce, C. S. (1994). Research students' early experiences of the dissertation literature review. Studies in Higher Education, 19(2), 217-229.

Cadman, K. (2000). 'Voices in the Air': evaluations of the learning experiences of international postgraduates and their supervisors. Teaching in Higher Education, 5(4), 475-491.

Caffarella, R. S., & Barnett, B. G. (2000). Teaching Doctoral Students to Become Scholarly Writers: the importance of giving and receiving critiques. Studies in Higher Education, 25(1), 39-51.

Chiang, I. (2003). Learning experiences of doctoral students in UK universities. International Journal of Sociology and Social Policy, 23(1/2), 4-32.

Collinson, J. A. (1998). Professionally trained researchers? Expectations of competence in social science doctoral research training. Higher Education Review, 31(1), 59-67.

Commission of Inquiry. (1997a). Commission of Inquiry Report. Oxford: University of Oxford.

Commission of Inquiry. (1997b). Supplementary Volume. Oxford: University of Oxford. Dawson, V. (1996). The (R)Evolution of my epistemology: my experience as a

postgraduate research student. Educational Action Research, 4(3), 363-374. Deem, R., & Brehony, K. J. (2000). Doctoral Students' Access to Research Cultures -

are some more unequal than others? Studies in Higher Education, 25(2), 149-165.

Delamont, S., Atkinson, P., & Parry, O. (2000). The Doctoral Experience. Success and Failure in Graduate School. London: Falmer Press.

Elton, L., & Pope, M. (1989). Research Supervision: the value of collegiality. Cambridge Journal of Education, 19(3), 267-276.

Evans, T. (2002). Part-time Research students: are they producing knowledge where it counts? Higher Education Research & Development, 21(2), 155-165.

Golde, C. M. (2000). Should I stay or should I go? Students descriptions of the Doctoral Attrition Process. The Review of Higher Education, 23(2), 199.

Graduate Careers Council of Australia (GCCA), & Australian Council for Educational Research (ACER). (2002). Postgraduate research experience questionnaire 2000. Parkville, Victoria.


The research experience of postgraduate research students at the University of Oxford Gurr, G. M. (2001). Negotiating the ‘Rackety Bridge’ a Dynamic Model for Aligning

Supervisory Style with Research Student Development. Higher Education Research & Development, 20(1), 81-92.

Haksever, A. M., & Manisali, E. (2000). Assessing supervision requirements of PhD students: the case of construction managemnt and engineering in the UK. European Journal of Engineering Education, 25(1), 19-32.

Harland, T., & Plangger, G. (2004). The Postgraduate Chameleon: Changing Roles in Doctoral Education. Active Learning in Higher Education, 5(1), 73-86.

Harman, K. (2002). The research training experiences of doctoral students linked to Australian Cooperative Research Centres. Higher Education, 44, 469-492.

Heath, T. (2002). A Quantitative Analysis of PhD Students' Views of Supervision. Higher Education Research & Development, 21(1), 41-53.

Holdaway, E., Debois, C., & Winchester, I. (1995). Supervision of Graduate Students. The Canadian Journal of Higher Education, 25(3), 1-29.

Johnson, L., Lee, A., & Green, B. (2000). The PhD and the Autonmous Self: gender, rationality and postgraduate pedagogy. Studies in Higher Education, 25(2), 135-147.

Kiley, M., & Mullins, G. (2003, August). Why conceptions of research might be significant in postgraduate education. Paper presented at the 10th Conference of the European Association for Research into Learning and Instruction, Padua.

Lee, D. (1998). Sexual Harrassment in PhD Supervision. Gender and Education, 10(3), 299-312.

Marsh, H. W., Rowe, K. J., & Martin, A. (2002). PhD Students' Evaluations of Research Supervision. The Journal of Higher Education, 73(3), 313-348.

Meyer, J. H. F., Shanahan, M., & Laugksch, R. C. (2003, August). Measuring differentiation in conceptions of research in postgraduate education. Paper presented at the 10th Conference of the European Association for Research into Learning and Instruction, Padua.

Morton, M., & Thornley, G. (2001). Experiences of Doctoral Students in Mathematics in New Zealand. Assessment and Evaluation in Higher Education, 26(2), 113-126.

Pearson, M., & Brew, A. (2002). Research Training and Supervision Development. Studies in Higher Education, 27(2), 135-150.

Pole, C. J., Sprokkereff, A., Burgess, R. G., & Lakin, E. (1997). Supervision of Doctoral Students in the Natural Science: expectations and experiences. Assessment and Evaluation in Higher Education, 22(1), 49-63.

Ramsden, P., Conrad, L., Ginns, P., & Prosser, M. (2003, August). Students' and Supervisors' Experiences of the Context for Postgraduate Study. Paper presented at the 10th Conference of the European Association for Research into Learning and Instruction, Padua.

Research Councils UK. (2001). Joint Statement of the Research Councils'/AHRB's Skills Training Requirements for Research Students: Research Councils UK.

Roberts, G. G. (2002). SET for success. The supply of people with science, technology, engineering and mathematics skills. London: HM Treasury.

Sayed, Y., Kruss, G., & Badat, S. (1998). Students' Experience of Postgraduate Supervision at the University of the Western Cape. Journal of Further and Higher Evaulation, 22(3), 275-285.

Seagram, B. C., Gould, J., & Pyke, S. W. (1998). An Investigation of Gender and Other Variables on Time to Completion of Doctoral Degrees. Research in Higher Education, 39(3), 319-335.

Smeby, J.-C. (2000). Same-gender relationships in graduate supervision. Higher Education, 40, 53-67.

Taylor, J. (2002). Changes in Teaching and Learning in the Period to 2005: the case of postgraduate higher education in the UK. Journal of Higher Education Policy and Management, 24(1), 53-73.


The research experience of postgraduate research students at the University of Oxford The Quality Assurance Agency for Higher Education. (2004). Code of Practice for the

Assurance of Academic Quality and Standards in Higher Education: Postgraduate Research Programmes. Gloucester: The Quality Assurance Agency for Higher Education.

Tinkler, P., & Jackson, C. (2002). In the dark? Preparing for the PhD viva. Quality Assurance in Education, 10(2), 86-97.

Trigwell, K., & Ashwin, P. (2003). Undergraduate Students' Experience of Learning at the University of Oxford. Oxford: University of Oxford.

UK GRAD Programme. (2004). A national review of emerging practice on the use of Personal Development Planning for postgraduate researchers (pp. 16).

University of Oxford. (2003). Examination Regulations 2003: Oxford University Press. Wright, J., & Lodwick, R. (1989). The process of the PhD: a study of the first year of

doctoral study. Research Papers in Education, 4(1), 22-56. Wright, T., & Cochrane, R. (2000). Factors Influencing Sucessful Submission of PhD

Theses. Studies in Higher Education, 25(2), 181-195.


Documents

The Research Experience of Postgraduate Research … · The research experience of postgraduate research students at the University of Oxford Headlines • The overall experience