Measuring Outcomes of Doctoral Programs:

Alumni and Exit Surveys at Western University

CAGSOctober 2015

Supported by

Higher Education Quality Council of Ontario

Full Report:http://www.heqco.ca/SiteCollectionDocuments/Doctoral%20Grad%20Survey%20ENG.pdf

Background & Overview:• Lack of data regarding career outcomes• Increasing view that recent doctoral graduates

are unemployed or underemployed

• Evaluated feasibility of two surveys: Graduate Studies Exit Survey Graduate Studies Alumni Survey

• Intent was that these surveys would form the basis for ongoing collection of outcome data

The SurveysGraduate Studies Exit Survey Graduate Alumni Survey

• Demographics

• Program information

• Quality of graduate program and supervision, and university services

• Participation in and quality of professional development opportunities

• Career preparation

• Plans following graduation

• Graduate learning outcomes

• Demographics

• Program information

• Employment and career path

• Continuing education and professional development

• Quality and impact of graduate education

Survey Administration

• Surveys were administered electronically

• Email invitations with a link to the survey

• All surveys were completed online and could be accessed only via the link in the email invitation

Doctoral Programs Included:• 50 doctoral programs across 10 Faculties:

• Arts & Humanities• Business• Education• Engineering• Health Sciences• Information & Media Studies• Medicine & Dentistry• Music• Science• Social Science

The SamplesGraduate Studies Exit Survey Graduate Alumni Survey

• Students who completed a PhD program at Western between September 1, 2013 and April 21, 2014

• 156 invitations distributed

• 25 students completed the survey

• Response rate of 16%

• Alumni who completed a PhD between January 1, 2008 and August 31, 2013

• 1,546* invitations to complete the survey (27% of emails were opened within 6 weeks; a second email was send to those who had not opened the first email within 6 weeks; 34% of the second emails were opened)

• 135 alumni completed the survey

• Response rate of 9%

* It is not known how many of the email addresses were active.

Results: Exit Survey – Sample

• Response rates were highest for Arts & Humanities/Social Science (25%) and Health Science/Medicine (28%)

• Response rates were lowest for Engineering/Science (8%) and Interdisciplinary Programs (9%)

• 64% male• 36% female

• 80% Canadians or Permanent Residents• 20% were International

Results: Exit Survey - Employment• 32% - secured employment prior to completion• 4% - planning further education

• 24% had not begun a job search yet

• Of those who had begun a job search, the most frequently used resources were personal contacts (79%), websites for professional organizations (58%), and employer websites (53%)

• Use of Career Services, Career Fairs and Recruitment Events were the least frequently reported resources (16% each)

Results: Exit Survey – Learning Outcomes

• On a scale of 1-5, respondents reported being: Most prepared for careers in academia (4.1) and

least prepared for careers in industry/business (2.8) and entrepreneurial careers (2.7)

Well prepared (≥ 4) regarding common doctoral learning outcomes, such as thinking analytically and critically, writing professionally and academically, conducting and evaluating research, understanding emerging issues and challenges, and demonstrating leadership

Results: Alumni Survey – Sample• Response rates were highest for most recent graduates

(2013 - 16%; 2012 - 12%) and lowest for the earliest graduates (2008 - 4%)

• Similar to the Exit Survey, response rates were highest for Arts & Humanities/Social Science (23%) and Health Science/Medicine (39%), followed by Engineering/Science (19%), Education/Media Studies/Business (12%) and Interdisciplinary Programs (4%)

• 79% employed full-time• 13% employed part-time• 5% not employed and looking for employment• 3% not employed and not looking

Results: Alumni Survey – Employment – All Respondents• Of the respondents employed:

95% - employed in jobs related to graduate studies 90% - satisfied or very satisfied with employment 49% - report salaries of $70,000 and higher 48% - employed in Ontario 27% - employed outside of Canada 58% - employed in Universities 16% - employed in health care 25% are employed as a Professor; 20% as a Postdoc;

6% as a sessional or term instructor

Results: Alumni Survey – Employment – Across Cohorts • Aspects of employment varied across cohorts:

Earlier grads had a higher rate of FT employment (89% for 2008-2010; 75% for 2013)

Employment tended to be related to graduate studies across all cohorts (lowest for 2011 at 87%)

Earlier grads were more likely to report salaries of $70,000 and higher

Employment as a sessional or term instructor was reported only for graduates of 2013 and 2012

Postdoc appointments were more frequently reported among more recent graduates, but 15% of 2008-2010 grad reported holding postdoc appointments

Results: Alumni Survey – Employment – Across Disciple Groups • Aspects of employment varied across discipline groups –

Arts/Humanities/Social Science (AHSS) & Medicine/Health Science (MHS)*: Higher rate of FT employment for MHS (83%; PT=6%) than AHSS (71%;

PT=26%) Employment was related to graduate studies in both groups (100% for

AHSS; 97% for MHS) MHS (93%) more satisfied with their employment than AHSS (84%) More MHS (48%) reported salaries ≥ $70,000 than AHSS (30%) Employment as a sessional/term instructor reported only for AHSS (17%) Postdoc appointments more frequently reported among MHS (24%) than

among AHSS (14%)

*only 9% of AHSS grads were 2008-2010, compared to 29% of MHS grads; other disciplines are not reported due to very low response rates

Results: Alumni Survey – Career Preparation• Presenting seminars and/or papers, writing a major paper or

thesis, writing manuscripts, preparing conference presentations, attending conferences, collaborating with faculty and working in a collaborative team were rated as the most important aspects of graduate education (≥ 3 on scale of 1-5)

• Respondents agreed that their graduate education prepared them to work independently, think analytically and critically, communicate effectively, write professionally, and think creatively.

• They disagreed that their program prepared them to become entrepreneurs.

Summary & Recommendations• Due to the small sample size and low

response rates, results must be interpreted with caution.

• Survey administration improvements: Use multiple reminders Include questions addressing leadership and

other skills/abilities often acquired as part of graduate studies and their application to employment