Upload
janelle-ligons
View
217
Download
1
Tags:
Embed Size (px)
Citation preview
How do you solve a problem like Impact? Summary of survey findings
Paul Redmond
Head of the Careers & Employability Service, University of Liverpool
Overview
• 46 completers (82% heads of service with between 4 – 7 years’ service).
• Majority (85%) located in student service-related department.
• 11% in a teaching-related department• 3.8% in an external-relations department• Spread of universities (post-1992, RG, etc.)
Reporting structure
• 70% of heads report to Head of Student Services
• 20% report to PVC• Academic deans / heads of T&L = 6.8%
respectively
Defining your Service’s purpose
• Providing services to students – nearly all cite ‘employability’ services to students as the primary goal, e.g.
• “Improving the employability of students and graduates and providing a central point of contact to support CEIAG across the campus …”
Defining your Service’s purpose
• Providing services to students – nearly all cite ‘employability’ services to students as the primary goal, e.g.
• “Improving the employability of students and graduates and providing a central point of contact to support CEIAG across the campus as well as delivering centrally …”
Rank your main stakeholders
1st 2nd 3rd 4th 5th
Students 93% (53) 7% (4) 0 0 0
Graduates 0 57.4% (31) 9.3% (5) 14.8% (8) 18.5% (10)
Employers 0 19.6% (11) 44.6% (25) 28.6% (16) 7.1% (4)
Academics 0 9.1% (5) 36.4% (20) 29.1% (16) 25.5% (14)
University managers
7.1% 8.9% (5) 10.7% (6) 28.6% (16) 44.7% (25)
How do you or your HEI measure the responsiveness, profile and visibility of your service with the above groups?
• Mixture of internal and external feedback measures• Student numbers / record of users• Referrals from academics• Profile with senior managers• DLHE – institutional comparators (ePI)• Student satisfaction / user surveys (various)• League tables• Involvement of CES in key projects /initiatives• Access to ‘prized’ networks• Employer evaluations• Hits on website• ‘Profile and visibility’ …• Matrix
How do you record and track usage of your Service?
• Central recording systems• On-line systems (e.g. Interfase)• ‘Numbers attending events’ records• Mixture of qualitative and quantitative research –
headcounts to ‘mystery shoppers’• We don’t!• Basic headcounts• Monthly MIS surveys / reports• Interview stats• Hits on website• Twitter traffic …
How do you measure your Service’s impact on students?
Numbers Satisfaction Impact Don’t measure
Curriculum based group work
82.1% (46) 85.7% (48) 28.6% (16) 1.8% (1)
Centrally based group work
81.8% (45) 90.9% (50) 20% (11) 1.8% (1)
One to one discussions
84.2% (48) 89.5% (51) 36.8%(21) 0
Online / self help 72.2% (39) 63% (34) 18.5% (10) 13% (7)
How do you measure your impact with employers?
Numbers Satisfaction Impact Don’t measure
Employer vacancy handling
86.8% (46) 62.3% (33) 30.2% (16) 7.5% (4)
Employer fairs / presentations / events on and off campus
86.8% (46) 94.3% (50) 18.9% (10) 1.9% (1)
Employer input into group work with students
66.7% (34) 64.7% (33) 13.7% (7) 19.6% (10)
Facilitating employer links with academic departments
41.2% (21) 39.2% (2) 23.5% (12) 43.1% (22)
Consultancy and other tailored services for employers
34% (16) 36.2% (17) 21.3% (10) 55.3% (26)
Other ways in which you measure your impact on employers
• None (several)• Numbers of vacancies filled / quantity and quality of
applications• Employer advisory board• Repeat business• Direct feedback from employers in relation to vacancies
and internships
Do you or your institution measure the impact of your service or your INSTITUTION’S REPUTATION AND SUCCESS on:
0.0% 20.0% 40.0% 60.0% 80.0%
Return on investment and value added
Income generation
Efficient use of resources, including staff time (VFM)
Contribution to professional development of academic staff
Contribution to student retention / progression
Contribution to improving graduate destinations
66.7%
35.3%
23.5%
52.0%
44.0%
27.1%
31.4%
54.9%
68.6%
40.0%
50.0%
50.0%
2.0%
9.8%
7.8%
8.0%
6.0%
22.9%
Yes
No
Don't know
How your careers service impacts on your institution’s reputation and success
• Provides comparative data on graduate employment (shows where HEI stands in league tables)
• Success in generating funded projects• Income generation• League Tables / DLHE – “Seen as our most important
function by senior management, sadly.”)• KPI’s in reports to management• Head produces full-cost recovery study to gauge VfM• Contribution to student retention• “Return on Investment … not sure about how this might
be specifically measured …”• “Nothing too explicit ..” … “None.”
How highly are you perceived in your HEI?
What KPIs and / or targets does your Service have?
• DLHE – top listed KPI• Employability P.I.• No’s of finalists / graduates seen• Increased numbers using the Service• Student satisfaction surveys• Listed in ‘Top 10’ surveys• Personal recommendation• No. of placements achieved• Matrix• Mixture of externally imposed and internally verified
Final comments …
• “I am still at a loss as to how to establish meaningful and real impact measures for most of the work we are engaged in, despite working at the problem for many years! Fortunately, I am rarely required to justify our impact.”
• “Be careful of using DLHE as a KPI or measure of impact.”
• “Our work does not necessarily have immediate effect – it may take many weeks, months or years for a client to really take on advice. How do we measure the impact of guidance?”
• “Collective approach from AGCAS in addressing this is very much welcomed.”
Reflection
• The Impact Paradox: By prioritising overwhelmingly services to students, which produce intangible outcomes, careers services find it difficult to measure impact. Those services which can be measured are often not those viewed as key priorities by services.
• Location, Location, Location: How crucial is institutional positioning?
• Across the sector, wide variations exist (and varying levels of measurement sophistication).
• But impact measurement is becoming a widely accepted issue of importance for most careers services. Nevertheless, methodologies remain limited.