Large scale surveys as campaigns: social media and the

Student Experience SurveySonia Whiteley

The Social Research Centre Seminar Series

Overview

1. About the QILT surveys

2. QILT & the QILT brand

3. Surveys as campaigns

4. Initial outcomes

QILT survey campaigns 2

About the QILT surveys

• The QILT survey suite consists of: The Student Experience Survey (SES), formerly

the University Experience Survey (UES)

The Graduate Outcomes Survey (GOS), and

The Employers Satisfaction Survey (ESS)

• All are cross sectional, point-in-time surveys except the GOS, which is longitudinal.

• All higher education providers are in-scope for QILT.

QILT survey campaigns 3

The Student Experience Survey

• The University Experience Survey (UES) / Student Experience Survey (SES) involves: Undergraduate higher education students

Who are onshore in Australia, and

Commencing their studies or in the final year of their degrees• More than 100,000 surveys completed each year• Data is collected online.

QILT survey campaigns 4

The Graduate Outcomes Survey

• The in-scope population is undergraduate and postgraduate higher education students who completed their qualification in the six months before

data collection

as an onshore student• Graduates are surveyed twice a year, approximately four

months after they have completed their qualification• Data is collected online using a similar approach to the

SES.

QILT survey campaigns 5

The Employers Satisfaction Survey

• All supervisors of graduates eligible for the GOS are in-scope for the ESS

• Two trials of the ESS were undertaken: a methodological trial in conjunction with the October 2014 AGS

a content trial (revised Employer Satisfaction Questionnaire – ESQ) in conjunction with the April 2015 AGS

• ‘Encouraging’ graduates to provide contact information for their supervisor remains a ‘challenge’ for the live implementation of the survey

QILT survey campaigns 6

QILT data collection periods

QILT survey campaigns 7

Domain 2015 2016 2017

Student Experience Survey August/September August/September August/September

Graduate Outcomes Survey

May/June & November/December

May/June & November/December

May/June & November/December

Employer Satisfaction Survey

May/June & November/December

May/June & November/December

May/June & November/December

Graduate Outcomes SurveyLongitudinal

- March/April March/April

The QILT ‘brand’

• QILT is the overarching or ‘corporate’ brand and is used as the anchor point to integrate the ‘brand family’ through The website Facebook Official communication Media releases Workshops and fora, and Conference sponsorship.

QILT survey campaigns 8

The QILT brand (2)

• The QILT brand is intended to be a constant, recognisable presence throughout the student’s higher education experience and their outcomes post-graduation

• QILT is positioned as an independent ‘feedback friend’ students and graduates expect to encounter throughout and after their studies

• Even if a particular survey/survey invitation is not immediately recognised by the recipient, the look, feel and tone should be familiar and reassuring while retaining a sense of ‘official importance’

QILT survey campaigns 9

The QILT brand family

• Student Experience Survey

• Graduate Outcomes Survey

• Employer Satisfaction Survey

QILT survey campaigns 10

The QILT brand family (2)

• The brand family or ‘products’ are the three individual surveys that have their own logos, colour schemes and brand guidelines.

• The SES brand is more informal, typically targeted at younger students / recent school leavers who may be less likely to engage with QILT

• The GOS brand has a more formal remit and has been designed to appeal to a broader group that includes more mature postgraduates

• The ESS is the closest in tone to the QILT brand with more of a sense of official, quality assurance

QILT survey campaigns 11

Surveys as ‘campaigns’

• Annual frameworks include: Institutional Liaison Strategy

Student/Graduate Engagement Strategy

Employer Engagement Strategy

• Each survey is supported by a: SES or GOS Communications & Marketing Plan, and an

Institutional specific plan

QILT survey campaigns 12

Wait a minute - Why bother?

• Combination of push and pull factors

• Pushed towards trying a new approach by Low online response rates

Declining online response rates

Problems with representativeness

• But also pulled into innovative activities by A large in-scope population for the surveys

Engaged (mostly) staff at institutions

Clients with vision and a flexible budget

QILT survey campaigns 13

Surveys as ‘campaigns’

• The QILT website is the start and the end point for all survey activity

• A background campaign runs throughout the year

• Awareness raising campaigns run two weeks prior to each survey

• Survey specific campaigns with advertising and sharable social media content are run during fieldwork

• Surveys are also supported by more traditional approaches to response maximisation

QILT survey campaigns 14

QILT website

• QILT website informs student choice through provision of survey results by institution and study area.

The initial indicators include:

• SES – Overall Educational Experience item, plus five scale scores

• CEQ – Overall Satisfaction Indicator, Good Teaching Scale and Generic Skills Scale

• GDS – employment rates, further study rates, and median salaries

QILT survey campaigns 15

QILT Home Page

QILT survey campaigns 16

Comparison chart

QILT survey campaigns 17

Table view

QILT survey campaigns 18

QILT Facebook Home

QILT survey campaigns 19

QILT Facebook Winners

QILT survey campaigns 20

QILT sharable content

QILT survey campaigns 21

QILT sharable content (2)

• https://www.youtube.com/watch?v=uvHBBGPOzUg

• https://www.youtube.com/watch?v=LdFFKYHUqKc

QILT survey campaigns 22

QILT coverage

• AFR launch - http://www.afr.com/news/policy/education/for-the-first-time-new-qilt-website-tells-students-about-salaries-after-graduation-20150915-gjnczj

• Australian, league tables under fire - http://www.theaustralian.com.au/higher-education/qilt-website-league-tables-under-fire/story-e6frgcjx-1227569138980

• AFR, lesser known job outcomes - http://www.afr.com/leadership/careers/jobs/lesserknown-universities-achieve-best-job-outcomes-20151012-gk6x16

• Queensland Times, USQ winning - http://www.qt.com.au/news/uni-tops-employment-stats-for-graduates/2808553/

• Business news, WA grads - https://www.businessnews.com.au/article/WA-grads-record-second-highest-median-salaries

• PS News - http://www.psnews.com.au/tvic/473/news/qilt-weaves-quality-comparisons

• AFR, sandstone Unis - http://www.afr.com/leadership/oops-graduates-of-sandstone-unis-dont-earn-less-melbourne-institute-admits-20150916-gjodvr

SRC Board 23

QILT & social media

QILT survey campaigns 24

QILT & social media (2)

QILT survey campaigns 25

QILT & social media (3)

QILT survey campaigns 26

Survey response maximisation

• Rolling prize draw for each of the four weeks of fieldwork• Seven email reminders • Reminder letter or postcard (to coincide with the second

email reminder)• SMS reminder(s)• Institution specific reminders• Survey links in Learning Management Systems• Promotional materials – website tiles, PowerPoint slides for

lectures, handouts & posters

QILT survey campaigns 27

Survey response ‘cycle’

QILT survey campaigns 28

Initial outcomes

• Still early days and challenging to identify what works best for each survey (ie SES vs GOS)

• Difficult to disentangle which strategy is having the most impact

• Very different outcomes for individual institutions – different levels of student and staff engagement

• QILT website not launched early enough to support the 2015 SES

QILT survey campaigns 29

Initial outcomes (2)

• Online fieldwork completed in four weeks

• 79 participating institutions

• More than 140,000 responses

• 38% response rate, up from 30% in 2014, 29% in 2013, and 21% in 2012

• Increase in the amount of reportable data on the website but…still issues with representativeness

QILT survey campaigns 30

Where to for 2016?

• Same prize draw and reminder schedule• Postcard rather than letter reminders• Main QILT website live (and hopefully updated)• New & refreshed videos• Paid and free social media activity• Engagement with mainstream media• Campaign website to support more effective tracking of

social media strategies• Early work to develop institutional level communication

strategies targeting underperforming unis/NUHEIs and study areas

QILT survey campaigns 31