Embed Size (px)
Citation preview
Staying Ahead of the Curve
Best Practices for
Collecting Career Outcomes Data Using
Presenters
Matt Berndt
Vice President,
Career Services & Research
CSO Research, Inc.
Max Wartel
Head of Research & Analysis
CSO Research, Inc.
25+ combined years collecting and analyzing career outcomes data
for colleges and universities
3
Staying Ahead of the Curve
AGENDA
The College Scorecard
The Outcomes Survey®
Six Best Practices for
Outcomes Data
Collection
Q&A
Next Steps
How many
graduates get jobs
What kinds of jobs
they get
How much they earn
At One Year At Five Years
How many graduates get jobs?
What kinds of jobs do they get?
How much do they earn?
R&D: 2012-2013
O Monitored state and federal initiatives and NACE efforts
around first destination career outcomes data collection
O Reviewed/studied reporting standards and best practices
O Gathered input/feedback from career services professionals
nationwide
O Developed, tested and revised beta version of The Outcomes
Survey
O 16 colleges and universities joined on as Partners in
Innovation prior to launch
128+ Schools in 38 States
National Outcomes Report
O Data at 6-months and at One Year
O Blends anonymized TOS client response
data and Chegg response data
O Students at TOS client schools do not
receive Chegg survey
The only national career outcomes report based on a nationally
standardized data collection and reporting methodology.
National Data
CAREER OUTCOMES QUESTIONS* O Employment/Self-Employment/Entrepreneur
O Graduate School/Additional Education
O Military Service, Service Program
O Unemployment (seeking & non-seeking)
O When did you begin your search?
O Where did you look for an occupation?
ENGAGEMENT QUESTIONS (optional)
O Job search resources used . . .
O Use of career services . . .
O Helpfulness of services . . . .
EXPERIENTIAL EDUCATION (optional)
O Internships, Co-ops, Assistantships
O Field Experiences, Practicums
O Clinical Experiences, Research
O Service Engagement, Volunteering
O And more
SATISFACTION QUESTIONS (optional)
O Relation of your occupation to your major
O Importance of correlation to major
O Relation of your occupation to your career goals
O Initial satisfaction
MOTIVATION QUESTIONS (optional)
O Income
O Employment opportunity
O Prestige
O Personal enrichment
O Job satisfaction
O Ability to impact the world
CUSTOM QUESTIONS (optional & local)
O Unique and editable
O School-specific
O Limited in scope
*Exceed College Scorecard data. Address anticipated federal reporting needs. Meet MBA CSEA & NACE reporting standards.
Features
O Takes from 3-7 minutes to complete O Outcomes module only: avg. 8 questions
O All modules: avg. 20-30 questions
O Skip logic and conditional questions optimize the survey experience
O Responses updateable throughout the entire survey period
O Graduates are prompted four times to respond/update their responses
O Web-based and mobile-friendly
O Secure, SSL-based system for students and administrators
O Informed Consent Statement based on IRB best practice examples
O Real-time data collection and reporting
How it works
~2 Months Prior to
Graduation
University Provides Student
Data
1 Month Prior to
Graduation
Surveying begins
At Graduation
1st set of responses
3 Months After
Graduation
2nd set of responses
6 Months After
Graduation
3rd set of responses
One Year After
Graduation
Final set of responses
Conduct The Outcomes Survey®
Update Student Data National Report National Report Import Data
24/7 Admin Dashboard provides:
Real-time access to alumni database, data and reporting tools
Mass email tools, response tracking tools
Multiple administrators/permission levels (fall 2015)
Administrator data entry tool for 2nd and 3rd party sourced data
Downloadable reports and data files
O Response Data: 30-100% expected Key Variables: Quality of email addresses provided and historical alumni engagement/affinity; Client engagement in data collection process and use of TOS tools and resources
O TOS Tools and Resources O Request my Link Tool
O Mass and Direct Email Tools & Analytics
O Best Practice Marketing Resources
O Admin-Entry and Data Upload Tools
O Optional Call Center Support (at additional expense from CSO partner Campbell-Rinker)
O Knowledge Rate: >65% is achievable Primary Variable: Client engagement in 2nd and 3rd party data collection
Expectations
Six Best Practices for staying ahead of the curve
1) Apply sound research methodology to ensure valid
and reliable data collection
2) Integrate data collection into graduation activities
3) Involve key influencers in awareness-building and
marketing
4) Offer incentives while student are still on campus
5) Conduct targeted follow-up with non-responders
6) Integrate into overall communication and marketing
activities and platforms
Best Practice #1
Apply sound research
methodology to ensure valid
and reliable data collection
Gold Standard Methodology
Our Methodology Works!
O “Gold Badge” institutions obtained response rates
as high as 100% (average response rate: 52%)
O 89% of respondents who start TOS complete TOS
O 75% of respondents provide salary/compensation
information
O 23% of respondents provide updates after
graduation
O Institutions that survey in real time obtain better
results
Collect multiple data types
O Response Data
Collected in a standardized manner directly from respondents
O Valid, reliable and consistent
O Defensible by Institutional Research
O Secondary Source Data
Collected indirectly and in a non-standardized manner from
faculty, family, social media, and other 2nd and 3rd parties.
O General Information
O Anecdotal and typically incomplete
FEMALE MALE
GENDER
NOT
REPORTED
HISPANIC/
LATINO
NATIVE
AMERICANASIAN
BLACK/
AFRICAN
AMERICAN
NATIVE
HAWAIIAN
/ PACIFIC
ISLANDER
WHITE/
CAUCASIA
N
INTERNATI
ONAL
MULTIETH
NIC/
OTHER
NOT
LISTED
ETNICITY
NOT
REPORTED
POPULATION (N= 62,102) 47.0% 30.1% 22.9% 7.7% 0.7% 5.1% 5.6% 0.5% 48.4% 4.3% 6.4% 21.3%
SAMPLE (n = 12,687) 55.9% 27.2% 16.8% 8.5% 0.6% 3.6% 5.8% 0.5% 51.8% 3.3% 5.6% 20.4%
0%
10%
20%
30%
40%
50%
60%
CLASS OF 2014-2015 DEMOGRAPHICS OF RESPONDENTS AS OF GRADUATION
Analysis shows respondent demographics are representative of the student population. By gender, ꭓ2(df = 2) = 3.59, p > .10 By ethnicity, ꭓ2(df = 8) = 1.99, p > .975
Collect representative data
Best Practice #2
Integrate data collection into
graduation activities
Flagler College
Roger Williams University
Best Practice #3
Involve key influencers in
awareness-building and
marketing
Northeastern State University
Strategies
O Engaging a C-level champion (President,
Provost, Vice President, Dean, etc.)
O Involving key departments in survey design
and custom question development
O Sending letters/emails from department
heads and favorite faculty encouraging their
students to respond
O Generating department/major-specific
reports to demonstrate value
Best Practice #4
Offer incentives while
students are still on campus
O Tabling in advance of
graduation
O Coffee mug &
goodies giveaway
O Use of “Request My
Link” tool
Prize Drawings
Best Practice #5
Conduct targeted follow up
with non-responders
Mass Email Tool
O SJFC reported a 5-10% boost in response
rates for each interim follow up
O Mass mailing tools allow rapid follow up
with non-responders between scheduled
emails
O Follow up can be customized to for
different majors, degrees, departments,
and more
Targeted Follow up
Best Practice #6
Integrate into overall
communication and
marketing activities and
platforms
Message Integration
Email signatures
Including the “Request my Link” URL in email
signature blocks
Six Best Practices
1) Apply sound research methodology to ensure valid and reliable data collection
2) Integrate data collection into graduation activities
3) Involve key influencers in awareness-building and marketing
4) Offer incentives while student are still on campus
5) Conduct targeted follow-up with non-responders
6) Integrate into overall communication and marketing activities and platforms
Usage/Purpose
Accreditation
Admissions recruiting
Advising: Academic and career
Alumni relations
Curriculum development
Media inquiries
Reporting: Federal, State,
Association, Other
Strategic Planning
Reporting Intervals
Before Graduation
At Graduation
At 3 Months
At 6 Months
At One Year
3-, 5-, 10-Years+
Define Your School’s Needs & Priorities | Tell Your Graduates’ Stories
Collection Standards
Response Data
Secondary Source Data
IRB Approval/Sign-off
Questions
The National Standard for Career Outcomes Data
Collection
1-866-705-4201, ext. 3
