Upload
gaetano-lavery
View
25
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Measuring Our Success: Providing Evidence for CTE Outcomes. KC Greaney , Santa Rosa Junior College Mallory Newell, De Anza College Terrence Willett, RP Group. RP Conference, Kellogg West, CA April 2013. - PowerPoint PPT Presentation
Citation preview
KC Greaney, Santa Rosa Junior College
Mallory Newell, De Anza College
Terrence Willett, RP Group
Measuring Our Success: Providing Evidence for CTE Outcomes
RP Conference, Kellogg West, CAApril 2013
2Measuring Our Success, RP Conference, April 2013
A Brief Overview
- CTE SURVEY OUTCOMES
- CTE LAUNCHBOARD PROJECT
- CTE USERS FEEDBACK SURVEY
- CTE OS DATA VISUALIZATION TOOL
3
2012 Statewide Effort:CTE Employment Outcomes Survey
• Pioneered at Cabrillo College over past 12+ years• Replicated at other individual colleges including
Chaffey, Los Rios, Riverside, Mt. SAC, Santa Monica
• Statewide effort instigated by Bay Area Vocational Deans
Measuring Our Success, RP Conference, April 2013
4
CTE Employment Outcomes Survey
• Collaborative effort among community college consortia, the Chancellor’s Office, and the RP Group
• Dual purposes• Provide insightful information for local program improvement
• Collect statewide data for documentation of CTE outcomes
• Practitioner-Driven Effort• Statewide meetings, conference calls, collaboration
• All methodology, implementation and survey questions vetted
• Report format developed in response to practitioner needs
Measuring Our Success, RP Conference, April 2013
5
CTE Employment Outcomes Survey
Methodology• 2 Surveys: Completers, Leavers• 3 Modalities: e-mail, then US mail, then phone calls• Survey conducted approximately 1.5 years after
completing/leaving
Sample• 15 Colleges• All students included who met the criteria
• By 2009/10, had either earned a vocational award of 6+ units OR met VTEA “concentrator” definition AND did not enroll in 2010/11
• Total = 15,105 (7,681 Completers and 7,424 Leavers)Measuring Our Success, RP Conference, April 2013
6
CTE Employment Outcomes Survey
Data• Chancellor’s Office MIS data wherever possible• Local data supplement (student contact information)• Survey data• Complete data set provided to colleges for further analysis
Measuring Our Success, RP Conference, April 2013
7
CTE Employment Outcomes Survey
Response Rates• Overall, 30% (Completers: 34%, Leavers: 26%)• By College, overall response rate ranged from 19% to
37%
Reports• Statewide reports for Completers and Leavers• College-level reports for Completers and Leavers
Measuring Our Success, RP Conference, April 2013
8
Statewide Findings: CompletersThe big questions…
• Are our CTE students employable?• Do they find jobs in the field in which we trained them?
• Do they see an increase in income?
…in other words, was the program worth it?
Measuring Our Success, RP Conference, April 2013
9
Statewide Findings: Completers
• 34% Response rate• 94% Satisfaction rate with education and training• 30% transferred (25% pursuing related field)• 77% working in their field of study (or “close”)• 73% are interested in taking more classes or training in their field in the near future
• 45% successfully engaged in a job search, 14% did not (the remainder were not looking or were already employed)
Measuring Our Success, RP Conference, April 2013
10
Findings: Completers
79% of respondents are employed for pay
Measuring Our Success, RP Conference, April 2013
11
Findings: Completers
More students worked full-time after completing studies and training
Full time (40+ hours per week)
Part time (20-39 hours per week)
Part-time (less than 20 hours per week)
0% 10% 20% 30% 40% 50% 60% 70%
After Before
Measuring Our Success, RP Conference, April 2013
12
Findings: CompletersThe hourly wage of degree/certificate earners increased 39% from their hourly wage before their studies ($17.78) to their hourly wage after completing a degree or certificate ($24.79)
Measuring Our Success, RP Conference, April 2013
13
Statewide Findings: LeaversThe big questions…
• Are our CTE leavers “losers”? Or have we failed them?• Are leavers employable?• Do they find jobs in the field in which we trained them?• Do they see an increase in income?
…in other words, were the courses and training worth it, even if the student did not complete a program of study?
Measuring Our Success, RP Conference, April 2013
14
Statewide Findings: LeaversSimilar to Completers, except:
• Lower Response Rate (26% versus 34% for Completers)• Fewer with the goal of earning a certificate or degree (60%
versus 80% for completers)• Twice as many with the goal of upgrading job skills and/or
renewing a license/permit (19% versus 10% for Completers)• Slightly higher transfer rate (34% versus 30% for Completers)• Slightly fewer report working in their field of study (or “close” –
70% versus 77% for Completers)• Earnings increase is lower (26% increase versus 39% for
Completers)
Measuring Our Success, RP Conference, April 2013
15
Basic DemographicsCompleters Leavers All CCCs
Gender: Female 60% 51% 54%
Asian 18% 17% 11%
Black 6% 5% 7%
Latino 19% 20% 30%
White 45% 47% 32%
DSPS 8% 4% ~
ESL 7% 5% ~
Acad. Disadvantaged 47% 40% ~
Migrant Worker 8% 6% ~
n=7,681 n=7,424 n=2.8 million
Measuring Our Success, RP Conference, April 2013
16
“Swirl”
Completers Leavers
Attended multiple colleges within the same district
13.5% 14.4%
Attended multiple districts 40.6% 39.7%
Attended only one college 44.6% 44.8%
Measuring Our Success, RP Conference, April 2013
17
CTE Survey: Changes in 2013• Implement all the lessons we learned from the pilot year
• 35 colleges participating• Move away from “Completers” versus “Leavers”• Move towards “CTE Skill Builders”• Broaden cohort to students who completed 9+ units of SAM-coded vocational courses (to include those who completed a certificate or vocational degree)
• Tighter timeline (reports out by the end of June)• Add questions re: industry certification, impact of coursework, more responses taking transfer into account
Measuring Our Success, RP Conference, April 2013
18
CTE Survey: Changes in 2014
• Santa Rosa Junior College will continue to administer survey, working directly with interested colleges
• Chancellor’s Office endorses survey and will continue to define cohort via MIS and provide basic data
• RP Group is supportive, but no longer directly involved• CCCAOE continues to support survey
• At Spring 2013 annual meeting, the CCCAOE “Excellence in Collaboration” award given to RP Group (Kathy Booth), Bay Area Vocational Deans Group (Rock Pfoetenhauer and Kit O’Dougherty) and Santa Rosa Junior College (KC Greaney)
• CalPASS+ will host CTEOS survey data in future
Measuring Our Success, RP Conference, April 2013
20
CTE User Feedback Survey• A link to the survey was emailed to all college contacts
who participated in the survey in the 2011-12 academic year
• Received 7 valid responses• Respondents suggested that the results had largely
been used to inform decision making in the areas of:
• Enrollment management
• Accreditation
• Perkins or other reporting
• Program improvements
• Changes to program degree offerings
Measuring Our Success, RP Conference, April 2013
21
CTE User Feedback SurveyAdditional Highlights
• Respondents found the following to be most valuable about the survey: • Ease of administration by the RP Group
• The multiple ways students were contacted to increase response rates
• The overall findings from the survey from my college
• The model report that was provided
• Respondents stated the following technical support would have been beneficial: • Assistance with the initial data pull of students
• Assistance with analyzing the results
• Examples of how other colleges shared the results and with whom
Measuring Our Success, RP Conference, April 2013
23
CTE Launchboard• 4 Community Colleges participating in the pilot• Includes data elements from “original” Launchboard
Metrics (OLB) and metrics from the “Common Metrics” (CMLB) associated with the Doing What Matters Framework: http://doingwhatmatters.cccco.edu/.
• Currently includes 67 data elements• All metrics are presented by program TOPS code • Some metrics presented by region as defined in the
Doing What Matters Framework• Includes data from: COMIS, CalPASS Plus, EDD,
Centers for Excellence, and data files submitted by colleges
Measuring Our Success, RP Conference, April 2013
24
CTE LaunchboardIncludes measures in the following areas:• Alignment – middle school to high school and high
school to CCC• Enrollments – course completion in high school and
CCC in CTE pathway• Capacity – FTEF, students served in CTE programs by
CCC, students served in similar programs in non CCC colleges
• Support – completed work readiness program, internship, took basic skills course, received financial aid, DSPS or EOPS
• Return on Investment – number of years of income to equal cost per students
Measuring Our Success, RP Conference, April 2013
25
CTE LaunchboardMilestones• Completed major requirements in a CTE area
(Primary Goal) (CMLB)• Earned more than 8 units in the same TOP code
within 3 years of first enrollment (OLB)• Term-to-term retention (OLB)• Completed college-level English or math within 3
years of first enrollment (OLB)• Course completion and success (OLB)
Measuring Our Success, RP Conference, April 2013
26
CTE LaunchboardCredentials• Industry-recognized certificates (OLB)• Non-credit certificates (OLB)• Chancellor’s Office approved credit certificates in
and not in CTE pathway (OLB)• Associate’s degree in and not in CTE pathway
(OLB)• Transfer in and not in CTE pathway (CMLB)
Measuring Our Success, RP Conference, April 2013
27
CTE LaunchboardEmployment:• Students are employed in same or similar field
(OLB)• Wage gain in field (OLB)• Wage gain for students earning 9+ units (OLB)• Average wage of students who are employed in
same or similar field (OLB) • Job retention (OLB)
Measuring Our Success, RP Conference, April 2013
28
CTE LaunchboardDemand:• Recent demand in Doing What Matters
Economic Region (OLB)• Projected demand in Doing What Matters
Economic Region (OLB)• Present demand in Doing What Matters
Economic Region (OLB)• Median (50th percentile) Hourly Earnings (OLB)
Measuring Our Success, RP Conference, April 2013
29
CTE Launchboard
Live look at the Launchboard:
http://www.calpass.org/
Measuring Our Success, RP Conference, April 2013
30
• Proposed• Help users access CTE Outcomes
Survey data with web based reporting tool
Measuring Our Success, RP Conference, April 2013
CTE OS Data Visualization Tool
32
Find Out More
KC Greaney, Santa Rosa Junior [email protected]
Mallory Newell, De Anza [email protected]
Terrence Willett, RP [email protected](831) 461-4586
http://www.rpgroup.org/projects/CTE-Employment-Outcomes-Survey