Embed Size (px)
Citation preview
Jocelyn Milner ([email protected]) Director of Academic Planning and Analysis
Sara Lazenby ([email protected]) Associate Policy and Planning Analyst
Mo Noonan Bischof ([email protected]) Assistant Provost
Aaron Brower ([email protected]) Vice Provost for Teaching and Learning
UW System President’s SummitApril 15, 2011
Essential Learning OutcomesAnd Post-Graduation Plans
ELOs and Post-Grad Plans
Why implement a survey?
Before 2008No institutional exit survey for bachelor’s degree recipients
Voluntary System of Accountability question on post-graduation plans
AAU Data Exchange Senior Exit Survey Higher Education Opportunity Act of 2008Essential Learning Outcomes (adopted in 2008)Campus leaders asked for it (AAC)
ELOs and Post-Grad Plans
Stage 1. Include the VSA question, AAUDE questionsCost-effective and sustainableIntegrated into the student recordPotential for very high response rateReporting considerations – InfoAccess
Stage 2. Need for indirect measure of ELOs
Criteria
ELOs and Post-Grad Plans
Have any of you solved this challenge?
How did you do it?
(5-7 minutes)
Many contributions to this project’s success:
Questionnaire DevelopmentVoluntary System of AccountabilityAAU Data Exchange Senior Exit Survey TeamNSSE John Ory, University of IllinoisUW-Madison Convergence TeamAACU’s LEAPUW-Madison Assessment FundJohn Stevenson, UW Survey CenterJennifer Dykema, UW Survey CenterJeremy Beach, UW Survey CenterStudent participants in focus groups
ELOs and Post-Grad Plans
Technical Development and ImplementationMichael Fay, DoITJames Helwig, DoITRob Lauer*, DoITWanjiru Pontes, DoITStephan Wahe, DoITBob Mayville, DoITBetty McIlwee, DoITJim Steele, Registrar’s OfficeKay Steiner, InfoAccess, DoIT
ELOs and Post-Grad Plans
Implementation *- Integrated into My UW student portal- Developer coded the questionnaire- Module is live from 10 days before the last day of the term until 20 days after the end of the term- Email notice goes to all graduating seniors on opening day- Module (annoyingly) stays live in student’s MyUW portal until student responds- Responses are saved in an ISIS/PeopleSoft bolt-on table- During the survey period, the data is extracted into an InfoAccess data warehouse table and refreshed nightly. This is available to anyone who also has access to the relevant tables
*Clever solution! Thanks to all who noodled this out!
Subsequent questions on - Work plans- Schooling plans- email address-“is there anything else you want to tell us?”
ELOs and Post-Grad Plans
Work Full-Time; 63%
Graduate School Full-Time; 22%
Work Part-Time; 10%
Other; 5%
Principal Activities after Graduation (Dec 2010)
http://www.provost.wisc.edu/content/WI_Exp_ELOs.pdf
ELOs and Post-Grad PlansUniversity of Illinois-Urbana Champaign, 2010 Senior Exit Survey Excerpt
University of Illinois-Urbana Champaign, 2010 Senior Exit Survey Excerpt
ELOs and Post-Grad Plans
Question Development -Indirect measures of learning, student perceptions- ELOs, UI-UC, NSSE were all influences-UW Survey Center did review and pre- testing, including layout and syntax (~2 months, ~$2,000)- Added questions to the post-graduation plans survey in December 2010 and into the future
ELOs and Post-Grad Plans
+.33
+.53
+.97
+.61
+.87
+.86
+.84
+.65
+.73
+.70
+.79
+1.34
+.86
+.63
+1.20
+1.04
+1.80
+.82
1 2 3 4 5
Communicate in a language other than English
Appreciate the arts such as literature, music, and fine arts
Understand cultures and societies outside of the United States
Contribute to the welfare of others
Speak effectively
Understand culture and society within the United States
Lead others effectively
Understand that science is relevant to everyday life
Work collaboratively in groups
Use computers and electronic technology
Empathize w/ind. differences based on culture, ethnicity, disability or sexual orientation
Apply knowledge and skills in real world settings
Write effectively
Develop a personal code of ethics and values
Find, organize and evaluate information from multiple sources
Draw conclusions after weighing evidence, facts, and ideas
Apply skills and knowledge of your chosen major(s)
Learn on your own
Essential Learning Outcomes: "How able were/are you to ..." When Entering At Graduation
Not at All A Little Somewhat Very Extremely
Note: Pilot semester; results are non-null mean of responses. December 2010 Graduates . (First full report available in October 2011). N=539 to 551, depending on question.
Gain
PIILOT RESULTS FOR ILLUSTATION ONLY – NOT FOR REFERENCE
ELOs and Post-Grad Plans
About using the data:- census fielding - fairly high response rates allows school/college and by major/program summaries- analysis delayed until October for May graduates
We involved a lot of creative people with a wide range of skills and experience. We took inspiration from our peers/competitors. We made use of our existing enterprise systems
ISIS/PeopleSoftMyUW student portalInfoAccess (data warehouse)
Census approach that has potential for high response rate.Cost-effective, sustainable collection and reporting system.
So far, it’s working!
