The Future of Student Learning Outcomes Assessment within the VSA

Embed Size (px)


The Future of Student Learning Outcomes Assessment within the VSA. Christine Keller VSA Executive Director Teri Lyn Hinds VSA Associate Director. North East Association for Institutional Research Bethesda, Maryland ▪ November 3-6, 2012. Presentation Outline. - PowerPoint PPT Presentation

Text of The Future of Student Learning Outcomes Assessment within the VSA

Transfer Student Presentation

The Future of Student Learning Outcomes Assessment within the VSANorth East Association for Institutional ResearchBethesda, Maryland November 3-6, 2012Christine KellerVSA Executive Director

Teri Lyn HindsVSA Associate DirectorSLO Workshop - Phoenix8/10/200911Presentation OutlineReview original goals of VSA, student learning outcomes pilot projectDiscuss selected findings, recommendations from NILOA evaluation of SLO PilotDescribe use of evaluation results: technical work group, communications advisory groupOutline next steps for VSA: wrap up pilot, new reporting options, refocus/rebranding

SLO Workshop - Baltimore7/27/20092

Voluntary System of Accountabilitythe undergraduate student experience through a common web report the College Portrait. Initiative by public universities to supply straightforward, comparable information on SLO Workshop - Phoenix8/10/20093BackgroundVSA developed, launched in 2007Sponsored by APLU and AASCU300+ participating universitiesThree original objectivesDemonstrate accountability, transparencyCollege search toolSupport innovation in the measurement and reporting of student learning outcomes

SLO Workshop - Baltimore7/27/20094Student Learning Outcomes PilotGoal: Directly measure, publicly report student learning gains (value-added) at institution level using a common methodSkills: critical thinking, analytic reasoning, problem solving, written communicationTests: CAAP, CLA, ETS Proficiency ProfileReporting includes link to institution specific learning outcomes dataPilot period ends December 2012SLO Workshop - Baltimore7/27/20095Evaluate Effectiveness of PilotConducted by National Institute for Learning Outcomes Assessment (NILOA)Focus groupsInterviewsSurveysGoogle AnalyticsCollege Portrait statisticsSLO Workshop - Baltimore7/27/20096

NILOAs mission is to document student learning outcomes assessment work, identify and disseminate best practices, and support institutions in their assessment efforts.

Surveys Web Scans Case Studies Focus Groups Occasional Papers Website Resources Newsletter Listserv Presentations Transparency Framework Featured Websites Accreditation Resources Assessment Event Calendar Assessment News Measuring Quality Inventory Policy Analysis Environmental Scan

7Findings IBoth participating and nonparticipating institutions agree that VSA in 2007 was a wise, timely, useful, necessary response to the accountability , transparency demands of the timeMany eligible institutions - about 1/3 - do not participate in the VSA50% of participating institutions have yet to post student learning outcomes informationThe student learning outcomes section of attracts little traffic

SLO Workshop - Baltimore7/27/20098Calculator and financial aid section most traffic 50%SLO second to last in pages visitedFindings IICollege Portrait: Information posted may not reflect the needs of prospective students, families or provide the information they seek to make decisions about where to attend college.Student Learning Outcomes: The standardized test measures of student learning outcomes lack broad credibility and acceptance in the higher education community, undermining institutional participation and engagement with the VSA and campus faculty and staff support of the VSA initiative.SLO Workshop - Baltimore7/27/20099RecommendationsMend it, dont end itExpand range of assessment tools and approachesFocus on specific audiences and communicating meaningful information SLO Workshop - Baltimore7/27/200910Working Group DeliberationsTechnical work group considered alternative measures of learning outcomes to recommend to VSA BoardConfirmed importance of student learning outcomes reporting within the VSANo perfect of measure of student learning exists for all audiencesExternal accountabilityInstitutional improvementCollege selectionSLO Workshop - Baltimore7/27/200911Working Group Deliberations IIContinue use of value-added measurement using CAAP, CLA, ETS Proficiency ProfileIntroduce option to use VALUE Rubrics based on AAC&U essential learning outcomeswritten communication critical thinkingIntroduce option to use aggregate scores from professional and graduate admissions exams such as the GRE, GMAT, LSAT, and MCATSLO Workshop - Baltimore7/27/200912Work Group Ratings of Recommended VSA Instruments in Key Areas Instrument1= lowest rating 10=highest rating CLA CAAP ETS PP(short form) ETS PP (long form) VALUE Rubrics GRE Allow comparisons, benchmarking across institutions? 6 8 9.5 9.5 6 7 Offer a representative sample? 7 7 7 7 7 2 Usefulness to faculty/students for learning improvement? inst-2class-7 5 2 2 9 3 Transparency of method, results 6 8 8 8 9 8 External accountability 8 7 7 7 5 5.5 Ease of administration on campus 5 6 7 6 4 9.9 Reasonable costs (time, resources) 5 7 7 6 startup -3 marginal- 8 9 Motivation for students to do well? 3 3 3 3 9 9.9 Likely interest to consumers? 2 2 2 2 5 6 Valid learning outcomes instrument for average student? 5.5 6 6 6 portfolios -9assign -7 3 SLO Workshop - Baltimore7/27/200913No perfect measureVSA Board DecisionsCollege Portrait re-focus from college selection tool to consumer information tool to demonstrate effectiveness of educational programs Instrument options:CAAP, CLA, ETS Proficiency Profile AAC&U VALUE rubrics critical thinking, written communication GRE General Test (currently under review)Reporting options: value-added, benchmarkingSLO Workshop - Baltimore7/27/200914Next Steps for SLO ReportingVSA participants describe, publish pilot project experiences using three part formRequired for institutions with no SLO results currently publishedStrongly encouraged for all institutionsData entry opens October 3, closes January 7New SLO reporting options released for 2012-13 data cycle (week of Jan 18, 2013)Deadline: Spring of 2015No data more than 3 years oldSLO Workshop - Baltimore7/27/200915Required Reporting for PilotReport overall experiences participating in the pilot whether 1 of the 3 value-added tests was administered or not. Provide information about an additional institutional assessment initiative on campus, including results from a recent administration and/or cycle. Disclose which of the newly approved SLO reporting options your campus is likely to consider for future VSA reporting.

SLO Workshop - Baltimore7/27/200916

Communication Advisory GroupIncludes individuals from public affairs, communications, government affairsDevelop outreach strategy to recruit new participants and connect with key audiencesIdentify and promote key elements on College Portrait: success and progress rate, cost of attendance, financial aid, future plansCreate streamlined view targeted at policy makers, government affairs professional

SLO Workshop - Baltimore7/27/200919The Heat is OnUnprecedented public challenges for higher education to be more accountable for results.Accompanying external demands for information about student and institutional performance are growing calls for institutions and accreditors to become more transparent about what they do and the results they achieve. (Ewell, 2010)SLO Workshop - Baltimore7/27/200920More InformationReports & Presentations

Christine Keller, VSA Executive Director

Teri Hinds, VSA Associate Director

SLO Workshop - Phoenix8/10/200921