VSA & Assessment of Learning Outcomes

Sally Frazee, Temple UniversityCriss Gilbert, University of Minnesota-Twin

CitiesSusan Morgan – Appalachian State University

V S A ?Voluntary System of

Accountability

2007AASCU

Association of American State

Colleges & Universities

“College Portrait” of Undergraduate Education

APLUAssociation of Public

& Land-grant Universities

College Portrait

Source of basic, comparable information about public colleges and institutions

presented in a common format.

No rankings, no spin…just the

facts!

Designed to be a trustworthy

source of reliable data for

prospective students and their families.

College Portrait

Student Characteristics

Undergraduate Admissions

Costs of Attendance & Financial Aid

Undergraduate Success & Progress Rate

Campus Life Degrees, Areas of Study & Future Plans

Student Experiences

Student Learning Outcomes

• Measure improvement in student abilities in three broad areas

• General cognitive skills that cut across disciplines: critical thinking, analytical reasoning, and written communication

• VSA sponsored review of sixteen Outcomes Assessment instruments…3 were selected - CAAP - CLA - Proficiency Profile

• Pilot project – expectation that best practices will emerge as data are collected

Student Learning Outcomes

How Does This Work?

Institutional learning outcomes as students enter

and exit

Cross-sectional methodology

Random sample of freshmen & seniorsAssess within 4

years, & then administer

every 3 years

University High Stakes

_____ Student

Low Stakes

Outcomes Measures Approved by VSA

Sampling

Costs

Administration

Student recruitment

Student motivation

Use of data

•CAAP

•CLA

•Proficiency Profile

Proficiency Profile (formerly MAPP)

•ETS

• Integrated test of general education skills

•Paper-and-pencil or online delivery

•Proctored and unproctored administrations

Standard Form108 items 2 hours

Abbreviated Form36 items 40 minutes

No minimum of test takers to produce aggregate reports

Score Reports include• a total score and subscores and

proficiency classifications for individual students

• total scores, norm-referenced subscores and criterion-referenced scores at the group level

• an alphabetical roster with 17 individual student scores

Minimum of 50 test takers to produce aggregate reports

Score Reports include• a total score, but no subscores, for

individual students

• total scores, norm-referenced (scaled) subscores and criterion-referenced scores (proficiency classifications) at the group level.

Outcomes Measure Journey at Temple

•Signed on for VSA in 2007

•2007 CAAP Pilot & 2008 MAPP Pilot▫Opportunity to try out administration▫Opportunity to try out recruitment▫Honoraria to fund faculty workshop

•2009: Faculty Workshop: Looking at Learning

•2010: Pilot Administration of the Proficiency Profile, with no VSA obligation to publish results

Faculty WorkshopClick icon to add picture

Multiple Ways of Looking at Assessment(Periodic Program Review)

VSA – How and Why(Vice Provost, MARC)

Using Published Tests(Chair, Educational Psychology)

A Look at Professional Exams(MARC, Institutional Research)

Fall 2010 Administration

• GOAL: Abbreviated version to 200 freshmen within 6 weeks of the start of the semester

• PLAN:▫ Email to all 4,300 incoming freshmen on 9/16

▫ Lots of convenient test sessions

▫ $20.14 in Diamond Dollars

▫ Spread the word – in-class & dorm presentations posters and plasma screens newspaper articles information desk & giveaways

• Budget will be $15,000-$20,000 per year

VSA & Assessment of Learning Outcomes

Questions?