Josipa Roksa

Academic SymposiumUniversity of Virginia

April 14, 2011

*This presentation is based on Academically Adrift by Richard Arum and Josipa Roksa. We thank the Carnegie Corporation of New York and the Lumina, Ford and Teagle Foundations for their generous financial support and the Council for Aid to Education for collaboration and assistance with data collection.

Facilitating Development of Critical Thinking, Analytical Reasoning, and

Writing on College Campuses

Source (University of Chicago Press, January 2011):

Longitudinal Design◦ Fall 2005, Spring 2007, Spring 2009, Spring 2010,

Spring 2011 (planned) Large Scale

◦ 2005-2007: 24 diverse four-year institutions; 2,341 students

◦ 2005-2009: 29 diverse four-year institutions, 1,666 students

Breadth of Information◦ Family background and high school information, college experiences and contexts, college

transcripts◦ Collegiate Learning Assessment (CLA)

Determinants of College Learning Dataset

Dimensions of learning assessed◦ critical thinking, complex reasoning, and written

communication Distinguishing characteristics

◦ Direct measures (as opposed to student reports)◦ NOT multiple choice◦ Holistic assessment based on open-ended

prompts representing “real-world” scenarios Used in other contexts

◦ One of the measures of learning used by VSA◦ Will be utilized in 2016 by OECD-AHELO project

Collegiate Learning Assessment (CLA)

Performance Task (example)Jamie Eager is a candidate who is opposing Pat Stone for reelection. Eager critiques the mayor’s solution to reducing crime by increasing the number of police officers. Eager proposes the city support a drug education program for addicts because, according to Eager, addicts are the major source of the city’s crime problem.

Performance Task (example, cont.) Students are provided with a set of materials (e.g.

newspaper articles, crime and drug statistics, research briefs, internal administrative memos, etc.) and asked to prepare a memo that addresses several issues, including a) evaluate the validity of Eager’s proposal and b) assess the validity of Eager’s criticism of the mayor’s plan to increase the number of officers.

http://www.collegiatelearningassessment.org/

CLA Performance: Faculty Expectations and Reading/Writing Requirements

Note: Predicting 2007 CLA scores while controlling for 2005 CLA scores, student characteristics, and institutions attended.

CLA Performance: Studying and Fraternities/Sororities

Note: Predicting 2007 CLA scores while controlling for 2005 CLA scores, student characteristics, and institutions attended.

0 5 10 15 201080

1100

1120

1140

1160

1180

studying alone studying with peersfraternity/sorority

Institutional Variation

23 percent of CLA growth between 2005 and 2009 occurs across institutions

23%

College Selectivity and CLA Performance

1080

1120

1160

1200

1240

1280

2005 2007 2009

High selectivity Low selectivity

Note: Based on a 3-level HLM model, controlling for a range of demographic/family characteristics.

CLA Performance: College Major

Note: Predicting 2009 CLA scores while controlling for 2005 CLA scores.

Business

Education/Social Work

Engineering/Computer Science

CommunicationsHealth

Social Science/Humanities

Science/math Other 1140

1160

1180

1200

1220

1240

1260

Summary of Findings

Importance of academic rigor, including course requirements, faculty expectations, and time on task (i.e., time spent studying alone).

There is substantial inequality in experiences and outcomes across institutions associated with college selectivity.

There is notable inequality in experiences and outcomes within institutions associated with students’ academic experiences, including college major.

http://highered.ssrc.org/

Josipa Roksajroksa@virginia.edu