Upload
rehan
View
29
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Measuring Up on College-Level Learning. Margaret Miller, Project Director September 2003. Measuring Up 2000. Learning in the States: Incomplete. [Add state map on incomplete]. Certification of individual students E.g., Texas ’ s TASP, Florida ’ s CLAST - PowerPoint PPT Presentation
Citation preview
Measuring Up on Measuring Up on College-Level College-Level
LearningLearning
Margaret Miller, Project Director
September 2003
Measuring Up 2000
Measuring Up 2000
Learning in the States: Incomplete
[Add state map on incomplete]
State Efforts to Measure Learning
(taxonomy: Peter Ewell, Change magazine)• Certification of individual students
– E.g., Texas’s TASP, Florida’s CLAST• Institutional assessment for
improvement– E.g., Tennessee's performance
measures– Missouri’s accountability program– Campus-based assessment
• Institutional assessment for accountability– E.g., S. Dakota and Arkansas
National Attention to College-Level Learning
• Pew’s Quality of Undergraduate Education and writing assessment projects
• American Association of Colleges and Universities’ general education assessment project
• Council on Higher Education Accreditation’s project on institutional effectiveness
• Secretary's Commission on Achieving Necessary Skills (SCANS) skills
• Equipped for the Future• National Skills Standards Board
Key Questions
What do the state’s college-educated citizens know and what can they do that contributes to the social good? What kind of educational capital do they represent?
and
Key Questions (cont.)
How well do the state’s public and private, two- and four-year colleges and universities collectively contribute to that capital? What do those whom they educate know, and what can they do?
Key Decisions
• Whose learning will we measure?• What learning will we measure?• How will we use the information?• What strategies will we pursue?
Whose Learning
The college-educated in the states
and
college students
• Whose learning• What learning• The policy uses for the information• Assessment strategies
What Learning
National Education Goal 6:
“By the year 2000, every adult American will be
literate and will possess the knowledge and skills
necessary to compete in a global economy and exercise
the rights and responsibilities of
citizenship”
• Whose learning• What learning• The policy uses for the information• Assessment strategies
What Learning (cont.)
National Goal 6, objective for college education:
“By the year 2000, every adult American will be
literate and will possess the knowledge and skills
necessary to compete in a global economy and exercise
the rights and responsibilities of
citizenship”
• Whose learning• What learning• The policy uses for the information• Assessment strategies
Policy Purposes
Higher education policyand
K-12 education + economic development +
adult literacy policy
• Whose learning• What learning• The policy uses for the information• Assessment strategies
Direct Strategies
• National Assessment of Adult Literacy
• Graduate-admissions and licensing exams
• General intellectual skills tests
• Whose learning• What learning• The policy uses for the information• Assessment strategies
National Assessment of Adult Literacy (NAAL)
concludes12/03 Disadvantages:• Labor-intensive,
expensive • Decadal federal
survey --timing• National sample
only, except in 6 states
• Not what colleges think they teach
Advantages:• Advanced literacy
levels of a good measure of educational capital
• Assesses general population
• Comparison group of non-college-educated
• Household survey – respondent motivation high
Existing Exams• Graduate-admissions exams
– Dental– Graduate Management– Graduate Record– Law School, – Medical College – Optometry– Pharmacy
• Licensing exams– Clinical Pathology– Dental Hygiene– Occupational Therapy– Physical Therapy– Physician Assistant– Nursing– Respiratory Therapy– Teaching
• Whose learning• What learning• The policy uses for the information• Assessment strategies
Existing Examsdata gathered by 03/04
Disadvantages:• Selection bias• Uneven coverage
by discipline• Variable (and
sometimes small) numbers of test- takers in each state
• Most in health professions
Advantages:• Established,
credible instruments
• Highly motivated test-takers
• Admissions tests assess general intellectual abilities
• Availability• Low cost
General Intellectual Skills Tests
administered fall 03
• WorkKeys to a sample of two-year students in each state– Applied Math– Locating Information– Reading for Information– Business Writing
• Collegiate Learning Assessment (CLA) to a sample of four-year students in each state
WorkKeys and CLA
Disadvantages:
• Institutional motivation
• Test-taker motivation
• Expense
Advantages:• Excellent tests
of general & functional intellectual skills
• Can impart useful information to student and school
Indirect MeasuresNSSE/CCSSE co-administered with
testsCRS summer through fall, 03
• National Survey of Student Engagement (NSSE)
• Community College Survey of Student Engagement (CCSSE)
• College Results Survey (CRS)
• Whose learning• What learning• The policy uses for the information• Assessment strategies
Surveys
Disadvantages:• Not direct
learning measures
• Not yet cross-correlated with direct measures
Advantages:• Excellent and
recently developed instruments
• Process measure could lead to improvement
• Both have face validity
• Respondent motivation good
Challenges
• Political instability in states: gubernatorial, SHEEO
• Personnel changes among key players
• Institutional skepticism• Faculty resistance• Data-collection hurdles• Test-taker motivation
General Timeline
• Measuring Up 2002: model tested with incomplete data from Kentucky
• 2002-2004: Five-state pilot to test assessment model: IL, KY, NV, OK, SC
• Measuring Up 2004: publish the results of the pilot
• Measuring Up 2006: if enough states adopt the model, grade states on learning
Reasons to Act
• It is the right thing to do.• We can determine how to
do it right.• This initiative will generate
information useful to states, institutions, and students.
• State-level analysis can promote collaborations to serve underachieving subpopulations or regions of the state.
• State resources can be effectively targeted.
http:///collegelevellearning.org