Creating Effective, Viable Plans to Assess Student Learning

Linda Suskie, Executive Associate DirectorMiddle States Commission on Higher Education

3624 Market Street, Philadelphia PA 19104Web: www.msche.org E-mail: LSuskie@msche.org

Pittsburgh Regional Assessment Workshops

September 7-8, 2005

Middle States Assessment Conference!

September 29-30, 2005Baltimore Marriott Inner Harbor at Camden YardsFeatured speakers

Ginny AndersonDoug EderPeter GrayJeff SeybertRandy SwingWendy Troxel

Visit www.msche.org & click on “Events”

This morning...

What is assessment?Four steps of program assessment Learning goalsCurricular alignmentThe assessment toolboxSetting targets for assessment results

What is Assessment of Student Learning?

Deciding what we want our students to learn

Making sure they learn it!

--Jane Wolfson, Director, Environmental Science & Studies Program, Towson University

The Teaching-Learning-Assessment Cycle

1. Learning Goals

4. Using Results

2. Learning Opportunities

3. Assessment

Learning Goals

What is a good learning goal?

Outcomes – what graduates should be able to DOObservable – action words

UnderstandClear – no fuzzy terms

Demonstrate (how?), critical thinking, communication skills

Skills and/or attitudes/valuesImportant - meet student/employer needs

Examples of Learning Goals• Chemistry: Design an experiment to test a hypothesis or

theory.

• Communication: Apply knowledge of media history and the principles of media criticism to understand and evaluate media messages.

• Finance: Identify financial problems, analyze their impact, and design solutions that will withstand critical examination.

• Theatre: Express through clear writing and analysis a unique point of view on theatre subjects.

• Environmental Science: Critically evaluate the effectiveness of agencies, organizations and programs addressing environmental problems.

More Examples

Demonstrate effective communication skills across disciplines.Demonstrate understanding of other cultures.Demonstrate ability to work effectively with computer components.

Time to Think!

Why do most colleges have a General Education requirement in the social sciences?

Why is it a good idea for college students to study the social sciences?

How should our students be different 5-10 years from now, because they completed this requirement?

Draft three learning goals for a General Education requirement in the social sciences.

Time to Think Again!

1. Draft three learning goals for one of your programs or Gen Ed requirements.

2. Critique & edit the goals of someone sitting next to you.

Clear? No fuzzy words?

Observable? Action words?

Outcomes?

Align Your Goals & Curriculum

Will every degree student, regardless of course choices, have ample opportunity to achieve each goal?

Aligning goals & curricula

Review syllabi for learning outcomes.All course syllabi must include the course’s learning outcomes!

Create a grid of program goals x courses.Use syllabi or have faculty check off goals in their courses.Look for underemphasized & “overkilled” goals.

Consider modifying syllabi.Include relevant program-level learning outcomes.Grid of assignments x learning outcomes

The Assessment Toolbox

Look for information already on hand.

TestsPapers, projects, performances

Especially from “capstones”Field experience supervisors’ evaluationsSurveys & self-ratings

Create embedded tools.

Useful and not too much work

Rubrics (rating scales/grading criteria)Test blueprintsReflective writing on attitudes & values

Useful but more time & work

Additional testsAdditional surveys Additional focus groupsAdditional portfolios

Understanding Assessment Results by Setting Appropriate Targets

Michael earned 65 points on the midterm.

Did he do well on the midterm?

To decide if Michael “did well,” we must compare his 65 against something else.

The “something else”depends on what we want the test to tell us.

Suppose 55 is passing and 70 is a perfect score.

Standards-based perspectiveaka competency-basedaka criterion-referenced

Question answered: Are my students meeting my standards?

Challenge: Establishing sound performance standards

Suppose the class average is 75.

Benchmarking perspectiveaka peer-referencedaka norm-referenced

Question answered:How do my students compare to peers?

Challenge:Identifying appropriate peers & collecting info from them

Suppose the university average is 75, but the average at South University is 95.

Best practice perspectiveaka best-in-class

Question answered:How do my students compare to the best of their peers?

Challenges:Commitment to improving teaching & learningIdentifying best-practice peers

Suppose Michael scored 35 a year ago.

Value-added perspectiveaka growth, change, improvement, pre-post

Question answered:Are my students improving?

Challenges:Imprecise assessments mask growth.Motivating students on pre-testIs growth due to us?

Suppose class average is 75 now and 40 three years ago.

Longitudinal perspectiveQuestion answered:

Is my teaching & curriculum improving?Challenge:

Using the same assessment

Suppose Michael is tone-deaf.

Capability perspectiveaka potential

Question answered:Are my students doing as well as they can?

Challenge:Determining potential

Recap: Six Questions that Assessments Can Answer

1. Are my students meeting my standards?

2. How do my students compare to their peers?

3. How do my students compare to the best of their peers?

4. Are my students improving?

5. Is my teaching & curriculum improving?

6. Are my students doing as well as they can?

Which perspective should you use?

Each has advantages and disadvantages.

Each gives a somewhat incomplete picture.

Multiple perspectives give the most balanced picture of student learning.