Measuring Student Learning Outcomes

VIKI P. KELCHNER, PH.D.UNIVERSITY OF CENTRAL FLORIDA

Why? Why is it important to measure student learning outcomes?

Why do we assess?

Implement strategies to respond to diverse needs To improve effectiveness by:

Measuring how students learn Measuring what students learn Develop new and diverse educational experiences Observe students mastery

Major Principles

Promotes collaboration Is dynamic and continuous Ensures quality education Focuses on learning

How, what and how well students learn Is integrated into our daily teaching and planning

Questions to Address

What should our students be learning? What are they actually learning?

What can we do to help our students learn?

What types of changes can we make to increase actual student

learning?(assignments, activities, materials)

Questions to Address

Are the teaching principles equally effective for diverse populations?

Are students gaining needed skills and values?

How do we improve our practice to address student success?

Can students apply to real world applications?

*Does this lead to employment?

Steps to Address Student Learning Outcomes

1. Identify student learning outcomes for your program 2. Determine practices used to achieve outcomes through

curriculum mapping 3. Determine appropriate methods of assessments 4. Gather evidence 5. “Connect the Dots”

Review & interrupt results

Recommend actions

Make changes

Measure effectiveness of changes

Good Educational Practices

Formative Assessments Intermittent Feedback that guides students

Summative Assessments Final analysis of students’ cumulative knowledge, skills

or attitudes Self-Reflection & Collaborative Priorities Create an environment for on-going campus/program

wide dialogue

Assessment is a Multi-Step Process1. Formulate a clear and succinct learning goal (or goals) for your

students.

2. Articulate those learning goals to your students.

3. Decide what your students should be able to do if they have met those learning goals.

4. Develop an assessment instrument (e.g., tests, essay, project) and a scoring rubric.

5. Administer the assessment instrument to your students.

6. Evaluate your students’ performance on the assessment instrument.

7. Assess your students’ mastery of the learning goals given their performance on the assessment instrument.

8. Reflect on why students did or did not master the learning goals, and develop strategies to help them be as or more successful in the future.

Top 4 Essential Skills to Succeed in Employment in Florida

National Association of Colleges and Employers (NACE, 2016)

NACE 8 Post-Secondary Education Competencies Framework

1. Career Management 2. Communications 3. Critical Thinking 4. Global Citizenship 5. Leadership 6. Professionalism 7. Teamwork 8. Technology

Ways to Measure Learning Outcomes

Methods of Assessments

Indirect Ask students to reflect on their learning rather then

demonstrate it Capture students’ perceptions of their learning and

the educational environment which supports learning Provides signs of student learning but does not

establish exactly what they are learning

Indirect Methods

Questionnaires/surveys Student

Alumni

Internship Sites

Employer

Parents/Guardians

Exit interviews Focus Groups (student/parent/internship supervisors) Job placement rates (graduation and post graduation) Course evaluations

Methods of Assessments

Direct Students display their knowledge and skills as they respond to

assessment instruments

Look at student work products or performances that demonstrate exact level of learning

Tangible, visible, self-explanatory evidence of what students have and have not learned

Direct Methods Program developed tests/test questions (NACE Standards) Rubrics for evaluating

Essays/Papers Exam questions Capstone projects (internship position presentation) Exhibits Performances/Presentations

Interview Role Plays Showcase Resume

Portfolio of student work

Standardized tests (what could this look like?) Certificate exams

Course Sequence & Assessments

Core Courses Develop of core courses all IES student will take 7 Courses

Strategies for Student Success

Career Planning I, II, & II

Personal Leadership

Intro to Communication

Internship x 5 (varying credit) Co-Curricular

Work Internship (on Campus)

Capstone Internship

Course Development What are the targeted learning outcomes? What are the skills students need for employment? Course Purpose Align Course Objectives with(NACE) Key attributes for Employability

Date Topics Reading/Study (to be completed before class)

Assignments due Course Objectives

NACE Standards

Week 1

Course Overview and Introduction

(in class activity: students will complete the “My Next Move” Interest Profiler)

Interest Profiler https://www.mynextmove.org/explore/ip (to be done in class)

Week 2

Defining Success and Recognizing Personal Responsibility

Pages 29-30, 43-47

1 1 & 2

Week 3

Getting to Know Yourself: Self-Talk, Self-Motivation, and Self-Management

(in class activity: students will create a Personal Calendar)

Pages 50-54 & 108-109

Journal Entry: “What is success and how will you reach it?”

6 9 & 10

Week 4

Analysis of Strengths and Opportunities

(in class activity: students will create a Personal Development Plan)

Pages 31-40

1 & 4 1 & 7

Instruments to Assess Student Learning

What does this look like within a course?

Date Topics Assignments due Course Objectives

NACE Standards

Week 1 Introductions & Expectations Review of Syllabus

Week 2 Group Internship SupervisionTime Management

Internship Site Contract Due 4, 6 6 & 7, 9 & 10

Week 3 Group Internship SupervisionWellness & Work Life Balance

Goal Statement 6 6 & 7

Week 4 Group Internship SupervisionCommunicating With Your Supervisors

Job Performance Presentation 1, 2, 4 1 & 2, 3 & 4, 6 & 7

Week 5 Group Internship SupervisionCommunicating With Your Coworkers

Wellness PlanWeekly Logbook Check

1, 2, 4 1 & 2, 3 & 4, 6 & 7

Week 6

Group Internship Supervision Job Performance Presentation 1, 7 1 & 2

Week 7 Group Internship SupervisionTaking Initiative

Site Supervisor Evaluation 4, 5 6 & 7, 8

Week 8 Group Internship SupervisionHow are meeting or not meeting our

goals

Job Performance PresentationMidpoint Review of Goal Statement

1 1 & 2

Week 9 Group Internship SupervisionProblem Solving

Job Performance Presentation 3 5

Week 10 Group Internship Supervision Job Performance PresentationWeekly Logbook Check

1 1 & 2

Week 11 Group Internship Supervision Job Performance Presentation 3 5

Week 12 Group Internship Supervision Job Performance Presentation 3 5

Week 13 Group Internship SupervisionExpectations of Terminating With Your

Internship Sites

Job Performance Presentation 2, 7 3 & 4, 5

Week 14 Group Internship SupervisionWriting Thank You Letters To Your

Internship Sites

Final Review of Goal Statement 2, 7 3 & 4, 5

Week 15 Internship Experience/ Final Class Final Site Supervisor EvaluationFinal Weekly Logbook Check

Satisfactory Academic Progress

What does this look like?

Attendance % Cumulative Grade Point Average Satisfactory Course Completion Pass rate 70% of all credit hours attempted NACE Employer Attributes

Use assessments (pre, mid, post)

Why is this important?

Align expected student learning outcomes Have Satisfactory Academic Progress standards Create a meaningful Program of Study that contains employment-

related credentials

Next Steps

Program Evaluation Tool Kit How do you evaluate program effectiveness?

Questions

Thank you for participating!

Please take a moment to share your feedback:

Follow the link in the chat box

or

Scan this QR code

ReferencesAllen, M.J. (2004). Assessing academic programs in higher education. Bolton, MA: Anker

Publishing. Allen, M.J. (2004). Initiating program assessment. The Department Chair, Spring 2004. Assessing student learning outcomes in the context of SACS re-accreditation standards.

Presentation to the Dean’s Council, The University of North Carolina at Chapel Hill; September 2, 2004.

Board of Directors, Council for Higher Education Accreditation. (September 2003). Statement of mutual responsibilities for student learning outcomes: Accreditation, institutions, and programs. Washington, DC: CHEA. www.chea.org.

Bresciani, M.J. (2006). Outcomes-based academic and co-curricular program review. Sterling, VA: Stylus.

Diamond, Robert M. (1998). Designing & assessing courses & curricula: A practical guide. 2nd edition. San Francisco: Jossey-Bass.

Huba, M.E. & Freed, J.E. (2000). Learner-centered assessment on college campuses: Shifting the focus from teaching to learning. Boston: Allyn and Bacon.

Lopez, C.L. (March 1996). Opportunities for improvement: Advice from consultant-evaluators on programs to assess student learning. Staff “White Paper”, North Central Association: Chicago, IL.

Maki, Peggy L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus.

Suskie, L. (2004). Assessing student learning: a common sense guide. Bolton, MA: Anker Publishing.