EOF Directors Professional Development Seminar Program Planning & Assessment EOF Directors...

EOF Directors’ Professional Development SeminarEOF Directors’ Professional Development Seminar“Program Planning & Assessment“Program Planning & Assessment””

June 13, 2013

Rider University

Creating and Using Rubrics to Assess Creating and Using Rubrics to Assess Program Level Student Learning Outcomes Program Level Student Learning Outcomes

Mildred A. Mihlon, Ph.D.Saint Peter’s University

Associate Vice President, Academic Affairs & Assessment

IntroductionsIntroductions

Who, What, Where &Level of Familiarity with

Assessment?

AssumptionsAssumptions

Effective program evaluation must be grounded in a solid assessment

plan that is inclusive of a clear vision/mission, relevant program goals and well-articulated student learning

outcomes that can be evaluated through direct and indirect measures against defined standards of success.

Writing Student Learning OutcomesWriting Student Learning Outcomes

Student Learning Outcomes…

Derived from the program mission and goals, student learning outcomes are

identified actions that a student is expected to demonstrate in terms of knowledge, skills, and attitudes upon

completion of a program, course, activity or process.

Student learning outcomes should be measurable, manageable, and

meaningful.

Writing Student Learning OutcomesWriting Student Learning Outcomes

Ask Yourself…

• What are the most important things a student should know, be able to do or demonstrate after completing my program or from utilizing my office/services?

• What does the end result look like? How will you know the students have learned what you want them to learn? How will you identify it? What will they be able to do, say, think, care about, or value after this experience.

Writing Student Learning Writing Student Learning OutcomesOutcomes

Transform these attributes into measureable, action-oriented

student learning outcome statements.Examples:

Students will… Identify critical campus resources required for a successful transition

to college life. Demonstrate engagement through participation in campus life

activities. Develop effective self-management skills. Employ the use of appropriate decision-making skills. Apply the appropriate use of study skills and time management

strategies.

Summer Bridge Programs Learning CommunitiesPeer Mentorships Service ActivitiesTransition Courses WorkshopsAcademic Advising Orientations

SLO 3

• Direct Measure• Indirect Measure

Measuring Student Learning OutcomesMeasuring Student Learning Outcomes

Indirect Measures = imply learning has occurred, but are not specific as to what has been learned; often subjective.

Direct Measures = tangible evidence that learning has occurred and to what extent through observed changes in skills, attitudes, values or knowledge base.

Middle States requires use of both!

Measuring Student Learning OutcomesMeasuring Student Learning Outcomes

Direct Measures Indirect Measures

Assignments/Exams Grades/GPAs

Papers/Essays Persistence/Graduation Rates

Projects Surveys (NSSE)

Portfolios Attendance

Student Reflections/Discussions Job/Internship Placements

Employers Ratings Interviews/Focus Groups

Capstone projects Scores on GREs/LSAT/etc…

Exhibits Honors/Awards

Presentations Participation Hours

Creating & Using RubricsCreating & Using Rubrics

Objectives:

Define RubricRubric FeaturesConstruct a RubricAggregate Rubric Data

Definition & Rubric PurposeDefinition & Rubric Purpose

A rubric is a scoring instrument for evaluating performance along a task-specific set of criteria rather than a single numeric score.

Purposes of rubrics:◦Describe what “outstanding” or “satisfactory” or

“inadequate” performance looks like◦Articulate the criteria against which student

learning is judged◦Provide an objective guide for assessment

Basic Rubric FeaturesBasic Rubric Features

A Rubric: Basic Features…• Stated Objective or Purpose - title• Scoring Criteria – characteristics of good performance on the task

• Levels of Performance – defined degrees of competency

• Descriptors –briefly define what is expected at each level of performance

Presentation Rubric http://www.rcampus.com/rubricshowc.cfm?code=Z79XC8&sp=true&Peer Evaluation: Group Work http://www.rcampus.com/rubricshowc.cfm?code=K78BBC&sp=true&Self Evaluation for Group Participation http://www.rcampus.com/rubricshowc.cfm?code=B78BAA&sp=true&Individual Portfolio http://www.rcampus.com/rubricshowc.cfm?code=K7369C&sp=yes&Blog http://www.rcampus.com/rubricshowc.cfm?code=N44XXC&sp=yes&Student Participation http://www.siue.edu/~deder/partrub.htmlTeam Work http://course1.winona.edu/shatfield/air/team%20work.pdfJournal http://course1.winona.edu/shatfield/air/reflectionrubric-1.pdf

Steps to Creating A RubricSteps to Creating A Rubric

1. Identify and define the assessment objective or purpose (title)

2. Select and write the needed number of scoring criteria (key attributes)

3. Select and write the desired levels of performance

4. Select and write the descriptors (features of scoring criteria)

Steps to Creating A RubricSteps to Creating A Rubric

1. Identify and define the assessment objective or purpose (title); statements which identify the specific knowledge, skill, or attitude the learner should gain and display as a result of the instructional activity.

Steps to Creating A RubricSteps to Creating A Rubric

2. Select and write the needed number of scoring criteria (key attributes); broad concepts or specific tasks the student should demonstrate when performing the activity.  

• Examples of dimensions for a group exercise analyzing a case study may include: • Contribute to the group discussion • Take responsibility for required work • Value others viewpoints • Analyze the study cooperatively• Present the outcome in a given format

Steps to Creating A RubricSteps to Creating A Rubric

3. Select and write the desired levels of performance; descriptive levels of quality starting with the worst quality up to the best quality.

--Example of a 3 level gradation: poor, average, excellent --Example of a 4 level gradation: beginning, developing, accomplished, exemplary--Example of a 5 level gradation: poor, fair, average, very good, excellent

Steps to Creating A RubricSteps to Creating A Rubric

4. Select and write the descriptors (features of scoring criteria); Start with the best quality of each dimension. Simply list the specific expectations you have for the student. Then, for each level below the best quality, identify the flaws or missing elements which will cause the student to lose points off the best quality performance.

Creating A RubricCreating A Rubric

Students are asked to write a 5-page reflection paper documenting their level of

college readiness following participation in summer

transition program.

Creating A RubricCreating A Rubric

Beginning 1 Developing 2 Accomplished 3 Exemplary 4

Writing Quality & Organization

Engagement

Study Habits

Resource Knowledge

Self-Efficacy

Creating A RubricCreating A Rubric

Group Activity:Develop a rubric to be used to

evaluate students’ leadership skills at the

conclusion of a peer mentorship program.

Aggregate Rubric DataAggregate Rubric Data

N = 75 Beginning 1 Developing 2 Accomplished 3 Exemplary 4

Writing Quality & Organization

15 (20%) 25 (34%) 30 (40%) 5 (6%)

Engagement 10 (13%) 15 (20%) 35 (47%) 15 (20%)

Study Habits 15 (20%) 15 (20%) 40 (29%) 5 (6%)

Resource Knowledge

5 (6%) 5 (6%) 60 (57%) 5 (6%)

Self-Efficacy 5 (6%) 10 (13%) 40 (54%) 20 (27%)

CONTACTCONTACTMildred A. Mihlon, Ph.D.

Associate Vice President, Academic Affairs & Assessmentmmihlon@saintpeters.edu