Articulating and Mapping Outcomes in Student Affairs

Articulating and Mapping Outcomes in Student Affairs

Dr. Marsha WatsonDirector of Assessment

Dr. Kenny RoyalAssistant Director of Measurement & Analysis

Dr. Julie JohnsonAssessment Specialist

The LEARNING Initiative Dual Track Implementation Strategy

Completion Dates Not actively engaged in program level assessment

Actively engaged in program level assessment

Sept 2009 Program level student learning outcomes revised and/or updated

Dec 2009 Assessment strategy in place

Jan-Mar 2010 Assessment strategy implemented

April 2010 Assessment results available for faculty reflection and action

May 2010 First cycle completed and improvement plans submitted

At least one cycle completed and improvement plans submitted

September 2010 First annual LEARNING Improvement awards announced

May 2011 Two cycles completed At least two cycles completed

August 2011 SACS Compliance Audit begins

September 2011 Second annual LEARNING Improvement awards announced

Academic units articulate and assess student learning outcomes aligned with the institution’s academic expectations

Where are the institution’s other expectations – like “tolerance,” “teamwork,” and “the ability to function in an increasingly diverse world” taught and assessed?Is “taught” even the right word?

Institutional Expectations

Substitute “learned” for “taught” – and the question becomes: Where are these other outcomes learned and assessed?

This shifts our perspective to:Student demonstration emphasis instead of

topic-coveringAbility to do or apply supersedes knowingShared responsibility for learningSite of learning becomes less specific;

boundaries become more porous

Shifting Perspectives

Learning is not exclusively classroom-basedMany important outcomes are:

Not learned in the classroom The result of processes outside the classroom

“Learning” is a process based on three interdependent student experiences:Understanding academic content and

processesStudent developmentIdentity formation

After Learning Reconsidered

Developmental ParadigmsPiaget: 4 stages of cognitive developmentPerry’s 9 stages of cognitive development

http://www.uky.edu/TASC/ED/perry.phpKegan: 6 stages of psychological

developmentKohlberg: 6 stages of moral development

Expands on Piaget’s two-stage theory of moral thought

Responsibility for learning:Exists outside the classroomDoesn’t always take the same form

Some entities on campus produce learning, some facilitate learning, some support learningThus, responsibility for assessing learning

exists outside the classroom as well

After Learning Reconsidered

Critical thinkingWorking with othersGlobal competenceCommunicationAdaptabilityPreparation for life-long learningEthical behavior

Learning in Non-Classroom Settings

The cognitive and/or affective competencies we intend students to be able to acquire with what they have learned

Achievement of outcomes indicates institutional and departmental effectivenessExamples of co-curricular learning outcomes

Co-Curricular Learning Outcomes

Co-Curricular Learning OutcomesMust be:

IntentionalCarefully planned and designedCoordinatedCarefully implementedPart of the structure of a student’s experienceAssessed

Co-Curricular AssessmentThe “Effectiveness Model”

Focus on student performance as a result of – or even within – a planned, structured co-curricular activity

Can be based on observation or objective measures

Requires carefully designed and consistent measuring practices

Formulate Statements ofIntended Learning

Outcomes

Develop or Select Assessment Measures

Create Experiences Leading

To Outcomes

Discuss and Use Assessment Results to

Improve LearningFormative-Based

Revisions

Activity #1 (15 minutes)The Evidence Inventory

Working in groups, identify direct and indirect student learning data you are already collecting

Report back

Questions to Ponder as You PlanWhat action/experience are you assessing?What specific outcome is it meant to develop?How are you assessing that development?What specific information about that

outcome’s development do you want to know?What will you do with the information you

gather?

Co-Curricular Assessment Example:

Group discussion of USC mini-case study(10 minutes)

15 Minute Break!!

Design Backward

Deliver Forward

IntendedLearning

Outcomes of theInstitution

IntendedLearning

Outcomes of theStudent Affairs

Division

IntendedLearning

Outcomes of theDepartment

IntendedLearning

Outcomes of theProgram

IntendedLearning

Outcomes of the Activity

Revised: Huba, M.E. and Freed, J.E. (2000).  Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning (p. 108).   Allyn & Bacon, Needham Heights, MA.

Activity # 2: Dr. JohnsonReview: Handout #4 (Design

Backward/Deliver Forward)Review University of Kentucky’s

Mission/VisionReview Values/Outcomes of Student AffairsReview New Student and Parent ProgramsReview K-Week Discuss Learning Outcomes and K-Week

Activity Mapping

Planning for the Long TermMapping and Coordinating

If the learning outcome is important, a single exposure isn’t enough

Map activities/experiences to outcomes, from first year to last year Determine how first-year experiences are different

from last-year ones What difference is expected in student response? Plan to assure student’s development of outcomes from

the first year to the lastConsider how co-curricular activities might be

coordinated with classroom instruction

Outcomes Activity or Learning Experience #1

Activity or Learning Experience #2

Activity or Learning Experience #3

Activity or Learning Experience #4

Activity or Learning Experience #5

Outcome 1I R E R

Outcome 2R R E

Outcome 3I E R

Outcome 4

Outcome 5E R R

Basic Mapping Template

I = Outcome is introducedR = Outcome is reinforcedE = Outcome is emphasized

Regular (Periodic) AssessmentOne-shot assessment produces haphazard

results that are usually insufficient for planning improvement

Tie assessments to logical stages of development, based on an outcomes map

Be consistent in approach to assessingOptions:

Standardized instruments Self-generated tools

Questions to Ask about EvidenceIs it relevant to the area’s stated mission and

function?Does it measure what we want it to measure?Does it deal in some way with outcomes?Is the information derived useful?Can the information be used to improve

either function or learning?

Developing Measures of EffectivenessIntentional Planning

Determine areas of responsibility: what office/function might be a logical place to contribute to particular learning outcomes?

Plan the outcomes-based purpose of the activity

Design non-passive activities Ex: Watching a film plus discussion; International

Days as more than food, costumes, dance …Develop/design outcome-focused opportunities

for processing

Surveys and QuestionairesLimitations:

Self-reporting, unvalidated opinionResponse rates“Opportunistic” dataSkewed samples

Surveys and QuestionairesMake them tools to (indirectly) assess

learningUse learning outcomes as basis for at least

some questionsValidate by cross-referencing outcomes with

different populations employers, alumni, graduates, current students, etc.

Emphasize the learning outcomes in design and analysis

Self-Generated ToolsObservationsExpert judgmentsStudent self-reflectionPeer assessmentsGroup/team discussionEmployer/supervisor judgments

University-Wide AssessmentIdeally, should use the same rubrics or

measures all who use them should have the same

understanding of its elementsWhen using the same tools is not possible, it

is essential that there be a way to extract information that is usable at the institutional level while still serving the needs of the functional area

Activity #3 Refer to Handout #8 (Activity #3 Worksheet)

Working in groups, revise/refine two co-curricular learning outcomes

Identify appropriate: student level (freshman, sophomore, etc) stage of cognitive development

Design a measurement(s) for each outcomeReport back the large groupMap the refined outcomes to learning

activities/experiences Handout #9

One last thing …Please complete the workshop evaluation

Thanks!