Upload
carmel-west
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Developing and Assessing Outcomes in the Co-Curricular Experience
Prepared for Auburn University’s Division of Student Affairs A-Team
April 2, 2013
Sandi Osters, Ph.D. Retired Director, Student Life Studies
Texas A&M University3001 Aztec Street
College Station, TX [email protected]
Workshop Overview
Student integrative learning Outcomes-based assessment Learning outcomes – the heart of
the matter Working sessions Assessment methods Working sessions Reflective dialogue
Today’s Program Outcomes
The presenter will: Work on issues important to participants Provide a general understanding about the
importance of outcomes based assessment in student affairs
Emphasize the importance of alignment with division and university mission, goals and planning
Emphasize the use of data to inform practice
Today’s Learning Outcomes
Participants will: Describe learning and program
outcomes for their own program, office or activity
Employ assessment methods for their outcomes
Develop initial action steps Articulate what they have learned and
what they still need to know
Handouts
PowerPoint Writing Outcomes Assessment Plan Template Rubrics Everything You Ever Wanted to
Know about Assessment – on One Page
It’s All About Students’ Integrative Learning A strong emphasis on student learning is
the primary key to retention through graduation
Learning is the primary activity and goal of the college environment – both inside and outside of the classroom
The integration of academic programs and co-curricular activities is important
The ability of students to integrate their learning both inside and outside of the classroom – be responsible for their own learning – is of the greatest importance
What Is Student Learning in the Co-curricular?
“Learning Reconsidered defines learning as a comprehensive, holistic, transformative activity that integrates academic learning and student development, processes that have often been considered separate, and even independent of each other.”“Student learning produces both educational and developmental outcomes; distinguishing them is pointless and potentially harmful, and the goal of higher education should be the integration of all domains of learning and of the work of all educators.”
Keeling, R. P. (Ed.) (2004). Learning Reconsidered. Washington, D.C.: NASPA and ACPA.
What is Assessment and Evaluation?
“Assessment is any effort to gather, analyze, and interpret evidence which describes institutional, departmental, divisional or agency effectiveness.”
“Evaluation is any effort to use assessment evidence to improve institutional, departmental, divisional, or agency effectiveness.”
Upcraft, M. & Schuh, J. (1996). Assessment in student affairs: A guide for practitioners. San Francisco: Jossey-Bass.
The Purpose of Outcomes Based Assessment To answer some simple questions:
What is it that we do? Why do we do it? How well do we do it? How do we know that we do it well? What improvements and changes do we make from
what we learn? Do the improvements work?
To genuinely engage faculty, staff and students in the day-to-day reflection of answering these simple questions
To demonstrate a commitment to systematic examination of the quality of all we do to improve our programs and departments
The Purpose of Outcomes Based Assessment, continued To celebrate our successes and learn from our
failures To inform our policy, programs, processes and
resource allocations Increase our confidence that we are putting our time
and energy into activities that results in the outcomes we and our students value
To have the data to support why our programs generate the learning Auburn wants for its students in times of financial downsizing
To be able to articulate exactly how student affairs professionals contribute to student learning and student development
To gather and display data that will allow us to satisfy accrediting agencies and program reviews (CAS or your own)
Advantages of Using Outcomes Language in Student Affairs
Make it clear to students who participate in our programs, activities and services what they can expect to gain from doing so
Make it clear to others what the program will accomplish and, where appropriate, what students will learn
Help staff select appropriate strategies and assessment measures/methods to reach the outcomes
Advantages of Using Outcomes Language in Student Affairs
Apply the results of our assessments for meaningful improvement and/or change
Move beyond student satisfaction and tracking the use of services as the sole means of describing student affairs effectiveness
Connect Student Affairs to the academic enterprise – we all are about student learning – we all are educators
What Any Assessment is NOT Assessment does not exist for
assessment’s sake It informs but does not drive where you
want to go and what you want to be as an office, activity, program
It is not used for personnel evaluation It is not a one time thing – but
systematic and iterative It does not assess all things, all the time
Outcomes Based Assessment and Decision Making
Most importantly, Outcomes based decision making and assessment should be
Understood = by faculty/professionals and students Meaningful = faculty/professional (i.e., expert) driven Inclusive = involve as many faculty, student affairs
professionals and students as possible Manageable = takes into account varying resources Flexible = takes into account assessment learning curves Truth-seeking/objective/ethical Iterative and systematic
Inform decisions for continuous improvement or provides evidence of proof
Promote a culture of accountability, of learning, and of improvement
Bresciani, J. J., Zelna, C. L., & Anderson, J. A. (2004). Techniques for assessing student learning and development: A handbook for practitioners. Washington, D.C.: NASPA, Inc.
Mission/Purpose
Goals
Objectives/Outcomes
Implement Methods to Deliver
Outcomes and Methods to Gather
Data
Gather Data
Interpret Evidence
Make decisions to improve programs; enhance student learning and development;
inform institutional decision-making, planning,
budgeting, policy, public accountability
Politics
Scho
lars
hi
p Accountability
Environment
Assessment Cycle
Budget
Writing Outcomes Outcomes – Detailed, specific, measurable or
identifiable, and personally meaningful statements that are derived from goals and articulate what the end result of a unit, program, course, activity, or process is. Learning Outcomes – An easily identified
action that a student is expected to demonstrate in terms of knowledge, skills, and attitudes upon completion of a program/activity/event.
Program/process outcomes – The end result of what a program or process is to do, achieve, or accomplish.
Write simply and with one outcome per statement
Learning Outcomes? Students will be satisfied with the
information received. Students will rate the service received
positively. Students will create their own personal
leadership philosophy based on three models taught in Lead 101 in the fall semester
After the service project, students will be able to articulate the root cause of homelessness in Auburn, Alabama.
Disability Services Example Learning outcome
Students will be able to describe their functional limitations in an academic setting
Assessment method Pre-test/post-test (intake form) Rubric
Criteria for success/achievement target Students post-scores will improve over pre-
test
Disability Services Example Learning outcome
Students will be able to describe how their disability may affect them in the workplace and what accommodations they may need to ask for from their employers
Assessment method Exit interviews with graduating students
Criteria for success/achievement target 80% of students in their graduating
semester will be able to identify at least one way their disability might effect them in the workplace.
Program/Process Outcomes? Students will like the conference banquet The professional development sub-
committee will offer two dining etiquette programs in the fall semester for graduating students
Develop and expand the Student Health Services web page in order to increase student access to health information, patient services and educational programming
Examples of Program Outcomes
The Department of Residence Life will be able to respond to maintenance requests within 24 hours of their walk-in time
The Student Health Service will be able to admit students for check-ups within 24 hours of their walk-in time.
The Financial Aid Office will have full award letters out to the on-time complete applicants by April 15th.
Program Outcome Example Program outcome
Develop and expand the Student Health Services webpage in order to increase student access to health information, patient services, educational programming
Assessment methods Data retrieval from webpage interactions Annual patient survey to determine student
utilization of the web page Survey instruments by Health Education program to
determine student utilization of web page Criteria for success/achievement target
Increase in the utilization of the SHS webpage regarding health information, patient services and educational programming.
Common Mistakes in Writing Learning Outcomes The learning outcomes don’t align with department,
division or university goals Outcomes include words that are hard or impossible to
measure (understand, appreciate, know about, become familiar with, learn about, become aware of)
Outcomes include too many skills in one statement Outcomes measure satisfaction or performance
evaluation rather than learning of the student There are too many learning outcomes Only one person wrote, reviewed, edited and
implemented the outcome
Working Session Work in small groups of two or three Develop at least two learning outcomes and
two program outcomes for the Division of Student Affairs Global Service Initiative (mission provided)
Don’t worry about being an expert on this initiative. Pretend you are. What’s important is working on the outcomes
Share time
Assessment Methods
Qualitative and quantitative One time and longitudinal Direct - requires students to display
their knowledge, behavior or thought processes
Indirect – asks students to reflect upon their knowledge, behaviors or though processes
Local (Auburn developed) and national assessments (like the NSSE or CIRP)
Direct Measures Benefits of direct measures to Student Affairs
Provides faculty and administrators more confidence in the measures and their results because it is the academic standard
Helps with accreditors who are looking for student learning in the co-curricular and measures of same
Challenges Time and expertise needed to develop Assessing the “fuzzies” – attitudes and values Small “n’s” and the validity of the studies
Bottom line Use multiple indirect measures to offset the lack of
direct measures
Common (and still viable) Assessment Measures Surveys to inform of needs and satisfaction
Surveys to assess outcomes Focus groups/interviews to inform future assessments or to dig more deeply into existing survey results
Tracking use and participation (numbers)
New Wave of Assessment Measures
Exit Interviews Learning contracts Photography Pre and post tests of knowledge Observations with documentation One minute papers Rubrics Reflective journals Reflective conversations/writing Combination of the above
Working Session Working individually, develop at least two
learning outcomes for a program or activity of interest to you. If you do not work directly with student learning, consider learning outcomes for your student workers
Develop at least one assessment measure for each outcome and address how you will know if you have been successful (criteria for success/achievement target)
Work in small groups and share your outcomes and measures.
As a group, develop an action plan (see Assessment Plan Template, Implementation of Assessment Process) for one of the learning outcomes in your group
Share
Sharing Your Assessment Data Have clear objectives/purpose Know who cares about your results Effectively analyze the data Effectively report/present results
Determine the best format – written reports, websites, newsletters, publications, oral presentations
Use the data yourself – role model to others Share with each other – assessment road
shows/poster sessions at division meetings/monthly during the A-Team meetings
Share with students!! What actions have you taken from what you have learned from them?
In Summation
Be clear about who you are, why you do what you do, how well you do what you do, and how you improve and change
Check in annually to be sure your program/activity and event outcomes align with your departmental mission/goals and that your departmental mission aligns with the division’s mission/goals and strategic plan and that your division mission/goals align with institutional mission/goals/strategic plan/QEP
Be patient; you will get better at writing outcomes; you always will have more to learn!
Practice, practice, practice and learn by teaching others Work with your students and student groups to
articulate outcomes for their programs Work with colleagues, faculty and others with expertise Celebrate your accomplishments
Contact Information
Sandi Osters, Ph.D. Retired Director, Student Life Studies
Texas A&M University3001 Aztec StreetCollege Station, TX 77845