Upload
aileen-burke
View
214
Download
0
Embed Size (px)
Citation preview
USING RUBRICS TO ASSESS THE PERFORMANCE OF NEW
PROFESSIONALS IN STUDENT AFFAIRS
Tony Ribera & Sarah FernandezA Presentation at the 2009 NASPA Region IV-E Conference,
Chicago, IL
AGENDA
State of assessment in student affairs Integrating rubrics into assessment efforts New professionals in student affairs and
using rubrics to assess their learning/performance
Going forward
PARTICIPANTS SHOULD GAIN…
An appreciation for rubrics as a way to communicate expectations and assess performance
An understanding of steps to develop a descriptive rubric in various functional areas of student affairs
Experience creating a descriptive rubric with knowledgeable colleagues from various institutions
ASSESSMENT IN STUDENT AFFAIRS
Who is responsible (ACPA, 2007) Questioning student learning (Creamer,
Winston, & Miller, 2001; Hanson, 1991; King, 2003; Love, 1995)
Systematic inquiry (Pascarella & Whitt, 1999) Looking toward the future (Torres & Walbert,
2009)
“One can argue there is a large gap between what student affairs practitioners say about the value and importance of research and assessment to their work and the extent to which such activities are actually conducted and used” (Pascarella & Whitt, 1999, p. 103).
WHAT IS A RUBRIC?
Assessment tool often used to grade assignments
Divides a larger assignment into smaller tasks
Describes criteria and sets expectations Allows for consistent scoring Provides appropriate feedback
(Stevens & Levi, 2005; Suskie, 2009)
TYPES OF RUBRICS (Suskie, 2009)
Checklists List of goals/outcomes
Rating Scales List of goals/outcomes plus rating scale
Descriptive Rubrics Includes description for each level of the rating
scale Holistic Scoring Guides
Narrative description of the ratings possible. Does not include list of goals/outcomes
DESCRIPTIVE RUBRICS (Stevens & Levi, 2005)
Most commonly used in higher education Made up of 4 parts:
Task Description Description of overall assignment
Scale Provides labels for the rating of performance
Dimensions Goals/outcomes. Skills that should be achieved
Description of Dimensions Expectations for each dimension at each rating scale
ASSESSMENT AND NEW PROFESSIONALS
“Learning must be at the center of the student affairs profession. We as student affairs professionals require the consistent development of our own learning. Thus, what we learn and how we learn it becomes critical to developing our role as facilitators of the student learning and development process” (ACPA, 2007, p.2).
WHAT DO WE KNOW ABOUT NEW PROFESSIONALS?
Graduate preparation (Waple, 2006) Competencies and responsibilities (Burkard,
Cole, Ott, & Stoflet, 2005) Values in recruiting and hiring (Kretovics,
2002) Desire for feedback (Renn & Jessup-Anger,
2008)
CHALLENGES FACING NEW PROFESSIONALS
Graduate preparation (Erwin & Sivo, 2001) Time/lack of resources (Johnson & Steele,
1984) Individual interest and motivation
Exemplary Competent Basic
Application of Professional Literature in program and intervention development
Use literature to understand new assessment techniques. Contribute to literature on assessment and evaluation.
Use literature to understand new assessment techniques.
Use literature to understand the importance of assessment and evaluation techniques
Facilitation of data collection to measure student learning
Design and implement high quality assessment and evaluation strategies appropriate for staff, departmental , and institutional efforts. Facilitate training of staff to participate in assessment techniques.
Recognize importance of assessment, and participate in training of colleagues. Identify most appropriate instruments to be used in assessment efforts for staff and departmental efforts. Construct basic assessment and evaluation tools with supervision.
Collect data to assess department wide educational efforts.Evaluate staff, collecting data from multiple sources.
Interpretation of assessment data
Facilitate the development of assessment and evaluation reports, which use data to create goals and plans of action for use of results. Use departmental resources to support high quality evaluation.
Participate in the analysis of collected data.Effectively interpret results to be reported to department.
Interpret data in order to use results of reports and studies.
Application of results to improve programs and practices
Ensure the use of results in improving and/or creating new departmental and institutional efforts, programs, activities. Emphasize importance of assessment to departmental and institutional efforts.
Ensure the use of results in improving and/or creating new departmental efforts, programs, activities. Emphasize importance of assessment to departmental efforts.
Use findings of assessment and evaluation in improving educational efforts.
Comprehension of qualitative research methods
Supervise the design of qualitative assessment and evaluation. Understand needs to be flexible in qualitative studies. Have mature understanding of transferability and trustworthiness .
Create qualitative sampling designs with supervision. Participate in analysis of qualitative data.Use techniques to ensure trustworthiness.
Understand qualitative research methods. Understand how qualitative findings can be used in work settings. Judge trustworthiness of study.
Comprehension of quantitative research methods
Supervise the design of quantitative assessment and evaluation. Understand how to disseminate results. Assess instruments for validity and reliability. Learn to use mixed methods of data collection.
Create quantitative designs with supervision. Participate in the analysis of quantitative data. Understand more complex methods, including multivariate techniques.
Understand quantitative research methods. Recognize issues of validity and reliability in quantitative methods.
STEPS FOR CREATING A DESCRIPTIVE RUBRIC (Suskie, 2009)
Identify learning outcomes-what do you want them to take away?
List criteria-what do you want them to accomplish? Focus on most significant skills. These will
become your dimensions Developing rating scale
At minimum levels for adequate and inadequate Write descriptions Test it out- make sure the standards and
descriptions are appropriate
RUBRIC DEVELOPMENT MODELS (Stevens & Levi, 2005)
Presentation Model Rubric presented without outside feedback
Feedback Model Rubric can be changed with feedback after
presented Pass-the-Hat Model & Post-it Model
Both parties work together to develop rubric 4 X 4 model
Students take full responsibility of development
SMALL GROUP WORK ACTIVITY AND LARGE GROUP DISCUSSION
MOVING FORWARD
Need for more scholar-practitioners (Hossler, 2001; Komives, 1998; Schroeder & Pike, 2001)
Engaging in a scholarship of teaching and learning (Huber & Hutchings, 2005) Questioning Gathering and exploring evidence Trying out and refining new insights Going public