1  

USING RUBRICS

Donna K. Korycinski This paper was completed and submitted in partial fulfillment of the Master Teacher Program, a 2-year faculty professional development program conducted by the Center for Teaching Excellence, United States Military Academy, West Point, NY, 2011. Abstract: Rubrics have been used for years by educators to evaluate and to provide feedback on student performance. Rubrics are ideal for this purpose; however, they must be constructed and used properly in order to gain the benefits. This literature review explores the pros and cons of rubric use, how to construct a rubric properly, teacher’s attitudes toward rubrics and lists some online resources. Key words: rubric, evaluate, assessment, grading.

A rubric as understood today within the educational realm is a tool that is used as an assessment and scoring device to judge the quality of student work. The term “rubric” itself has an interesting progression. Popham (1997) writes, “The original meaning of rubric had little to do with the scoring of students’ work. The Oxford English Dictionary tells us that in the mid-15th century, rubric referred to headings of different sections of a book. This stemmed from the work of Christian monks who painstakingly reproduced sacred literature, invariably initiating each major section of a copied book with a large red letter. Because the Latin word for red is ruber, rubric came to signify the headings for major divisions of a book.” Cooper & Gargan (2009) add, “the Catholic Church has long employed the term for the directions for conducting the Mass, which are printed in red and inserted into liturgical books” and “in law, a rubric is a heading or title of a statute or section of a legal code (again, originally printed in red ink).” This reference to red may have carried over into current usage as teachers commonly grade or correct student papers using red ink so that the markings are readily distinguishable from the original student work that is usually expected to be written in blue or black ink.

The use of rubrics in the classroom became popular in the 1970’s and has gained momentum ever since and for good reason. When constructed and used properly, rubrics are highly effective tools not only to assess and evaluate student performance but also to provide constructive and informative feedback. Specifically, Moskal (2000) states, “First, they [rubrics] support the examination of the extent to which the specified criteria has been reached. Second, they [rubrics] provide feedback to students concerning how to improve their future performances.” Andrade (2000) and Cooper & Gargan (2009) provide reasons why instructors should want to use rubrics:

1. Easy to use and explain. 2. Makes teachers’ expectations very clear to students, parents, teachers, educators and

others. 3. Helps teachers think carefully and critically about what they are teaching and what

students need to learn. 4. Provides students with more informative feedback about their strengths and areas in need

of improvement than other forms of assessment; provides opportunity for reflection. 5. Supports continued learning, the development of skills and understanding and good

thinking.

2  

Making a high-quality, useful rubric takes thought, time and effort. All rubrics have two things in common: a list of evaluation criteria or expectations for the assignment and descriptions of degrees of quality associated with each criterion that identifies if the work has met the requirements to be considered “excellent”, “above average”, “average” or “below average”. A general rule of thumb is to use four to five levels of quality; three levels or less does not provide enough differentiation while six or more may be too detailed and difficult to distinguish the gradation. If applicable, weights or a weighting scheme can also be applied to the criteria if one or more of the criteria are thought to be more important than the others. The type of scoring strategy used can be either holistic or analytic (or a combination of the two). Popham (1997) states, “Using a holistic strategy, the scorer takes all of the evaluative criteria into consideration but aggregates them to make a single, overall quality judgment. An analytic strategy requires the scorer to render criterion-by-criterion scores that may or may not ultimately be aggregated into an overall score.” A rubric can be constructed solely by an instructor to grade a single event (performance rubric) or it can be constructed with input from the students, also known as “negotiable contracting” [Boston, 2002]. When a rubric is negotiated with students and used throughout a course for interim feedback then for a final evaluation of some course product, it is often referred to as an instructional rubric. [Andrade, 2000] The steps for constructing a rubric are outlined in many sources and consist of:

1. Look at examples of good and poor work on a course deliverable (or similar project). 2. List criteria. 3. Revise criteria, if needed. 4. Articulate levels of quality for the criteria. 5. Draft the rubric. 6. Revise the draft.

There are several teacher resource websites that assist in the development of rubrics. Some are free; some are not. Many have free trials. Here are several (all accessed on 22 April 2011):

http://rubistar.4teachers.org/ “RubiStar is a free tool to help teachers create quality rubrics. Copyright. © 2000-2008, ALTEC at University of Kansas”

http://www.rcampus.com/indexrubric.cfm “Individual educators and students can use iRubric and hundreds of other free RCampus features at no charge. All we ask is to tell others, especially your school, about our easy-to-use and powerful tools.” Copyright © 2011 Reazon Systems, Inc.

http://www.teach-nology.com/web_tools/rubrics/ “TeAch-nology - The Art and Science of Teaching with Technology®” ©1999 - 2010 Teachnology, Inc.

http://teacher.scholastic.com/tools/rubric.htm “Rubric Maker: A great tool for assessment and evaluation!” TM ® & © 2011-1996 Scholastic Inc.

http://myt4l.com/index.php?v=pl&page_ac=view&type=tools&tool=rubricmaker “Use the Rubric Maker to make customized assessments for student work. You can create rubrics for primary, elementary, middle, and high school.” ©2011 Tech4Learning, Inc.

http://www.rubrics4teachers.com/ “A complete guide for Educational Teacher Rubrics and Assessment” Copyright © 2001 - 2011 TeacherPlanet.com®

http://www.rubricbuilder.on.ca/ “Rubric Builder is a simple, powerful tool which helps teachers create meaningful, high quality assessments. With thousands of searchable criteria to choose from, it is so fast and simple that you will be creating rubrics in

3  

minutes. ” ©Copyright 2006, Gateway Software Productions, created by Gateway Software Productions

http://www.teacherweb.com/ “TeacherWeb® offers website solutions for teachers, schools, and districts. TeacherWeb® sites are completely customizable and easy-to-use.” Copyright © 2000 - 2010 TeacherWeb, Inc., - Patent 7,167,903

There are many benefits to using rubrics for evaluation; they include inter-rater reliability

(same or similar ratings when used by different people) [Boston, 2002], consistency (same or similar ratings when used by the same person) [Greville, 2009], validity (when aligned with reasonable and respectable standards within the curriculum being taught) [Andrade, 2005] and equity (must be checked to ensure that ratings are not slanted toward gender, race, ethnicity or socioeconomic status) [Andrade, 2005]. In Andrade (2005), one teacher commented that “Rubrics keep me honest.”

While rubrics have several benefits as described, if constructed incorrectly, they may also have some drawbacks. In the article “What’s Wrong—and What’s Right—with Rubrics”, Popham (1997) outlines the following flaws: task specific evaluative criteria (“need evaluative criteria that capture the essential ingredients of the skill being measured, not the particular display of the skill applied to a specific task”); excessively general evaluative criteria; dysfunctional detail (be succinct in capturing the key evaluative criteria and qualities); and equating the test of the skill with the skill itself (“Test-focused instruction, especially if it mimics the test in every detail, will often stifle the student’s general mastery of the skill. Students may, indeed, learn how to do well on a given performance test, but if asked to tackle a different performance test—a test derived from the same skill—they may stumble.”).

Other drawbacks include “restricting students’ problem solving, decision-making and

creativity” [Chapman, 2009]. It is thought that many students will adhere to the strict guidelines of a rubric and will choose not to venture too far from outlined expectations for fear of losing points, therefore stifling individual creativity. Chapman notes, “. . . we found little flexibility and even less encouragement of individual initiative: almost every rubric, along with samples of student work, pointed to a bland sameness.” [Chapman, 2009] In addition, to be effective, students must first be introduced to a particular rubric (or assist in its construction), understand how to use it, understand the criteria and the differences between the quality levels, and understand how the final score is obtained and its meaning.

Additionally, teachers who use or are expected to use rubrics in the classroom for assessment and grading purposes need to understand the rubrics power and become comfortable using rubrics; this comfort translates into a positive attitude when working with and using rubrics and a much more comprehensive use of the instrument. Kutlu observes, “. . . the teachers with positive attitudes used the information they obtained from the rubrics for giving feedback about the students’ products and observing to what extent the[y] students used their knowledge and skills in real life situations, the teachers with negative attitudes primarily used them for grading the students.” [Kutlu, 2010]

4  

References Andrade, H. (2000). Using Rubrics To Promote Thinking and Learning. Educational Leadership, 57(5), 13-18. Retrieved from EBSCOhost. Andrade, H. (2005). Teaching with Rubrics: The Good, the Bad, and the Ugly. College Teaching, 53(1), 27. Retrieved from EBSCOhost. Boston, C., & ERIC Clearinghouse on Assessment and Evaluation, C. D. (2002). Understanding Scoring Rubrics: A Guide for Teachers. Retrieved from EBSCOhost. Chapman, V. G., & Inman, M. (2009). A Conundrum: Rubrics or Creativity/Metacognitive Development?. Educational Horizons, 87(3), 198-202. Retrieved from EBSCOhost. Cooper, B. S., & Gargan, A. (2009). Rubrics in Education: Old Term, New Meanings. Phi Delta Kappan, 91(1), 54-55. Retrieved from EBSCOhost. Greville, E. C. (2009). A Rose by Any Other Name: Grading and Assessment. Assessment Update, 21(5), 1-2. Kutlu, O., Yildirim, O., & Bilican, S. (2010). The comparison of the views of teachers with positive and negative attitudes towards rubrics. World Conference on Learning, Teaching and Administration 2010, Procedia Social and Behavioral Sciences, 9, 1566-1573. Moskal, B. M., & ERIC Clearinghouse on Assessment and Evaluation, C. D. (2000). Scoring Rubrics Part I: What and When. ERIC/AE Digest. Retrieved from EBSCOhost. Popham, W. (1997). What's Wrong--and What's Right--with Rubrics. Educational Leadership, 55(2), 72-75. Retrieved from EBSCOhost. Annotated readings: Boston, C., & ERIC Clearinghouse on Assessment and Evaluation, C. D. (2002). Understanding Scoring Rubrics: A Guide for Teachers. Retrieved from EBSCOhost. This guide for teachers is a comprehensive look at rubrics, what they are, how to use them as a teaching tool, their validity and reliability, and includes a step-by-step procedure on how to design and develop rubrics. It also includes a chapter on “Creating Rubrics Through Negotiable Contracting” written by Andi Stix (see Stix 1996). Brown, G. L., Glasswell, K., & Harland, D. (2004). Accuracy in the Scoring of Writing: Studies of Reliability and Validity Using a New Zealand Writing Assessment System. Assessing Writing, 9(2), 105-121. Retrieved from EBSCOhost. These authors researched the use of scoring rubrics in the United States and the United Kingdom in order to develop scoring rubrics that could be used consistently in assessing writing samples for the New Zealand Ministry of Education’s national assessment program. It concludes that

5  

training and monitoring of the raters is necessary to increase consistency and reliability of the rubrics. Jonsson, A., & Svingby, G. (2007). The Use of Scoring Rubrics: Reliability, Validity and Educational Consequences. Educational Research Review, 2(2), 130-144. Retrieved from EBSCOhost. This article discusses the fact rubrics have the potential to improve instruction and to promote learning, and the common misconception that they will also increase reliability and validity. While this can be the case when rubrics are used correctly, the authors caution that in order to increase reliability and validity, the rubric should be analytic, topic specific and be coupled with examples and/or training in its proper use. Kutlu, O., Bilican, S., & Yildirim, O. (2010). A study on the primary school teachers’ attitudes towards rubrics with reference to different variables. World Conference on Educational Sciences 2010, Procedia Social and Behavioral Sciences, 2, 5398-5402. Using Analysis of Variance (ANOVA) techniques, the authors studied teacher attitudes towards the use of rubrics based on the following variables: number of students in the classroom, level of individual information on rubrics (structure, features, preparation, implementation and scoring), and the purposes of using the results. It was concluded that if there were less students in the classroom, if the teacher had a sufficient level of information on the rubrics and used the rubrics to provide feedback to students and to assist in the development of advanced thinking skills, this translated into a positive attitude towards using rubrics; on the other hand, if the teachers did not feel that they had sufficient information or knowledge of rubrics and who only used the rubrics for grading purposes, this translated into a negative attitude toward using rubrics. Loveland, T. R. (2005). Writing Standards-Based Rubrics for Technology Education Classrooms: The Use of Rubrics Goes Beyond the Simple Need for Objective Grading in Classrooms. Technology Teacher, 65(2), 19. Retrieved from EBSCOhost. This article tests a theory from another author (Kachergis, 2004) that increased use of rubrics in the classroom translates into improved academic performance on standardized tests [specifically, the science portion of the Connecticut Academic Performance Test (CAPT)]. The author did report a correlation between schools having “the highest levels of strategy and rubric use” and “high CAPT gains and increasing CAPT scores over time”. Maxwell, S., & National Centre for Vocational Education, R. (2010). Using Rubrics to Support Graded Assessment in a Competency Based Environment. Occasional Paper. National Centre for Vocational Education Research (NCVER), Retrieved from EBSCOhost. This paper’s author participated in a 2008 community of practice scholarship program through the National Centre for Vocational Education Research. Her research entailed completing a literature review on rubrics, mentoring teachers in the development and use of rubrics, then interviewing them about their thoughts on rubrics. Her conclusion was that the teachers thought that rubrics were a useful tool for assessment but also for relaying their expectations to students. Rezaei, A., & Lovorn, M. (2010). Reliability and Validity of Rubrics for Assessment through Writing. Assessing Writing, 15(1), 18-39. Retrieved from EBSCOhost.

6  

This is an interesting article that highlights that although educators have adopted the grading rubric in an attempt to have a more reliable and less biased grading mechanism, this is often not the case if a rubric is used to grade a written product unless the raters are training on how to design and employ a rubric effectively. Stix, A. (1996). Creating Rubrics through Negotiable Contracting. Retrieved from EBSCOhost. This paper addresses the popular practice of allowing students to be participants in the rubric development process known as “negotiable contracting” to create student buy-in. It is based on the idea that if students feel that they are a valued stakeholder in the assessment process, they will be more likely to want to meet the expectations laid out in the rubric. It discusses how this can be accomplished, and it also provides several examples of rubrics and assessment sheets. Extended bibliography of related publications: Andrade, H. (1999). The Role of Instructional Rubrics and Self-Assessment in Learning To Write: A Smorgasbord of Findings. Retrieved from EBSCOhost. Hitt, A. M., & Helms, E. C. (2009). Best in Show: Teaching Old Dogs to Use New Rubrics. Professional Educator, 33(1), Retrieved from EBSCOhost. Mckeown, R. (2011). Using Rubrics to Assess Student Knowledge Related to Sustainability: A Practitioner's View. Journal of Education for Sustainable Development, 5(1), 61-74. Retrieved from EBSCOhost. Reddy, Y., & Andrade, H. (2010). A Review of Rubric Use in Higher Education. Assessment & Evaluation in Higher Education, 35(4), 435-448. Retrieved from EBSCOhost. Stix, A. (1996). Creating Rubrics through Negotiable Contracting and Assessment. Retrieved from EBSCOhost. Turley, E. D., & Gallagher, C. W. (2008). On the "Uses" of Rubrics: Reframing the Great Rubric Debate. English Journal, 97(4), 87-92. Retrieved from EBSCOhost.