The impact of framing effect on student preferences for university grading systems

Embed Size (px)

Text of The impact of framing effect on student preferences for university grading systems

  • pStudies in Educational Evaluation 35 (2009) 160167




    , a





    Contents lists available at ScienceDirect

    Studies in Educati

    journa l homepage: wwwAlmost all courses at the tertiary level involve grading, andtherefore involve a scheme for assigning grades. Universities varysubstantially in the policies they have for such grading practices. Atmost universities, the options for grades to be assigned (e.g., A, B, C,D, F, and then plusses and/or minuses) are determined for theuniversity as a whole; however, determining which assessmentsand assignments are counted for grades, and how those arecombined to form grades is usually left to the discretion of theindividual facultymember. In the United States, it is most often thecase that assessments and assignments are given weights of sometype, and then graded according to a scale, after which the scoresand theweights are combined and reduced to one score on a 0100scale, with the following general conversions to letter grades (seee.g., Dixon, 2004):

    A 90100 B 8090 C 7080 D 6070 F Below 60

    Usually the top 2 or 3 points in each range merit a plus, and thebottom 2 or 3 points merit a minus. Some faculty operate solelythrough averaging letter grades, but most use a number systemand then convert numerical averages to letters (Dombach & Smith,2004). The number systems used vary widely, but tend to fall intothree broad categories, discussed below.

    Perhaps the most common approach is to assign weights toeach of the assessments/assignments to be used for grading, withthe weights summing to 100%. Each assessment is graded on a 0100 scale, and then multiplied by the weight to determine itscontribution to the total grade.

    A second approach assigns a point value to each assessment/assignment with the total of these points summing to 100. Thissystem is functionally identical to the rst system, but theweight of the assessment/assignment has been turned into thetotal point value. This obviates the need to multiply the weighttimes the score received on the 0100 score scale whencalculating a nal grade.

    A third approach is a variation of the previous twoapproaches. It assigns each assessment/assignment a naturalnumber of points based on how much the faculty member thinkseach aspect of each assessment/assignment should be worth,without worrying what the total number of points will sum to atthe end of the semester. To establish how the number of pointsearned will translate to nal grades, the total number of possiblepoints is determined, and this total is multiplied by .90 to get


    Motivation with regard to theirmotivation, condence, effort, accuracyof assessment, andhowwell the studentmight

    do using those assessments. Multivariate and univariate analyses of variance indicated that the framing

    conditions signicantly affectedmotivation, condence, and effort, but not the perception of howwell the

    student would do in the course or whether the grading system provides an accurate assessment of

    performance. Results are discussed in terms of application to college grading, the relationship to framing

    effect theory, and the potential for decision theory to inform assessment practice.

    2009 Elsevier Ltd. All rights reserved.

    * Corresponding author at: University of Otago, Educational Studies and

    Professional Practice, PO Box 56, Corner of George and Union, Dunedin, Otago

    9001, New Zealand. Tel.: +64 3 479 5467; fax: +64 3 479 7550.

    E-mail address: (J.K. Smith).

    0191-491X/$ see front matter 2009 Elsevier Ltd. All rights reserved.doi:10.1016/j.stueduc.2009.11.001The impact of framing effect on studentgrading systems

    Jeffrey K. Smith *, Lisa F. Smith

    University of Otago, New Zealand

    A R T I C L E I N F O

    Article history:

    Received 4 April 2009

    Received in revised form 4 November 2009

    Accepted 6 November 2009


    Framing theory

    Grading systems

    A B S T R A C T

    Kahneman and Tverskys (1

    to student preferences for gr

    assigned to 1 of 3 framing

    system, a percentage system

    were the same, and the rela

    approach tohowmany total

    performancewould yield the

    theywere assigned to and threferences for university

    9, 2000; Tversky & Kahneman, 1981) work in decision-makingwas applied

    ing practices. Undergraduate psychology students (n = 240)were randomly

    nditions related to how a university course might be graded: a 100 point

    nd an open point system of assessment. The assignments in each condition

    e weights assigned to each were the same. The only difference was in the

    intswere involvedandhowthenal gradeswere calculated. Thus, the same

    amenal grade in each condition. Participants reviewed the grading system

    were asked to rate a series of possible assessments to be used in the course

    onal Evaluation

    . e lsev ier .com/stueduc

  • J.K. Smith, L.F. Smith / Studies in Educational Evaluation 35 (2009) 160167 161the A/B cut, by .80 to get the B/C cut, etc. We consider this to bean open point system, where the number of total points in thecourse can vary, but within any particular course, would bespecied. Each of these three approaches is summarized inFig. 1, along with the way the scores would work out for animaginary student.

    It can be seen that these three approaches are functionallyidentical, varying only in terms ofwhen themathematics is appliedto the grading system, and when scores are rounded off intopercentages. Note that in the open point system (which we haveinstantiated to be 600 points in this research), ner gradations ofscores on assessments/assignments can be made, and then thepoints are basically rounded into the grade categories. In the 100point system, some of this rounding is done within eachassessment/assignment (there is no equivalent of a score of 63in the 600 point system when using the 100 point system, unlessthe faculty allows for scores such as 10.5). The percent systemactually allows for even ner gradations, especially on assess-ments/assignments that carry a small weight. These are fairlynuanced points in an overall approach to grading; fundamentallythe three systems are the same.

    Fig. 1. Example of approaAssessment Percent


    100 point


    600 point


    Students scores

    Percent 100 pts 600 pts

    Homework 20% 20 pts 120 pts 80% 16 pts 96 pts

    Midterm exam 20% 20 pts 120 pts 90% 18 pts 108 pts

    Class presentation 10% 10 pts 60 pts 60% 6 pts 36 pts

    Term paper 25% 25 pts 150 pts 88% 22 pts 132 pts

    Final exam 25% 25 pts 150 pts 92% 23 pts 138 pts

    aTotal 100% 100 pts 600 pts 85% 85 pts 510 pts

    a Grade calculation

    Percent : 80% 20% 90% 20% 60% 10% 88% 25% 92% 25%


    The students nal grade is 85%, which would be a B.

    100pts : 16 18 6 22 23 85

    The students nal grade is 85 points, which would be a B.

    Openpoint 600pts : 96 108 36 132 138 510

    ches to assessment.

  • J.K. Smith, L.F. Smith / Studies in Educational Evaluation 35 (2009) 160167162The student earned 510 points on a 600 point scale.

    90% 600 540or above A

    80% 600 480; 480 539 B

    Therefore, the students nal grade is a B.Alternatively, 510/600 = 85, which would be a B.

    It can be seen that the three approaches to grading are the samefrom a quantitative perspective. But do students perceive them asbeing the same? This question arose from our practical experienceas university faculty. One of us consistently uses the 100 pointsystemunder the argument that it eliminates the need to engage incalculations to determine a nal grade and thus is easier forstudents to understand where they stand in a course at all times.But, in a discussion of the system in an assessment course, it wasdiscovered that the students did not like that approach. It seemedto them that they were, in the words of one student, starting thecourse with 100 points and losing points away from that with eachnew assessment/assignment that was not perfect. That is, theysaw the grading system as one in which each grade on anassignment or assessment took them farther away from the 100points that they had to start with, or the neutral point (Tversky &Kahneman, 1974) where they had started.

    Although the grading systemmay seem to be little more than anecessary evil, for many students it is the payoff for the work donein the course (Brookhart, 1993b, 2004). Students are sensitive tothe grading system used, because the grades they receivesummarize their efforts and reect upon their achievement inthe course (Brookhart, 1993a). Thus, careful consideration of theassessment system, and the procedures used for tallying grades is afactor that needs to be considered carefully in instructionalsettings (Harlen, 2007). It is natural that some students wouldprefer examinations taken in the course while others prefer termpapers and still others prefer projects done in groups (Smith,2009). Students will prefer whatever they think they are best atand what ts with their course work, study style, and timeallocated for the course. Some prefer whatever they perceive as theeasiest road to the highest grade. But, why should they prefer onemethod of tallying grades over another if the various systems arefunctionally identical?

    To look for an answer to this question, we turn to thedecision-making literature, and in particular to the work ofKahneman and Tverskys (1979, 2000; Tversky & Kahneman,1981). There are many aspects to the life work of thesescientists; in this research we focus on the concept of framingeffects and how people behave in situations where they perceivea potential loss and a potential