The impact of framing effect on student preferences for university grading systems

  • Published on

  • View

  • Download

Embed Size (px)


<ul><li><p>pStudies in Educational Evaluation 35 (2009) 160167</p><p>97</p><p>ad</p><p>co</p><p>, a</p><p>tiv</p><p>po</p><p>s</p><p>en</p><p>Contents lists available at ScienceDirect</p><p>Studies in Educati</p><p>journa l homepage: wwwAlmost all courses at the tertiary level involve grading, andtherefore involve a scheme for assigning grades. Universities varysubstantially in the policies they have for such grading practices. Atmost universities, the options for grades to be assigned (e.g., A, B, C,D, F, and then plusses and/or minuses) are determined for theuniversity as a whole; however, determining which assessmentsand assignments are counted for grades, and how those arecombined to form grades is usually left to the discretion of theindividual facultymember. In the United States, it is most often thecase that assessments and assignments are given weights of sometype, and then graded according to a scale, after which the scoresand theweights are combined and reduced to one score on a 0100scale, with the following general conversions to letter grades (seee.g., Dixon, 2004):</p><p> A 90100 B 8090 C 7080 D 6070 F Below 60</p><p>Usually the top 2 or 3 points in each range merit a plus, and thebottom 2 or 3 points merit a minus. Some faculty operate solelythrough averaging letter grades, but most use a number systemand then convert numerical averages to letters (Dombach &amp; Smith,2004). The number systems used vary widely, but tend to fall intothree broad categories, discussed below.</p><p>Perhaps the most common approach is to assign weights toeach of the assessments/assignments to be used for grading, withthe weights summing to 100%. Each assessment is graded on a 0100 scale, and then multiplied by the weight to determine itscontribution to the total grade.</p><p>A second approach assigns a point value to each assessment/assignment with the total of these points summing to 100. Thissystem is functionally identical to the rst system, but theweight of the assessment/assignment has been turned into thetotal point value. This obviates the need to multiply the weighttimes the score received on the 0100 score scale whencalculating a nal grade.</p><p>A third approach is a variation of the previous twoapproaches. It assigns each assessment/assignment a naturalnumber of points based on how much the faculty member thinkseach aspect of each assessment/assignment should be worth,without worrying what the total number of points will sum to atthe end of the semester. To establish how the number of pointsearned will translate to nal grades, the total number of possiblepoints is determined, and this total is multiplied by .90 to get</p><p>Assessment</p><p>Motivation with regard to theirmotivation, condence, effort, accuracyof assessment, andhowwell the studentmight</p><p>do using those assessments. Multivariate and univariate analyses of variance indicated that the framing</p><p>conditions signicantly affectedmotivation, condence, and effort, but not the perception of howwell the</p><p>student would do in the course or whether the grading system provides an accurate assessment of</p><p>performance. Results are discussed in terms of application to college grading, the relationship to framing</p><p>effect theory, and the potential for decision theory to inform assessment practice.</p><p> 2009 Elsevier Ltd. All rights reserved.</p><p>* Corresponding author at: University of Otago, Educational Studies and</p><p>Professional Practice, PO Box 56, Corner of George and Union, Dunedin, Otago</p><p>9001, New Zealand. Tel.: +64 3 479 5467; fax: +64 3 479 7550.</p><p>E-mail address: (J.K. Smith).</p><p>0191-491X/$ see front matter 2009 Elsevier Ltd. All rights reserved.doi:10.1016/j.stueduc.2009.11.001The impact of framing effect on studentgrading systems</p><p>Jeffrey K. Smith *, Lisa F. Smith</p><p>University of Otago, New Zealand</p><p>A R T I C L E I N F O</p><p>Article history:</p><p>Received 4 April 2009</p><p>Received in revised form 4 November 2009</p><p>Accepted 6 November 2009</p><p>Keywords:</p><p>Framing theory</p><p>Grading systems</p><p>A B S T R A C T</p><p>Kahneman and Tverskys (1</p><p>to student preferences for gr</p><p>assigned to 1 of 3 framing</p><p>system, a percentage system</p><p>were the same, and the rela</p><p>approach tohowmany total</p><p>performancewould yield the</p><p>theywere assigned to and threferences for university</p><p>9, 2000; Tversky &amp; Kahneman, 1981) work in decision-makingwas applied</p><p>ing practices. Undergraduate psychology students (n = 240)were randomly</p><p>nditions related to how a university course might be graded: a 100 point</p><p>nd an open point system of assessment. The assignments in each condition</p><p>e weights assigned to each were the same. The only difference was in the</p><p>intswere involvedandhowthenal gradeswere calculated. Thus, the same</p><p>amenal grade in each condition. Participants reviewed the grading system</p><p>were asked to rate a series of possible assessments to be used in the course</p><p>onal Evaluation</p><p>. e lsev ier .com/stueduc</p></li><li><p>J.K. Smith, L.F. Smith / Studies in Educational Evaluation 35 (2009) 160167 161the A/B cut, by .80 to get the B/C cut, etc. We consider this to bean open point system, where the number of total points in thecourse can vary, but within any particular course, would bespecied. Each of these three approaches is summarized inFig. 1, along with the way the scores would work out for animaginary student.</p><p>It can be seen that these three approaches are functionallyidentical, varying only in terms ofwhen themathematics is appliedto the grading system, and when scores are rounded off intopercentages. Note that in the open point system (which we haveinstantiated to be 600 points in this research), ner gradations ofscores on assessments/assignments can be made, and then thepoints are basically rounded into the grade categories. In the 100point system, some of this rounding is done within eachassessment/assignment (there is no equivalent of a score of 63in the 600 point system when using the 100 point system, unlessthe faculty allows for scores such as 10.5). The percent systemactually allows for even ner gradations, especially on assess-ments/assignments that carry a small weight. These are fairlynuanced points in an overall approach to grading; fundamentallythe three systems are the same.</p><p>Fig. 1. Example of approaAssessment Percent</p><p>system</p><p>100 point</p><p>system</p><p>600 point</p><p>system</p><p>Students scores</p><p>Percent 100 pts 600 pts</p><p>Homework 20% 20 pts 120 pts 80% 16 pts 96 pts</p><p>Midterm exam 20% 20 pts 120 pts 90% 18 pts 108 pts</p><p>Class presentation 10% 10 pts 60 pts 60% 6 pts 36 pts</p><p>Term paper 25% 25 pts 150 pts 88% 22 pts 132 pts</p><p>Final exam 25% 25 pts 150 pts 92% 23 pts 138 pts</p><p>aTotal 100% 100 pts 600 pts 85% 85 pts 510 pts</p><p>a Grade calculation</p><p>Percent : 80% 20% 90% 20% 60% 10% 88% 25% 92% 25%</p><p> 85%</p><p>The students nal grade is 85%, which would be a B.</p><p>100pts : 16 18 6 22 23 85</p><p>The students nal grade is 85 points, which would be a B.</p><p>Openpoint 600pts : 96 108 36 132 138 510</p><p>ches to assessment.</p></li><li><p>J.K. Smith, L.F. Smith / Studies in Educational Evaluation 35 (2009) 160167162The student earned 510 points on a 600 point scale.</p><p>90% 600 540or above A</p><p>80% 600 480; 480 539 B</p><p>Therefore, the students nal grade is a B.Alternatively, 510/600 = 85, which would be a B.</p><p>It can be seen that the three approaches to grading are the samefrom a quantitative perspective. But do students perceive them asbeing the same? This question arose from our practical experienceas university faculty. One of us consistently uses the 100 pointsystemunder the argument that it eliminates the need to engage incalculations to determine a nal grade and thus is easier forstudents to understand where they stand in a course at all times.But, in a discussion of the system in an assessment course, it wasdiscovered that the students did not like that approach. It seemedto them that they were, in the words of one student, starting thecourse with 100 points and losing points away from that with eachnew assessment/assignment that was not perfect. That is, theysaw the grading system as one in which each grade on anassignment or assessment took them farther away from the 100points that they had to start with, or the neutral point (Tversky &amp;Kahneman, 1974) where they had started.</p><p>Although the grading systemmay seem to be little more than anecessary evil, for many students it is the payoff for the work donein the course (Brookhart, 1993b, 2004). Students are sensitive tothe grading system used, because the grades they receivesummarize their efforts and reect upon their achievement inthe course (Brookhart, 1993a). Thus, careful consideration of theassessment system, and the procedures used for tallying grades is afactor that needs to be considered carefully in instructionalsettings (Harlen, 2007). It is natural that some students wouldprefer examinations taken in the course while others prefer termpapers and still others prefer projects done in groups (Smith,2009). Students will prefer whatever they think they are best atand what ts with their course work, study style, and timeallocated for the course. Some prefer whatever they perceive as theeasiest road to the highest grade. But, why should they prefer onemethod of tallying grades over another if the various systems arefunctionally identical?</p><p>To look for an answer to this question, we turn to thedecision-making literature, and in particular to the work ofKahneman and Tverskys (1979, 2000; Tversky &amp; Kahneman,1981). There are many aspects to the life work of thesescientists; in this research we focus on the concept of framingeffects and how people behave in situations where they perceivea potential loss and a potential gain (Tversky &amp; Kahneman,1986). Tversky and Kahneman (1986, p. 258) presented thefollowing simple experiment to illustrate this:</p><p>Potential gain setting: assume yourself richer by $300 than youare today. Choose between: Option K: a sure gain of $100 orOption L: a 50% chance to gain $200 and a 50% chance to gainnothing.Potential loss setting: assume yourself richer by $500 than youare today. Choose between: Option K0: a sure loss of $100 orOption L0: a 50% chance to lose $200 and a 50% chance to losenothing.Note that in both of the K situations, you end up with a sure$400, and in both of the L situations, you have a 50% chance toend up with $500 and a 50% chance to end up with $300.</p><p>In Tversky and Kahnemans (1986) study, they found that in thepotential gain setting, 72% of respondents took the $100 sure gain(putting them at an assured $400). In the potential loss setting, 36%of the subjects took the $100 sure loss (putting them at an assured$400). It appears to be the case that individuals would rather take arisk to avoid a perceived loss than to anticipate a potential gain (inthis case, they were twice as likely). This nding, and ones similarto it, has been replicated in a variety of studies (Johnson, Hershey,Meszaros, &amp; Kunreuther, 1993; Takemura, 1992; Yamagishi, 2002).The prospect of loss appears to be viewed more negatively byindividuals than the prospect of gain is viewed positively.</p><p>Now, in real life, individuals are usually not presentedwith suchclear-cut alternatives and options. Often, they are not presentedwith alternatives at all, but go about their lives navigatingsituations that are not entirely under their control. But they doreact to these situations in fashions that are predictable from howthe situations are perceived (Chong &amp; Druckman, 2007). Withregard to our original set of course grading schemes describedabove, it appears to be the case that students perceive the 100point system as one of successive losses from the perfect gradethey had to begin with, as opposed to the open point system andthe percentage system, which they perceive to be more of abuilding or gaining toward a grade rather than losing points awayfrom it. In part, we believe that this is due to the 100 point systemengendering a sense that the neutral point (Tversky &amp; Kahneman,1974) is 100; in the percent and 600 point system, the neutral pointis either zero, or perhaps simply not well-dened in the studentsmind. One possibility in the latter two settings is that the studentsuse their anticipated grade in the course as a neutral point. Thepurpose of this study was to see if, in fact, our observations fromteaching would be conrmed in an experimental setting, and if theframing effect described by Kahneman and Tversky offers a helpfulexplanation of the phenomenon.</p><p>The studywas designed to allow for an evaluation the followinghypothesis: the 100 point grading system is perceived by studentsas a loss scenario in comparison to an open point system and apercentage system. This would, in turn, engender a more negativeattitude toward the course as a whole than either of the other twoapproaches. The negative attitude resulting from this framingeffect would inuence students response to aspects of the coursethat were logically independent of how the grades were tallied. Toinvestigate this hypothesis, we set up a study to explore the ideathat the framing of the grading scheme inuenced not just thestudent response to the scheme directly, but had a broader effecton how students perceived aspects of the evaluation system notdirectly related to how the points were tallied.</p><p>Method</p><p>Participants were provided with a course scenario (describedbelow) involving the 100 point system, the percent system, or theopen point system (referred to as the 600 point system hereafter),and then asked about different kinds of approaches to assessmentthatmight be involved in such a system. That is, we did not directlyask how much they reacted to the grading system, but how theywould react to taking multiple choice tests, essay tests, shortanswer tests, and the like under the scenario to which they wereassigned. How the assessments are combined into grades shouldnot be related to the assessmentmethods used in the course unlessthe framing of the course grading system affects how the studentsview the course more generally.</p><p>Participants</p><p>The participants for the studywere 240 undergraduate studentsat a public university in New Jersey. The participants were enrolledin psychology classes and received credit for their participation.Forty-seven were male and 193 were female; 198 were betweenthe ages of 1824, with 52 being 25 or older. One hundred twenty-eight self-identied as Caucasian, 41 were African American, 34</p></li><li><p>randomly assigned to, and a self-rating of college grade pointaverage category (2.002.99, 3.003.49, and 3.504.00) asindependent variables in a 3 3 analysis. This allowed us to rsttest the hypothesis across all students, and to see if students ofdiffering achievement levels reacted differently to the variousscenarios. Our interest was in the main effect for the differentframes, and then in the interaction term.Wewere not interested inthe main effect for grade point average here. Following eachmultivariate anova, we examined the univariate anovas for eachvariable.</p><p>It was hypothesized that students would perceive the 100 pointsystem as what Kahneman and Tversky (2000) would see as a</p><p>J.K. Smith, L.F. Smith / Studies in Educational Evaluation 35 (2009) 160167 163were Hispanic, 19 were Asian or Pacic Islanders, 2 were NativeAmericans,...</p></li></ul>


View more >