Understanding the RelationshipBetween Language Performanceand University Course GradesAlan V. BrownUniversity of Kentucky

Abstract: This article presents preliminary data correlating students’ scores onmeasures of speaking, listening, and reading with their grades in Spanish courses. Asmight be expected, students with higher grades generally scored higher on performanceassessments than did classmates with lower course grades. However, large score rangesresulted within a single letter grade. In addition, some students with low course gradesscored rather well on certain measures of performance while some students with highcourse grades had low scores. In sum, the relationship between course grades, GPA, andperformance level as evidenced by these data were far from predictable and underscoredthe need for (1) more research using a variety of methodologies, and (2) more carefulalignment of course grades with demonstrated abilities to use language in interpretiveand interpersonal settings.

Key words: assessment, GPA, program evaluation, Spanish proficiency, universityforeign language course grades

IntroductionMost contemporary foreign language (FL) educators, scholars, and researchers agreeon the importance of helping students to develop the skills they need to use languagefor real communicative purposes in non‐instructional settings. This overarchingphilosophy reflects increased understanding of the complexities of the communica-tive process (Bachman, 1990; Canale, 1983; Canale & Swain, 1980; Hymes, 1972)and the essential contributions of input (Krashen, 1985; VanPatten, 1996), output(Swain, 1995), and interaction (Gass & Varonis, 1994; Long, 1996) to effectivesecond language (L2) acquisition. In order to help students become “informed andcapable interlocutors” (Modern Language Association, 2007), proficiency‐oriented,communicative‐based language teaching has become the preferred instructionalapproach for L2 classrooms inmany countries and contexts. It is unclear, however, towhat extent course grades also reflect this emphasis. This article explores the extentto which course grades reflect students’ actual level of performance in the language.

Alan V. Brown (Ph.D., University of Arizona) is assistant professor of Spanish atthe University of Kentucky, Lexington, Kentucky.Foreign Language Annals, Vol. 46, Iss. 1, pp. 80–87. © 2013 by American Council on the Teaching of ForeignLanguages.DOI: 10.1111/flan.12014

80 SPRING 2013

Literature ReviewThe design of assessment instruments thatdirectly and authentically evaluate thedesired construct(s) by minimizing theimpact of construct‐irrelevant variance andconstruct underrepresentation (Messick,1989, 1994, 1996) is an extremely difficultundertaking. Although much attention hasbeen given to the usefulness and construct‐sensitive development of language tests(Bachman, 1990; Bachman & Palmer,1996) and the interpretation of resultingscores (Messick, 1989), much less attentionhas been paid to how the results fromvarious assessment instruments and practi-ces across an entire sequence of instruction,e.g., a university semester, are given expres-sion in course grades. In other words, theway in which final course grades arecomposed and the reporting of classroomFL learning have been rather neglected inthe literature on L2 and (FL) teaching andlearning. For example, do letter grades andoverall GPA in world language classes alignmore closely with measures of overalllanguage proficiency, academic achieve-ment, grammatical accuracy, work ethic/perseverance, or effective use of communi-cation strategies? To what extent do tradi-tional letter and grade scales reflect theknowledge and skills students require inorder to successfully communicate in aworld language both within and beyondthe classroom while also reflecting societaland institutional values and norms?

For many educators and scholars, de-constructing the composition of university‐level FL grades and what they represent isan extremely complex undertaking, pri-marily because multifaceted constructs suchas academic achievement, FL proficiency,and communicative competence should orcould all be reflected to some degree in afinal grade in a world language course.Ironically, in spite of the field’s lack ofunderstanding and agreement regarding FLcourse grades and best practices for theircomposition, grades continue to be used todetermine if students can advance fromone course to the next higher course. They

are also used by universities and employersin making high‐stakes decisions regardingundergraduate and graduate admissions,scholarships, fellowships, program advance-ment, academic awards, internships, andemployability. Although previous research(Magnan, 1986) correlating proficiency andlevel of university study dates back to the1980s, the relationship between coursegrades and students’performanceon externalmeasures of L2 ability has not been carefullyexplored.

From the field of general education,Guskey and Bailey (2001) conceded thatgrading and grade reporting represent a“murky, disagreement‐fraught quagmire”(p. 1) but also argued that the topic mustbe broached because related issues such asgrade inflation cannot be addressed untilgrades are better understood. Guskey alsoeloquently argued that grade inflation is notthe primary concern educators must face inachieving transparency in reporting studentlearning; rather, it is the grades themselvesand their interpretability: “The problem ofgrade inflation is not simply that morestudents are receiving high grades. It is thatwe’re not sure what those grades mean”(Guskey, personal communication).

Fundamentally, what Guskey and Bai-ley (2001) proposed is that educators shoulddecide to emphasize process (work habits,study skills, etc.), product (achievement,performance, academic products), progress(amount of learning), or some weightedcombination of all three components. Somescholars have presented strong arguments infavor of clearly limiting achievement gradesto a student’s attainment of criterion‐referenced learning objectives (Brown &Abeywickrama, 2010; Gronlund, 1998)without considering such idiosyncratic fac-tors as progress over time or processes oflearning. For example, Gronlund proposedthat grades should not be “contaminated”(p. 174) by nonlearning issues such astardiness, effort, behavior, or attendance.Guskey and Bailey urged educators todistinguish between learning and nonlearn-ing issues but not to place undue emphasis

Foreign Language Annals � VOL. 46, NO. 1 81

on the final product or performance. Theyencouraged the reporting of separate marksfor product and progress and for productand process. For these authors, a studentwho improves grades over the course of theinstructional sequence from D’s to B’s mayhave learned much more—relatively speak-ing—than a student who received A’s fromthe beginning and in reality saw littleprogress toward improved proficiency.

While contributions from scholars suchas Gronlund (1998) and Guskey (1996,2009) are useful for framing the debate, theunique nature of L2 acquisition requiresfurther research. Micro issues in L2 assess-ment at the individual instrument level havebeen dealt with extensively by L2 testers(Bachman, 1990; Bachman & Palmer, 1996;Brown & Abeywickrama, 2010; Hughes,2003; Liskin‐Gasparro, 1995; McNamara,2000), and to a lesser extent, macro issuessuch as program evaluation (Alderson &Beretta, 1992; Lynch, 1996; Norris, Davis,Sinicrope, &Watanabe, 2009) have receivedattention as well. Conspicuously absentfrom such research efforts and their con-clusions are evidence‐based recommenda-tions for meaningful ways to combinemultiple assessment events and constructswhen assigning FL letter grades so as to bemaximally meaningful in regard to class-room achievement and real‐world languageability.

StudyThis analysis of the correlation betweenSpanish students’ final course grades, cu-mulative GPA, and language ability focuseson the following research question: Whatrelationship exists between Spanish FLstudents’ performance on interpersonaland interpretive communication tasks, theircourse grades, and their cumulative GPA?Clearly, sheer communicative FL ability isnot the only relevant construct that shouldcontribute to the assignment of a final FLcourse grade as other attributes and abilitiessuch as willingness to communicate, perse-verance, and work ethic surely contribute to

successful language learning and academicachievement. However, in the currentpedagogical climate in which authentic,contextualized, and performance‐basedlearning is valued, proficiency is a keypredictor of future success both withinand beyond the classroom.

MethodThis article reports exploratory findingsderived from three data sets representingstudents’ scores on performancemeasures inthree of the four traditional language skills:speaking, reading, and listening. Data Set 1comes from advisory ACTFL oral profici-ency interviews (OPIs) conducted by twoACTFL‐trained raters at the University ofKentucky with students from three differentlevels in the university’s Spanish languageprogram. Data Sets 2 and 3 reflect under-graduate Spanish students’ performancefrom the same undergraduate program ononline reading (Data Set 2) and listening(Data Set 3) tests designed and administeredby Avant Assessment, associated with theUniversity of Oregon’s Center for AppliedSecond Language Studies (CASLS).

Data AnalysisThe oral proficiency data set (Data Set 1)included each student’s oral proficiencyrating as the dependent variable and a rangeof independent variables taken from anacademic and demographic questionnaire.For the purposes of this study, severalvariables of interest, including SpanishGPA, cumulative GPA, self‐assessed OPIrating, and study abroad experience, weretested for correlation (association) with oralproficiency ratings. In addition, a generallinear model (GLM) was used to model oralproficiency rating in terms of study abroad,GPA, and the interaction of the two. Whilemore complex interaction models wereconsidered, these were not pursued due tothe reduced sample size and exploratorynature of the research.

In the case of the reading (Data Set 2)and listening (Data Set 3) assessments, a

82 SPRING 2013

percentage of correct responses rather than araw score was calculated and used as thedependent variable because each form of thetest included a different number of totalitems. The statistical procedures that wererun were predominantly descriptive, e.g.,range, median, and standard deviation, inaddition to tests for correlation, i.e., Spear-man and Pearson correlation coefficient,rather than more complex tests given thatthe sample of students was of modest size.

Results

Data Set 1—Interpersonal OralCommunicationData from 30 students from all of thedifferent courses and levels were analyzedas a single group. First, a marginallysignificant correlation existed betweenSpanish GPA and interpersonal oral perfor-mance (r ¼ 0.370, p < 0.044). Second,students’ self‐assessment of their oral skillswas significantly correlated with both theirSpanish GPA (r ¼ 0.552, p < 0.002) andcumulative GPA (r ¼ 0.471, p < 0.009).The highest correlations in this 4 � 4matrix were found between cumulativeGPA and Spanish GPA (r ¼ 0.752,p < 0.000) and between oral performanceratings and students’ self‐assessed level ofinterpersonal performance (r ¼ 0.761,p < 0.000). In addition, while graduating/graduated majors’ self‐assessments of inter-personal oral performance correlated signif-icantly with the researcher’s ratings(r ¼ 0.556, p < 0.031), the self‐assess-ments of their second‐year counterpartsdid not (r ¼ 0.399, p < 0.253). One finalanalysis from the data on oral performanceexamined the relationship between studyabroad, Spanish GPA, and oral performancerating. A univariate general linear modelshowed that, for those students who had notspent time abroad, Spanish GPA had aminimal effect on their oral proficiencyrating (slope ¼ 0.333 OPIpts./GPApts.,when abroad ¼ 0). On the contrary, forstudents who had travelled abroad, increasesin Spanish GPA exerted a significantly

positive impact on their proficiency rating(slope ¼ 6.974 OPIpts./GPApts., whenabroad ¼ 1).

Data Sets 2 and 3—Reading andListeningAlthough students were enrolled in twoconsecutive course levels (first‐year courses101/102 or second‐year courses 201/202),scores were analyzed as a group becausestudents in each course use the sametextbook and the four courses emphasizesimilar objectives, content, and pedagogy.Scores for students in two additionalcourses, SPA 210 (grammar/writing) and211 (conversation), were also grouped whencorrelated with students’ cumulative GPAbecause these two courses are both fifth‐semester bridge courses, represent the samestage in the program’s curriculum, and canbe taken concurrently. However, coursegrades for students enrolled in SPA 210and 211 were analyzed separately due to thediffering course objectives and content.

Several intriguing tendencies emergedfrom the statistical analysis. It was interest-ing to note the range of percentages on boththe interpretive listening and reading examsfor first‐year students (SPA 101/102) whoreceived a grade of A (Table 1). For Astudents, these ranges exceeded 30 points onboth exams. This range might simply reflectscores for the lowest 101 student and thehighest 102 student. In the case of studentsenrolled in the fifth‐semester bridge courses,however, a difference in course level cannotexplain the 33 percentage–point range onthe reading exam between A students (SPAin 210) and the 38 percentage–point rangein SPA 211 on the listening exam between Bstudents.

In only one case (SPA 201/202 reading)did both the median and mean percentagescores from a lower letter grade (C) surpassthe median/mean of the next highest lettergrade (B), a fact that might be attributed tothe very small number of test‐takers in thesesubgroups, two and three respectively(Table 2). Interesting trends surfaced with

Foreign Language Annals � VOL. 46, NO. 1 83

SPA 210 and 211, in which an A or B isrequired for students to advance to the firstmajor course within the program. Very fewstudents received a C, D, or F in SPA 210and 211, particularly in 211 (IntermediateSpanish Conversation), where only oneperson among those who participated inthe study received a C.

One encouraging finding, albeit ratherpredictable, was that the median/meanpercentage scores on each skill (listening/reading) for students receiving an A in-creased by course level. Spanish 101/102students who received an A performed at amedian/mean level of 57/58 on the listeningexam and 56/56 on the reading, while their201/202 counterparts’ median/mean per-centage score was 68/67 on the reading

and 66/65 on the reading. The median/meanpercentage scores for Spanish 210 and 211students, both of which are fifth‐semestercourses, were higher with 210 studentsscoring 72/71 on the listening exam and 81/79 on the reading exam and 211 studentsscoring 82/79 on the listening exam and 81/81 on the reading exam.

Overall, there were very few significantcorrelations between students’ performanceon the performance exams and their finalcourse grade or cumulative GPA. Finalcourse grades were significantly correlatedwith performance scores in only two of eighttotal comparisons (Table 1), while cumula-tive GPA significantly correlated with scoresin two of six comparisons (Table 2). That isto say that from a total of 14 comparisons,

TABLE 2

Correlation Between Exam Percentage Score and Cumulative GPA,

by Course Level

Listening Exam % Scores Reading Exam % Scores

n ¼ r ¼ p ¼ n ¼ r ¼ p ¼101–102 21 0.331 0.143 28 0.374 0.05201–202 28 0.734 <0.001

�11 0.297 0.376

210/211 20 0.036 0.880 49 0.512 < 0.001�

�significant at an alpha level of p < 0.05

TABLE 1

Correlation Between Exam Percentage Score and Course Grade,

by Course Level

Listening Exam % Scores Reading Exam % Scores

n ¼ r ¼ p ¼ n ¼ r ¼ p ¼101–102 21 0.55 0.01

�29 0.262 0.169

201–202 28 0.574 0.001�

11 0.146 0.668210 12 0.317 0.315 35 0.292 0.089211 8 0.387 0.344 14 0.385 0.174

�significant at an alpha level of p < 0.05

84 SPRING 2013

only four resulted in significant correlations.Three of the four significant relationshipsinvolved the listening exam andwere amongstudents in the first two years of universitystudy, i.e., 101–202.

DiscussionThe underlying assumption of this researchwas that students’ ability to use language forreal communicative purposes is one of themost important overall goals of postsecond-ary FL classroom education. Thus, at least tosome degree, communicative performanceshould be reflected in course grades andGPA so as to facilitate effective coursearticulation, subsequent classroom achieve-ment, and use of language for personalenjoyment and in professional settings.Although this article reports an exploratorystudy of the relationship between L2 coursegrades and L2 ability measured by externalinstruments of ability, several interestingtrends emerged. In regard to oral perfor-mance (Data Set 1), the oral performanceratings for students in this sample did notcorrelate significantly with their cumulativeGPA, but a significant correlation (r ¼0.044) did result between Spanish GPA andoral performance rating, suggesting thatSpanish grades were a slightly better predic-tor of oral interpersonal skill than cumula-tive GPA for this group of students. Also ofinterest was the finding that there was apositive relationship between Spanish GPAand performance for students who hadstudied abroad, while increases in SpanishGPA had no significant effect on perfor-mance for those who had not spent timeabroad, perhaps because students who studyabroad are more academically or personallymotivated.

The results from the listening andreading assessments indicated that studentswho received the same letter grade in aparticular course demonstrated a very largerange of performance scores: A 33 percent-age–point difference was found amongstudents receiving an A in SPA 210 on thereading exam, and a 38 percentage–point

difference resulted among students receiv-ing a B in SPA 211. Correlation analysesdemonstrated few significant relationshipsbetween course grade and reading/listeningtest score (4 of 14 total comparisons), withthree of the four significant correlationsoccurring on the listening exam amongfirst‐ and second‐year classes.

Apparently, students’ performance onthe listening exam, which contained authen-tic speech samplespresentedatnormal speed,more closely paralleled their course gradesthan did their performance on the readingexam.Whenpresented as an interpretive skillvia a recorded speech sample rather thaninteractively through interpersonal commu-nication, listening appears, not surprisingly,to be more difficult for lower‐level learnersthan reading because when engaged ininterpretive listening, learners’ short‐termmemory is taxed as well as their ability tosuccessfully parse the speech signal intomeaningful chunks. With literate test‐takers,the static nature of the printed text facilitatesform‐meaning connections.

ConclusionThe deconstruction of FL course grades andtheir correlation with constructs such aslanguage ability will not be an easy task andmust be undertaken at the program level.This exploratory study and its preliminarydata are indeed only a first step towardempirically examining the relationship be-tween course grades and external measuresof language ability, but they appear toindicate that, in this particular programwith this sample of students, the relation-ship is rather unpredictable. Future re-searchers should assess a larger sample ofstudents and collect performance and profi-ciency data in a more systematic longitudi-nal fashion, following a cohort of studentsthroughout a series of classes, or in a cross‐sectional fashion by looking at grade andperformance correlations at different pointsin the program’s curriculum. Furthermore,grades must be correlated with other rele-vant constructs including instructors’ beliefs

Foreign Language Annals � VOL. 46, NO. 1 85

about assessment and assessment practicesand “process”‐oriented (study habits, hardwork, homework completion, attendance,extra credit), and “progress”‐oriented per-spectives (learning gains over the instruc-tional sequence) so that variability in lettergrades may be more accurately explained.Finally, deconstructing course grades willallow FL learners and faculty to explicitlyidentify personal and institutional philoso-phies, values, unwritten norms, and idio-syncratic perceptions of grading behaviors,and it will encourage dialogue about theappropriateness of assessment types andtasks as well as their respective weights inthe final grade.

The researcher agrees with Norris(2006) that FL educators must presentmore compelling evidence, in a moretransparent and intelligible way, of thelearning gains that students experience asa result of classroom instruction. FL in-structors need not be psychometricians ormathematicians, but must clearly articulatehow each component of a student’s gradedirectly relates to (1) achievement of explicitcourse and program learning goals, and (2)increases in global proficiency. While it maynot be feasible, desirable, or equitable todemand that classroom grades at thepostsecondary level reflect only students’scores on external measures of languageability, all students at all stages in theirprogram of study are entitled to consistentinformation about what their grades com-prise and how such information is used togenerate an interpretable final grade thatreflects the program’s goals, institutionalvalues, and the student’s probable level ofsuccess in using language for varied pur-poses beyond the classroom. Likewise,upper‐level course instructors are equallyentitled to smoother articulation betweenlevels so that the heterogeneity of studentability does not require severe alteration ofclassroom pedagogy, course content, orboth. In short, FL course grades must betteralign with current understandings of com-municative competence and FL proficiencywhile serving the purposes of students,

instructors, program administrators, andsociety at large.

References

Alderson, J. C., & Beretta, A. (Eds.). (1992).Evaluating second language education.Cambridge, UK: Cambridge University Press.

Bachman, L. F. (1990). Fundamental consider-ations in language testing. Oxford: OxfordUniversity Press.

Bachman, L. F.,& Palmer, A. (1996). Languagetesting in practice. Oxford: Oxford UniversityPress.

Brown, H. D., & Abeywickrama, P. (2010).Language assessment: Principles and classroompractices. White Plains, NY: Pearson Education.

Canale, M. (1983). From communicativecompetence to communicative language peda-gogy. In J. C. Richards&R.W. Schmidt (Eds.),Language and communication (pp. 2–27). NewYork: Longman.

Canale, M., & Swain, M. (1980). Theoreticalbases of communicative approaches to secondlanguage teaching and testing. Applied Linguis-tics, 1, 1–47.

Gass, S., & Varonis, E. (1994). Input, interac-tion and second language production. Studiesin Second Language Acquisition, 16, 283–302.

Gronlund, N. E. (1998). Assessment of studentachievement. Needham Heights, MA: Allyn &Bacon.

Guskey, T. R. (Ed.). (1996). Communicatingstudent learning. 1996 Yearbook of theAssociation for Supervision and CurriculumDevelopment. Alexandria, VA: Association forSupervision and Curriculum Development.

Guskey, T. R. (Ed.). (2009). Practical solutionsfor serious problems in standards‐based grading.Thousand Oaks, CA: Corwin Press.

Guskey, T. R., & Bailey, J. M. (2001).Developing grading and reporting systems forstudent learning. Thousand Oaks, CA: CorwinPress.

Hughes, A. (2003). Testing for languageteachers. Cambridge, UK: Cambridge Univer-sity Press.

Hymes, D. (1972). Models of the interaction oflanguage and social life. In J. Gumperz & D.Hymes (Eds.), Directions in sociolinguistics:The ethnography of communication (pp. 35–71). New York: Holt, Rinehart & Winston.

86 SPRING 2013

Krashen, J. (1985). The input hypothesis: Issuesand implications. London: Longman.

Liskin‐Gasparro, J. E. (1995). Practical ap-proaches to outcomes assessment: The under-graduate major in foreign languages andliteratures. ADFL Bulletin, 26, 21–27.

Long, M. (1996). The role of the linguisticenvironment in second language acquisition.In W. Ritchie & T. Bhatia (Eds.), Handbook ofsecond language acquisition (pp. 413–468). SanDiego: Academic Press.

Lynch, B. K. (1996). Language programevaluation: Theory and practice. London: Cam-bridge University Press.

Magnan, S. S. (1986). Assessing speakingproficiency in the undergraduate curriculum:Data from French. Foreign Language Annals,19, 429–438.

McNamara, T. (2000). Language testing.Oxford: Oxford University Press.

Messick, S. (1989). Validity. In R. L. Linn (Ed.)Educational measurement. (3rd ed., pp. 13–103). New York: Macmillan/American Councilon Education.

Messick, S. (1994). The interplay of evidenceand consequences in the validation of perfor-mance assessments. Educational Researcher,23, 13–23.

Messick, S. (1996). Validity and washback inlanguage testing. Language Testing, 13, 241–256.

Modern Language Association. (2007). For-eign languages and higher education: Newstructures for a changed world [Electronicversion]. Retrieved May 9, 2012, from http://www.mla.org/flreport

Norris, J. M. (2006). The why and (how) ofstudent learning outcomes assessment incollege FL education. Modern Language Jour-nal, 90, 576–583.

Norris, J. M., Davis, J. M., Sinicrope, C., &Watanabe, Y. (Eds.). (2009). Toward usefulprogram evaluation in college foreign languageeducation. Honolulu: National Foreign Lan-guage Resource Center.

Swain, M. (1995). Three functions of output insecond language learning. In G. Cook & B.Seidelhofer (Eds.), Principle and practice inapplied linguistics: Studies in honor of H.G.Widdowson. (pp. 125–144). Oxford: OxfordUniversity Press.

VanPatten, B. (1996). Input processing andgrammar instruction in second language acqui-sition. Norwood, NJ: Ablex.

Received August 19, 2011

Accepted January 30, 2013

Foreign Language Annals � VOL. 46, NO. 1 87