Designing and Sustaining aForeign Language WritingProficiency Assessment Programat the Postsecondary LevelElizabeth BernhardtStanford University
Joan MolitorisStanford University
Ken RomeoStanford University
Nina LinStanford University
Patricia ValderramaStanford University
Abstract: Writing in postsecondary foreign language contexts in North America hasreceived far less attention in the curriculum than the development of oral proficiency.This article describes one institutions process of confronting the challenges not only ofrecognizing the contribution of writing to students overall linguistic development, butalso of implementing a program-wide process of assessing writing proficiency. Thearticle reports writing proficiency ratings that were collected over a 5-year period formore than 4,000 learners in 10 languages, poses questions regarding the proficiency
Elizabeth Bernhardt (PhD, University of Minnesota) is Professor of GermanStudies and John Roberts Hale Director of the Language Center, StanfordUniversity, Stanford, CA.Joan Molitoris (PhD, Columbia University) is Lecturer in Spanish and AssociateDirector of the Language Center, Stanford University, Stanford, CA.Ken Romeo (PhD, Stanford University) is Lecturer in English for Foreign Studentsand Academic Technology Specialist for the Language Center, Stanford University,Stanford, CA.Nina Lin (MA, Stanford University) is Lecturer in Chinese in the LanguageCenter, Stanford University, Stanford, CA.Patricia Valderrama (PhD candidate, Stanford University) is Graduate TeachingAssociate in Comparative Literature, Stanford University, Stanford, CA.Foreign Language Annals, Vol. 48, Iss. 3, pp. 329349. 2015 by American Council on the Teaching of ForeignLanguages.DOI: 10.1111/flan.12153
Foreign Language Annals VOL. 48, NO. 3 329
levels that postsecondary learners achievedacross 2 years of foreign language instruction,and relates writing proficiency scores to Sim-ulated Oral Proficiency Interview ratings for asubset of students. The article also articulatesthe crucial relationship between professionaldevelopment and writing as well as the role oftechnology in collecting and assessing writingsamples.
Key words: assessment, oral proficiency,technology, writing proficiency
IntroductionWriting in postsecondary foreign languagecontexts in North America has received farless attention in the curriculum than thedevelopment of oral proficiency. While atleast one extensive volume on the impor-tance of advanced writing in foreign lan-guage contexts exists (Byrnes, Maxim, &Norris, 2010), there remain a number ofreasons that account for the lack of bothresearch-based and instruction-based atten-tion to the development of foreign languagewriting throughout the early years of lan-guage acquisition. First, the focus of foreignlanguage instruction at the postsecondarylevel in the United States is most often onoral proficiency goals. Most foreign lan-guage programs intend to prepare learnersto speak and listen and to read so that theyare able to negotiate overseas foreign set-tings with confidence. Hence, oral profi-ciency is emphasized and writing in thesecontexts is often relegated to exercises thatreveal learners acquisition of grammaticalforms or developing breadth of vocabulary,to the thank-you note to foreign hosts or thepersonal resume, or as a means of measur-ing syntactic complexity. In upper-levelcourses, programs may even allow compo-sitions to be written in the native languageof students, English in the American con-text, in order to facilitate deeper literary andcultural interpretations. Even dissertationsproduced in foreign language departmentsin many American universities are written
in English. This phenomenon stands instark contrast to the field of English as asecond language, which attempts to prepareEnglish language learners with the skillsthat they will need to pursue bachelors orpost-bachelors degrees in English-speakingcountriesa project that necessarily entailscopious amounts of academic writing. Anumber of research studies such as Leki(1995) and Leki and Carson (1997) existon this topic and have been synthesizedsuccinctly by Hedgcock (2005).
Another speculation for the lack of fo-cus on foreign language writing is that writ-ing is a planned language performance, incontrast to the spontaneous language per-formance of oral proficiency, and, hence, isviewed as less demanding. The languagedevelopment research that has dominatedstudies in second language acquisition hasbeen principally rooted in oral assessments,with literacy (writing or reading) rarely ac-knowledged as an important dimension ofinput (Bernhardt, 2011). With the excep-tion of Byrnes and colleagues (2010), whocontended that writing provides learnerswith the ability to perform in genres andhence is a particularly valued indicator ofoverall FL development toward upper levelsof ability (p. 4), most research has ignoredlearners abilities to integrate spoken andwritten texts; even fewer studies referredto foreign language writing in terms of rela-tively lengthy connected discourse. Reichelt(1999) thoroughly reviewed these and otherdilemmas confronted by the context of for-eign language writing.
The development of the foreign lan-guage profession itself from the 1980s on-ward has lacked perspective on writingdevelopment. In the early years of the pro-ficiencymovement, the focuswas exclusive-ly on oral proficiency and centered on aconsistent measure of student performance,most especially in their oral performance.Admittedly, the ACTFL Proficiency Guide-lines entailed reading, writing, listening,and speaking from their initial publicationin 1982, with the primary emphasis on oralproficiency. This was evidenced by a full-
330 FALL 2015
scale certification process, attached only tooral proficiency interviewing and ratingwith concomitant recertification possibili-ties, that was developed throughout the1980s. The potential for assessment in read-ing, listening, and writing, comparable tooral proficiency assessment, remained un-tapped for more than a decade. Writingproficiency has, of course, been includedfrom the inception of the formal discussionaround proficiency rooted in the ForeignService Institute (FSI) guidelines. An earlyversion of the ACTFL guidelines that in-cluded all four skills appeared in 1986,and a further version also accompaniedthe revised guidelines for oral proficiencyin 1999 (Breiner-Sanders, Lowe, Miles, &Swender, 2000). In 2001, the PreliminaryProficiency GuidelinesWriting Revised(Breiner-Sanders, Swender, & Terry,2002) were published. In parallel to theprocess involving oral proficiency, the writ-ing guidelines were widely distributed, andultimately a training and certification pro-cedure was put into place by 2008. Further,a writing protocol, the Writing ProficiencyTest (WPT), was developed and made avail-able commercially via online delivery aswell as in hard copy. In spite of the consid-erable research that exists on oral proficien-cy attainment, i.e., the number of hours ofinstruction related to proficiency levels(Omaggio, 1986) and the ultimate attain-ment of language majors in their oral profi-ciency (Glisan, Swender, & Surface, 2013;Swender, 2003), limited published data ex-ist with regard to the role of writing inforeign language programs or the use ofthe ACTFL writing guidelines.
In light of the complexities of consider-ing the role of writing in the postsecondaryforeign language programs, the limited re-search base, and a continued focus on oralproficiency, this article describes a pro-gram-wide approach to assessing studentswriting proficiency. It offers writing profi-ciency data from for 4,476 postsecondarylearners in 10 languages across 2 years offoreign language instruction. Further, itcompares those writing data with oral
proficiency ratings for a subset of learnersand provides insight into the role of tech-nology and its influences on writing perfor-mance. Specifically, it poses the followingquestions:
1. What writing proficiency ratings arelearners able to achieve across the levelsof a 2-year foreign language sequence?Are there differences based in Englishcognate vs. English noncognatelanguages?
2. What is the relationship between foreignlanguage learners oral proficiency rat-ings and their writing proficiency rat-ings? Are there differences in therelationship based in English cognateand English noncognate languages?
3. Does technology use influence foreignlanguage writing assessment?
Literature ReviewSeveral studies have examined foreign lan-guage writing as a support mechanism forlanguage learning and have used or men-tioned the ACTFL guidelines as bench-marks. Armstrong (2010) comparedgraded compositions; for-credit online dis-cussion boards; and ungraded, not-for-cred-it essays in order to determine the effect ofgrades on foreign language writing in afourth-semester Spanish class. Specifically,she tried to understand differences in theaccuracy, fluency, and complexity betweengraded and ungraded assignments, buildingon previous work done with the ACTFLwriting guidelines in the curriculum; Arm-strong suggested that more frequent andungraded writing assignments should beincorporated into the foreign languageclassroom, since assessment had little effecton student writing. Brown, Brown, and Eg-gett (2009) also looked to writing as amechanism for enhancing language devel-opment. They described a curriculum forthird-year foreign language courses thatwas grounded in content-based instructionto enhance written argumentation and to
Foreign Language Annals VOL. 48, NO. 3 331
differentiate it from oral production. Theaim of the curricular shift was to help stu-dents cross the Intermediate-Advanced bor-der, as defined by the writing guidelines.They found that a focus on Advanced-and Superio