Performance of English Language Learners as a Subgroup in Large-Scale Assessment: Interaction of Research and Policy

Performance of EnglishLanguage Learners as aSubgroup in Large-ScaleAssessment: Interaction ofResearch and PolicyJamal Abedi and Patricia Gandara, University of California, Davis

Factors that contribute to the performance gap betweensubgroups and mainstream students deserve special attention.Different subgroups are faced with different sets of challenges.To understand and control for factors leading to the performancegap between subgroups and mainstream students one mustclearly understand the issues specific to each subgroup. Thispaper focuses on assessment and performance issues forEnglish language learner (ELL) students as a subgroup.Identifying factors affecting the performance gap between ELLand non-ELL students may help gain insight into assessmentissues for other subgroups of students as well as strengthenassessment of this group.

Keywords: assessment, language, English language learners,accountability, accommodation

PerspectiveCurrent legislative mandates in theNo Child Left Behind (NCLB Act of2002) require the states to reportperformance of all students, includ-ing students in major ethnic groups,economically disadvantaged students,students with disabilities, and Englishlanguage learners (ELL). While thisrequirement to report by major sub-groups brings highly needed attentionto achievement issues for these stu-dents, it also introduces a whole newset of technical and ethical issues inassessment.

Different subgroups are faced withdifferent sets of challenges. Therefore,to understand and control for factorsleading to the performance gap be-tween subgroups and mainstream stu-dents, one must clearly understand the

issues specific to each subgroup. Identi-fying factors affecting the performancegap between ELL and non-ELL studentsmay help gain insight into assessmentissues for other subgroups of studentsas well. Thus, the focus of this paperis on the specific issues associated withELL academic performance.

English language learners nownumber over 4.5 million students; theyrepresent about 8% of all K-12 studentsnationally (Zehler et al., 2003) and arethe fastest growing subgroup in thenation (Kindler, 2002). Consequently,fairness and validity issues relating totheir assessment must be among thetop priorities of the national educa-tion agenda. ELL students have histori-cally lagged behind their English profi-cient peers in all content areas, partic-

ularly academic subjects that are highin English language demand. The litera-ture suggests that this performance gapis explained by many different factorsincluding parent education level andpoverty (Abedi, Leon, & Mirocha, 2003),the challenge of second language acqui-sition (Hakuta, Butler, & Witt, 2000;Moore & Redd, 2002), and a host of in-equitable schooling conditions (Abedi,Herman, Courtney, Leon, & Kao, 2004;Gandara, Rumberger, Maxwell-Jolly, &Callahan, 2003) in addition to measure-ment tools that are ill-equipped to as-sess their skills and abilities.

English language learners are morelikely to be taught by teachers withoutappropriate teaching credentials andwith little classroom experience thanother students (Rumberger & Gandara,2004). A recent study of 4800 teachersof ELL students in California respond-ing to a survey about the challengesthey faced in teaching ELLs in theirclassrooms found large percentages ofthese teachers expressing the concernthat they were not prepared to teachthese students (Gandara et al., 2003).

While many ELL students are nativeborn, many also have at least one parentwho is an immigrant, and overwhelm-ingly they are reared in homes in whichEnglish is not the primary languagespoken. Moreover, a disproportionately

Jamal Abedi, School of Education,University of California, Davis, OneShields Avenue, Davis, CA 95616-5270;[email protected].

Patricia Gandara, School of Educa-tion, University of California, Davis, OneShields Avenue, Davis, CA 95616-5270;[email protected].

36 Educational Measurement: Issues and Practice

0

10

20

30

40

50

60

Reading Math General knowledge

Per

cent

abo

ve 5

0th

perc

enti

le

English Non-English

0

10

20

30

40

50

60

Reading Math General knowledge

Per

cent

abo

ve 5

0th

perc

enti

le

English Non-English

FIGURE 1. Cognitive skills of beginning kindergarteners by language back-ground, 1998.Source: U.S. Department of Education. National Center for Education Statis-tics. America’s Kindergartners, NCES 2000-070, by Kristin Denton, ElviraGermino-Hausken. Project Officer, Jerry West, Washington, DC: 2000. Re-trieved January 10, 2006 from http://nces.ed.gov/pubs2000/2000070.pdf.

high percentage of these studentsare raised in poverty and attend poorschools where they receive weakinstruction (Gandara & Rumberger, inpress). ELL students begin school sig-nificantly behind their English speak-ing peers and so require extra time andinstruction from the very beginning ofschool to catch up (See Figure 1).

Within this context, ELL studentsmust take on the challenges of learningEnglish and U.S. culture, in addition tolearning the academic content of thecurriculum. The process of developingnew language skills is difficult andrequires time and effort that other-wise could be spent on acquiring aca-demic content knowledge. For exam-ple, a number of studies have shownthat it takes from five to seven yearsand even more for most ELLs to gainsufficient mastery of academic Englishto join English speaking peers in tak-ing full advantage of instruction in En-

glish (Hakuta, Butler, & Witt, 2000).During this time, learning cannot oc-cur at the same rate as for a nativespeaker of English when that instruc-tion is offered only in English. LimitedEnglish proficiency may also make it dif-ficult to benefit fully from the teacher’sinstructions and to understand assess-ment questions.

Because ELL students, by definition,do not have a strong command of theEnglish language, both learning and as-sessment are affected by their limitedEnglish proficiency, and both learningand assessment conditions must be ad-dressed to help close the performancegap. Standardized achievement teststhat have been constructed for main-stream students, but which do not takeinto account the special needs of En-glish Learners, can provide a furtherchallenge for these students, and maynot provide a good indication of whatthese students know and can accom-

plish. Findings from studies suggestthat a major factor contributing to theperformance gap between ELL and non-ELL students is the impact of linguis-tic complexity of assessments (Abedi,2006a; Abedi, Hofstetter, & Lord, 2004).Sometimes ELL students are providedwith accommodations to help offset thelinguistic challenge of the test, butthese accommodations may not be rel-evant or helpful, and almost always fallshort of ideal.

In this paper, we will discuss cog-nitive and non-cognitive (conative andaffective) factors that contribute toa performance-gap between ELL andmainstream students, how such factorsinteract with academic achievement ofELL students and how research findingscan help resolve some of these issues.

While the main thrust of this paperis on the cognitive aspect of academicachievement of ELL students, we willalso touch on the conative and affec-tive factors that may influence ELL stu-dents’ academic progress. In the cogni-tive part of this paper we will first dis-cuss the development of reading skillsamong English language learners andthen examine the impact of languagefactors on the academic achievementof ELL students, in particular, on theassessment and accountability systemsfor these students. We will then elabo-rate on how research findings, thoughlimited in content and scope, can helpaddress some of the challenges that ELLstudents are faced with in their aca-demic careers.

Cognitive Factors1. The Development of Reading Skillsamong English Language Learners

Because reading is implicated inmost assessments of cognitive skill, andbecause it is the fundamental windowthrough which we have access to cog-nitive processes, we begin with the lit-erature on learning to read. The newlyreleased report from the National Liter-acy Panel (August & Shanahan, 2006)calls attention to the problems of as-sessment for English language learners.They note that

Most assessments do a poor jobof gauging individual strengths andweaknesses. The research on the de-velopment of English literacy stronglysuggests that adequate assessmentsare essential for gauging the individ-ual strengths and weaknesses of lan-guage minority students. . . .[U]nfortunately, existing assess-ments are inadequate to the need

Winter 2006 37

in most respects. For example,most measures do not predict howwell language-minority students willperform over time on reading orcontent-area assessments in English(August & Shanahan, 2006, ExecutiveSummary, p. 6).

Theories about how second languagelearning takes place often identify fac-tors that may impact ELL students’assessment outcomes. For example,several cognitive processes are as-sumed to be imperative to the devel-opment of reading skills in English.These processes include phonologicalprocessing, syntactic awareness, andworking memory (for a review of thisliterature, see Siegel, 1993). Althoughthere is a dearth of research that ex-amines whether these same processescharacterize the reading developmentof English language learners (Lesaux &Siegel, 2003), we cite a few studies thatare relevant to the topic of assessmentof second language learners.

One study by Chiappe, Siegel, andWade-Woolley (2002) investigated theprocesses involved in learning to readamong a sample of 727 native Englishspeakers and 131 English languagelearners. While the English languagelearners performed more poorly thannative speakers of English on phonolog-ical and linguistic processing tasks, thedevelopment of literacy skills followed asimilar trajectory for both groups. Thissuggests that while the process of learn-ing to read may follow the same pattern,more time may be necessary for ELLsto actually acquire the necessary skills.The implications for test makers arethat they must be cognizant of these fac-tors when developing, administering,and scoring test items for ELL students.

Some research has focused specif-ically on the transfer of phonologicalawareness skills from the first languageto the second language. Durgunoglu,Nagy, & Hancin-Bhatt (1993) exam-ined 27 Spanish-speaking first gradersto find whether phonological aware-ness in a child’s native language isrelated to their word recognition inthe second language. In addition, theresearchers investigated whether orallanguage competence in the secondlanguage impacts acquisition of read-ing in the second language. The re-sults of this study indicated that Span-ish phonological awareness predictedreading acquisition in the second lan-guage, suggesting that transfer of pre-reading skills can occur across lan-guages. Moreover, as August and Shana-han (2006) conclude, “the research

indicates that instructional programswork when they provide opportunitiesfor students to develop proficiency intheir first language” (Executive Sum-mary, p. 5). These findings have par-ticular implications for use of primarylanguage assessments in order to betterpredict which students will need extrahelp in English reading. Unfortunately,policy in many states serving large num-bers of English learners has workedagainst the development of strong pri-mary language assessments.

Most recently, a study by Gottardo,Pannucci, Kuske, & Brettin (2003)showed evidence for simultaneousphonological processing in English andthe primary language for English lan-guage learners. A sample of 65 childrenwhose first language was Cantonese andsecond language was English was ex-amined to determine the relationshipbetween phonological skill and first andsecond languages. The study found thatphonological skill was correlated be-tween the first and second language.This is an interesting finding consid-ering that Cantonese, unlike English,is not written in alphabetic orthogra-phy. This provides further evidence ofthe transfer of basic pre-reading skillsacross languages and the utility of as-sessing at least some skills in students’primary languages.

The areas of syntactic awareness andworking memory have been studied verylittle in connection to English languagelearners’ acquisition of reading. How-ever, a study by Geva, Yaghoub-Zadeh,and Schuster (2000) briefly addressedsyntactic awareness. The findings ofthis study indicated that when com-pared to native English speaking stu-dents, average English language learn-ers in reading had a deficit in syntacticawareness skills. The lesson from thisresearch appears to be that test makerswould do well to reduce syntactic com-plexity in test items that will be used forELL students, eschewing atypical sen-tence formats.

Working memory as well as oral En-glish proficiency are two competenciesthat bode well for English reading de-velopment. In a study of working mem-ory and second language acquisitionconducted by Geva and Siegel (2000)among a sample of English speakerslearning Hebrew, the researchers foundthat verbal memory was a significantpredictor of basic reading skills. Like-wise, August and Shanahan (2006)summarize one of the primary findingsof the National Reading Panel as thecritical importance of oral proficiency

in English and in the primary languageas precursors of strong reading skills inEnglish. Again, this research suggeststhat the testing of verbal memory andoral proficiency in either the first or sec-ond language will yield important infor-mation about students’ readiness to un-dertake reading instruction in English.

While the summary presented aboveclearly shows the challenges that En-glish language learners are faced within learning a new language, it alsosuggests that students are transfer-ring their pre-reading skills across lan-guages and so the time they spendlearning in any language is going to beroughly predictive of learning to read.As such, knowledge of what studentsknow in both primary and secondarylanguages can yield important infor-mation about the cognitive processingskills they have for learning to read.

2. Language Factors and Assessment ofELL Students

Title III of No Child Left BehindAct (NCLB 2002) requires states tomeasure ELL students’ level of En-glish proficiency (in addition to theTitle I assessment requirements forthese students) in four domains: read-ing, writing, listening, and speaking(and comprehension). Therefore, stu-dents’ level of reading proficiency ob-viously plays a major role in their as-sessment outcomes since without pro-ficiency in reading, students will havedifficulty understanding test questions.Proficiency in other domains is also es-sential for a valid assessment. In thissection we use the term “English profi-ciency” in a broader sense to includevarious aspects of students’ level ofproficiency.

There are many factors affecting aca-demic progress of ELL students thatare influenced by students’ level of En-glish proficiency in general. For exam-ple, linguistic complexity of assessmentmay impact the performance outcomeof these students and increase the per-formance gap between ELL studentsand their native English speaker peers.Invalid assessment results may affectdecisions regarding classification andinclusion of ELL students and these allmay impact the validity and fairnessof the accountability system for thesestudents. We first discuss the impactof language factors on the assessmentoutcomes of ELL students and then pro-vide some research-based recommen-dations on how to improve assessmentfor these students by controlling forlanguage factors that could impact the


validity of assessment for these stu-dents. We will also discuss language-based accommodations that would helpELL students with their English lan-guage needs.

Impact of language factors on the valid-ity of assessment for ELL students. TheNational Research Council has warnedthat the use of achievement tests devel-oped for English-speaking students willnot likely yield valid results for studentswho are not proficient in English. Asthe NRC noted, “if a student is not pro-ficient in the language of the test, herperformance is likely to be affected byconstruct-irrelevant variance—that is,her test score is likely to underestimateher knowledge of the subject matter be-ing tested (Huebert & Hauser, 1999,p. 225). We know that consistently lowtest scores often lead to placement inremedial and low-level instruction thatfurther disadvantages already disad-vantaged learners (Heubert, 2000) andexacerbates existing achievement gaps.Hence the consequences of such testingcan be serious for these students.

Findings from several studies con-ducted recently show the impact oflanguage factors on the assessment ofELL students (Abedi, 2006a; Abedi, Hof-stetter, & Lord 2004; Maihoff, 2002;Kiplinger, Haug, & Abedi, 2000). Thesefindings suggest that unnecessary lin-guistic complexity may hinder ELLstudents’ ability to express their knowl-edge of the construct being measured.Results from these studies question theappropriateness of mainstream assess-ments for English language learners, anissue that has long been debated. Ed-ucational psychologists, measurementexperts, and educational practitionershave raised concerns over testing ELLstudents with assessment tools that aredeveloped for native speakers of En-glish, and which have been field testedmainly for these students:

For all test takers, any test that em-ploys language is, in part, a mea-sure of their language skills. This isof particular concern for test takerswhose first language is not the lan-guage of the test. Test use with indi-viduals who have not sufficiently ac-quired the language of the test mayintroduce construct-irrelevant com-ponents to the testing process. Insuch instances, test results may notreflect accurately the qualities andcompetencies intended to be mea-sured . . . Therefore it is importantto consider language background in

developing, selecting, and adminis-tering tests and in interpreting testperformance. (American EducationalResearch Association, American Psy-chological Association, & NationalCouncil on Measurement in Educa-tion, 1999, p. 91)

In the assessment of ELL students,research suggests that unnecessary lin-guistic complexity introduces a sourceof measurement error and is consideredas a construct-irrelevant factor in theassessment (Abedi, 2006a; Abedi, Leon& Mirocha, 2003). Thus, language fac-tors may seriously confound the out-comes of instruction and assessmentin content-based areas (Abedi, Lord, &Hofstetter, 1998; Cocking & Chipman,1988; De Corte, Verschaffel, & DeWin,1985; Kintsch & Greeno, 1985; Larsen,Parker, & Trenholme, 1978; Lepik, 1990;Mestre, 1988; Munro, 1979; Noonan,1990; Orr, 1987; Rothman & Cohen,1989; Spanos, Rhodes, Dale, & Crandall,1988).

The results of a series of studieson the impact of language on assess-ment of ELLs resulted in two majorconclusions: (1) reducing the linguisticcomplexity of assessment tools helpedELL students to perform significantlybetter because it reduced the perfor-mance gap between ELL and non-ELL,and (2) the process of reducing linguis-tic complexity of test items did not al-ter the construct under measurement.Therefore by reducing the linguisticcomplexity of assessment tools, the va-lidity of assessment was not compro-mised (see Abedi, Lord, Hofstetter, &Baker, 2000; Abedi & Lord, 2001; Abedi,Courtney, Mirocha, Leon & Goldberg,2005; Kiplinger, Haug, & Abedi, 2000;Abedi, Lord, and Hofstetter, 1998;Abedi, Lord, and Plummer, 1997).

In other studies on the impact oflanguage factors on the assessment ofELL students, extant data from theNational Assessment of EducationalProgress (NAEP) were analyzed. Re-sults (see, for example, Abedi, Lord,& Plummer, 1997) suggested that ELLstudents had difficulty with test itemsthat were linguistically complex. Theresults also indicated that ELL studentsexhibited substantially higher numbersof omitted/not-reached test items.

Linguistic features that researchersidentified in test items slow down thereader, make misinterpretation morelikely, and add to the reader’s cognitiveload, thus interfering with concurrenttasks. Indexes of language difficulty in-

clude word frequency/familiarity, wordlength, and sentence length. Other lin-guistic features that may cause diffi-culty for readers include long nounphrases, long question phrases, pas-sive voice constructions, comparativestructures, prepositional phrases, sen-tence and discourse structure, subordi-nate clauses, conditional clauses, rel-ative clauses, concrete versus abstractor impersonal presentations, and nega-tion (Abedi, Lord, & Plummer, 1997).

3. Language Factors and Accountabil-ity System for ELL Students

The impact of language factors on theassessment outcomes for ELL studentshas direct consequences on the ac-countability system for these students.Under the NCLB Title I accountabil-ity requirements, ELL students are as-sessed with the same tests that are de-veloped and field tested mainly for na-tive speakers of English. Earlier in thispaper we discussed challenges that ELLstudents are faced with when takingthese tests. Due to the complex linguis-tic structure of these tests, ELL stu-dents’ performance outcomes are verylikely to be underestimated. Unfortu-nately, however, there is no simple ap-proach to determine the level of under-estimation that may occur due to thelinguistic complexity of these tests.

The large performance gap betweenELL and non-ELL students—whichis caused by many factors, includingparent education, poverty, the chal-lenge of second language acquisition,and inequity in opportunity to learn—establish a substantially lower baselinescore for ELL students. That is, fromthe very beginning, these students areclearly at risk of failing. It is quite clearthat with starting low and progress-ing lower than their English proficientpeers, ELL students cannot make therace for reaching the intended goal of100% proficiency by the 2014 target dateor even any time soon after that date.

Data from the California Departmentof Education for 2005 reveal how keysubgroups are performing on NCLB ac-countability measures. Of the Englishlearners tested in California, 79% weredesignated as not meeting proficiencystandards. In addition, of the studentswith disabilities tested, 83% were clas-sified as non-proficient (California De-partment of Education, 2006).

NCLB also requires states to annuallytest English language learners on theirprogress toward gaining proficiency in

Winter 2006 39

English, and some states have devel-oped their own tests of English profi-ciency for this purpose. In California,the state developed test is the Cali-fornia English Language DevelopmentTest (CELDT). As concern over the lackof progress in closing the achievementgap between English language learnersand English speakers has intensified,debates have ensued about the appro-priate level of proficiency as measuredby the CELDT at which to place stu-dents into the regular English-only cur-riculum (“basic”, “early advanced”, or“advanced”) (Parrish et al., 2006). Pres-sure to mainstream students at lowerlevels of proficiency has increased.Katz, Low, Stack, & Tsang (2004), intheir study of San Francisco ELL stu-dents, also found little relationship be-tween proficiency on the CELDT andstudents’ ability to navigate the con-tent of English-only standardized tests.They argued that the CELDT and stan-dardized tests of academic competen-cies are either measuring very differ-ent constructs or measuring the sameconstructs at very different levels of dif-ficulty, but in either case, English profi-ciency levels established by English lan-guage proficiency tests such as CELDTshould not be used to determine stu-dents’ readiness to take on English-onlycoursework that requires academic asopposed to social English. Gandara andRumberger (in press) likewise foundthat 60% of tenth grade English lan-guage learners in 2005 were able to passthe CELDT at a level of early advancedor advanced English proficiency, butonly 3% of these students could pass thestate test of English Language Arts ata level of proficient—again suggestingthat the tests measure different types oflanguage proficiency, or have differentstandards of language proficiency.

Linn (in press) indicated that forlarge schools with diverse student pop-ulations, the disaggregated require-ment can create major obstacles inreaching their AYP goal. “A school witha large enough number in, say, threeracial/ethnic groups, students with lim-ited English proficiency, economicallydisadvantaged students, and studentswith disabilities, would have a total of29 hurdles to clear, 4 for each of the6 subgroups plus the 5 that all schoolshave for the total student body” (p. 21).Sunderman and Kim (2004) expressedconcerns over the racial and ethnic eq-uity of NCLB in states with a large num-ber of minority enrollments. Nichols,

Glass, and Berliner (2005) found thatstates with a larger number and pro-portion of minority students are un-der more pressure. They concludedthat problems associated with high-stakes assessment disproportionatelyaffect minority students.

4. Accommodation Issues in the Assess-ment of ELL Students

To assist ELL students with theirEnglish language limitations, severalaccommodation strategies have beenproposed. However, results of stud-ies on the assessment and accommo-dations for ELL students have raisedconcerns over appropriateness of someof these accommodations. For exam-ple, researchers have found that manyof the commonly used accommoda-tions for ELL students are neither ef-fective in helping ELL students withtheir language barriers nor are the re-sults of these accommodated assess-ments valid (see, for example, Abedi,Lord, Hofstetter, & Baker, 2000; Abedi,Hofstetter, & Lord, 2004; Sireci, Li, &Scarpati, 2003).

To investigate the relevance of ac-commodations used for ELL students,Abedi (2006b) analyzed a list of 73accommodations that have been usednationally in the assessment of ELL stu-dents (Rivera, 2003). The results indi-cated that of the 73 accommodationsused by different states only 11 (or 15%)of them were highly relevant for thesestudents. The main questions here are:How do states decide what accommoda-tion to use for English language learn-ers and students with disabilities? Whattype of criteria they use for such deci-sions? Do states use research findingsas criteria in their decisions?

To answer these questions and tounderstand the states’ policies andpractices regarding inclusion and ac-commodations for students with dis-abilities (SDs) and ELL students, eightstates with large numbers of ELLsand/or students with disabilities weresurveyed (Pitoniak, Lutkus, Cahalan-Laitusis, Cook, & Abedi, 2006.) Statetesting directors were asked to identifycriteria used in their decision about thetype of accommodations offered to stu-dents and about their policies regardinginclusion of both SDs and ELLs in NAEPand state assessments. Reponses of theparticipating states across the two as-sessments (state and NAEP) were sim-ilar. Therefore, we discuss the resultsof these interviews regarding state as-

sessments both in terms of decisionsregarding accommodations and the in-clusion of students. In this paper, welimit our discussion to the results forELL students.

In general, the interview results sug-gest that decisions regarding both in-clusion of and accommodations for ELLstudents were mainly based on theguidelines and recommendations fromthe state department of education. Allof the eight participating states in thisinterview indicated that accommoda-tions for ELL students are “determinedby following guidelines provided by thestate department of education” and onlytwo of the eight indicated that “accom-modations must be language relatedto assist LEP students with their lan-guage needs.” This raises serious con-cerns over the appropriateness of ac-commodations for these students.

It is important to examine the pat-tern of responses in these interviewsin terms of recent research findings inthe area of accommodations and inclu-sion of English language learners. Itdoes not seem that research findingsgreatly influenced the decision makingprocess. For example, when states wereasked whether they determine accom-modations based on opinion or researchevidence, only two of the eight statesindicated that they determine NAEPaccommodations based on research ev-idence.

When states were asked about spe-cific research evidence affecting theirdecisions regarding accommodationsand inclusion of ELL students, a ma-jority of them did not find researchevidence to influence their decisions.There are, however, some examplesof research findings that have a di-rect bearing on issues of accommoda-tion and inclusion. Results from experi-mentally controlled studies suggest ac-commodations that are language-basedor consistent with students’ languageneeds are more effective and valid forELL students than those originally cre-ated and proposed specifically for stu-dents with disabilities (see, for exam-ple, Abedi, Lord, Hofstetter, & Baker,2000; Abedi, Hofstetter & Lord, 2004).In contrast, the interviews show onlytwo of the eight states indicated thatthey use accommodations that addressELL students’ language needs. Simi-larly, research evidence supports theuse of the length of time students havebeen instructed in the United Statesas well as level of English proficiency


as valid criteria for decisions regardingappropriate accommodations for ELLstudents (Abedi, Lord, Hofstetter, &Baker, 2000; Abedi, Hofstetter, & Lord,2004). The results from the interviewssuggested that only three of the eightstates reported using such criteria forselecting appropriate accommodationsfor ELL students.

5. Research Findings that may Help Im-prove the Validity of Assessment and Ac-countability Systems for ELL Students

Results from studies on the assess-ment and accommodations of ELL stu-dents can help to identify assessmentissues and provide research-based rec-ommendations for improving assess-ment and accountability systems forELL students. For example, research onthe impact of language factors on the as-sessment of ELL students demonstratesthat unnecessary linguistic complexityof assessment may jeopardize the va-lidity of assessment and accountabil-ity system and provides suggestions forimproving the assessment and account-ability systems by reducing the unnec-essary linguistic complexity of assess-ments.

Abedi and Lord (2001) examinedthe effects of the linguistic modifica-tion approach with 1,031 eighth-gradestudents in Southern California. Inthis study, NAEP mathematics itemswere modified to reduce the complex-ity of sentence structures and to re-place potentially unfamiliar vocabu-lary with more familiar words. Content-related terminologies, i.e., mathemati-cal terms, were not changed. The re-sults showed significant improvementsin the scores of ELL and non-ELL stu-dents in low and average-level mathe-matics classes, but no effect on scoresamong other non-ELL students. Amongthe linguistic features that appeared tocontribute to the differences were low-frequency vocabulary and passive voiceverb constructions.

In another study, Abedi, Lord, andHofstetter (1998) examined the impactof language modification on the math-ematics performance of English learn-ers and non-English learners on a sam-ple of 1,394 eighth graders in schoolswith a large number of Spanish speak-ers enrolled. Results showed that mod-ifications to the language of items con-tributed to improved performance on49% of the items. The ELL studentsgenerally scored higher on shorter/lesslinguistically complex problem state-ments. In another study on a sample of

946 eighth graders Abedi, Lord, Hofstet-ter, & Baker (2000) found that amongfour different accommodation strate-gies for ELL students, only the linguis-tically modified English form narrowedthe score gap between English learnersand other students.

Another study (Abedi, Courtney, &Leon, 2003) examined 1,594 eighthgrade students using items from NAEPand TIMSS. Students were given one ofthe following accommodations: a cus-tomized English dictionary (words wereselected directly from test items), abilingual glossary, a linguistically mod-ified test version, or the standard testitems. Only the linguistically modifiedversion improved the ELL students’scores without affecting the non-ELLstudents’ scores.

Other researchers have employeda language modification approachbased on CRESST studies. Maihoff(2002) found linguistic modification ofcontent-based test items to be a validand effective accommodation for ELLstudents. Kiplinger, Haug, and Abedi(2000) found linguistic modification ofmath items to help improve the perfor-mance of ELL students in math. Riveraand Stansfield (2001) compared En-glish learner performance on regularand simplified fourth and sixth gradescience items. Although the small sam-ple size did not show significant differ-ences in scores, the study did demon-strate that linguistic simplification hadno effect on the scores of English-proficient students. This indicates thatlinguistic simplification is not a threatto score comparability.

Research findings can also help iden-tify accommodations that make assess-ments more accessible to ELL students.The results from several studies havedemonstrated that some accommoda-tions may impact the validity of as-sessments. For example, some formsof accommodation strategies, such asglossary use plus extra time, raisedperformance of both ELL and non-ELL students. The level of increasedue to such accommodation strategieswas higher for non-ELL students. Thisraised concerns regarding the validityof these accommodations (Abedi, Lord,Hofstetter, & Baker, 2000; Abedi, Lord,& Hofstetter, 1998).

English and bilingual dictionarieswere used as different forms of ac-commodation strategies. The results ofthese studies suggested that by gain-ing access to the definition of content-related terms, recipients of dictionar-

ies may be advantaged over those whodid not have access to dictionaries. Thismay jeopardize the validity of assess-ments. The dictionary as a form of ac-commodation suffers from yet anothermajor limitation, the feasibility issue.For a variety of reasons, it was lo-gistically difficult to provide this formof accommodation to students (Abedi,Courtney, Mirocha, Leon, & Goldberg,2005).

There are however, some accommo-dations that help ELL students withtheir English language needs that donot affect the performance of non-ELL students. For example, studies byCRESST researchers along with otherstudies nationwide suggested that lin-guistically modified versions of testsitems are an effective and valid ac-commodation for ELL students (Abedi,Lord, & Hofstetter, 1998; Abedi, Lord,Hofstetter, & Baker, 2000). Duncanet al. (2005), using NAEP mathemat-ics test data for 402 Spanish-speakingeighth-grade students, found that thestudents appreciated and preferreddual-language test booklets, with no un-due advantage using the booklet. Theseinvestigators found the method hope-ful, but cautioned that it was importantto do more study to determine for whichcategories of ELLs the dual-languagetest booklets appear to be mostuseful.

Non-Cognitive Factors1. Communicative Competence andSocio-cultural Factors

In this article, we use the conceptof linguistic complexity in a more com-prehensive manner, as it interacts witha complex set of social, cultural, andpsychological factors. We believe En-glish language learners who inadver-tently suffer the most from languagecomplexity in instruction and assess-ments are not only different from theirnative English speaker peers in theirlanguage background, but in their cul-tural, family, and personal characteris-tics. We believe, as research results inthe past 10 years have clearly suggested,that linguistic and cultural factors mayhinder a student’s ability to fully under-stand the instructional materials anddo well on assessments. Celce-Murcia’sModel of Communicative Competence(Celce-Murcia, 1995) provides a com-prehensive view of linguistic and cul-tural issues that may affect students’academic performance.

Winter 2006 41

Celce-Murcia, Dornyei, & Thurrell(1995) proposed a model which is anelaboration of the earlier communica-tive competence model (Canale, 1983)modified by Celce-Murcia (1995).This model is based on the follow-ing six components: linguistic compe-tence, strategic competence, discoursecompetence, socio-cultural compe-tence, formulaic competence, andparalinguistic competence (see alsoBachman, 1990; Bachman & Palmer,1996). Celce-Murcia (1995) empha-sizes the dynamic aspect of her modeland indicates that the different compo-nents in the model interact with eachother:

It should be emphasized that the in-teractions and overlaps schematicallyrepresented in Figure 3 [Figure 2 inthis article] are integral aspects ofthe proposed model of communica-tive competence. It is not sufficientsimply to list all the components aswas done in Figure 2; it is important toshow the potential overlaps, interrela-tions and interactions, and to realizethat discourse is where all the compe-tencies most obviously reveal them-selves. Discourse thus is the compo-nent in which (or through which)all the other competencies must bestudied—and ultimately assessed—ifone is concerned with communicativecompetence, which is not a hierarchi-cal system of discrete competenciesor abilities but a dynamic, interactiveconstruct. (p. 704)

Figure 2 shows Celce-Murcia’smodel. In this model, socio-culturalcompetence represents the speaker/listener’s background knowledge of thetarget community (e.g., understand-ing communications, beliefs, values,conventions, and taboos of the tar-get community), which makes informedcomprehension and communicationpossible. The linguistic/grammaticalcompetence and formulaic competencecomplement each other and allow theuser to express original and creativemessages or to draw upon conventional-ized prepackaged and mundane formu-laic messages. A comprehensive view ofthe communicative competence model(CCM) and its related issues is be-yond the scope of this paper. How-ever, the main reason for presentingthis brief introduction to CCM is toindicate how complex the role of lan-guage factors is in the instruction andassessment of ELL students and howdifferent factors interact within theframework.

2. Affective Factors in Learning andAssessment

There is a multitudinous literatureon the role of affect and learning. Ban-dura’s (1993) very large body of workon social learning theory is based onthe notion that learners who feel theyhave control over their own learningare more apt to excel and that thesense of control is bound up in per-ceptions of self-efficacy—how they feelabout themselves and their abilities.Likewise, Nolen-Hoeksema, Gurgus, &Seligman (1986) showed how a lackof feeling of self-efficacy could lead to“learned helplessness,” resulting in afailure to even try to accomplish tasksthat are perceived as too difficult—whether in fact they are or not. En-glish Learners are fundamentally learn-ers, like all other students, and so arevulnerable to the same social and psy-chological factors that affect learning.If they come to feel that they are notcapable of learning either a new lan-guage or a new content area, they will

Strategic Competence

Discourse Competence

Socio-CCompetence

ultural

Linguistic Competency

Formulaic Competence

Paralinguistic Competence

FIGURE 2. Celce-Murcia’s Model of Interaction among components of commu-nicative competence.

falter. Similarly, lack of confidence inone’s skills has been shown to affecttest performance as well.

Steele (1997), through a series ofnovel experiments, demonstrated thatstereotypes or beliefs about how othersview their ability can affect the test per-formance of both women and minori-ties. Often subjects are told that oth-ers do not believe they are particularlystrong in the material being tested, andthis results unwaveringly in depressedtest performance. It is difficult for thetest maker to account for these affec-tive influences on test performance inthe building of a test, but the inter-pretation of test results can certainlytake into account the error that arisesfrom these affective factors in particu-lar learners, such as those who are stilllearning English.

High levels of pressure can alsonegatively impact learning. The levelof pressure on all students, and partic-ularly on ELL students due to languagebarriers, increases as the stakes asso-ciated with assessment increases. For


example, ELL students, their teachersand their parents may be blamed partlywhen their schools are not meetingthe AYP requirements. Nichols, Glass,and Berliner (2005) indicated thatthe theory of action underlying theNCLB accountability system is thatthe pressure of high-stakes testing willincrease student achievement. Theauthors created the Pressure RatingIndex (PRI) and correlated the PRIoutcome with the NAEP results from1990 to 2003 in 25 states. They foundthat “States with greater proportionsof minority students implement ac-countability systems that exert greaterpressure” and “High-stakes testingpressure is negatively associated withthe likelihood that eighth and tenthgraders will move into twelfth grade.”They also found that “increases intesting pressure are related to largernumbers of students being held backor dropping out of schools” (p. ii).

3. Conative Factors: Motivational andVolitional Aspects of Learning

Non-cognitive factors—such as at-titudes, beliefs, and motivation—havebeen shown to be related to studentachievement (Schreiber, 2002). For in-stance, using an attitudes inventory,Tapia and Marsh (2000) found thatstudents failing in mathematics scoredlowest on self-confidence, motivation,value, and enjoyment.

Using nationally representative dataof eighth-grade students from the Na-tional Education Longitudinal Study of1988, Singh, Granville, & Dika (2002)found significant positive effects of mo-tivation factors. Research has foundthat students who struggle with read-ing grow to have less desire and moti-vation to read as a direct result of theirstruggles (O’Brien & Dillon, 2002).

Snow (1994) advocated that cog-nitive processes alone could not ac-count for students’ learning, but thatboth affective and conative (motiva-tional) processes had to be viewedas interacting with cognitive processesto produce learning. There is consid-erable literature to demonstrate thispoint in the research on second lan-guage learning. A number of studies(see Bialystok & Hakuta, 1994) haveshown that individuals who are moti-vated to learn a language, whether bynecessity or desire, are more effectivelearners of that language. Likewise, weknow that the context of second lan-guage learning can have powerful ef-fects on motivation, as when the learner

perceives that she is being asked to re-ject her identity by displacing her pri-mary language with a new language.In such cases, second language learn-ing can be retarded (Schmidt, 2000).It was, in fact, this very principle—that students who were made to feelthat their primary language and cul-ture had value in the society would bemore motivated to learn a second lan-guage than those who felt their nativelanguage threatened—that was behindthe development of many bilingual–bicultural programs during the 1970s.Adolescents, especially those who maybe acutely concerned about how theyappear as they struggle with a secondlanguage, can experience embarrass-ment that leads to lack of motivation touse the language, which in turn delaysdevelopment of the second language.(Tse, 2001).

Just as language learning is affectedby motivation, so too can test perfor-mance be affected. Students who do not“care” about learning are not likely to“care” about how they perform on testsof that learning. Gandara et al. (2003)found that many Latino high schoolstudents (especially males) who feltmarginalized in their schools also pro-fessed not to care how they did in schooland routinely failed their courses andmade little or no effort to study fortests.

Given these findings, it is importantto examine how policy issues regardingassessment measures consider possibledifferences in cognitive, conative, andaffective factors for the ELL subgroup.Recent studies on the assessment andaccommodations of English languagelearners and students with disabilitieshave raised concerns over a variety ofissues related to academic assessmentof these students.

DiscussionFairness demands equal educationalopportunities for all students includ-ing ethnic minorities, students with dis-abilities, low-income students, and En-glish language learners. However, thesesubgroups of students have historicallylagged behind their mainstream peerson test scores due, in part, to factorsthat may not be closely related to theiracademic achievement but do influencetheir performance outcomes. In orderto begin reducing the performance gapsbetween subgroups and mainstreamstudents, improvements must be madein both instruction and assessment by

identifying and controlling for thesefactors (often referred to as nuisancevariables) that affect a subgroup’s aca-demic achievement.

The four subgroups identified in theNCLB legislation are different in theirbackground and are faced with dif-ferent issues in their academic ca-reers. While such differences are rel-evant to consider, in this paper we fo-cused on assessment and accountabil-ity issues for ELL students, a key sub-group of students that is faced withserious challenges. Understanding is-sues that are involved in the instruc-tion and assessment of ELL studentswill help us realize the limitationsof the nation’s current accountabilitymeasures for all the subgroups underconsideration.

Many different factors such as parenteducation, poverty, and schooling con-ditions contribute to the existing per-formance gap between ELL and non-ELL students. However, language fac-tors have a greater impact on ELL stu-dent performance than any other fac-tors. While ELL students perform be-hind non-ELL students in almost allsubject areas in school, the perfor-mance gap between ELL and non-ELLis largest in areas with higher levelsof language demand. Because ELL stu-dents, by definition, do not have a strongcommand of the English language, bothlearning and assessment are affected bytheir limited English proficiency, andboth learning and assessment condi-tions must be addressed to help closethe performance gap.

The concept of language as it af-fects ELL students’ performance is trulymultidimensional and complex in na-ture. As elaborated in Celce-Murcia’smodel of communicative competence,language interacts with many differentsocio-cultural factors. By understand-ing the complex nature of language fac-tors, one can then discuss the impact oflanguage on the academic achievementof ELL students.

The challenges of second languageacquisition, such as learning in a lan-guage in which students are not quiteproficient and working with measure-ment tools that are ill-equipped to as-sess their skills and abilities, have muchmore impact on ELL students’ aca-demic performance than is acknowl-edged by many researchers and edu-cational practitioners. For example, aselaborated earlier in this paper, whencompared to native English speakingstudents in reading, average English

Winter 2006 43

language learners had a deficit in syn-tactic awareness skills.

Furthermore, the process of develop-ing new language skills is difficult andrequires time and effort. Thus, it takesa long time for ELL students to becomeproficient enough in English to under-stand teachers’ instructions and testquestions in a language with an unfa-miliar structure and vocabulary. Basedon some research evidence it may takefrom five to seven years or longer formost ELLs to gain sufficient mastery ofacademic English to join their Englishspeaking peers in taking full advantageof instruction in English. During thisperiod, ELL students may not be ableto learn at the same rate as their na-tive speaking peers when the instruc-tion is offered only in English. LimitedEnglish proficiency may also make it dif-ficult to benefit fully from the teacher’sinstructions and to understand assess-ment questions. This has a direct im-pact on these students’ performance onassessments.

It is ironic that during the period thatELL students are struggling to learnEnglish, they are expected to progressat the same rate as their native En-glish speaking peers. In addition, theirschools and teachers may receive puni-tive corrective action for a lack of ad-equate progress. Perhaps one of themost challenging tasks in ELL students’academic life is dealing with the oftencomplex linguistic structure of assess-ment tools. Standardized achievementtests that have been constructed formainstream students do not take intoaccount the special needs of Englishlearners. This can be a major source offrustration for these students and cancontribute substantially to the perfor-mance gap between ELL and non-ELLstudents as there is no evidence to sug-gest that the basic abilities of ELL stu-dents are different from non-ELL stu-dents.

Sometimes ELL students are pro-vided with accommodations to help off-set the linguistic challenges of assess-ment instruments, but many of theseaccommodations may not be relevantfor these students since they were origi-nally designed for other subgroups withcompletely different backgrounds andneeds. That is, there is not enoughevidence that these accommodationsmake the assessments more accessiblefor ELL students. Moreover, results ofexperimentally controlled studies showthat some accommodations may alter

the construct being measured whichconsequently may affect the validity ofthe assessment.

Problems in the assessment and in-struction of ELL students, and theimpact language background variableshave on students’ learning and achieve-ment outcomes undermine the authen-ticity of the nation’s accountability sys-tem for ELL students. The account-ability system developed under NCLBhas the expressed intent to pressureschools to be more responsive to theneeds of at-risk students who havehistorically fared poorly in America’spublic schools. By exerting pressurethrough sanctions, the belief has beenthat schools can be made to help thesestudents improve academically. This iscertainly a laudable and important goal,although there is far from universalagreement on the degree to which thisstrategy can achieve the goal. As Linn(2003) indicated, NCLB offers manycontributions to the nation’s account-ability system. For example, special at-tention given to groups of students whohave had the lowest achievement in thepast is encouraging. However “the goalsthat NCLB sets for student achievementwould be wonderful if they could bereached, but, unfortunately, they arequite unrealistic” (Linn, 2003, p. 10).

It is important to consider that thereare many other factors beyond the cog-nitive domain in general and languagefactors in particular that are affectingthe academic achievement of ELL stu-dents. Conative factors (motivationaland volitional aspects of learning) aswell as affective factors can greatly im-pact students’ performance in school.Lack of progress due to serious chal-lenges that ELL students are faced withmay seriously impact their level of moti-vation. Undue pressure to perform wellunder the nation’s accountability sys-tem and other sources of pressure inthe academic life of ELL students maycause frustration and impact their aca-demic performance.

Research findings can help to resolvesome of the major issues in the instruc-tion and assessment of ELL studentsand consequently improve the account-ability system for these students. Whileresearch in this area is very limited incontent and scope, and some of the find-ings may be inconclusive, some of thesefindings can help improve the existingassessment and accountability systems.For example, by identifying the sourcesof linguistic complexity of assessments,

and by providing ways to reduce thelevel of unnecessary linguistic complex-ity, research findings have contributedsignificantly to improving assessmentsfor ELL students.

Similarly, by identifying factors thatimpact the effectiveness and validityof accommodated assessments and byproviding research-based recommen-dations for selecting the most appro-priate accommodations for ELL stu-dents, research findings can help cre-ate a more effective assessment systemfor ELL students. For example, as ex-plained in this paper, research findingshave identified some accommodationsthat help ELL students with their En-glish language needs without compro-mising the validity of the assessment.

In this paper we discussed the chal-lenges that English language learnersare faced with in learning a new lan-guage. We also elaborated on the factthat ELL students transfer their pre-reading skills across languages. Thisimplies that the time they spend learn-ing in any language is going to beroughly predictive of learning to read.Therefore, knowledge of what studentsknow in both primary and secondarylanguages can yield important infor-mation about the cognitive process-ing skills they can apply to learningto read. Including such informationin the assessment and accountabilitysystems for these students would behelpful.

Instruction and assessments for ELLstudents are affected by many differ-ent factors; therefore, it is not possibleto present a comprehensive picture ofall of the issues involved in the edu-cational experiences of these students.In this article we tried to present afew tangible examples of factors influ-encing the academic achievement ofthese students. We hope this discus-sion might help policy makers, edu-cational researchers, and educationalpractitioners to understand the com-plex nature of the academic life of ELLstudents in particular, and all four sub-groups identified in NCLB in general,and plan their curriculum, instruction,and assessment accordingly.

References

Abedi, J. (2006a). Language issues in itemdevelopment. In S. M. Downing & T. M.Haladyna (Eds.), Handbook of test devel-opment. Mahwah, NJ: Lawrence ErlbaumAssociates.


Abedi, J. (2006b). Are accommodations usedfor ELL students valid? Presented at the2006 Annual Meeting of the American Ed-ucational Research Association, San Fran-cisco.

Abedi, J., Courtney, M., & Leon, S. (2003).Effectiveness and validity of accommoda-tions for English language learners inlarge-scale assessments (CSE Tech. Rep.No. 608). Los Angeles: University of Cal-ifornia, National Center for Research onEvaluation, Standards, and Student Test-ing.

Abedi, J., Courtney, M., Mirocha, J., Leon, S.,& Goldberg, J. (2005). Language accom-modations for English language learnersin large-scale assessments: Bilingual dic-tionaries and linguistic modification (CSETech. Rep. No. 666). Los Angeles: Univer-sity of California, National Center for Re-search on Evaluation, Standards, and Stu-dent Testing.

Abedi, J., Herman, J. L., Courtney, M., Leon, S.,& Kao, J. (2004). English language learn-ers and math achievement: A study onclassroom-level opportunity to learn. LosAngeles: University of California: Center forthe Study of Evaluation/National Centerfor Research on Evaluation, Standards, andStudent Testing.

Abedi, J., Hofstetter, C., & Lord, C.(2004). Assessment accommodations forEnglish-language learners: Implicationsfor policy-based empirical research. Re-view of Educational Research, 74(1), 1–28.

Abedi, J., Leon, S., & Mirocha, J. (2003). Im-pact of students’ language background oncontent-based assessment: Analyses of ex-tant data (CSE Tech. Rep. No. 603). LosAngeles: University of California, NationalCenter for Research on Evaluation, Stan-dards, and Student Testing.

Abedi, J., & Lord, C. (2001). The lan-guage factor in mathematics tests. AppliedMeasurement in Education, 14(3), 219–234.

Abedi, J., Lord, C., & Hofstetter, C. (1998). Im-pact of selected background variables onstudents’ NAEP math performance (CSETech. Rep. No. 478). Los Angeles: Univer-sity of California, National Center for Re-search on Evaluation, Standards, and Stu-dent Testing.

Abedi, J., Lord, C., Hofstetter, C., & Baker, E.(2000). Impact of accommodation strate-gies on English language learners’ test per-formance. Educational Measurement: Is-sues and Practice, 19(3), 16–26.

Abedi, J., Lord, C., & Plummer, J. (1997).Language background as a variable inNAEP mathematics performance (CSETech. Rep. No. 429). Los Angeles: Univer-sity of California, National Center for Re-search on Evaluation, Standards, and Stu-dent Testing.

American Educational Research Association,American Psychological Association, & Na-tional Council on Measurement in Edu-cation (1999). Standards for educational

and psychological testing. Washington, DC:American Educational Research Associa-tion.

August, D., & Shanahan, T. (2006) (Ed.).Developing literacy in second-languagelearners. Mahwah, NJ: Lawrence ErlbaumAssociates.

Bachman, L. F. (1990). Fundamental consid-erations in language testing. Oxford: Ox-ford University Press.

Bachman, L. F., & Palmer, A. S. (1996). Lan-guage testing in practice: Designing anddeveloping useful language tests. Oxford:Oxford University Press.

Bandura, A. (1993). Perceived self efficacycognitive development and functioning.Educational Psychologist, 28, 117–148.

Bialystok, E., and Hakuta, K. (1994). In OtherWords. New York: Basic Books.

California Department of Education (2006).State AYP report. Retrieved June 6, 2006from www.ayp.cde.gov/reports.

Canale, M. (1983). From communicative com-petence to communicative language ped-agogy. In J. C. Richards & R. W. Schmidt(Eds.), Language and communication.New York: Longman.

Celce-Murcia, M. (1995, March). The elabora-tion of sociolinguistic competence: Impli-cations for teacher education. Presentedat the Georgetown University Round Tableon Languages and Linguistics Conference,Washington, DC.

Celce-Murcia, M., Dornyei, Z., & Thurrell,S. (1995). Communicative competence: Apedagogically motivated model with con-tent specifications. Issues in Applied Lin-guistics, 6(2), 5–35.

Chiappe, P., Siegel, L. S., & Wade-Woolley,L. (2002). Linguistic diversity and the de-velopment of reading skills: A longitudi-nal study. Scientific Studies of Reading,6(4), 369–400. Retrieved June 6, 2006, fromPsycINFO database.

Cocking, R. R., & Chipman, S. (1988). Concep-tual issues related to mathematics achieve-ment of language minority children. In R.R. Cocking & J. P. Mestre (Eds.), Linguis-tic and cultural influences on learningmathematics (pp. 17–46). Hillsdale, NJ:Erlbaum.

De Corte, E., Verschaffel, L., & DeWin, L.(1985). Influence of rewording verbal prob-lems on children’s problem representationsand solutions. Journal of Educational Psy-chology, 77(4), 460–470.

Duncan, T., Parent, L., Chen, W-H., Ferrara,S., Johnson, E., Oppler, S., & Shieh, Y-Y. (2005). Study of a dual-language testbooklet in eighth-grade mathematics. Ap-plied Measurement in Education, 18, 129–161.

Durgunoglu, A., Nagy, W., & Hancin-Bhatt, B.(1993). Cross-language transfer of phono-logical awareness. Journal of EducationalPsychology, 85, 453–465.

Gandara, P., Rumberger, R., Maxwell-Jolly, J., & Callahan, R. (2003). En-glish learners in California schools:Unequal resources, unequal outcomes.

Education Policy Analysis Archives,11(36). Retrieved June 14, 2006 fromhttp://epaa.asu.edu/epaa/v11n36/.

Gandara, P., & Rumberger, R. (in press). Im-migration, language, and education: Howdoes language policy structure opportu-nity? In J. Holdaway & R. Alba (Eds.), Ed-ucation of immigrant youth: The role ofinstitutions and agency. New York: SocialScience Research Council.

Geva, E., & Siegel, L. S. (2000). Orthographicand cognitive factors in the concurrent de-velopment of basic reading skills in twolanguages. Reading and Writing: An Inter-disciplinary Journal, 12, 1–30.

Geva, E., Yaghoub-Zadeh, Z., & Schuster, B.(2000). Understanding differences in wordrecognition skills of ESL children. Annalsof Dyslexia, 50, 123–154.

Gottardo, R., Pannucci, J. A., Kuske, C. R.,and Brettin, T. (2003). Statistical analysisof microarray data: A Bayesian approach.Biostatistics, 4, 597–620.

Hakuta, K., Butler, Y., & Witt, D. (2000). Howlong does it take English learners to at-tain proficiency? (Policy Report 2000-1).University of California: Linguistic Minor-ity Research Institute.

Huebert, J. P. (2000). High-stakes testing: Op-portunities and risks for students of color,English-language learners, and studentswith disabilities. In M. Pines (Ed.), Thecontinuing challenge: Moving the youthagenda forward (Policy Issues Monograph00-02, Sar Levitan Center for Social PolicyStudies). Baltimore, MD: Johns HopkinsUniversity Press.

Huebert, J., & Hauser, R. (1999). Highstakes: Testing for tracking, promotion,and graduation. Washington, DC: NationalResearch Council.

Katz, A., Low, P., Stack, J., & Tsang, S.-L.(2004). A Study of content area assessmentfor English language learners. Preparedfor the Office of English Language Acqui-sition and Academic Achievement for En-glish Language Learners, U.S. Departmentof Education. Oakland, CA: ARC Associates.

Kindler, A. L. (2002). Survey of the states’limited English proficient students andavailable educational programs and ser-vices, 2000-2001 summary report. Wash-ington, DC: National Clearinghouse for En-glish Language Acquisition and LanguageInstruction Educational Programs.

Kintsch, W., & Greeno, J. G. (1985). Under-standing and solving word arithmetic prob-lems. Psychological Review, 92(1), 109–129.

Kiplinger, V. L., Haug, C. A., & Abedi, J.(2000). Measuring math—not reading—on a math assessment: A language ac-commodations study of English languagelearners and other special populations.Presented at the annual meeting of theAmerican Educational Research Associa-tion, New Orleans, LA.

Larsen, S. C., Parker, R. M., & Tren-holme, B. (1978). The effects of syntacticcomplexity upon arithmetic performance.

Winter 2006 45

Educational Studies in Mathematics, 21,83–90.

Lepik, M. (1990). Algebraic word problems:Role of linguistic and structural variables.Educational Studies in Mathematics, 21,83–90.

Lesaux, N., & Siegel, L. (2003). The develop-ment of reading in children who speak En-glish as a second language. DevelopmentalPsychology, 39(6), 1005–1019.

Linn, R. L. (in press). Scientific evidence in ed-ucational policy and practice: Implicationsfor adequate yearly progress. In C. Dwyer(Ed.), Measurement and research in theaccountability era. Mahwah, NJ: LawrenceErlbaum Associates.

Linn, R. (2003). Accountability: Responsibilityand reasonable expectations. EducationalResearcher, 32(7), 3–13.

Maihoff, N. A. (2002). Using Delaware data inmaking decisions regarding the educationof LEP students. Presented at the Coun-cil of Chief State School Officers 32nd An-nual National Conference on Large-ScaleAssessment, Palm Desert, CA.

Mestre, J. P. (1988). The role of language com-prehension in mathematics and problemsolving. In R. R. Cocking & J. P. Mestre(Eds.), Linguistic and cultural influenceson learning mathematics (pp. 200–220).Hillsdale, NJ: Erlbaum.

Moore, K. A., & Redd, Z. (2002). Childrenin poverty: Trends, consequences, andpolicy options. Washington, DC: ChildTrends.

Munro, J. (1979). Language abilities and mathperformance. Reading Teacher, 32(8),900–915.

Nichols, S. L., Glass, G. V., and Berliner, D.C. (2005). High-stakes testing and studentachievement: Problems for the No Child LeftBehind Act. Tempe, AZ: Education PolicyStudies Laboratory.

Nolen-Hoeksema, S., Gurgus, J., & Seligman,M. (1986). Learned helplessness in chil-dren: A longitudinal study of depression,achievement, and explanatory style. Jour-nal of Personality and Social Psychology,51, 435–442.

Noonan, J. (1990). Readability problems pre-sented by mathematics text. Early ChildDevelopment and Care, 54, 57–81.

O’Brien, D. G., & Dillon, D. R. (2002). Motiva-tion and engagement in reading (Moduleprepared for the Minnesota Reading Excel-lence Act Grant and the Minnesota ReadingFirst Grant). Minneapolis, MN: Universityof Minnesota and the Minnesota State De-partment of Education.

Orr, E. W. (1987). Twice as less: Black Englishand the performance of black students inmathematics and science. New York: W. W.Norton.

Parrish, T. B., Merickel, A., Perez, M., Lin-quanti, R., Socia, M., Spain, A., Speroni, C.,Esra, P., Brock, L., & Delancey, D. (2006).Effects of the implementation of Proposi-tion 227 on the education of English learn-ers, K-12: Findings from a five-year eval-uation. Palo Alto, CA: American Institutesfor Research and WestEd.

Pitoniak, M., Lutkus, A., Cahalan-Laitusis, C.,Cook, L., & Abedi, J. (2006). Are inclu-sion policies and practices for state assess-ment systems and NAEP state assessmentsaligned? Princeton, NJ: Educational Test-ing Service.

Rivera, C. (2003). State assessment policiesfor English language learners. Presentedat the 2003 Large-Scale Assessment Con-ference.

Rivera, C., & Stansfield, C. W. (2001). Theeffects of linguistic simplification of sci-ence test items on performance of lim-ited English proficient and monolingualEnglish-speaking students. Presented atthe annual meeting of the American Ed-ucational Research Association, Seattle,WA.

Rothman, R. W., & Cohen, J. (1989). The lan-guage of math needs to be taught. AcademicTherapy, 25(2), 133–42.

Rumberger, R., & Gandara, P. (2004). Seek-ing equity in the education of California’sEnglish learners. Teachers College Record,106, 2031–2055.

Schmidt, R. (2000). Language policy andidentity politics in the United States.Philadelphia: Temple University Press.

Schreiber, J. B. (2002). Institutional and stu-dent factors and their influence on ad-vanced mathematics achievement. Jour-nal of Educational Research, 95(5), 274–286.

Siegel, L. S. (1993). The development of read-ing. In H. W. Reese (Ed.), Advances inchild development and behavior (pp. 63–97). San Diego, CA: Academic Press.

Singh, K., Granville, M., & Dika, S. (2002).Mathematics and science achievement: Ef-fects of motivation, interest, and academicengagement. Journal of Educational Re-search, 95(6), 323–332.

Sireci, S. G., Li, S., & Scarpati, S. (2003). Theeffects of test accommodation on test perfor-mance: A review of the literature (Centerfor Educational Assessment Research Re-port No. 485). Amherst, MA: University ofMassachusetts.

Snow, R. E. (1994). Abilities in academic tasks.In R. J. Sternberg & R. K. Wagner (Eds.),Mind in context: Interactionist perspec-tives on human intelligence (pp. 3–37).Cambridge, England: Cambridge UniversityPress.

Spanos, G., Rhodes, N. C., Dale, T. C., & Cran-dall, J. (1988). Linguistic features of math-ematical problem solving: Insights and ap-plications. In R. R. Cocking & J. P. Mestre(Eds.), Linguistic and cultural influenceson learning mathematics (pp. 221–240).Hillsdale, NJ: Erlbaum.

Steele, C. M. (1997). A threat in the air:How stereotypes shape intellectual identityand performance. American Psychologist,52(6), 613–629.

Sunderman, G., & Kim, J. (2004). Inspiringvision, disappointing results: Four studieson implementing the No Child Left BehindAct. Boston: Harvard University, The CivilRights Project.

Tapia, M., & Marsh, G. E. (2000). Effect ofgender, achievement in mathematics, andethnicity toward mathematics. Presentedat the annual meeting of the Mid-SouthEducational Research Association, BowlingGreen, KY.

Tse, L. (2001) Why don’t they learn English?Separating fact from fallacy in the U.S. lan-guage debate. New York: Teachers CollegePress.

Zehler, A., Fleischman, H., Hopstock, P.,Stephenson, T., Pendzick, M., & Sapru, S.(2003). Descriptive study of services to lim-ited English proficient students. Washing-ton, DC: Development Associates.


Documents

Performance of English Language Learners as a Subgroup in Large-Scale Assessment: Interaction of Research and Policy