Embed Size (px)
Citation preview
Assessment and Accountability Issues for Assessment and Accountability Issues for English Language Learners and English Language Learners and
Students With DisabilitiesStudents With Disabilities
Oregon Department of EducationOregon Department of EducationOctober 4, 2007October 4, 2007
Jamal AbediJamal Abedi
University of California, DavisUniversity of California, Davis
National Center for Research on Evaluation, Standards and Student Testing National Center for Research on Evaluation, Standards and Student Testing (UCLA/CRESST)(UCLA/CRESST)
Accountability QuestionsAccountability QuestionsAre there specific ELL/SWD subgroup features that affect the accountability system?
Yes No
Could the current accountability system for ELLs/SWDs be improved?
Yes No
Do research findings help inform assessment & accountability systems for these students?
Yes No
Should Schools Test Children Should Schools Test Children with Disabilities?with Disabilities?
Assessment outcomes may not be valid because their disabilities interfere with content knowledge performance
Test results affect decisions regarding promotion or graduation
They may be inappropriately placed into special educational programs where they receive inappropriate instruction
SWD students may not have received the same curriculum which is assumed for the test
Yes
Students with disabilities (SWD) can be placed at a disadvantage because:
Should Schools Test English Should Schools Test English Language Learners?Language Learners?
Yes
Assessment outcomes may not be valid because their low level of English proficiency interferes with content knowledge performance
Test results affect decisions regarding promotion or graduation
They may be inappropriately placed into special educational programs where they receive inappropriate instruction
ELL students may not have received the same curriculum which is assumed for the test
English language learners (ELLs) can be placed at a disadvantage because:
ProblemsDue to the powerful impact of assessment on instruction, ELL and SWD students’ quality of instruction may be affectedIf excluded, they will be dropped out of the accountability pictureInstitutions will not be held responsible for their performance in schoolThey will not be included in state or federal policy decisionTheir academic progress, skills, and needs may not be appropriately assessed
Should Schools Test English Language Should Schools Test English Language Learners and Students with Disabilities?Learners and Students with Disabilities?
No
Problems in AYP Reporting for Problems in AYP Reporting for ELL StudentsELL Students
1. Problems in classification/ reclassification of ELL students (moving target subgroup)
2. Measurement quality 3. Low baseline 4. Instability of the ELL subgroup5. Sparse ELL population6. ELL cutoff points (Conjunctive vs.
Compensatory model)
Percent of ELL student in 2000-Percent of ELL student in 2000-2001 2001 (Kindler, 2002)(Kindler, 2002)
• California 1,511,646 25.0%• New Mexico 63,755 19.9%• Arizona 135,248 15.4%• Texas 570,022 14.0%• Nevada 40,131 11.8%• Florida 254,517 10.7%• Utah 44,030 9.3%• Oregon 47,382 8.7%
Composition of SWD Population:Composition of SWD Population:National StatisticsNational Statistics
•Grade 4
•11% Nationally
•Ranging between 4% to 17% by different states
•Grade 8
•10% Nationally
•Ranging between 6% to 14% by different states
Some recent statisticsSome recent statistics
• In 2005-2006 there were a total of 559,215 students were enrolled in the K-12 public schools in Oregon
• Of the total, 11.7% (65,239) were ELLs (as
compared with 8.7% in 2000-2001) and 12.8% (71,517) were Special Education students.
• Of the 65,239 ELL students, (76.9%) had Spanish as their language of origin which is about 9% of the total students enrolled
Population ChangePopulation ChangeRace/ Ethnicity
1992-1993 2005-2006 1992-3 to 2005-6
Number of Students
Percent of All
Students
Number of Students
Percent of all
Students
Change in Number
Percent Change
White 446,251 87.5 401,086 71.7 -45,165 -10.1
African-American
12,220 2.4 16,742 3.0 +4,522 +37.0
Hispanic 27,115 5.3 84,244 15.1 +57,129 +210.7
Asian/ Pacific Islander
15,360 3.0 25,204 4.5 +9,844 +64.1
Native American
9,176 1.8 11,678 2.1 +2,502 +27.3
TOTAL 510,122 100.0 538,954 96.4 +28,832 +9.6
Special Education Special Education
1992-1993 1997-1998 2001-2002 2005-2006
Special Education
54,952 63,097 70,902 71,517
Total Enrollment
510,122 540,359 551,679 559,215
% of Total Enrollment
10.8 11.7 12.9 12.8
Language DiversityLanguage DiversityLanguage of
OriginNumber of Students
Percent of Students
Percent of All Students
559,215
Spanish 50,143 76.9% 9.0%Russian 3,558 5.5% 0.6%Vietnamese 1,987 3.0% 0.4%Ukrainian 957 1.5% 0.2%Korean 719 1.1% 0.1%Chinese, Yue 639 1.0% 0.1%Other 7,236 11% 1.3%
TOTAL 65,239 100.0% 11.7%
A Point for DiscussionA Point for Discussion : :
o This large majority of ELL students speaking the same language makes Oregon one of the most eligible state to use native language testing (in Spanish).
o However, the decision is not that simple. Why not, what are the issues?
o Alignment of language of assessment and language of instruction
o Technical issues in translation or trans-adaptation
o Comparability Issues
Students with Disabilities and Students with Disabilities and Statewide Testing in OregonStatewide Testing in Oregon
Students with Disabilities being served with active Individual Education Plans (IEP) or 504 plans have a set of choices for participation in Oregon’s Statewide Assessment System.
• Implementing accommodations and/or modifications
• Out of level testing (no longer an option in Oregon)
• Alternative Assessments for those with profound cognitive disabilities
Alternative AssessmentsAlternative Assessments(Oregon)(Oregon)
• The Alternate Assessment is comprised of tasks designed to measure basic skills which are anchored to the Oregon standards.
• The Alternate Assessment is administered for the same age groups (in the Spring).
• Tasks can be administered in a variety of ways and students can respond with considerable latitude (e.g., pointing, sign language).
Challenges in Statewide Testing of Challenges in Statewide Testing of ELLs & Students with Disabilities ELLs & Students with Disabilities
• Providing effective and valid accommodations
• Developing reliable and valid alternate assessment
• Most importantly, comparability issues
Why Should English Language Why Should English Language Learners be Accommodated?Learners be Accommodated?
Their possible English language limitations may interfere with their content knowledge performance.Assessment tools may be culturally and linguistically biased for these students.Linguistic complexity of the assessment tools may be a source of measurement error.Language factors may be a source of construct-irrelevant variance.
Why Should Students with Why Should Students with Disabilities be Accommodated?Disabilities be Accommodated?
Their disabilities put them at disadvantage
Accommodations must be provided to level the playing field
Means and Standard Deviations of Total Correct Scores for 3rd Grades
Reading SAT 9 (30 items) Reading DL (40 items)
Mean SD Mean SD
Gender
Females 19.5 5.9 32.6 11.3
Males 19.6 6.1 32.8 11.5
LEP
Non-LEP 19.6 5.9 32.9 11.4
LEP 12.6 6.6 22.1 10.9
SES
No 21.1 5.6 35.9 10.6
Yes 17.0 5.8 27.6 10.8
Title 1
No 19.9 6.0 33.5 11.4
Yes 17.4 5.6 28.3 10.4
SWD
Non-SWD 20.3 5.6 34.3 10.7
SWD 15.2 6.1 24.4 11.5
Performance Difference Between SWD and Non-SWDPerformance Difference Between SWD and Non-SWD
Reading Math Math Calculation
Math Analytical
Non-LEP-SWD
Mean 45.63 49.30 49.09 48.75
SD 21.10 20.47 20.78 19.61
N 9217 91.18 9846 92.50
LEP only
Mean 20.26 36.00 39.20 33.86
SD 16.39 18.48 21.25 16.88
N 692 687 696 699
SWD only
Mean 18.86 27.82 28.42 29.10
SD 19.70 14.10 15.76 15.14
N 872 843 883 873
LEP/SWD
Mean 9.78 21.37 22.75 22.87
SD 11.50 10.75 12.94 12.06
N 93 92 97 94
Site 4 Grade 8 Descriptive Statistics for the SAT 9 Test Scores by Strands
ReadingReading Science Science Math Math MM SDSD MM SDSD MM SDSD
Grade 10Grade 10SWD onlySWD only 16.416.4 12.712.7 25.525.5 13.313.3 22.522.5 11.711.7ELL onlyELL only 24.024.0 16.416.4 32.932.9 15.315.3 36.836.8 16.016.0ELL & SWDELL & SWD 16.316.3 11.211.2 24.824.8 9.3 9.3 23.623.6 9.8 9.8Non-ELL/SWD 38.0Non-ELL/SWD 38.0 16.016.0 42.642.6 17.217.2 39.639.6 16.916.9All studentsAll students 36.036.0 16.916.9 41.341.3 17.517.5 38.538.5 17.017.0
Grade 11Grade 11SWD OnlySWD Only 14.914.9 13.213.2 21.521.5 12.312.3 24.324.3 13.213.2ELL OnlyELL Only 22.522.5 16.116.1 28.428.4 14.414.4 45.545.5 18.218.2ELL & SWDELL & SWD 15.515.5 12.712.7 26.126.1 20.120.1 25.125.1 13.013.0Non-ELL/SWD 38.4Non-ELL/SWD 38.4 18.318.3 39.639.6 18.818.8 45.245.2 21.121.1All StudentsAll Students 36.236.2 19.019.0 38.238.2 18.918.9 44.044.0 21.221.2
Normal Curve Equivalent Means & Standard Deviations for Students in Grades 10 and 11, Site 3 School District
Stanford 9 Sub-scale Reliabilities (Alpha), Grade 9Stanford 9 Sub-scale Reliabilities (Alpha), Grade 9Non-LEP Students
Sub-scale (Items) Hi SES Low SES
English Only
FEP RFEP LEP
Reading, N= 205,092 35,855 181,202 37,876 21,869 52,720
-Vocabulary (30) .828 .781 .835 .814 .759 .666
-Reading Comp (54)
.912 .893 .916 .903 .877 .833
Average Reliability .870 .837 .876 .859 .818 .750
Math, N= 207,155 36,588 183,262 38,329 22,152 54,815
-Total (48) .899 .853 .898 .898 .876 .802
Language, N= 204,571 35,866 180,743 37,862 21,852 52,863
-Mechanics (24) .801 .759 .803 .802 .755 .686
-Expression (24) .818 .779 .812 .804 .757 .680
Average Reliability .810 .769 .813 .803 .756 .683
Science, N= 163,960 28,377 144,821 29,946 17,570 40,255
-Total (40) .800 .723 .805 .778 .716 .597
Social Science, N= 204,965 36,132 181,078 38,052 21,967 53,925
-Total (40) .803 .702 .805 .784 .722 .530
Grade 11 Stanford 9 Reading and Science Structural Modeling Results, Site 3Grade 11 Stanford 9 Reading and Science Structural Modeling Results, Site 3
All Cases (N=7,176)
Even Cases (N=3,588)
Odd Cases (N=3,588)
Non-LEP (N=6,932)
LEP (N=244)
Goodness of Fit
Chi Square 1786 943 870 1675 81
NFI .931 .926 .934 .932 .877
NNFI .898 .891 .904 .900 .862
CFI .932 .928 .936 .933 .908
Factor Loadings
Reading
Composite 1 .733 .720 .745 .723 .761
Composite 2 .735 .730 .741 .727 .713
Composite 3 .784 .779 .789 .778 .782
Composite 4 .817 .722 .712 .716 .730
Composite 5 .633 .622 .644 .636 .435
Math Variables
Composite 1 .712 .719 705 709 660
Composite 2 .695 .696 .695 .701 .581
Composite 3 .641 .628 .654 .644 .492
Composite 4 .450 .428 .470 .455 .257
Factor Corr
Reading/ Math .796 .796 .795 .797 .791
Fundamental QuestionsFundamental Questions
• Why should ELLs and SWDs be accommodated? (to level the playing field)
• What are the major characteristics of accommodations that would level the playing field?
• Do the most commonly used accommodations possess those characteristics
The major characteristics of The major characteristics of accommodations that would level the accommodations that would level the
playing fieldplaying field
EffectiveValidConsistent with students’ backgroundFeasible Relevant
From now on our focus will be on ELL students
Problems with Accommodation UsageProblems with Accommodation Usage
• Accommodations for English language learners are often selected based on feedback from teachers and bilingual coordinators without enough influence from research findings
• Several studies have identified some accommodations that may provide unfair advantage to the recipients of the accommodations and may render invalid results
Examples of Accommodations for ELL Examples of Accommodations for ELL Students that May Alter the ConstructStudents that May Alter the Construct
• Providing an English dictionary (Abedi, Courtney, & Leon, 2003; Abedi, Lord, Boscardin, & Miyoshi, 2000)
• Providing extra or extended time (Abedi, Lord, Hofstetter, & Baker, 2000; Hafner, 2001; Thurlow, 2001)
• Translating tests into students’ native language (Abedi, Lord, Hofstetter, & Baker, 2000)
• By gaining access to definition of content-related terms, recipients of a dictionary may be advantaged over those who did not have access to the dictionaries. This may compromise the validity of assessment (Abedi, Courtney, Mirocha, Leon, & Goldberg, 2005)
Native language testingNative language testing
Problems in creating parallel forms of the test
Translation issue Alignment of language of
assessment and language of instruction
Glossary with extra time raised the performance of both ELL and non-ELL students (Abedi, Hofstetter, Lord, and Baker, 1998, 2000)
ELL students’ performance increased by 13% when they were tested under glossary with extra time accommodation.
While this looks promising, it does not present the entire picture.
Non-ELL students also benefited from this accommodation, with an increase of 16%.
English and bilingual dictionaries recipients may be advantaged over those without access to dictionaries. This may jeopardize the validity of assessment.
Glossary Plus Extra TimeGlossary Plus Extra Time
How can accommodations be How can accommodations be examined for validity?examined for validity?
Only through experimentally-controlled research where:
ELL and non-ELL students are randomly assigned to experimental and control groups
Both ELL and non-ELL students are observed under accommodated and non-accommodated assessmentsUsing existing data?
How the validity of accommodations How the validity of accommodations can be tested in an experimentally can be tested in an experimentally
controlled condition?controlled condition? Accommodated Non-
Accommodated
ELL
Non-ELL
Characteristics of ELL studentsCharacteristics of ELL students• ELL students constitute a very diverse and
heterogeneous population (SES, cultural and linguistic backgrounds).
• They can be vastly different in their level of proficiency in their native or home language
• They are quite different in their level of proficiency in English
• Studies show that the level of English language proficiency of these students range from high (even higher than some native English speakers) to very low
The Most Commonly Used Accommodations The Most Commonly Used Accommodations for ELL Students Include:for ELL Students Include:
(Rivera, 2003)(Rivera, 2003)
• Extended time (42 of the 48 states)
• Use of glossary (26 states)
• Use of an English dictionary (33 states)
• Use of a bilingual dictionary (22 states)
• Linguistically-simplified test items (12 states)
The Most Commonly Used Accommodations The Most Commonly Used Accommodations for Students With Disabilities Include:for Students With Disabilities Include:
(Thurlow, et al, 2001; Tindal et al, 2000)(Thurlow, et al, 2001; Tindal et al, 2000)
• Braille (allowed by 33 of the 48 states studied)• Computerized assessment (34 states)• Dictation of responses to a scribe (32 states)• Extended time (37 states)• Translation of instructions (40 states)• Allowance for marking answers in the test
booklets (33 states)• Test items read aloud (34 states)• Simplified test directions (31 states)• Test breaks (33 states).
Do we have enough evidence on the following Do we have enough evidence on the following characteristics of these most commonly used characteristics of these most commonly used
accommodation?accommodation? EffectiveValidConsistent with students’ backgroundFeasible Relevant
Accommodations for Accommodations for ELLs and Students with DisabilitiesELLs and Students with Disabilities
• Can the same accommodations used for students with disabilities be used for ELLs?
• Can the same accommodations used for ELLs be used for students with disabilities?
SY 2000-2001 Accommodations Designated for ELLs Cited in States’
Policies
There are 73 accommodations listed:
N: Not Related
R: Remotely Related
M: Moderately Related
H: Highly Related
From: Rivera (2003) State assessment policies for English language learners. Presented at the 2003 Large-Scale Assessment Conference
There are 73 Accommodations Listed
47 or 64% are not related
7 or 10% are remotely related
8 or 11% are moderately related
11 or 15% are highly related
Samples accommodations used Samples accommodations used for ELL studentsfor ELL students
Test-taker marks answers in test booklet
Copying assistance provided between drafts
Test-taker indicates answers by pointing or other method
Paper secured to work area with tape/magnet
Physical assistance provided
Samples accommodations used Samples accommodations used for ELL studentsfor ELL students
Enlarged answer sheets providedBreaks provided Test individually administeredTest administered in small groupTest administered in location with
minimal distraction
Assessment Options for all ELLs Assessment Options for all ELLs in Oregonin Oregon
• Students may take the test under standard administration with or without accommodations
• Students may take side-by-side English/Spanish, English/Russian versions where provided.
• Eligible students may respond on the Writing Assessment in Spanish.
• Students may take the test under modified conditions. • Students in grade 3 may take the Aprenda (Spanish
reading test at grade 3).• Students may use the Juried Assessment process for
reading or writing in another language.
Test Decisions for ELLsTest Decisions for ELLs(Oregon)(Oregon)
• A teacher and instructional team who know the student make the decision to test under standard conditions or modify test
• Consult parent/guardian• Each student must be considered individually for
each assessment• Best interest of student• Not on participation in a program nor
identification as an ELL• How about research evidence?
Accommodations Tables:Accommodations Tables:Changes in Timing or SchedulingChanges in Timing or SchedulingKnowledge & Skills Test
Writing Test ELPA Test
• Extended Time
• Frequent Breaks
• Divide into sessions
• Most beneficial time of day
• Extended Time
• Frequent Breaks
• Most beneficial time of day
• Extended Time
• Frequent Breaks
• Divide into sessions
• Most beneficial time of day
Accommodations Tables:Accommodations Tables:Changes in Test DirectionsChanges in Test Directions
Knowledge & Skills Test Writing Test ELPA
• Read or reread• Sign• Translate orally• Provide written
version of oral directions
• Simplify language • Clarify• Highlight words• Auditory
amplification devices
•Read or reread•Sign•Translate orally•Provide written version of oral directions•Simplify language •Clarify•Highlight words•Auditory amplification devices
•Read, reread or repeat•Sign•Translate orally•Provide written version of oral directions
•Clarify
•Auditory amplification devices
Accommodations Tables:Accommodations Tables:Changes in How the Test Questions are PresentedChanges in How the Test Questions are Presented
Knowledge & Skills Test Writing Test ELPA
• Large print version• Braille version• Read aloud (not
reading test)• Student reads test
aloud or sub-vocalizes
• Proctor highlights vowel combinations (not reading test)
• Student highlights vowel combinations
• Visual magnification devices
• Large print version• Braille version• Read aloud • Sign • Student reads test
aloud or sub-vocalizes
• Electronic word-for-word, text-to-voice scanning
• Proctor highlights vowel combinations
• Visual magnification devices
• Student reads test aloud or sub-vocalizes
• Visual magnification devices
Accommodation Tables:Accommodation Tables:Changes in How the Student RespondsChanges in How the Student RespondsKnowledge & Skills Test Writing Test ELPA
• Answers marked in test booklet or recorded & transcribed onto answer sheet
• Use of assistive technology device that serves as primary communication mode
• Point to or dictate responses to a scribe
• Student retells story Student vocalize first
• Use a recording device to record/playback
• Students who require increased spacing can do so
• Respond in Braille• Use of assistive
technology device that serves as primary communication mode (with certain computer features disengaged)
• Function keys may not be used
• Student vocalize first
• Student retells story to proctor before responding to questions
• Point to or dictate responses to a scribe (in English or native language)
• Use of assistive technology device that serves as primary communication mode (with certain computer features disengaged)
• Function keys may not be used
Accommodation Tables:Accommodation Tables:Changes in Test SettingChanges in Test Setting
Knowledge & Skills Test Writing Test ELPA
• Test student in a separate location
• Test small group of students in separate location
• Minimize distractions• Encourage student’s
work habits during test
• Stabilize test material/papers
• Use sensory processing techniques
• Use adaptive furniture or positioning
• Test student in a separate location
• Test small group of students in separate location
• Minimize distractions• Encourage student’s
work habits during test
• Stabilize test material/papers
• Use sensory processing techniques
• Use adaptive furniture or positioning
• Physical setting• Use of physical
assistance or devices
• Test student in a separate location
• Test small group of students in separate location
• Minimize distractions• Physical setting
(seating, lighting)• Stabilize test
material/papers• Use sensory
processing techniques• Use adaptive furniture
or positioning
How are we doing in practice nationally?
Are states and districts across the nation cognizant of this important principle of using accommodations that are appropriate for ELLs?
Are there any objective national criteria to help states to select appropriate accommodations for ELL students?
Or, is the assignment of accommodations to these students based on temporary and subjective decisions?
Comparability IssuesComparability Issues• If accommodated assessment is not valid then
the outcome may not be comparable with the non-accommodated assessment (a major peer-review concern)
• To report AYP for ELL students, it is imperative to establish the validity of accommodated assessment
• Construct-irrelevant sources such as linguistic and cultural biases should be controlled before reporting AYP based on accommodated assessments
Are there accommodations that Are there accommodations that would benefit both ELLs and SWDs?would benefit both ELLs and SWDs?
• Assessment can be designed in a way to be accessible to both groups.
• For example, long tests, crowded pages, tables and texts would create frustration and anxiety for everyone, particularly for ELLs and SWDs.
• Complex linguistic structure of assessment would be a major nuisance variable for ELLs and SWDs (Tindal, G., Anderson, L., Helwig, R., Miller, S., & Glasgow, A. (2000).
Accommodating students with learning disabilities on math tests using language simplification. Eugene: University of Oregon, RCTP)
3rd Graders Frequencies of Special Education Groupings
Overall % Non-LEP % LEP %Regular Education 84.1 84.0 93.9
Educable Mentally Handicapped 1.1 1.1 0.0
Seriously Emotionally Disturbed 0.4 0.4 0.0
Learning Disability 10.0 10.1 5.1
Trainable Mentally Handicapped 0.0 0.0 0.0
Severely Mentally Handicapped 0.0 0.0 0.0
Physically Impaired 0.7 0.7 0.0
Hard of Hearing – Partially Deaf 0.2 0.2 0.0
Blind 0.0 0.0 0.0
Partially Sighted 0.0 0.0 0.0
Autistic 0.0 0.0 0.0
Deaf and Blind 0.0 0.0 0.0
Speech 3.4 3.4 1.0
Total 100.0 100.0 100.0
A Sample Representing a Subgroup of Students with DisabilitiesA Sample Representing a Subgroup of Students with Disabilities
8TH Graders Frequencies of Special Education Groupings
Overall % Non-LEP % LEP %Regular Education 86.8 86.7 92.3
Educable Mentally Handicapped 1.3 1.2 3.6
Seriously Emotionally Disturbed 0.7 0.7 0.0
Learning Disability 10.0 10.1 3.6
Trainable Mentally Handicapped 0.1 0.1 0.0
Severely Mentally Handicapped 0.1 0.1 0.0
Physically Impaired .05 .05 0.0
Hard of Hearing – Partially Deaf 0.1 0.1 0.0
Blind 0.0 0.0 0.0
Partially Sighted 0.0 0.0 0.0
Autistic 0.0 0.0 0.0
Deaf and Blind 0.0 0.0 0.0
Speech 0.3 0.3 0.0
Total 100.0 100.0 100.0
A Sample Representing a Subgroup of Students with DisabilitiesA Sample Representing a Subgroup of Students with Disabilities
A Clear Language of Instruction A Clear Language of Instruction and Assessment Works for and Assessment Works for ELLs, SWDs, and EveryoneELLs, SWDs, and Everyone
•What is language
modification of test items?
Learning Disability & Language of Learning Disability & Language of AssessmentAssessment
Students in the Learning Disability category may have difficulty processing complex language in assessmentSimplifying the language of test items will help students with disabilities, particularly those with learning disabilitiesAs the sample page suggests, a large majority of students with disabilities are in the Learning Disability category
Examining Complex Linguistic Features in Content-Based Test Items
Feature Feature Description Categories Combined
1 I tem length 1, 2, 4, 45
2 Vocabulary 3, 26, 27
3 Nominal heaviness 5, 6, 29, 30, 31, 32
4 Verb voice 7, 33
5 Modal 8, 34
6 Relative clause 9, 10, 11, 35, 36, 37
7 Adverbial modification 12, 13, 14, 15, 16, 17, 38, 39, 40, 41
8 Conditional clause 18, 19
9 Complement clause 20, 44
10 Sentence structure 28, 42, 43, 46
11 Preferred argument structure 22, 23, 47, 48
12 Question form 21
13 Global difficulty 24
14 Content interest 25
Familiarity/frequency of non-math vocabulary: unfamiliar or infrequent words changed
census > video gameA certain reference file > Mack’s company
Length of nominals: long nominals shortened last year’s class vice president > vice presidentthe pattern of puppy’s weight gain > the pattern above
Question phrases: complex question phrases changed to simple question words
At which of the following times > Whenwhich is best approximation of the number >
approximately how many
Linguistic Modification Concerns
Conditional clauses: conditionals either replaced with separate sentences or order of conditional and main clause changed If Lee delivers x newspapers > Lee delivers x newspapers
If two batteries in the sample were found to be dead > he found three broken pencils in the sample
Relative clauses: relative clauses either removed or re-cast A report that contains 64 sheets of paper > He needs 64
sheets of paper for each report
Voice of verb phrase: passive verb forms changed to activeThe weights of 3 objects were compared > Sandra
compared the weights of 3 rabbitsIf a marble is taken from the bag > if you take a marble
from the bag
Linguistic Modification cont.
Harriet, Jim, Roberto, Maria, and Willie are in the same eighth Harriet, Jim, Roberto, Maria, and Willie are in the same eighth grade class. One of them is this year’s class president. Based grade class. One of them is this year’s class president. Based on the following information, who is the class president?on the following information, who is the class president?
The class president was last year’s vice president and lives The class president was last year’s vice president and lives on Vince Street. on Vince Street.Willie is this year’s class vice president.Willie is this year’s class vice president.Jim and Maria live on Cypress Street.Jim and Maria live on Cypress Street.Roberto was not last year’s vice president.Roberto was not last year’s vice president. A. A. JimJimB. HarrietB. HarrietC. RobertoC. RobertoD. MariaD. MariaE. WillieE. Willie
Original ItemOriginal Item
Modified ItemModified Item Harriet, Jim, Roberto, Maria, and Willie ran for president of
their eight-grade class. One of them won. Who is president? The president now was vice president last year and lives on Vince Street.Willie is vice president now.Jim and Maria live on Cypress Street.Roberto was not vice president last year. A. JimB. HarrietC. RobertoD. MariaE. Willie
Original:Original:The census showed that three hundred fifty-six thousand, The census showed that three hundred fifty-six thousand,
ninety-seven people lived in Middletown. Written as a ninety-seven people lived in Middletown. Written as a number, that is: number, that is:A. A. 350,697350,697B. 356,097B. 356,097C. 356,907C. 356,907D. 356,970D. 356,970
Modified:
Janet played a video game. Her score was three hundred fifty-six thousand, ninety-seven. Written as number, that is: A. 350,697B. 356,097C. 356,907D. 356,970
Interview StudyInterview Study• Table 1. Student Perceptions Study: First Set (N=19)• Item # Original item chosen Revised item chosen• 1 3 16• 2 4 15• 3 10 9• 4 11 8• Table 2. Student Perceptions Study: Second Set (N=17)• Item # Original item chosen Revised item chosen
5 3 14• 6 4.5a 12.5• 7 2 15• 8 2 15
Many students indicated that the language in the revised item was easier:
“Well, it makes more sense.”
“It explains better.”
“Because that one’s more confusing.”
“It seems simpler. You get a clear idea of
what they want you to do.”
“It’s easier to read, and it gets to the point, so
you won’t have to waste time.”
“I might have a faster time completing that
one ’cause there’s less reading.”
“Less reading; then I might be able to get to
the other one in time to finish both of them.”
“’Cause it’s, like, a little bit less writing.”
The revised items need less time for response:
Conclusions and RecommendationsConclusions and RecommendationsAccommodations:
Must be relevant in addressing assessment issues for ELL students Must be effective in reducing the performance gap between accommodated and non-accommodated studentsShould not alter the construct being measuredThe accommodated results can be aggregated with the assessments under standard conditionsMust be feasible in national and state assessments
Conclusions and RecommendationsConclusions and Recommendations
Providing a customized dictionary is a viable alternative to providing traditional dictionaries.
The linguistic modification of test items that reduce unnecessary linguistic burdens on students is among the accommodations that help ELL students without affecting the validity of assessments.
Computer testing with added extra time and glossary was shown to be a very effective, yet valid accommodation (Abedi, Courtney, Leon, and Goldberg, 2003)
Examples of research-supported accommodations:
Conclusions and RecommendationsConclusions and Recommendations
Without information on important aspects of accommodations such as validity, it would be extremely difficult to make an informed decision on what accommodations to use and how to report the accommodated and non-accommodated results.
It is thus imperative to examine different forms of accommodations before using them in state and/or national assessments.
Accountability QuestionsAccountability QuestionsAre there specific ELL/SWD subgroup features that affect the accountability system?
Yes No
Could the current accountability system for ELLs/SWDs be improved?
Yes No
Do research findings help inform assessment & accountability systems for these students?
Yes No
For more information, please For more information, please contact Jamal Abedi at contact Jamal Abedi at
UC Davis/CRESST:UC Davis/CRESST:
(530) 754-9150
or
