Upload
schoolpsychology
View
17.632
Download
2
Embed Size (px)
DESCRIPTION
Mike Vanderwood, Ph.D. University of California, RiversideUsing Response to Intervention with English Language Learners.Recent changes in federal legislation and California code provide educators an opportunity to implement response to intervention (RtI) approaches in general and special education. RtI decision making is particularly promising for English Language Learners (EL) because this model places a heavy emphasis on prevention strategies, and provides skill acquisition data that can be used to examine special education eligibility. The primary expected learning outcome is for participants to acquire a basic understanding how to use RtI with ELs.
Citation preview
1
Best Practices in Assessment and Intervention for ELLMike Vanderwood, Ph.D.University of [email protected]
2
ELL Growth
9.6% of students in the US public schools are ELLs 25.2% of students in California schools are ELL, and
85.3% of these students speak Spanish ELLs’ literacy skills wane in comparison to non-ELL
students (NCES, 2005) ELLs have a higher risk of being placed into special
education across elementary grades, and are 40-50% more likely than their White peers to qualify for SLD. (Artiles, et al, 2005)
Schools often delay examination for Special Ed because they want to eliminate language interference
3
Current Practice Very few teachers and psychologists are
trained to work with students who have diverse language backgrounds
Some suggest that we have a national shortage of qualified staff and the shortage will continue to grow.
4
Foundation Of Practice Growing understanding that educational
practices should be guided by high quality research and standards of practice
New standards for determining whether a practice is “evidence based”
IES implementation of “What Works Clearinghouse”
AERA/APA/NCME Test Standards
5
Foundations: Test Standards and ELL
6
LEP Assessment (English language learners) Students are considered Limited English
Proficient when: they were not born in the U.S. and native
language is not English, or Come from environments where English is not
dominant language Bilingual individuals do not necessarily have
equal proficiency in both languages
7
Standards Must have validity evidence for the purpose
used Pg. 95. If student is not from the culture or
linguistic that produced norms, scores may not provide a valid comparison
Standard 9.3, bilingual students should be assessed to determine the proficiency in both languages
8
Standards (cont) Standard 9.10 determining language
proficiency needs to be based on a range of language features
Standard 9.11, interpreters need to be fluent in both languages, and have some knowledge of assessment
9
Test Bias for LEP (ELL) Significant flaws in research in this area Primary issue is determination of English
proficiency Studies should account for: acculturation, cultural
background, quality of instruction and educational history
Current evidence does not indicate bias, yet results are questioned by many
10
Foundations: Research Standards
11
Scientifically Supported Interventions Dept. of Education was reorganized to put a focus on
high quality research Institute of Education Sciences (IES) Created What Works Clearinghouse
http://www.whatworks.ed.gov/ WWC evaluates educational interventions Focused on applying randomized controlled
experiments as methodology to determine causation
12
What Works ELL Report
13
ELL Literacy Assessment
14
Prevention is Critical Several authors (Gersten, 2005; Vaughn,
2005; What works, 2007; Vanderwood, 2008) suggest RtI is an exceptionally appropriate service delivery approach for ELL
15
RTI Defined RtI is the practice of (1) providing high-
quality instruction/intervention matched to student needs and (2) using learning rate over time and level of performance to (3) make important educational decisions. Source: Response to Intervention: Policy
considerations and implementation (National Association of State Directors of Special Education-2005).
16
Quick Summary of What We Know for NS Early screening and Intervention impacts performance and
can reduce those who need special education Reading Performance is best described by performance in:
Phonological Awareness Phonics Comprehension Fluency
Progress monitoring with instruments that have strong reliability and validity can improve outcomes for students with academic problems
17
Does this literacy knowledge apply for ELLs? Additional Focus on oral language proficiency Influence of culture?
ELL Students are Often not included in literacy screens. Why?
Research suggests, we can accurately assess English early literacy skills as early as Kindergarten for ELLs.
18
Language Proficiency Degree of control one has over the language in
question Basic Interpersonal communication skill (BICS)
Communicating in socially related situations First area to develop in new language
Cognitive academic language proficiency skills (CALPS) Developed through academic activities
19
Language of Instruction Center for Research on the Education of Students Placed At
Risk (CRESPAR) Very few studies that meet current research standards Native language instruction appears to be most beneficial,
and does not harm English Language Dev. Early exit appears to work as well as late exit (research is not
exceptioinally clear).
http://www.csos.jhu.edu/crespar/techReports/Report66.pdf
20
Critical Point It is not necessary to wait to deliver English
Language literacy support/interventions until the native language is established
In fact, all ELL students’ literacy skills should be assessed as soon as possible (i.e., kindergarten).
21
Previous ELL Reading Research Chiappe, Siegel, & Wade-Woolley (2002)
858 Kindergarteners, 23 languages A significant relationship between PA and improved reading
was reported in this study for monolingual and ELL students Linan-Thompson & Hickman-Davis (2002)
Supplementary reading instruction with at-risk, low SES second grade monolingual and ELL students
30-minute supplementary instruction: phonological awareness, vocabulary, decoding, comprehension, and word analysis strategies
Positive outcomes for reading for monolingual and ELL students
22
Lesaux & Siegel (2003) Suggest that PA may be a better predictor of reading
development than oral language proficiency Development of reading for ELL students was not predicted
by English Language Proficiency PA instruction was effective for ELL students EL group developed quickly and by grade 2 were performing
as well or better on most tasks as EO students Metal-linguistic advantage for ELL may explain rapid
improvement PA is an important skill for EL. For poor performers, similar
profile exists to EO poor performers
23
Literacy Research with typical RtI measures What type of evidence to we need
Ability to identify students at risk Sensitivity and specificity
Sensitive to growth during intervention Use across curricula and interventions Minimally impacted by language proficiency?
24
Nonsense Word Fluency (NWF) Standardized, individually administered test of alphabetic principles Intended for children from mid to end of kindergarten through the
beginning of second grade Children are given 1 minute to read as many nonsense words as
possible Example:
Read whole word, such as “lut” or say the individual sounds, such as /l/ /u/ /t/
Psychometrics One month alternate form reliability is .83 Predictive validity in first grade with CBM ORF first grade
is .82, and CBM ORF second grade is .60
25
NWF Study # 1: Purpose of Study To examine the appropriateness of using the
NWF in assessing students’ reading readiness skills for an EL, urban population
26
Site Demographics 1 school site 100% Free/ Reduced Lunch Ethnicity:
83% Hispanic (n=165) 11% African American (n=21) 6% other (n=13)
Home language: 82% Spanish (n=186) 14% English (n=32) 5% other (n=12)
27
EL Status (Determined by CELDT): Beginning LEP N = 49 (21%)
Intensive/structured-English classes Early Intermediate Range N = 58 (25%)
Structured English class Intermediate Range N = 111 (47%)
structured-English/English only classes Early Advanced/Advanced range N = 16 (7%)
Excelled class
28
Mean Differences (All Signficant)
22.19 25.17 52.98
104 103 94
17.516 19.926 27.323
32.23 43.16 72.71
44 43 42
14.325 19.897 32.722
44.98 47.85 81.61
56 55 59
21.410 27.292 34.601
59.63 91.69 100.78
16 16 16
37.845 48.202 42.372
32.72 39.39 68.54
220 217 211
23.277 30.530 35.180
Mean
N
Std. Deviation
Mean
N
Std. Deviation
Mean
N
Std. Deviation
Mean
N
Std. Deviation
Mean
N
Std. Deviation
English LanguageLearner LevelBeginning LEP
Early Intermediate
Intermediate
Advanced
Total
Time 1 Median Time 2 Median Time 3 Median
29
Measuring Growth
31.5109 1.2220
92 91
23.90756 .92448
40.5000 1.4886
42 42
26.00211 .95634
37.5273 1.5337
55 53
26.37680 1.06314
41.1563 1.3290
16 15
41.86793 1.37047
35.7195 1.3679
205 201
26.82908 1.00874
Mean
N
Std. Deviation
Mean
N
Std. Deviation
Mean
N
Std. Deviation
Mean
N
Std. Deviation
Mean
N
Std. Deviation
English LanguageLearner LevelBeginning LEP
Early Intermediate
Intermediate
Advanced
Total
Total Growth from Time 1 to Time 3 Weekly Growth from Time 1 to Time 3
30
Results of Study NWF was able to significantly predict reading scores
on a state-mandated achievement test above and beyond EL level
There were significant mean differences on all NWF scores by EL Level.
NWF alone significantly predicted SAT9 reading composite at 11 of 12 opportunities
Growth was not significantly different by EL group
31
NWF Study # 2: SAMPLE METHODS 3-year longitudinal study Spanish-speaking ELL population
90% ELL in Grade 1 58% ELL in Grade 3
Assessed first-grade students: Nonsense Word Fluency SAT9
Assessed third-grade students: ORF MAZE CAT6
32
Correlation of Grade 1 Measures to Grade 3 Measures
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
NWF (Beginning)
NWF (Middle)
NWF (End)
SAT9
CAT6 ORF MAZE
.36
.38
.34
.17
.50
.57
.64
.25.26
.24
.14
.23
33
Specificity and Sensitivity Analysis VP = Valid positive; FN = False negative; FP =
False positive; VN = Valid negative; Pos. PV = Positive predictive value (VP/(VP + FP); Neg. PV = Negative predictive value (VN/(VN +
FN); Sensitivity = VP/(VP + FN); Specificity = VN/(VN + FP); Hit rate = (VP + VN)/(VP + FN + VN + FP).
34
Specificity and Sensitivity of NWF to ORFOutcome Measure NWF At Risk NWF Not at
RiskIndices
CBM ORF < 25th percentile VP = 19 FN = 15 Sensitivity = 55%
CBM ORF > 25th percentile FP =18 VN =82 Specificity = 82%
Pos. PV = 51% Neg. PV = 84% Hit Rate = 75%
35
Specificity and Sensitivity of NWF to MazeOutcome Measure NWF At Risk NWF Not at Risk Indices
Maze < 25th percentile VP = 30 FN = 40 Sensitivity = 43%
Maze > 25th percentile FP = 7 VN = 57 Specificity = 89%
Pos. PV = 81% Neg. PV = 59% Hit Rate = 65%
36
Specificity and Sensitivity of NWF to CAT6Outcome Measure NWF At Risk NWF Not at Risk Indices
CAT6 < 25th percentile VP = 16 FN = 18 Sensitivity = 47%
CAT6 > 25th percentile FP = 21 VN = 79 Specificity = 79%
Pos. PV = 43% Neg. PV = 81% Hit Rate = 71%
37
Progress Monitoring - Intervention Study #1: Research Question What is the impact of a phonological
awareness intervention implemented with at-risk ELL students?
How effective is progress monitoring in English with ELL.
Healy, Vanderwood, & Edelston. (2005)
38
Methods Participants
15 Low SES ELL first grade students (7 male, 8 female) with PSF and NWF baseline scores 30 and below
Materials Sounds and Letters for Readers and Spellers
(Greene, 1997) Progress Monitoring
PSF and NWF weekly
39
Methods (cont.) Procedures
Small group (max. 5) ½ Hour Sessions x2 per week Exited after PSF > 50 and NWF > 45 12 – 25 Sessions
40
Results 6 students exited at first exit point after 12 sessions 12 students (80%) exited by conclusion 2 students (13.3%) met exit criteria for PSF, but not
NWF 1 student (6.7%) met exit criteria for NWF, but not
PSF The group as a whole went from mean PSF and
NWF scores that were considered to be in the at risk range to mean PSF and NWF scores that were in the mastery level range
41
0
10
20
30
40
50
60
Assessment Dtaes
Co
rre
ct
Se
gm
en
ts
Goal Line
BaselineIntervention
Trendline
Example of PSF scores of a participant who was exited from the intervention
42
0
10
20
30
40
50
60
1/2
1/2
004
1/2
8/2
004
2/4
/200
4
2/1
1/2
004
2/1
8/2
004
2/2
5/2
004
3/3
/200
4
3/1
0/2
004
3/1
7/2
004
3/2
4/2
004
3/3
1/2
004
4/7
/200
4
4/1
4/2
004
4/2
1/2
004
4/2
8/2
004
5/5
/200
4
5/1
2/2
004
5/1
9/2
004
5/2
6/2
004
6/2
/200
4
6/9
/200
4
Assessment Dates
Co
rre
ct
So
un
ds
Goal Line
Trendline
Baseline Intervention
A participant’s NWF scores who met the exit criteria for PSF, but not NWF
43
0
5
10
15
20
25
30
35
40
45
50
Assessment Dates
Co
rre
ct
Se
gm
en
ts
Goal Line
Trendline
BaselineIntervention
A participant’s PSF scores who met the exit criteria for NWF, but not PSF
44
Progress Monitoring/Intervention Study #2: To examine the extent to which a direct and
explicit PA intervention impacts at-risk ELLs’ early literacy skills.
To replicate findings by Healy, et al., 2005 To examine the impact of a bilingual, direct
and explicit PA intervention with at-risk ELLs. To assess the extent to which progress
monitoring is effective with ELLs.
45
Method
10 Participants (8 males, 2 females) CELDT Levels (beginner, early intermediate, & intermediate levels) Three Pronged Screening Method
Bottom 25% of students on Developmental Reading Assessment DIBELS PSF & NWF (at-risk levels)
Intervention Curriculum Sounds and Letters for Readers and Spellers (Greene, 1997) with
modifications Progress Monitoring
One time per week using PSF and NWF tests
46
Procedures 2 groups of 5 students 30 minutes intervention 2x per week Reinforcement: Token Economy Treatment Integrity
A 10-item checklist was developed Assessed 30% of intervals with 90% accuracy Curriculum manual was used 100% of observations
Added Bilingual support and Spanish PA intervention
47
Results Baseline
PSF: 0 – 6 (M = 3, SD = 2.75) NWF: 0 – 32 (M = 10, SD = 9.75)
Final 3 Monitoring Data Points PSF: 19 – 72 (M = 56.8, SD = 20.6) NWF: 49 - 66 (M = 56.8, SD = 23.4)
Effect Size [Cohen’s d (1988); Pooled (Swanson & Saches-Lee (2000)]
PSF: 19.8; 1.9 (Large) NWF: 4.8; 1.3 (Large)
48
Conclusion In 20 sessions of supplemental English PA instruction, 8 of
10 students met or exceeded PSF and NWF goals This study provides further support for the use of RTI to
determine LD status among ELL students at-risk for reading failure.
3 of 3 students met PSF goals using the bilingual PA intervention
Trendline analysis indicates that the Bilingual PA intervention was as and more effective than English PA alone for this very small sample, but only for PA skills.
49
Oral Reading Fluency and Middle School Students, including EL Learners
50
CBM per Week Gains by English Fluency
00.10.20.30.40.50.60.70.80.9
1
EO FEP ELL
IL gain
GL gain
EO
FEP ELL
Grade level Word Per Week Gain
.57 N=1084
.60 N=294
.68 N=1415
Instructional level Word Per Week Gain
.40 N=841
.44 N=247
.56 N=1150
51
Assessment Conclusions Initial evidence that typical measures used for
monitoring progress for EO students can be used with ELL
Initial evidence that measures can pick up growth caused by an intervention
Initial evidence that rate of growth for ELL is similar to EO
52
Instructional Practices for ELL (Gersten & Geva, 2003) Explicit teaching Promotion of English Language Learning Phonemic awareness and decoding Vocabulary development Interactive teaching that maximizes student
engagement Opportunities for accurate responses with
immediate feedback
53
Vanderwood et al. English Intervention studies Used Dibels to select group Small group instruction (3 to 5) Explicit instruction in phonemic awareness and
phonics Frequent use of Choral responding Behavioral reinforcement techniques to maintain
high levels of engagement 30 minutes of intervention 2X per week Between 87-93% of students moved established
level on Dibels measures within 16 weeks
54
Experimental ELL PA Intervention All 12 Sessions include a Vocabulary Section,
and 5 Phonological Awareness Activities in the following order: Phoneme Production/Replication; Phoneme Segmentation and counting, Phoneme Blending, Phoneme Isolation, and Rhyming.
55
Sessions 1-3 Session 1: / t /; / s /; / m /; / b /; / k /; / f /,
Short / a /. Session 2: Review: / t /; / s /; / m /; / b /; /
k /; / f /, Short / a /. New : / r /; / h /; / j /; / n /; / l /; / p /.
Session 3: Review: / t /; / s /; / m /; / b /; / k /; / f /; / r /; / h /; / j /; / n /; / l /; / p /, Short / a /. New : / g /; / d /; / v /.
56
Spanish Intervention (Vaughn et al., 2006a) 69 ELL (Spanish) 1st graders at risk for
reading difficulty Screening was conducted in Spanish
(Woodcock Language Proficiency Battery-Spanish—LWID) and Experimental measure of Spanish word reading ability)
Randomly assigned by school (7 schools, 20 classrooms) to treatment or comparison.
57
Treatment: Systematic, explicit instruction in oral language and reading (Lectura Proactiva, Mathes, et al., 2003) for groups of 3 to 5 for 50 minutes a day.
In Spanish to match core literacy instruction Comparison: School’s standard reading
intervention (e.g., guided reading, reading recovery, tutoring)
58
Vaughn et al., 2006a Results Pre-Post design
No group differences in pre-test scores Assessed Spanish & English literacy & oral
language Treatment group scored higher on most
Spanish outcome measures No group differences on English outcome
measures
59
Intervention Starting Points (Vaughn, et al., 2006b) Started with assumption that ELL learn to read like
monolingual students: through phonological recoding and spelling sound patterns
Assumed that those students have problems learning how to read are struggling because they have not mastered the alphabetic principle.
Decodable text used throughout instruction Sight words were taught (words that are less
phonetically regular in English)
60
Instructional Design (Vaughn, et al., 2006b) Systematic and explicit instruction for
Phonemic awareness Phonemic decoding skills Word recognition fluency Construction of meaning Vocabulary Spelling Writing
Used the same intervention as others had used with EO students, but interspersed language support activities
61
Population 2 districts in Texas 4 schools
English intervention to at least 2 classes of ELL first graders (48 to 99% Spanish speaking)
At least 60% of school population was Latino 80% or more of 3rd grade students passed state
reading tests
62
Screening Assessment 14 first grade classrooms 2 tests in English, 2 in Spanish
Letter word Identification from Woodcock Language Proficiency Battery in English and Spanish
Word lists in English and Spanish Note: this is not currently an empirically supported
approach to screening 56 students met criteria in both languages, 48
participated in the study 24 treatment, 24 comparison
63
Intervention Structure 48 student Groups of 3 to 5 50 minutes per day from October to May Supplemental to core reading instruction (not as
replacement!) Teachers received 12 hours of professional development
before intervention and 6 hours after first 6 weeks Staff development throughout the year occurred and lessons
were videotaped Conducted intervention validity checks
64
Intervention implementation Fast paced, all students responding, followed
by individual turns Provided immediate feedback
65
Results Intervention students outperformed
comparison group on English letter naming, phonological awareness, other language skills and reading achievement
Differences were less significant on Spanish measures
66
Evaluating Interventions for ELL Structure: Direct Instruction is critical Size: Small group is essential Feedback process: immediate and often Content: similar to monolingual students with
enhanced vocabulary
67
Special Ed Eligibility Issues for ELL
68
Special Ed Assessment Issues Eligibility determination issues:
Non-verbal IQ tests
69
IQ/ Processing Assessment Techniques bilingual examiner Translator Translated version developed with norms translated version with English norms Translated version with foreign country norms non-verbal assessment Problem-solving
70
Testing Options English developed Test translated to foreign
language May not measure same characteristic in the foreign
language Questions may have changed in meaning Cultural groups may differ in conceptualization of
construct Violates Test Standards
Foreign norms with American translated test Different culture, issue of equivalence
71
Non-verbal for ELL Many lack appropriate psychometric evidence Predictive power is limited and not as strong
as traditional IQ tests Not normed with ELL population
72
Issues related to LD identification for ELL We need to improve our system at identifying and
intervening with ELL students who have academic problems. We need to identify students who are most at risk of
academic difficulties earlier than 3rd grade, preferably kindergarten.
We need to use resources in general and special education more efficiently.
We need to provide an integrated continuum of services that uses high quality data for entry to succeeding levels.
We need to re-conceptualize the construct of LD to integrate language proficiency.
73
We need to better connect our initial eligibility assessments to instruction and intervention (assessment to intervention link) and address issues of literacy development in L1 and L2
We need to integrate recent assessment science that suggests that progress monitoring can have substantial positive impact on academic outcomes.
74
RTI special education decision making Disability and Need Concept of Dual Discrepancy Disability:
Lack of response to high quality intervention Torgesen’s work shows about 6%
Need: Convergent Data District determined Norm referenced Individualized achievement test Large scale assessment Alternative assessment local norms Percentage for cutoff ranges from 6 to 10%
75
What about culture? MAMBI Acculturation surveys?
76
Conclusions Initial evidence early literacy measures can be
used to identify EL students at risk of literacy problems.
Initial evidence tools can be used assess growth during intervention
77
What we don’t know How does language proficiency affect the
ability of the measures to achieve their purposes?
How do we integrate culture into the decision making process?
How do we integrate native language knowledge into the process?