Embed Size (px)
Citation preview
Hill Reading Achievement Program Student Achievement Results for
Brunswick County Schools, 2009-2011
Presented by:Tamara M. Walser, Ph.D., Lead Evaluator & Emily R. Grace, M.Ed., Project Coordinator
The University of North Carolina Wilmington
March 29, 2012
Research Questions and Measures
Do students who receive HillRAP instruction improve academic achievement in reading, overall, and by student groups?
• Woodcock-Johnson III Tests of Achievement (WJ-III)• North Carolina End-of-Grade Reading Comprehension Test (NC EOG)
Is there a relationship between the number of HillRAP instructional hours a student receives and achievement in reading? • Attendance of HillRAP Sessions
Do teachers who receive HillRAP training effectively implement HillRAP in a public school setting?
• Classroom Observations of Teachers
Students, Schools, and TeachersStudents
First Year 2009-2010 Sample (n = 325)
Two Year 2009-2011 Longitudinal Sample(n = 164)
Schools and Teachers2009-2010: 10 Elementary, 4 Middle Schools
32 Teachers2010-2011: 10 Elementary, 3 Middle, 3 High
Schools37 Teachers
Woodcock-Johnson III Tests of Achievement Results
Description of Subtests Used
Letter-Word Identification: List of individual words for student to read aloud
Reading Fluency: Short sentences for student to read quickly and identify as true or false
Passage Comprehension: Short sentences containing a blank for student to read and supply appropriate word
Word Attack: List of “nonsense” words for student to read aloud, measuring phonics skills
What are Age-Referenced Standard Scores?
Age-referenced standard scores account for expected academic growth.
A student who makes expected growth would have the same standard score at pretest and posttest.
An increase in a standard score indicates greater than expected growth; a decrease indicates the opposite.
The average standard score for the WJ-III is 100; the average range is 90-110.
Means, Standard Deviations (SD), and Change Scores for First Year Sample and Two Year Longitudinal Sample Age-Referenced Standard Scores on Woodcock Johnson Tests of
Achievement III Reading Subtests
SubtestFirst Year Sample
(n = 325)aTwo Year Longitudinal Sample
(n = 164)b
Fall 2009 Mean
Spring 2010Mean
2009–2010
Change
Fall 2009Mean
Spring 2010Mean
Spring 2011Mean
2009–2011
Change
Letter-Word Identification
87.39(12.39)
89.07(12.34)
1.67** 85.46(12.76)
87.24(12.68)
87.34(13.02)
1.88**
Reading Fluency 84.74(11.24)
87.80(11.78)
3.06** 83.67(11.35)
86.46(11.81)
86.88(11.71)
3.21**
Passage Comprehension
81.70(11.25)
84.92(11.64)
3.22** 79.55(11.72)
83.15(11.73)
83.50(11.71)
3.95**
Word Attack 88.89(10.24)
92.09(9.70)
3.20** 87.17(10.46)
90.95(9.11)
92.48(10.60)
5.31**
an size for First Year Sample is 324 for Reading Fluency and Word Attack subtests*p <.05. **p < .01
0
10
20
30
40
50
60
70
80
90
100
110
Fall 2009Spring 2010Spring 2011
All Students in Two Year Longitudinal Sample (n = 164)
Elementary Students Fall 2009 & Spring 2010 (n = 257) Spring 2011 (n = 113)
Letter W
ord Identifica
tion
Reading Fluency
Passage Comprehensio
n
Word Attack
0
20
40
60
80
100
Fall 2009Spring 2010Spring 2011
Middle School Students Fall 2009 & Spring 2010 (n = 68) Spring 2011 (n = 51)
Letter W
ord Identifica
tion
Reading Fluency
Passage Comprehensio
n
Word Attack
0
20
40
60
80
100
Fall 2009Spring 2010Spring 2011
White Students Fall 2009 & Spring 2010 (n = 170) Spring 2011 (n = 90)
Letter W
ord Identifica
tion
Reading Fluency
Passage Comprehensio
n
Word Attack
0
20
40
60
80
100
Fall 2009Spring 2010Spring 2011
Black Students Fall 2009 & Spring 2010 (n = 99) Spring 2011 (n = 42)
Letter W
ord Identifica
tion
Reading Fluency
Passage Comprehensio
n
Word Attack
0
20
40
60
80
100
Fall 2009Spring 2010Spring 2011
Hispanic Students Fall 2009 & Spring 2010 (n = 32) Spring 2011 (n = 14)
Letter W
ord Identifica
tion
Reading Fluency
Passage Comprehensio
n
Word Attack
0
20
40
60
80
100
Fall 2009Spring 2010Spring 2011
Students Identified to Receive Exceptional Children’s Services Fall 2009 & Spring 2010 (n = 109) Spring 2011 (n = 68)
Letter W
ord Identifica
tion
Reading Fluency
Passage Comprehensio
n
Word Attack
0
20
40
60
80
100
Fall 2009Spring 2010Spring 2011
Students Not Identified to Receive Exceptional Children’s Services Fall 2009 & Spring 2010 (n = 216) Spring 2011 (n = 96)
Letter W
ord Identifica
tion
Reading Fluency
Passage Comprehensio
n
Word Attack
0
20
40
60
80
100
Fall 2009Spring 2010Spring 2011
North Carolina End-Of-Grade Reading Comprehension Test
Results
NC End-Of-Grade Reading Achievement Levels for Students in First Year Sample
2009 (n = 244) 2010 (n = 302)
0
0
0% 20% 40% 60% 80% 100%
62.7%
31.1%
32.8%
44.4%
4.1%
22.9%
0.4%
1.7%
#REF! #REF! #REF! #REF!
NC End-Of-Grade Reading Achievement Levels for Students in Two Year Longitudinal Sample2009 (n = 117) 2010 (n = 158), 2011 (n = 154)
2009
2010
2011
0% 20% 40% 60% 80% 100%
70.1%
32.9%
26.6%
24.8%
43.7%
41.6%
4.3%
22.2%
28.6%
0.9%
1.3%
3.2%
Level I Level II Level III Level IV
NC End-Of-Grade Reading Achievement Levels for White Students in Two Year Longitudinal Sample
(n = 58)
2009
2010
2011
0% 20% 40% 60% 80% 100%
70.7%
27.6%
27.6%
20.7%
43.1%
41.4%
6.90%
25.86%
27.59%
1.72%
3.45%
3.45%
Level I Level II Level III Level IV
NC End-Of-Grade Reading Achievement Levels for Black Students in Two Year Longitudinal Sample
(n = 32)
2009
2010
2011
0% 20% 40% 60% 80% 100%
65.6%
31.3%
28.1%
31.3%
56.3%
40.6%
3.13%
12.50%
28.13%
Level I Level II Level III Level IV
NC End-Of-Grade Reading Achievement Levels for Students Identified to Receive EC Services in Two
Year Longitudinal Sample (n = 44)
2009
2010
2011
0% 20% 40% 60% 80% 100%
70.5%
31.8%
22.7%
18.2%
47.7%
36.4%
9.09%
15.91%
34.09%
2.27%
4.55%
6.82%
Level I Level II Level III Level IV
NC End-Of-Grade Reading Achievement Levels for Students Not Identified to Receive EC Services in
Two Year Longitudinal Sample (n = 69)
2009
2010
2011
0% 20% 40% 60% 80% 100%
69.6%
26.1%
33.3%
29.0%
49.3%
44.9%
1.45%
24.64%
20.29% 1%
Level I Level II Level III Level IV
ConclusionAnnual and longitudinal results provide support for the effectiveness of HillRAP in improving student academic achievement in reading.
Overall, students demonstrated greater than expected growth on reading subtests of the Woodcock-Johnson III Tests of Achievement; NC End-of-Grade Reading Comprehension Test results showed a similar pattern of improved student reading achievement for HillRAP Students.
Related PresentationsWalser, T. M., Parker, M. A., Grace, E. R., Bell, C. A., Dowcett, E. T., & Tart, M. E. (2012, April). The Hill Reading Achievement Program model replication: Results and implications. To be presented at the American Educational Research Association annual conference, Vancouver, BC.
Walser, T. M., Parker, M. A., Grace, E. R., Dowcett, E. T., & Hodges, D. M. (2012, April). The value of replication: Learning about an educational model and evaluation practice. To be presented at the American Educational Research Association annual conference, Vancouver, BC.
Walser, T. M., & Parker, M. A. (2012, April). Hill Reading Achievement Program evaluation project: Results and reflections. To be presented at the Watson School of Education Scholarship Brown Bag Series, Wilmington, NC.
Walser, T. M., Grace, E. R., Bell, C. A., Parker, M. A., & Tart, M. E. (2012, February). Hill Reading Achievement Program Studies: What We Learned about Impacts on Student Achievement, Program Implementation, and Conducting Research in Schools. Presented at the North Carolina Association for Research in Education annual conference, Winston-Salem, NC.
Related PresentationsParker, M. A., Walser, T. M., Tart, M. E., & Bell, C. A. (2012, February). Methodological questions about aggregating results from Hill Reading Achievement studies across North Carolina. Presented at the Eastern Educational Research Association annual conference, Hilton Head Island, SC.
Walser, T. M., & Parker, M. A. (2011, November). How to train your dragon: One story of using a quasi-experimental design element in a school-based evaluation study. Paper presented at the American Evaluation Association annual conference, Anaheim, CA.
Walser, T. M., Parker, M. A., Grace, E. R., & Hodges, D. M. (2011, November). The role of replication in evaluating complex systems in education. Paper presented at the American Evaluation Association annual conference, Anaheim, CA.
Walser, T. M., Hodges, D. M., Wetherill, K., Grace, E. R. (2010, October). The role of program evaluation in replicating an educational model. Presented at the Consortium for Research on Educational Accountability and Teacher Evaluation annual conference, Williamsburg, VA.
Walser, T. M. (2009, November). Context and quasi-experiments in schools: The case for cohort comparison groups. Paper presented at the American Evaluation Association annual conference, Orlando, FL.
Thank You!
