Embed Size (px)
Citation preview
The Impact of Literacy Coaching on Teachers’ Value-Added to Student Learning in
Literacy CollaborativeGina Biancarosa, University of Oregon
Anthony S. Bryk, The Carnegie Foundation for the Advancement of Teaching
Allison Atteberry, Stanford UniversityHeather Hough, Stanford University
Institute of Education SciencesAnnual Conference
June 2010
Key Features of Literacy Collaborative• Comprehensive school reform program designed to
improve elementary children’s reading, writing, and language skills primarily through school-based coaching
• Used in over 700 elementary schools in 200 districts across 26 states
• Intensive professional development of coaches (selected from school faculty)– Trained over one year (Lesley University and the Ohio State
University)– Ongoing support from local and national network
• Coach’s role and duties– Half-time teaching, half-time coaching– In-school professional development courses– One-on-one coaching sessions
Main Research Questions
• Does Literacy Collaborative improve the value-added to student literacy learning?
• Can Literacy Collaborative effects be attributed to coaching, either directly or indirectly?
Student Data• Value-added analyses focused on grades exposed to LC
professional development (K-2)• Sample: 8576 children, 341 teachers, and 17 coaches in
17 public schools across 8 states in the Eastern U.S. • Children tested in fall and spring for 4 years to measure
change over time in students’ literacy learning using:– Dynamic Indicators of Basic Early Literacy Skills (DIBELS)– Terra Nova in spring
Low Income 46.0%
Race/Ethnicity African-American
LatinoOtherWhite
15.5%5.8%7.2%70.6%
Limited English Proficiency 4.0%
Accelerated Longitudinal Cohort Design6 cohorts studied over 4 years
Year of Study
First Year Second Year Third Year Fourth Year
Fall Spring Fall Spring Fall Spring Fall Spring
K C C D D E E F F
1 B B C C D D E E
2 A A B B C C D DGra
de
Training yearYear 1 of
implementationYear 2 of
implementationYear 3 of
implementation
Our early literacy scale
• Equal differences on scale imply equal differences on the trait measured at any level
• Reported in logits (which describe the probability of a student with a given ability level getting a particular item right or wrong)
• But what do they mean given the particular assessments used?
1
2
3
4
Mean at K entry Names about 30 letters in a minuteVery low phonemic awareness (PA)
Mean at K end & 1st grade entryAccurate and fast letter recognitionGood initial sound PALittle evidence of decoding
Mean at 1st grade end & 2nd grade entryAccurate (not fast) PAReads 50-60 wpmAnswers 1/3 of 1st grade comprehension questions correctly
Mean at 2nd grade endMastery of component skillsReads 90 wpmAnswers 2/3 of 1st grade comprehension questions correctly, 1/3 of 2nd grade questions correctly
Value-added Hierarchical Cross-classified Effects Modeling
• Four Levels – time : (students x teachers) : school
– Repeated measures on students (level 1) – Students (level 2) who cross Teachers (level 3) over time – All nested within Schools (level 4)
• The analysis model can be conceptualized as a joining of 2 separate multi-level models
– One two-level model for individual growth in achievement over time, and
– A second two-level model which represents the value-added that each teacher in a school contributes to student learning in that school in a particular year.
Value-added effects by year (prior to adding coaching as predictor)
Year 1 Year 2 Year 3
Average value-added (overall)
.164 .280 .327
Performance improvement
16% 28% 32%
Effect size .22 .37 .43
School 95% plausible value-added range ±.23 ±.28 ±.37
Teacher 95% plausible value-added range ±.51 ±.71 ±.91
Average student learning growth in an academic year (1.02 logits)
Explaining variability in value-added effects
• Tested models with cumulative number of coaching sessions per year (derived from coach logs)– Per teacher– Averaged across teachers at school-level
• Also tested a variety of controls thought to influence teachers’ openness to, participation in, and selection for coaching– Prior use of reform literacy practices– Role conception– School commitment– New to school
Summary of findings
• Coaching at the teacher level significant• Coaching at the school level not significant• Teacher expertise of implementation not
significant• Only one teacher characteristic significant
(role conception), but only in one year
Conditional value-added effects
Year 1 Year 2 Year 3
Average value-added for teacher receiving NO coaching
0.26*** 0.17* 0.14ns
Role conception -.01ns .04* .01ns
Teacher expertise 0.02ns -0.03ns 0.03ns
Value-added per coaching session (cumulative)
-.026* .012* .012*
Comparing Coaching Value-added to Unconditional Mean Value-Added
Year 1 Year 2 Year 3
Value-added per coaching session (cumulative)
-.026 .012 .012
Effect size per session -0.03 0.02 0.02
Mean cumulative coaching sessions
2.60 8.96 15.70
Mean coaching value-added
-0.07 0.09 0.19
Unconditional mean value-added
.164 .280 .327
Proportion accounted for by coaching
NA 0.32 0.57
0
10
20
30
40
Cu
mu
lativ
e N
umb
er
of
Se
ssio
ns
pe
r T
ea
che
r
JanY2FebY2
MarY2AprY2
MaJuY2AuSeY3
OctY3NovY3
DecY3JanY3
FebY3MarY3
AprY3MaJuY3
AuSeY4OctY4
NovY4DecY4
JanY4FebY4
MarY4AprY4
MaJuY4
Month of the Study
*Note: Each line represents 1 of the 18 schools in the study
Across the Eighteen Schools, Over TimeAverage Number of Coaching Sessions Accumulated Per Teacher
Across Seventeen Schools, Over Time
17
Variability in Coaching between Schools
Variability of Coaching between and within Schools: A Tale of Two Schools
School 10: Riverside• Staff size = 14• Value-added:
Y1 Y2 Y30.07 0.22 0.25
• Starts below average and improves
• Variability between teachers decreases from Y1 to Y3
School 16: Tyson William• Staff size = 14• Value-added:
Y1 Y2 Y30.17 0.13 0.01
• Starts at average and deteriorates
• Variability between teachers increases from Y1 to Y3
How can we make sense of what happened in these two schools?
School 10: Coaching Sessions Accumulated per Teacher
0
5
10
15
20
25
30
35
40
45
Cum
ulat
ive
Num
ber
of C
oac
hin
g S
ess
ions
Rec
eive
dby
mo
nth
i fo
r te
ach
er j
in s
choo
l k
JanY
2
FebY2
Mar
Y2
AprY2
MaJ
uY2
AuSeY
3
OctY3
NovY3
DecY3
JanY
3
FebY3
Mar
Y3
AprY3
MaJ
uY3
AuSeY
4
OctY4
NovY4
DecY4
JanY
4
FebY4
Mar
Y4
AprY4
MaJ
uY4
23 Months of the Study
Riverside Elementary
School 16: Coaching Sessions Accumulated per Teacher
0
5
10
15
20
25
30
35
40
45
Cum
ulat
ive
Num
ber
of C
oac
hin
g S
ess
ions
Rec
eive
dby
mo
nth
i fo
r te
ach
er j
in s
choo
l k
JanY
2
FebY2
Mar
Y2
AprY2
MaJ
uY2
AuSeY
3
OctY3
NovY3
DecY3
JanY
3
FebY3
Mar
Y3
AprY3
MaJ
uY3
AuSeY
4
OctY4
NovY4
DecY4
JanY
4
FebY4
Mar
Y4
AprY4
MaJ
uY4
23 Months of the Study
Tyson William
Network Analysis• Conducted by Allison Atteberry & Tony Bryk• Pre- and post-teacher surveys asked who
talked to about instruction and student problems in literacy (up to 7) and how often
Floyd (0) Tanya (na)Fiona (9.8)
Claire (na)
Sarah (3.5)
Sonia (only Y1)
Tabitha (na)
Suzy (12.1)
Kerri (13.3)
Tessa (na)
Kaleb (12.9)
Ford (9.8)
Stacy (8.6)
Kegan (12.5)
Trudy (na)
Ken (10.2)
Felicity (0)Kyle (11)
Faith (14.1)
Shape color = grade levelShape size = PD “dosage”
Lines and arrows = reported tiesArrow weight = reported frequency of consultation
Black box = coach
Floyd (0) Tanya (na)Fiona (9.8)
Claire (na)
Sarah (3.5)
Sonia (only Y1)
Tabitha (na)
Suzy (12.1)
Kerri (13.3)
Tessa (na)
Kaleb (12.9)
Ford (9.8)
Stacy (8.6)
Kegan (12.5)
Trudy (na)
Ken (10.2)
Felicity (0)Kyle (11)
Faith (14.1)
School 10Riverside 2005
Grade Taught
K
K/1
1st
1/2
2nd
2/3
3rd
LC
Sophia (2)
Claire (na)
Stella (6.3)
Fern (3.5)
Katya (1.3)
Tessa (na)
Floyd (only Y1)
Simon (2.7)Sonia (1.9)
Trudy (na)
Karen (2)
Frieda (3.5)
Kristin (1.6)
Twyla (na)
Fiona (2)
Finn (2)
Kerri (7.8)
Fred (only Y1)
Tyler (na)
School 16Tyson William Elem 2005
Grade Taught
K
K/1
1st
1/2
2nd
2/3
3rd
LC
Frieda (14)Fiona (9.8)
Claire (na)
Finn (4.7)
Sarah (3.5)
Tabitha (na) Suzy (12.1)
Kerri (13.3)
Tessa (na)
Kaleb (12.9)
Ford (9.8)
Stacy (8.6)
Kegan (12.5)
Trudy (na)
Ken (10.2)
Kyle (11)
Faith (14.1)Twyla (na)
Sandra (13)
Grade Taught
K
K/1
1st
1/2
2nd
2/3
3rd
LC
School 10Riverside 2008
Sophia (2)
Claire (na)
Stella (6.3)
Fern (3.5)
Tessa (na)
Simon (2.7)
Trudy (na)
Karen (2)
Frieda (3.5)
Kristin (1.6)
Twyla (na)
Fiona (2)
Finn (2)
Kerri (7.8)
Seth (4.7)
Kaleb (9)
School 16Tyson William Elem 2008
Grade Taught
K
K/1
1st
1/2
2nd
2/3
3rd
LC
Summary of findings
• Evidence that the mechanism for improved value-added shifts from over time– Year 1: Coaching has no value-added– Year 2: Coaching begins to add to value-added for
student learning– Year 3: Coaching becomes the primary mechanism for
value-added to student learning• Cumulative coaching explains differences in
teacher value-added effects, but not school effects
Implications
• Coaching largely mediates teachers’ value-added to student learning
• Not in Year 1, but in Year 2 and especially 3– Selection effects?– Dosage effects?– Developmental/expertise effect for teachers?– Developmental/expertise effect for coaches?
Future Steps
• Exploration of the role of coach – Expertise and its development– Coach decision-making, especially what influences
whom coaches target
• Exploration of the influence of school context– Teacher influence in school decisions– Principal leadership and supportiveness– Trust
Limitations
• Limited sample, especially at school level, limits ability to explore contextual mechanisms
• Professional development for coaches is more intense than in most other models
• Coaching embedded in a school-wide reform model that included– Professional development courses– Detailed literacy instruction framework
Variability in school value-added, year 1Average student gain per academic year
No effect
Year 1 mean effect (.16)
High value-added schools
Low value-added schools
Variability in school value-added, year 2Average student gain per academic year
No effect
Year 1 mean effect (.16)
Year 2 mean effect (.28)
Variability in school value-added, year 3Average student gain per academic year
No effect
Year 1 mean effect (.16)
Year 2 mean effect (.28)Year 3 mean effect (.33)
Variability in teacher value-added within 2 schools
Average student gain per academic year
No effect
Year 1 mean effect (.16)
Year 2 mean effect (.28)
Year 3 mean effect (.33)
