1 Los Angeles Unified School District Program Evaluation and Research Branch January 9, 2009...

1

Los Angeles Unified School District

Program Evaluation and Research Branch

January 9, 2009

Evaluation of ELD PracticumYear 1 Report

2006-07

Gojko Vuckovic, Ph.D., Kathy Hayes, Ph.D., & Jesus Salazar, Ph.D.

2

This report examines implementation of the ELD program in 3rd and 4th grade classrooms looking at:

Regular Into English! lessons

Enhanced Into English! Task Based Language Teaching (TBLT) lessons

Introduction

3

Sample

25 randomly selected schools

School Characteristic Index (SCI) 100 randomly selected classrooms

Two Grade 3 classrooms per school

Two Grade 4 classrooms per school

4

Data collection

Two classroom observations, once per semester

If ELD was taught, two additional consecutive days were observed

Open-ended teacher interviews and administrator surveys

5

Data gathering strategies used in observations

Fieldnotes 30-minute Timeline of Classroom

Interaction Structures Observation Summary Statement Observation Summary Scale Interview Protocols Administrator Survey

6

Data reduction and analysis

Application of a coding scheme to field notes and teacher interviews

The coded data were analyzed for descriptive statistics, frequencies, and crosstabs

7

The evaluation objective # 1

To document:

The capacity of the schools to support ELD Program implementation in 3rd and 4th grade classrooms serving English learners.

8

The evaluation objective # 2

To document:

The extent and nature of ELD Program

implementation in 3rd and 4th grade classrooms serving English learners.

9

The evaluation objective # 3

To document:

The effectiveness of ELD instruction as measured by student progress on CELDT and ELA CST.

10

Capacity – Professional development

83% of teachers completed the ELD Practicum.

Of remaining 17%, 10% had ELD 5 day Institute completed, 3% were trained by EL Coordinator, and 4% had no ELD training.

Teachers found demonstration lessons to be the most beneficial aspect of the professional development.

11

Capacity – ELD materials

82% of teachers had all of the ELD materials.

18% of teachers were missing some materials such as ballparking forms, Language Logs, lesson pictures, picture cards, CDs, and student books.

74% of teachers used supplemental materials, such as realia, visuals, literature, and the internet.

12

Implementation criteria

Number of lessons taught Number of objectives met Materials used ELD strategies used Students on task (engagement) Time spent on ELD

13

Specific time ranges dedicated to ELD instruction

Average minutes per day

Classrooms

30 to 45 60

More than 45 19

Less than 30 16

No ELD taught 5

14

Time by program and level

ELD Lesson TypeTime in minutes

Implemented Not Implemented

Enhanced 39 23

Regular 35 33

Regular and Enhanced 37 n.a.

Special Cases 44 38

Average 39 26

15

3rd Grade implementation

Semester ImplementedNot

ImplementedTotals

First 43 7 50

Second 37 13 50

16

4th Grade implementation

Semester Implemented Not

ImplementedTotals

First 37 13 50

Second 40 10 50

17

General factors contributing to no or very low implementation – Second semester (23 classes)

School site issue 17

Substitute teacher (2 also school site issue)

6

Professional development (2 also school site issue)

4

18

Specific factors contributing to no or very low implementation –

Second semester

No ELD instruction of any kind was observed

2

No ELD Program materials were used 4

Teacher cited timing issues 4

Only one lesson taught over 3 days 5

Grade level or student population issues 2

Other 6

19

ELD implementation by ELD program observed - Semester 1

ELD Program Implemented Not Implemented

N % N %

Enhanced 37 86% 6 14%

Regular 21 81% 5 19%

Enhanced & Regular

9 82% 2 18%

Special Cases 13 81% 3 19%

No ELD Program 0 0% 4 100%

Total 80 80% 20 20%

20

ELD implementation by ELD program observed - Semester 2

ELD ProgramImplemented Not Implemented

N % N %

Enhanced 48 94% 3 6%

Regular 8 57% 6 43%

Enhanced & Regular

11 100% 0 0%

Special Cases 10 53% 9 47%

No ELD Program 0 0% 5 100%

Total 77 77% 23 23%

21

Definitions of “Special cases”

Implemented Regular lessons taught with TBLT framework

Not Implemented Regular lessons taught with TBLT framework Made up lessons taught with TBLT framework Teacher self-created lesson with no connection

to the IE! books

22

Use of strategies Enhanced Into English!

ELD Strategies

Semester 1 (43 classes)

Semester 2 (51 classes)

N % N %

Backward Buildup 0 0% 1 2%

Ballparking 12 28% 8 16%

Corrective Recasting 19 44% 10 20%

Pull Out and Talk/Write 38 88% 45 88%

RASP 6 14% 2 4%

Think-Pair-Share 41 95% 48 94%

Thinking Maps 42 98% 50 98%

Vocalized Reading 33 77% 35 69%

23

Use of strategies Regular Into English!

ELD StrategiesSemester 1 (26 classes) Semester 2 (14 classes)

N % N %

Audio 14 54% 7 50%

Chant 11 42% 4 29%

Choral Reading 15 58% 6 43%

Collaborative Reading 1 4% 1 7%

Demonstrating 14 54% 12 86%

Echo Reading 4 15% 1 7%

Games 2 8% 1 7%

Graphs/Charts 16 62% 8 57%

Prior Knowledge 11 42% 7 50%

Realia 9 35% 2 14%

Think-Pair-Share 17 65% 7 50%

TPR 6 23% 2 14%

Visual/Illustrations 23 88% 13 93%

24

Effectiveness of ELD instructionCELDT

Grade Implemented Not Implemented

Gain N Gain N

3rd grade .03 361 -.19 112

4th grade .25 483 -.02 124

25

Effectiveness of ELD instructionELA CST

Grade Implemented Not Implemented

Gain N Gain N

3rd grade .00 379 -.32 118

4th grade -.06 505 -.35 118

26

Effect sizes

Grade ELA CSTImpl. vs Not Impl.

CELDTImpl. vs Not Impl.

3rd grade .32 .22

4th grade .29 .27

27

10 classes in which ELD levels 4 & 5 significantly outperformed lover

ELD levels

Differentiation

Classroom

# 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10

Suggested number 8 1 6 7 6 3 4 6 6 4

Completed number 3 1 1 0 0 0 1 0 0 1

28

10 classes - Differentiation practices not or rarely used

Questioning techniques based on ELD level Mapping Discussion prompts Activity options by level Different grouping for reading Generating different student responses by

facilitating simple, elaborate and complex sentences (some observed)

29

10 classes – Quality of teaching and learning analysis

Learning environment typically not student centered

No extra oral opportunities when needed Vocabulary needed for better understanding not

covered No assessment practices Rarely accessed prior knowledge No evidence of listening to students to inform

teaching

30

15 high performing classrooms

Classroom and implementation level (I*, NI**)

Grade

ELD Levels

Num. of ELs

CST ELA

Gains

CELDTGains

Classroom # 1(I) 3 1,2,3,4,5 17 .66 .31

Classroom # 2(I) 3 1,2,3,4,5 11 .37 .44

Classroom # 3(I) 4 1,2,3,4,5 17 .38 .76

Classroom # 4(I) 4 1,2,3,4,5 23 .56 .09

Classroom # 5(I) 3 3,4,5 6 .42 .25

Classroom # 6(I) 3 1,2,3,4,5 12 .65 .23

Classroom # 7(I) 3 3,4,5 6 .50 .70

Classroom # 8(I) 4 3,4,5 12 .61 .17

Classroom # 9(I) 4 1,2,3,4,5 13 .60 .28

Classroom # 10(NI) 4 1,2,3,4,5 25 .30 .41

Classroom # 11(NI) 3 1,2,3 11 .53 .33

Classroom # 12(I) 3 4,5 5 1.40 .26

Classroom #13(I) 4 4,5 16 .44 .71

Classroom # 14(I) 4 4,5 13 .85 .27

Classroom # 15(I) 3 4,5 4 .48 .39

31

15 classes - Quality of teaching and learning analysis

Lessons student centered Topics relevant to the students’ lives, needs and

interests Pacing plan adapted to meet the needs of the

students Extra oral opportunities provided Vocabulary in context taught Prior knowledge accessed Evidence of listening to the students

32

Recommendations

Implementation of the ELD program had a positive effect on student performance

LD superintendants and directors of school services should plan for the professional development of school administrators and monitor the capacity of schools to implement and support the ELD program.

33

Recommendations (continued)

Principals and assistant principals in partnership with school site leadership bodies should develop detailed plans for the implementation of the ELD program, provide necessary support, and closely monitor program implementation in each classroom with English learners.

They must pay regular visits to the classrooms, observe instructional practices, and provide directions.

34

Recommendations (continued)

Principals and assistant principals should take responsibility for their own personal development and training in ELD programs and practices, as well as the PD of school staff and teachers.

The Language Acquisition Branch should find ways to strengthen the professional development design, curriculum, and the way the program is implemented in order to address the needs of students at lower ELD levels.

35

Recommendations (continued)

The Local Districts in partnership with the Language Acquisition Branch should continue providing PD opportunities for teachers and administrators.

The Language Acquisition Branch should provide additional training in core differentiation practices.

Principals, local district superintendents and directors of school services should take a lead in creating conditions to ensure that there are no more then two consecutive ELD levels per class.

36

Recommendations (continued)

Additional steps should be made by the Division of Professional Development and Leadership and the Language Acquisition Branch to incorporate high quality teaching strategies across all core content professional developments in our District.

www.lausd.net/lausd/offices/perb