Assistive Technology Capacity Building Initiative - Endline Report -

Reading for Ethiopia’s Achievement Developed Technical Assistance (READ TA)

Cooperative Agreement No.: AID-663-A-12-00013

October 2017

DISCLAIMER This document was produced with support from the U.S. Agency for International Development through the USAID/Ethiopia READ TA project under Cooperative Agreement No. AID-663-A-12-00013.

This publication was produced for review by the United States Agency for International Development (USAID) and was prepared by RTI International. The authors’ views expressed in this publication do not necessarily reflect the views of USAID or the United States Government.

iii

Table of Contents

List of Exhibits ............................................................................................................................... ivAcknowledgements ......................................................................................................................... vAcronyms and Abbreviations ........................................................................................................ vi1 Introduction ......................................................................................................................... 12 Goals and Objectives .......................................................................................................... 23 Methodology ....................................................................................................................... 3

3.1 Study Participants ............................................................................................................. 33.2 Final Study Sample .......................................................................................................... 5

3.3 Measures ........................................................................................................................... 5

3.4 Data Collection Methods and Analysis .......................................................................... 10

4 Baseline Findings .............................................................................................................. 114.1 Prevalence of Vision and Hearing Impairment .............................................................. 11

4.2 Teachers’ Barriers to the Implementation of Inclusive Education ................................. 12

4.3 Teachers’ Attitudes and Self-Efficacy in Inclusive Education at Baseline .................... 14

4.4 Teachers’ Inclusive Practices in the Classroom at Baseline .......................................... 15

4.5 Students’ Experience in Inclusive Classrooms and their Reading Skills at Baseline .... 17

5 Endline Findings and Outcomes of the Assistive Technology Intervention ..................... 185.1 Teachers’ Adoption of the Assistive Technology .......................................................... 18

5.2 Changes in Teachers’ Attitudes and Self-Efficacy in Inclusive Education ................... 19

5.3 Changes in Teachers’ Inclusive Practices in the Classroom .......................................... 22

5.4 Changes in Students’ Reading Skills ............................................................................. 23

6 Proof of Concept Study Limitations ................................................................................. 267 Conclusions and Recommendations ................................................................................. 26Bibliography ................................................................................................................................. 29

iv

List of Exhibits Exhibit 1. PeekVision Visual Categories and LogMAR Scores ..................................................... 7Exhibit 2: HearScreen Hearing Categories and Related Loss in Decibels ..................................... 7Exhibit 3: Number and Percentage of Students Identified by Visual Category ........................... 12Exhibit 4: Number and Percentage of Students Identified by Hearing Category ......................... 12Exhibit 5: Barriers to Inclusive Education and Percentage of Teachers Who Agreed or Strongly Agreed to Each (n = 109) .............................................................................................................. 13Exhibit 6: Teacher Responses to Individual Attitude and Self-Efficacy Statements (n = 109) .... 15Exhibit 7: Teachers Inclusive Classroom Practices at Baseline and Endline (n = 375)* ............. 16Exhibit 8: Teacher Attitude and Self-efficacy to Inclusive Education at Baseline and Endline (n = 101) ............................................................................................................................................... 20Exhibit 9: Percent of Teachers Agreeing or Strongly Agreeing with Attitude and Self-Efficacy Statements at Baseline and Endline (n = 101) .............................................................................. 21Exhibit 10: Teachers Inclusive Classroom Practices at Baseline and Endline (n = 325) ............. 22Exhibit 11: Students’ Zero Scores at Baseline and Endline (n = 624) ......................................... 24Exhibit 12: Students’ Zero Scores at Baseline and Endline (n = 624) ......................................... 25

v

Acknowledgements It is a pleasure to acknowledge the many people who contributed to the implementation of the Assistive Technology Capacity Building Initiative (ATCBI) and the development of this report.

Most READ TA staff were involved with ATCBI due to the large scope of this activity and, thus, deserve thanks for their contribution to this effort. Special gratitude is due to the READ TA Gender and Inclusive Education Team and Monitoring and Evaluation (M&E) Team whose strong support included providing advice, collecting data, and training. The teams were composed of the following individuals:

• Ali Ahmed Abdulahi, Gender & Inclusive Education Specialist (Ethio-Somali) • Jemal Abdulkedir, Reading and Curriculum Specialist (Oromia) • Beshir Sheik Abdurahman, Regional Manager (Ethio-Somali) • Getachew Assefa, Capacity Building and Policy Specialist (Oromia) • Meskerem Abera Ayele, Gender & Inclusive Education Specialist (SNNPR) • Habtamu Mekonnen Balderas, READ TA Gender & Inclusive Education Advisor • Desta Daniel, Reading and Curriculum Specialist (SNNPR) • Fasika Tibebu Dessie, Gender & Inclusive Education Specialist (Amhara) • Asmelash Baraki Gebreyohans, Gender & Inclusive Education Specialist (Tigray) • Hailu Melesse Hailu, Sr. M&E Advisor • Nuria Ibrahim Jado, Gender & Inclusive Education Specialist (Oromia) • Deneke Lefebo, Reading and Curriculum Specialist (SNNPR) • Abshir Mohammed, Capacity Building and Policy Specialist (Ethio-Somali) • Labena Tona, Capacity Building and Policy Specialist (SNNPR) • Tewodros Wendmneh Worku, M&E Director

In addition, Carmen Strigel, RTI International’s Director for Technology for Education and Training, designed and monitored the overall initiative and wrote this report; Jennifer Pressley lead RTI’s data analysis; and RTI consultant, Wykia Macon, provided advice and support in the implementation of the activity in Ethiopia, as well as in review of all its documentation and reports.

Most importantly, READ TA would like to thank the participating students, teachers, school leadership, ATCBI working group members, RSEB, and Ministry of Education staff for their invaluable advice, collaboration, active engagement, dedication, and commitment to this groundbreaking effort.

READ TA would also like to acknowledge the critical contribution and support by the Ethiopia National Association of the Blind, Ethiopia National Association of the Deaf, and the Federation of Ethiopian National Associations of Persons with Disabilities.

vi

Acronyms and Abbreviations ATCBI Assistive Technology Capacity Building Initiative CLSPM correct letter sounds per minute CWPM correct words per minute CTE College of Teacher Education dB decibels HI hearing impairment IMLP inclusive multimedia lesson plan IRB Institutional Review Board MOE Ministry of Education MT mother tongue READ TA Reading for Achievement of Ethiopia’s Development Technical

Assistance RSEB Regional State Education Bureau SNNPR Southern Nations, Nationalities, and Peoples Region UNESCO United Nations Educational, Scientific and Cultural Organization USAID United States Agency for International Development VI visual impairment WHO World Health Organization

1

1 Introduction The Reading for Ethiopia’s Achievement Developed Technical Assistance’s (READ TA) project is a 5-year (2012–2017) initiative funded by the United States Agency for International Development (USAID) and implemented by RTI International (RTI) and its partners. READ TA aims to improve reading and writing for 15 million Ethiopian children in seven mother tongues (MTs) and English. To achieve this objective, the project built the capacity of local actors through its revision of the national curriculum for MT reading and writing instruction, development of new teacher guides and student textbooks, and training of teacher trainers in the implementation of the new curriculum. READ TA also revised the relevant pre-service teacher education courses.

READ TA’s mandate also included the consideration of modest technologies and supplementary teaching aids in direct support of the revised curriculum for reading and writing in MT at Colleges of Teacher Education (CTEs) and School Cluster Centers. The key to this consideration is relevance, cost-effectiveness, and practicability of such resources in the Ethiopian context and their specific application to enhance programmatic objectives of the project. With this in mind, READ TA has worked closely with the Ministry of Education (MOE), Regional State Education Bureaus (RSEBs), and CTEs to make decisions about the most feasible and sustainable technology interventions. READ TA’s approach is aimed at building the capacity of Ethiopian institutions to determine which technologies

• are locally relevant (i.e., for a particular region or CTE), • are potentially useable in the existing institutional environments, • offer the best potential instructional value-added, and • have reasonable costs and can be implemented cost-effectively.

A cross-cutting objective of the project was inclusive education1 and especially the consideration of children with disabilities2 in regular classrooms. This objective is in alignment with Ethiopia’s inclusive education policy and strategy (MOE, 2012) and new master plan for inclusive education (MOE, 2016a). Despite these existing inclusive education policies and strategies, children with disabilities in Ethiopia are under-represented in the primary school population. For the 2014/2015 school year, the Government of Ethiopia reported 71,001 (about 0.38% of the gross enrollment number) children with disabilities enrolled in its primary schools (MOE, 2016b).

In addition to the enrollment cited above, one should consider that the most recent data from 21,572 households, just in Kersa District of the Oromia region, indicated that 2.6% of children aged 0–14 years live with a known disability (Geda, Berhane, Assefa, & Worku, 2016). Further, a nationally representative study of 30,022 households conducted in Ethiopia in 2005 found that among the 1 For the purpose of this report, “inclusive education” is understood as an approach that “seeks to address the learning needs of all children, youth, and adults with a specific focus on those who are vulnerable to marginalization and exclusion” (United Nations Educational, Scientific and Cultural Organization [UNESCO], 2003, p. 4), with an emphasis on children with disabilities. Practically, inclusive education is understood as the integration of students with disabilities into regular classrooms for 80% or more of the school day. 2 For the purpose of this report, “disabilities” are defined as “long-term physical, mental, intellectual or sensory impairments which in interaction with various barriers may hinder [individuals’] full and effective participation in society on an equal basis with others” (United Nations, 2006, p.4).

2

entire sample of children and adults in the study, the prevalence rate for low vision was 3.7%, while blindness was found in 1.6% of the sample—with significant regional variations (Berhane et al., 2007). Rather than assessing the prevalence of known disabilities, the Berhane et al. data is based on a test of visual acuity administered by ophthalmologists and ophthalmologic nurses. These data indicate significant gaps in the enrollment of children with disabilities in Ethiopia’s primary schools. For the East African region, notable factors of the low school enrollment of children with disabilities include poverty (Mitra, Posarac, & Vick, 2013), limitations in the operationalization of national policies (Polat, 2011), traditional sociocultural barriers and the stigma attached to disabilities (Stone-MacDonald, 2012), gaps in teacher professional development (Ojok & Wormnaes, 2013), and lack of specialized materials and resources (Hofman & Kilimo, 2014). Empirical evidence specifically exploring reading achievement of children with disabilities in Ethiopia has not been established to date.

RTI’s READ TA team designed an assistive technology capacity building initiative (ATCBI) to target READ TA's objective of improving reading and writing for all children in mainstream Ethiopian classrooms, while informing the potential drivers to the problem of low school enrollment of children with disabilities at the classroom level. The drivers that are within the scope of READ TA’s mandate specifically include the gaps in teacher professional development and sociocultural barriers to inclusive education of children with disabilities in the country. Given READ TA’s technical assistance mandate, the ATCBI also builds the capacity of the participating RSEBs, disabled persons organizations, and CTE staff (organized into regional working groups) in identifying, implementing, and monitoring the initiative.

2 Goals and Objectives

A comprehensive literature review conducted in 2016 highlighted a dearth of empirical evidence on inclusive education in Ethiopia among peer-reviewed articles published in English-language journals. As a first of its kind in Ethiopia, the ATCBI functions foremost as a proof of concept to explore the following research questions:

• What is the prevalence of a potential vision impairment (VI) and hearing impairment (HI) in participating classrooms?

• What are barriers for teachers in the implementation of inclusive education policies? • To what degree would public primary school teachers adopt technology, specifically

locally available smartphones, for instruction in inclusive classrooms? • To what degree do provision of phone-based pedagogical support materials and training

promote adoption of inclusive instructional practices among those teachers? • To what extent do such technology and practices help remove barriers of teacher attitude

and self-efficacy in teaching students who are hard of hearing or have low vision?

Given the experimental nature of the initiative, consideration of recent empirical evidence on the prevalence of disabilities in Ethiopia (Geda et al., 2016), and in consultation with regional working groups, READ TA focused the initiative on children who are hard of hearing or who have low vision. The ATCBI does not focus on other disabilities, or children who are fully blind or deaf. It

3

is also important to stress that the initiative was a screening effort only and did not constitute, or replace, a medical diagnosis. Referral slips were given to school principals for each child identified for a potential vision impairment or hearing impairment, to inform parents of the potential condition of their child and encourage appropriate medical follow-up.

The ATCBI consisted of the following components: • Baseline data collection (December 2016/January 2017). • Two days of initial teacher training (February 2017) and two days of refresher training

(April 2017). • One smartphone device and headset for each participating teacher. The smartphone was

equipped with digital versions of 16 weeks of scripted lesson plans for MT reading instruction with explicit accommodation instructions for students who were hard of hearing or had low vision; embedded audio files allowing play-back of phonemic awareness and read-aloud elements in the lessons; and clinically validated vision and hearing screening tools.

• Activity implementation with monthly monitoring visits by the working group to each participating classroom (February–May 2017).

• Endline data collection (May/June 2017). • Data analysis, reporting, and sharing (June–September 2017).

This report documents activities and findings from the ATCBI baseline data collection, which took place from December 2016 through January 2017 and the endline data collection, which took place in May/June 2017.

3 Methodology

3.1 Study Participants

To answer the ATCBI research questions, data were collected from grade 2 teachers in Ethiopia who taught reading and writing in one of seven MTs at regular public schools and who had students in their classrooms who were hard of hearing or had low vision. Grade 2 was selected for convenience, as existing audio recordings, in alignment with the teacher’s guide, had already been developed earlier in the project and were, thus, readily available for all seven MTs.

Selected teachers had to have undergone the READ TA 10-day teacher training on the revised MT curriculum for reading and writing as a condition for participation in the ATCBI. They also had to have already received the new teacher guide and new student textbooks for their classrooms. Data were collected from the teachers’ grade 2 classroom students, as well as their school principals. All data collection instruments were translated into seven MTs.

In adherence to project parameters, data were collected from schools in READ TA’s five focal regions (Amhara; Ethio-Somali; Oromia; Southern Nations, Nationalities, and Peoples Region [SNNPR]; and Tigray) and seven focal languages (Afaan Oromo, Af-Somali, Amharic, Hadiyyisa, Sidaamu Afoo, Tigrinya, and Wolayttatto). Given that this initiative was designed as a proof of

4

concept and focused on building the capacity of the implementing regional working groups, the study sample was purposefully kept small.

Furthermore, a pre-requisite for participating classrooms was that teachers in the classrooms were selected in a multistage sampling procedure within each of the five regions. In SNNPR, the selection started at the zonal level as the determinant of the language of instruction. The selection was done by READ TA staff in consultation with the participating RSEBs. The data collection aimed to reach at least 55 schools and 110 classrooms in the five regions: 10 schools each in the Amhara, Ethio-Somali, Oromia, and Tigray regions, and 15 schools (five per language) in SNNPR.

Selection of Zones. In the Amhara, Ethio-Somali, Oromia, and Tigray regions, two zones each were selected for participation. In SNNPR, the activity was implemented in three zones (one each for Hadiyyisa, Sidaamu Afoo, and Wolayttatto, respectively). Selection of zones (in all but SNNPR) was first done by the zone’s achievement in distributing books to students and the comprehensiveness by which the zone has ensured participation of its teachers in the new teacher guide teacher training earlier on in the project. Apart from these two factors, zones were selected by convenience, i.e., the ability to reach a zone within a half-day drive from the regional capital. Among the remaining zones, the selection was random.

Selection of Clusters. In each zone, two clusters were selected. Selection of clusters was, again, by degree of coverage of grade 2 classrooms with the new textbooks and the teachers’ past participation in the training. Among the eligible clusters, the selection was random.

Selection of Schools. In one cluster, three schools were selected, in the other cluster, two schools were selected. Selection of schools within each cluster was, by degree of coverage of grade 2 classrooms with the new textbooks and teachers’ past participation in the training. Selected schools also had to have at least two grade 2 classrooms and known attendance of grade 2 children who were hard of hearing or had low vision. Among the eligible schools, the selection was random.

Selection of Classrooms. In each school, two classrooms were selected. Selection of classrooms within each school was the degree of coverage of grade 2 classrooms with the new textbook and the teacher’s past participation in the training. Selected classrooms also had to have at least three children with a potential hearing or vision impairment—either already known or identified following the ATCBI screenings. Among the eligible classrooms in the school, the selection was random.

For ethical reasons, classrooms were ineligible for participation if they included children who were fully blind or deaf, as no specific accommodation or modification for these children was included in this initial pilot effort.

If a selected school did not have two classrooms meeting the criteria established, data were collected in just one classroom. If the selected school did not have at least one classroom meeting the criteria established, another school was selected, applying the same criteria. If the appropriate classrooms and number of schools were not found in a selected cluster, another cluster was selected, applying the same criteria. If the appropriate classrooms, schools, or clusters were not found in a selected zone, another zone was selected, applying the same criteria.

5

3.2 Final Study Sample The final baseline sample included participants from a total of 63 public primary schools in five regions of Ethiopia, specifically from 12 schools in Oromia; 10 schools each in Amhara, Tigray, and Ethio-Somali; 8 schools each in Hadiya and Sidama; and 5 schools in Wolayita. Participating schools, on average, had 1,436 students, with a range between 287 and 3,440 students per school. The average class size of the participating grade 2 classrooms was 58 students, with a range of 32 to 147 students in each class.

A total of 3,728 grade 2 students participated in the vision screening, of which 47% were girls. For the hearing screening, a total of 3,725 grade 2 students participated, of which 47% were girls. A subset of 727 students, 48% girls, were interviewed—30% were identified as having a potential vision impairment and 25% a potential hearing impairment. The interviewed students were, on average, 9.5 years old, with an age range of 6 to 17 years.

A total of 109 grade 2 teachers were interviewed, of which 82 (75%) were women. Over half of the participants (53%) reported their age to be between 25–34 years. On average, participating teachers had been teaching in grades 1–4 for about 10 years, with the range being between 2 and 38 years. Nearly 90% of teachers reported having a diploma as their highest educational training level. Regarding specific training on inclusive education or special needs education, nearly 39% of participants reported never having received any specific training on the topic. Forty percent of participating teachers reported having attended one college course, while 10% had attended two college courses. There were three teachers who reported having received a certificate or diploma in special needs education. In addition to the student screenings and teacher interviews, the READ TA data collectors completed 375 classroom observations.

At endline, the same 63 schools were visited and a total of 101 teachers were interviewed. Eight teachers that participated at the time of baseline data collection were not available at the time of endline data collection, which led to a reduction in the sample size. In addition, 624 students participated in the endline reading assessment and interview, of which 328 had been those identified as possibly having a vision and/or hearing impairment at baseline. The 296 students without a hearing or vision impairment participating in the endline data collection were selected randomly from the same classroom. Therefore, the total student number may not include the same students without vision or hearing impairment that had participated in the baseline data collection. READ TA data collectors also conducted 325 classroom observations at endline.

3.3 Measures The research design adopted for this initiative was both explanatory and evaluative (Schutt, 2015). The initiative drew primarily on quantitative methods to inform the research questions stated above, specifically

• prevalence of potential vision and hearing impairment in participating classrooms, • teachers’ barriers in implementing inclusive education, • teachers' attitudes and self-efficacy in inclusive education,

6

• teachers' actual utilization of inclusive practices for students who are hard of hearing or have low vision in their classes, and

• students’ reading outcomes.

READ TA used six instruments in the baseline data collection: (1) a clinically-validated vision screening tool, (2) a clinically-validated hearing screening tool, (3) a teacher interview questionnaire, (4) a classroom observation tool, (5) a student reading assessment and interview instrument, and (6) a principal interview questionnaire. The purpose of the principal interview was mainly to collect demographic and contextual information on the school. Findings from the principal interviews are not reported on in this report.

Except for the screening tools, all instruments were drafted in English by an international 4-person team comprising RTI evaluation and education experts. The tools were then reviewed by a 3-person team comprising Ethiopian special needs education and evaluation specialists. Finally, the instruments were translated into the seven MTs, rendered electronically onto RTI’s open source data collection platform, Tangerine®, and loaded on 7-inch tablet devices.

A field test of the instruments in two local languages, Amharic and Afaan Oromo, was conducted in December 2016. Six schools, three in Amhara and three in Oromia were visited, and a total of 318 students screened; 12 classrooms observed; and 64 students, 12 teachers, and 6 principals interviewed. Data collectors for the instrument test noted procedural issues with the tools and their application using a systematic instrument review guide. RTI statisticians analyzed the data for potential issues with the electronic rendering, variable definitions, and data values, as well as potential problems with specific questions and their analysis. As a result of the instrument field test, several questions and answer categories were deleted or changed.

Measuring Vision and Hearing Impairment Prevalence. To measure the prevalence of potential vision and/or hearing impairments in participating classrooms, READ TA used existing, clinically-validated tools customized for operation on smartphone devices. Although these tools are clinically validated, they do not constitute or replace a medical diagnosis.

To screen for a potential visual impairment (VI), a visual acuity test was conducted with participating students using the PeekVision vision screening app for distance vision (T. Carter, personal communication, November 20, 2016). Based on the test results, each student’s vision was classified by one of five severity levels: (1) normal, (2) mild impairment, (3) moderate impairment, (4) severe impairment, and (5) blindness. The five levels correspond to numerical LogMAR scores of visual acuity, as outlined in Exhibit 1. These categories are in line with those used by the World Health Organization (WHO; 2010). Only students identified with mild, moderate, or severe visual impairments were considered a target population for the initiative. The students’ weaker eye (as determined by the vision test) was used to establish each student’s visual category.

7

Exhibit 1. PeekVision Visual Categories and LogMAR Scores Visual Category LogMAR Score | Snellen

Normal 0–0.3 | 6/6–6/12 Mild visual impairment >0.3–0.48 | >6/12–6/18 Moderate visual impairment >0.48–1.0 | >6/18–6/60 Severe visual impairment >1.0 | >6/60–3/60 Blindness >1.3 | >3/60

To screen for a potential hearing impairment, a hearing test was conducted with participating students using the HearScreen hearing screening app (D.W. Swanepoel, personal communication, October 27, 2016). Based on the test results, each student’s hearing was classified by one of six severity levels: (1) normal, (2) slight impairment, (3) mild impairment, (4) moderate impairment, (5) moderate-severe impairment, and (6) severe impairment. The six levels correspond to numerical scores of hearing loss measured in decibels (dB), as seen in Exhibit 2, for three frequencies measured in hertz (Hz): 1,000Hz, 2,000Hz, and 4,000Hz. The HearScreen categorizations are similar to those used by WHO (2017). WHO considers slight and mild loss to be between 26dB and 40dB, moderate loss between 41dB and 60dB, severe loss between 61dB and 80dB, and anything over 81dB as profound loss. Only students identified with mild, moderate, moderate-severe, or severe (but not deaf) hearing impairments, as measured by the HearScreen app, were considered a target population for the initiative. The student’s weaker ear (as determined by the hearing test) on one or more frequencies was used to establish a student’s hearing category.

Exhibit 2: HearScreen Hearing Categories and Related Loss in Decibels Hearing Category dB loss (one or more frequency)

Slight loss <35 dB Mild loss 35 to 44 dB Moderate loss 45 to 54 dB Moderate-severe loss 55 to 64 dB Severe loss >65 dB

Measuring Teacher Attitude and Self-efficacy. To measure participating teachers’ attitude and self-efficacy in inclusive education, Likert scale questions related to inclusive education were given to teachers as part of a larger teacher interview instrument. Existing scales on teacher attitude and self-efficacy in inclusive education provided the source for select items, but no single existing scale was deemed appropriate for the specific purpose of this research and context.

New scales designed to measure teacher attitude were established by a 4-person team made up of evaluation experts, inclusive education experts, and education experts with Ethiopia expertise. The team examined existing attitude scales in the peer-reviewed literature3 and selected and adapted 22 attitudinal items with responses on a 5-point Likert scale for inclusion in the initial teacher

3 The literature reviewed for this purpose was as follows: Agbenyega, 2007; Cullen, Gregory, & Noto, 2010; Forlin, Earle, Loreman, & Sharma, 2011; Hofman & Kilimo, 2014; and Saloviita, 2015.

8

interview instrument. Following the baseline data collection with 109 teachers, factor analysis was conducted to reduce the scale to the most relevant items from the data collection, which together explained more than 76% of the variance. The final scale contained 10 items and had a Cronbach alpha of internal consistency of 0.71. Only these 10 items were used in the remaining data analysis related to the teacher self-efficacy measure. To facilitate communication of findings to the intended target audience, READ TA measured the indicator for teacher attitudes to inclusive education of children with vision or hearing impairment as the calculated mean score of participating teachers’ responses to the 10 items in the final scale (minimum = 10; maximum = 50), rather than as a weighted or normalized score.

As with the attitude scales, existing scales on teacher self-efficacy in inclusive education were deemed inappropriate for the purpose and context. The same specialist team also examined existing self-efficacy scales in the peer-reviewed literature4 and selected and adapted nine self-efficacy items. For all items, responses entailed a 5-point Likert scale.

As noted above, following the data collection, factor analysis was conducted to reduce the self-efficacy scale to what emerged as the most relevant items, which in this case explained more than 97% of the variance. The final scale contained six items and had a Cronbach alpha of internal consistency of 0.73. Only those six items were used in the remaining data analysis related to the teacher self-efficacy measure. To facilitate communication of findings to the intended target audience, READ TA measured the indicator for teacher self-efficacy in inclusive education of children with vision or hearing impairment as the calculated mean score of participating teachers’ responses to the six items in the final scale (minimum = 6; maximum= 30), rather than as a weighted or normalized score.

In addition to the attitude and self-efficacy scales, the teacher interview questionnaire contained demographic questions on teachers’ gender, age, teaching experience, training on inclusive education/special needs instruction, and knowledge of the prevalence of disability among their students. The questionnaire also contained items querying teachers on the challenges they face in the implementation of inclusive education for children who are hard of hearing or have low vision in their mainstream reading classroom. The original English language teacher questionnaire is attached in Appendix A.

Measuring Teachers’ Inclusive Practices in the Classroom. To measure teachers' actual utilization of inclusive practices for students who are hard of hearing or have low vision in their classrooms, READ TA utilized two data collection methods and corresponding instruments. First, lesson observations were conducted using a lesson observation instrument. Second, students were interviewed about their teacher’s instructional practices.

No existing classroom observation instrument (e.g., Soukakou, Winton, West, Sideris, & Rucker, 2014) was found that seemed appropriate for the purpose and context of this research in the literature review conducted. Thus, the same 4-person specialist team mentioned above (see Measuring Teacher Attitude and Self-efficacy section) developed a new instrument, drawing on 4 The literature reviewed for this purpose was as follows: Hofman & Kilimo, 2014; Malinen et al., 2013; and Sharma, Loreman, & Forlin, 2011.

9

years of expertise in the development of classroom observation instruments for reading and writing instruction in mother tongue, as well as the team’s expertise in special needs instruction and Ethiopia.

The new classroom observation instrument contained three parts. Part one captured classroom demographics; part two captured observations on individual students in the classroom, e.g., their placement in the classroom, interaction with peers, and instructional engagement; and part three captured observations of the teacher's behaviors and practices, including speaking clearly, repeating questions and responses, and describing pictures or illustrations used for the lesson. The final English language classroom observation instrument is attached in Appendix B.

Measuring Students’ Experience with Disabilities and their Reading Skills. To triangulate data sources with teachers’ inclusive practices (Krefting, 1991) READ TA combined teachers' self-reported data and the classroom observation data with data collected directly from students. As with the classroom observation instrument above, no existing student interview instrument on inclusive education (e.g., Diamond, 2001) was found to fit the purpose and context of this study. Thus, a new student interview questionnaire was developed, following the same process as for the classroom observation instrument outlined above. The student instrument contained four parts. In part one, the instrument captured basic demographic data on the student; in part two, the instrument contained a reading assessment; part three contained a set of questions about the student’s home and reading environment; and part four had questions about the student’s experience with disabilities.

The reading assessment contained three tests, a letter/syllable sound identification test, an oral reading fluency test, and a reading comprehension test. The letter sound identification test measured the child’s knowledge of the alphabet and the sounds of the letters. For this test, the child was presented with a sheet containing 100 randomly arranged letters and asked to read as many letters as they could in 60 seconds. The oral reading fluency test measured the child’s speed and accuracy of reading connected text. For this test, the child was given 60 seconds to read a grade-appropriate story out loud. The reading comprehension test assessed the child’s comprehension of the story they had read for the oral reading fluency test. The child was asked to orally respond to questions about the text they had read (up to five questions were asked if the child read the entire story). The reading tests had already been developed in all seven languages by RTI (2014) and others (American Institutes for Research, 2015) for previous early grade reading assessments in the country, and have thus already been validated for each language. The final English language student instrument, which served as the starting point for local language adaptation, is attached in Appendix C.

Institutional Review Board (IRB) Review. RTI’s IRB exempted the activity from review, given that it took place in an established education setting and involved practices that are considered reflective of this environment for the participating children. Participation in the data collection and initiative was voluntary for principals, teachers, and students. No student names were collected or stored in a way that would allow for identifying individual's screening data or interview data on the smartphones or tablets. Oral consent was sought from all participants in advance of the data collection to ensure participants understood who was conducting the research, its purpose, what

10

was expected of them, the potential risks and benefits of their participation, and whom to contact should there be concerns or questions. The consent statements, combined with their respective instruments, are attached in Appendices A–C.

3.4 Data Collection Methods and Analysis For the baseline data collection, READ TA engaged 33 assessors consisting of project staff, ATCBI working group members, and contracted data collection specialists. Assessors were fluent in at least one of the seven mother tongues in which the instruments had been translated. The project conducted a 3-day assessor training in December 2016 to familiarize data collectors with the initiative, the purpose of the baseline data collection, and the instruments. Assessors for the student reading assessment were selected based on their previous training and field experience in conducting early grade reading assessments. Participants practiced the application of each instrument, including with principals, teachers, and students, in two schools in Addis Ababa.

For the actual data collection, assessors formed 11 teams: two teams per language for Afaan Oromo, Af-Somali, Amharic, and Tigrinya and one team per language for Hadiyyisa, Sidaamu Afoo, and Wolayttatto. Data collection took place in each participating region and zone from December 19, 2016 to January 5, 2017. The RSEB in each region informed participating schools on the arrival of the data collection teams in advance of their visit. Upon arrival of the data collection teams, the team leader, a READ TA staff member, introduced the initiative to the principal and obtained oral consent from the principal to conduct the initiative and all parts of the data collection in their school.

Once consent was received, data collection teams screened all grade 2 children in the selected classrooms for a potential hearing or vision impairment. All children who were identified as having a potential vision or hearing impairment were retested to confirm results. Next, data collectors conducted the student interviews with all participating students identified to have a vision or hearing impairment, as well as an equal number of children not identified with these impairments for contrast. Following the student interviews, usually the next day, data collectors observed the participating grade 2 classrooms. To ensure the data collectors’ ability to concentrate on the teacher-student interactions, no data collector observed more than four students in the same classroom for the same lesson. Therefore, up to three data collectors, depending on the number of students identified, observed the same classroom at the same time. The data collectors used the same instrument but identified the individual children they were responsible for using the child’s unique identification number, which matched the one from the screening and student interview. Lastly, data collectors conducted the teacher and principal interviews.

Data collection teams spent between 2 and 3 days at each school, depending on the number of children in the participating classrooms. Where classrooms or schools proved to be ineligible for participation, e.g., because the teachers had not yet received the 10-day teacher training or there were less than three students screened for a vision or hearing impairment in the classroom, replacement schools/classrooms were sought as outlined in the Section 3.1.

All data were collected electronically—the vision and hearing screening data using the HearScreen and PeekVision screening tools and other data on Tangerine. When there was sufficient

11

connectivity to the Internet, data collectors uploaded their data, in encrypted form, to a central server. Other teams uploaded their data upon their completion of the data collection. The HearScreen, PeekVision and Tangerine platforms then allowed for the export of all data, by instrument, as a comma-separated values file.

Data were analyzed by an RTI statistician and the ATCBI activity lead, both based in RTI’s home office in North Carolina. All data were cleaned (e.g., correcting school names or student identification numbers to uppercase, where appropriate), then data sets were merged for analysis. The RTI statistician used Stata data analysis and statistical software for all data analysis. Most of the analysis entailed the calculation of frequencies. For the teacher attitudes and self-efficacy measures, the statistician also conducted inferential statistical analysis, specifically regressions to determine predictors of positive teacher attitude and self-efficacy.

At endline, READ TA applied a similar data collection method and analysis approach as was applied at baseline. Specifically, a total of 33 assessors (most were the same assessor as those who had participated in the baseline data collection) from across the five regions and seven language- communities were engaged. Assessors again formed 11 data collection teams. READ TA held a 2-day data collection refresher training on May 18–19, 2017, while endline data collection took place from May 22 to June 2, 2017. The endline data collection took less time than the baseline data collection because no screenings for vision or hearing impairment were conducted. In addition to utilizing similar analysis techniques as for the baseline data, endline data analysis also included comparison of group means to establish differences between baseline and endline, as well as tests for statistical significance of mean differences.

4 Baseline Findings

This section presents results from the baseline data collection. The specific research questions pertaining to the baseline data collection included the following:

• What is the prevalence of vision and hearing impairment in participating classrooms? • What are barriers for teachers in the implementation of inclusive education policies? • What are teachers’ current attitude towards self-efficacy in teaching children who are hard

of hearing or have low vision? • What is teachers’ current level of inclusive instructional practices in the classroom? • What are students’ current experiences in inclusive classrooms and what is their current

reading level, with or without a vision and/or hearing impairment?

Results are presented in line with theses research questions in the following sections.

4.1 Prevalence of Vision and Hearing Impairment Among the 3,728 children screened for a potential vision impairment, 5.37% of children were identified to have some form of vision impairment, of which 53% were girls. Exhibit 3 outlines the number and percentage of children identified by visual category.

12

Exhibit 3: Number and Percentage of Students Identified by Visual Category Visual Category Number of Students Percentage

Normal 3,518 94.36% Mild visual impairment 70 1.88% Moderate visual impairment 104 2.79% Severe visual impairment 26 0.7% Blindness 0 0% No response 10 0.27% Total 3,728 100.00%

Among the 3,725 children screened for a potential hearing impairment, 4.86% were identified to have some form of hearing impairment, of which 51% were girls. Exhibit 4 outlines the number and percentage of children identified by hearing category.

Exhibit 4: Number and Percentage of Students Identified by Hearing Category Hearing Category Number of Students Percentage

Normal 3,544 95.14% Mild loss 79 2.12% Moderate loss 27 0.72% Moderate-severe loss 26 0.7% Severe loss 49 1.32% Total 3,725 100.00%

Among the sample, 14 participating students were identified for both a potential vision and hearing impairment, resulting in a total of 367 unique children (or 9.84% of the total sample of 3,728 students). When participating teachers were asked about their students’ disabilities before the student screenings took place, they responded that only 2% of their students were known to have any kind of disability. This finding indicates a significant gap between the known (by the teacher) and actual prevalence rates of disability in participating classrooms. The screened 9.84% of students with a potential vision and/or hearing impairment is also a significantly larger proportion compared to the known (by the caregiver) 2.6% prevalence rate of any kind of disability that was found by Geda et al. (2016). These numbers are also higher than the 3.7% prevalence rate for vision impairment (among children and adults) that Berhane et al. found (2007).

As outlined above, in Ethiopia, the current, enrollment rate among children with disabilities is estimated to be 0.38%. This figure is based on data reported by school principals to the regional state education bureau for inclusion in the national education statistics published by the MOE. Although the prevalence data found in this data collection do not measure enrollment rate among out of school children, they do confirm that there may also be a significant proportion of children with undiagnosed disabilities already in school.

4.2 Teachers’ Barriers to the Implementation of Inclusive Education Data from the baseline data collection indicate that participating teachers face a wide range of challenges in implementing inclusive education in their classrooms. Among the most dominant

13

challenges are the shortage of instructional materials for vision and hearing impairment, lack of teacher training, insufficient government support, and lack of parental support. Exhibit 5 outlines the challenges and the percentage of teachers who reported to agree or strongly agree with each challenge presenting a barrier for implementing inclusive education in their classrooms.

Exhibit 5: Barriers to Inclusive Education and Percentage of Teachers Who Agreed or Strongly Agreed to Each (n = 109)

Challenge Percentage Shortage of teaching and learning materials for visual or hearing impairment 97.24% Lack of training 93.58% Insufficient government support 88.07% Lack of parental support 86.24% Large number of students in classes 77.98% High teaching load for teachers 76.15% Poor working environment 66.06% Lack of support from school leadership in implementing the MOE inclusive education strategy document

54.13%

General teacher attitudinal problems 52.3% Severity of disability 50.45% Lack of clarity on the MOE inclusive education strategy 47.71% Lack of guidance in implementing the MOE inclusive education strategy 47.71%

Teachers’ responses regarding challenges, such as teacher attitudes and severity of disabilities, as well as those related to the inclusive education strategy document, show high levels of variance among respondents. Although more than 52% of teachers surveyed agreed or strongly agreed that teacher attitudes are a problem in implementing inclusive education, 42.2% disagreed or strongly disagreed with this statement. Similarly, although more than 50% of teachers surveyed believed that the severity of disability presents a challenge in implementing inclusive education, 38.54% disagreed or strongly disagreed with this assertion. Further, at least 16% of the participating teachers indicated not being aware of the MOE inclusive education strategy document.

Teachers also mentioned challenges with the identification of students with special needs as a barrier to better serving them in their classrooms and schools. Several teachers indicated relying on the parents or students to inform them of a disability. Other teachers noted that there are no specialized identification tools or approaches and they identify children with disabilities mostly through observation of student behavior and performance. In addition to these issues with identification, teachers raised problems with the physical infrastructure of many of their schools. These problems included a lack of desks, benches, toilet facilities, clean water, and safe access ways or transportation to school.

The findings from the baseline data collection established gaps in teacher professional development and a lack of specialized materials and resources among the most notable challenges in the implementation of inclusive education in participating schools in Ethiopia, confirming results from other research in the region (e.g., Hofman & Kilimo, 2014; Ojok & Wormnaes, 2013).

14

4.3 Teachers’ Attitudes and Self-Efficacy in Inclusive Education at Baseline Concerning teachers’ attitudes to inclusive education, analysis of the baseline data indicated participating teachers’ attitude to inclusive education to be slightly positive. The mean score of teachers’ responses was 34.5 (minimum =10; maximum = 50; n =109), equivalent to the 61st percentile of the score. Participating teachers’ self-efficacy in inclusive education was also slightly positive. The mean score of teachers’ responses was 22.1 (minimum = 6; maximum = 30; n = 109), equivalent to the 67th percentile of the score.

READ TA conducted additional analysis to establish what factors may be predictors of positive teacher attitude or self-efficacy in inclusive education. Having a child with a known hearing impairment in the classroom was found to be a predictor of a more positive attitude to inclusive education among participating teachers. Other variables, including teacher gender, age, training background, self-efficacy, or class size were not found to be statistically significant. No single variable, including teacher gender, age, training background, teacher attitude, or class size was found to be a predictor of participating teachers’ self-efficacy in inclusive education at baseline. At endline, teachers responding positively to “does your headteacher provide any specific support to you about inclusive education?” was statistically significantly correlated with more positive teacher attitudes.

Analyzing teachers’ responses to individual attitude or self-efficacy statements in more detail (see Exhibit 6), sheds light on the complexity of the issue for participating teachers. Although 70% of participating teachers agreed or strongly agreed that inclusion promotes more academic growth of students with moderate visual or hearing impairment, 65% felt that the needs of students with moderate visual or hearing impairment can best be served through special, separate classes. Similarly, although 76% of teachers surveyed agreed or strongly agreed that they felt comfortable designing learning tasks appropriate for children with visual or hearing impairments and stated that they could use a variety of assessment strategies to evaluate these students (74% of teachers agree with this), 62% felt that students with visual or hearing impairments may develop academic skills more rapidly in a special class.

15

Exhibit 6: Teacher Responses to Individual Attitude and Self-Efficacy Statements (n = 109)

These findings indicate that while, overall, teachers may hold a slightly positive attitude to teaching children with a hearing or vision impairment in their classrooms, and a slightly positive sense of self-efficacy in doing so, there are shared prevailing opinions that the needs of these students may be best served outside of the regular classroom and by special teachers instead. These results support sociocultural factors, including teacher attitudes and opinions, as barriers to inclusive education in participating classrooms in Ethiopia, complementing previous research in the country and the East African region (e.g., Arbeiter & Hartley, 2002; Dagnew, 2013; Hofman & Kilimo, 2014).

4.4 Teachers’ Inclusive Practices in the Classroom at Baseline A key element of the ATCBI research was to measure potential changes in teachers’ adoption of foundational inclusive practices in the classroom, including speaking clearly, repeating questions and responses, or describing pictures or illustrations used for the lesson, which benefit all students in the classroom. Results from the classroom observations (see Exhibit 7) indicated that in 36% of the observations, the teachers at least frequently ensured that students were paying attention before beginning an activity, while in 33% of the observations this was rarely or never the case. In most observations (67%), teachers frequently, or always, spoke clearly and loudly enough to be

22%

29%

21%

4%

9%

3%

18%

11%

31%

17%

16%

20%

14%

37%

3%

3%

5%

7%

8%

37%

28%

37%

42%

40%

44%

25%

28%

9%

25%

34%

30%

32%

12%

0% 20% 40% 60% 80% 100%

IbelievethattheneedsofVI/HIstudentscanbestbeservedthroughspecial,separateclasses

TheextraattentionthatVI/HIstudentsrequirewillbetothedetrimentoftheotherstudents

IbelievethataVI/HIstudentwilldevelopacademicskillsmorerapidlyinaspec.class

IbelievethattheinclusionofstudentswithVI/HIcanbebeneficialforstudentswithout

disabilities.

IthinkthatinclusionpromotesmoreacademicgrowthofVI/HIstudents

IfeelcomfortabledesigninglearningtasksappropriateforchildrenwithVI/HI

IbelievethatIhavesufficientexpertise,knowledge,andskillstoteachVI/HIstudentsin

myreg.class

StronglyDisagree Disagree Neutral Agree StronglyAgree

16

heard in the back of the class (this was rarely or never the case in 18% of observations). In nearly 67% of classroom observations, teachers were facing the class while talking without obstructing their mouths, while this was rare or never the case in 18% of observations. In just over half of the observations (53%), the teacher wrote questions on the board, but this didn’t occur in more than two-thirds of classroom observations (64%). In more than a third of the observations (38%), teachers rarely or never repeated a question or response. In 155 of the 247 observations where a picture was used during the lesson, teachers rarely or never orally described a picture or illustration. Furthermore, few of the observed teachers (16%) used self-made teaching aids to support instruction, and fewer used materials specifically in support of children with disabilities (5%). Finally, in 83% of the classroom observations, the students were using the new student textbooks for mother tongue reading and writing. However, in more than half (53%) of the classroom observations, teachers did not use the new teacher guide during the lesson.

Exhibit 7: Teachers Inclusive Classroom Practices at Baseline and Endline (n = 375)*

* The percentage for each practice do not add to 100% as a fifth answer option, “sometimes” is not reflected in this graph

Regarding student seating arrangements, 41% of the observed children (which were those identified for a potential a vision or hearing impairment during the screening procedure) sat at the front of the room and more than 71% sat where lighting was best. Most of the observed students were seated in a circle with their peers to facilitate communication during group work. In just under 50% of observations, questions were rephrased or repeated for the observed students. In 35% of the observations, extra time was given to students to respond to questions and in 25% of the observations, students were permitted to respond verbally, in writing, or with sign language.

Among the students who had been identified for a potential vision and/or hearing impairment, assessors found that more than half of these children (56%) appeared on-task (e.g., focused on the

36%

67% 67%53%

25%38%

25% 16%33%

18% 18% 25%

64%

38%

63%84%

0%20%40%60%80%

100%

Ensuresstudentsarepayingattentionbeforeactivity

Speaksclearlyandloudly

Speaksfacingthe

class

Writesquestionsontheboard

Writeslesson

objectiveontheboard

Repeatsquestionsatleastonce

Orallydescribespicture(n=247)

Usesaself-made

teachingaid

Teacherfrequentlyorallthetime:

Frequentlyorallthetime Rarelyornever

17

teacher, class, or work assigned), while 41% were off task during the lesson and (5%) were identified as disruptive (e.g., speaking out of turn, making noise, or fighting or talking with other students).5 In most cases (55% of observations), assessors noted positive interactions between the students with potential impairments and their peers, e.g., talking appropriately with classmates, engaging in group discussion, or taking turns. However, 11% of the observed students identified for a potential impairment were reported to interact negatively with their classmates, e.g., speaking inappropriately or disrupting group work. Over a third of observed children were sitting alone and exhibited no interaction with their peers.

These results from the classroom observations should be considered with caution. The significant discrepancy between the number of children with a disability known to the teacher and the number identified from the screenings indicates that teacher practices or classroom arrangements that were indexed in this data collection as inclusive, such as speaking loudly or seating children with a impairment in the front of the classroom, may be part of teachers’ usual practices or simple coincidence, rather than indicative of a systematic accommodation. Similarly, some of the behaviors of the students identified with a potential impairment, including being off-task, were observed not as unique to this group only, but as behaviors also exhibited by other children in the classroom.

Despite those limitations, findings from the classroom observations indicate that a significant proportion of teachers rarely implemented even foundational instructional accommodations to the benefit of children with vision or hearing impairment, as well as all other children in their classroom. Such practices included writing lesson objectives and key questions on the board, repeating questions and responses, and ensuring that students were paying attention before beginning an activity.

4.5 Students’ Experience in Inclusive Classrooms and their Reading Skills at Baseline As outlined in earlier sections of this report, the potential prevalence of vision and hearing impairment among participating students was found to be significantly higher than that known by their teachers. When asked directly whether they had trouble seeing what the teacher wrote on the blackboard or hearing what the teacher said, responses from participating students identified with a potential vision or hearing impairment were inconclusive. Among those students who had been identified with a potential moderate or severe vision impairment, 47% (i.e., 42 students) confirmed that they had difficulty seeing what the teacher wrote on the blackboard. This potentially indicates that some teachers wrote in a large enough font to mitigate the vision loss. Two thirds (i.e., 39 students) that were identified with moderate or severe hearing impairment confirmed having trouble hearing what the teacher said in class. Again, it is possible, and results from the classroom observations support this, that teachers are naturally speaking clearly and loudly enough to compensate—at least to some extent—for the hearing loss.

5 For a few students, enumerators noted more than one behavior as dominant, which led to a total percentage of 102%.

18

Regarding students’ experience with disabilities, more than 80% of participating students indicated that they did not know anyone in their class with a disability and more than 90% indicated that they did not have a friend with a disability.

Students’ reading skills were assessed to add a learning outcome perspective to the study. At the time of the baseline data collection, three months into the school year, the study found that there were already significant differences in students’ knowledge of the letter/syllable sounds in their language. For example, in all but one of the languages, students identified with a potential vision or hearing impairment, on average, identified less letter/syllable sounds correctly per minute and read fewer correct words per minute compared to their peers without an impairment. Given the very small sample size of participating children in each of the languages and severity categories, these learning outcome results need to be considered with care as the sample size is not large enough to be considered representative of these subpopulations. Details of students reading outcomes are further discussed in Section 5.4.

5 Endline Findings and Outcomes of the Assistive Technology Intervention

This section presents results from the endline data collection on the ATCBI. The focus for the endline data collection was to answer questions about teachers’ adoption of assistive technology, potential changes in teachers’ attitudes and self-efficacy in inclusive education between baseline and endline, and changes in teachers’ adoption of inclusive practices in the classroom between baseline and endline. This section also compares baseline and endline results from the student reading assessments. As outlined earlier, the specific research questions for the endline data collection included:

• How would public primary school teachers adopt technology, specifically locally available smartphones, for instruction in inclusive classrooms;

• To what degree do provision of phone-based pedagogical support materials and training promote adoption of inclusive instructional practices among those teachers; and

• To what degree do such technology and practices help remove barriers of teacher attitude and self-efficacy in teaching children who are hard of hearing or have low vision.

Results below are presented in line with these research questions.

5.1 Teachers’ Adoption of the Assistive Technology A key question for the ATCBI was to what degree Ethiopian public school teachers in the five regions would use assistive technologies—in the form of one smartphone device with digital screening and lesson plan resources and one headset—to support their mother tongue reading and writing instruction in inclusive classrooms.

More than 85% of participating teachers reported owning a cell phone before the start of the ATCBI; however, 64% reported never having used a smartphone before. Results from the first monitoring visits (conducted one month after starting smartphone implementation in the classroom) indicated that many of the participating teachers were struggling with using the phones and digital, instructional resources they contained. Although a significant part of the initial 2-day

19

teacher training contained practice on using the phones and digital resources, it was not sufficient to give teachers confidence in their use. In consideration of these monitoring results, READ TA organized a 2-day refresher training immediately following those first monitoring visits, focused nearly exclusively on guided practice in using the phones and digital resources for screening and inclusive instruction.

Following the refresher training, the situation changed with notable improvement in technology adoption and use. This was noted by ATCBI working group representatives during a second monitoring visit, which took place shortly after the refresher training.

At endline, 70% of teachers reported having become very comfortable and 25% somewhat comfortable in using the phone for screening students for vision or hearing impairment. Further, more than 90% of participating teachers reported having become very comfortable and 9% somewhat comfortable in using the phone and the inclusive multimedia lesson plan (IMLP) to teach their lessons.

During endline classroom observation, 76% of observations indicated that participating teachers used the smartphone-based resources during the lesson. Of these, nearly half (49%) were observed to be very confident and 41% were observed to be somewhat confident in using the smartphone. Of those using the smartphones, only 9% were observed not to appear confident in its use.

Classroom observations also found that in more than half of the observations, participating teachers made use of the audio files embedded in the IMLP. In fact, 26% of the observations saw participating teachers give the phone directly to a student or groups of students to listen to the audio files. For this action, more than 90% of observations indicated that the teachers managed this phone transition to students with confidence and that it did not distract the other students when they did so.

During interviews, students of participating teachers were asked whether their teacher used the smartphone during a lesson in the week prior to the observation. More than 90% of students confirmed that their teachers had used the smartphone, while more than 69% of participating students further indicated that their teachers also used the phone to play the stories or sounds embedded in the lesson plans during the lesson.

In summary, results indicated that not all participating teacher used the phones at all times, and not all teachers felt entirely comfortable operating the smartphone—especially with using the screening tools. However, data from teachers’ self-reports, their students’ reports, and the classroom observations conducted by READ TA assessors indicated very high levels of adoption of the smartphone and its resources, specifically the IMLP, for mother tongue reading and writing instruction.

5.2 Changes in Teachers’ Attitudes and Self-Efficacy in Inclusive Education

Analysis of the data collected at baseline and endline indicated significant changes in teachers’ attitudes towards self-efficacy in inclusive education as measured by the scales developed. At endline, teacher attitudes towards teaching children with mild to severe vision or hearing impairment (but not fully blind of deaf) in their regular classroom improved from a mean score of

20

34.5 to 40.97 (minimum = 10; maximum + 50; n = 101). Concerning teacher self-efficacy, at endline, teacher self-efficacy in teaching children with mild to severe vision or hearing impairment (but not fully blind of deaf) in their regular classroom improved from a mean score of 22.1 to 24.1 (minimum = 6; maximum = 30; n = 101). Both improvements were found to be statistically significant (p <0.001). Exhibit 8 highlights the changes.

Exhibit 8: Teacher Attitude and Self-efficacy to Inclusive Education at Baseline and Endline (n = 101)

Exploring responses on individual items in each scale highlighted notable changes in teachers’ responses. Exhibit 9 lists select statements from the teacher attitude and self-efficacy scales and the percentage of teachers who at baseline and endline agreed or strongly agreed with each statement.

21

Exhibit 9: Percent of Teachers Agreeing or Strongly Agreeing with Attitude and Self-Efficacy Statements at Baseline and Endline (n = 101)

Participating teachers appear to have notably changed their assessment of what type of placement (i.e., regular classroom versus separate classroom) may best meet the needs of students with vision and/or hearing impairments. For example, at baseline, more than 64% of participating teachers felt that the needs of a student with a visual impairment or hearing impairment may best be met in a special, separate classroom, at endline, only 13% of participating teachers still held this belief. Similarly, at baseline, 61% of participating teachers felt that the academic skills of students with a vision impairment or hearing impairment may be developed more rapidly in a special, separate classroom, but this percent dropped to 21% at endline.

Regarding their self-efficacy, 37% of teachers at baseline felt that they had sufficient expertise, knowledge, and skills to teach students with a visual impairment or hearing impairment in their regular classroom—this proportion increased to 70% at endline. More teachers also reported feeling comfortable designing learning tasks appropriate for children with vision or hearing impairments.

In summary, across regions, teachers participating in the ATCBI appear to have significantly improved their attitudes and levels of self-efficacy towards teaching children with vision and/or hearing impairments in their regular classrooms. Even beyond statistical significance, the practical and contextual importance implied in the nuanced responses they provided to specific questions is highly noteworthy and promising. Teachers responses indicated notable changes in their thinking about their own ability to teach children with visual impairment and/or hearing impairment and the value of these children’s inclusion in their regular classes. Illustratively, one teacher stated, “before the training, I considered those students as lazy and mentally retarded students, but now

64%

61%

37%

76%

37%

71%

13%

21%

19%

93%

70%

91%

IbelievethattheneedsofVI/HIstudentscanbestbeservedthroughspecial,separateclasses

IbelievethataVI/HIstudentwilldevelopacademicskillsmorerapidlyinaspec.classthaninareg.classTheextraattentionthatVI/HIstudentsrequirewill

betothedetrimentoftheotherstudents

IfeelcomfortabledesigninglearningtasksappropriateforchildrenwithVI/HI

IbelievethatIhavesufficientexpertise,knowledge,andskillstoteachVI/HIstudentsinmyreg.class

IthinkthatinclusionpromotesmoreacademicgrowthofVI/HIstudents

Percentofteachersthatagreeorstronglyagreetothefollowingstatements:

Baseline Endline

22

after they [were] identified, I have changed their sitting [and] my methodology in the classroom. I have started to believe that they can achieve and learn as other students. I have also observed change [in] them in the classroom. They started to participate, they improved their result.”

5.3 Changes in Teachers’ Inclusive Practices in the Classroom Beyond teachers’ self-reports, classroom observations at endline allowed for actual observation of teachers’ practices. Exhibit 10 compares classroom observation results from the baseline data collection with those from the endline data collection on items such as teachers’ use of voice, teachers repetition of questions or responses, and teachers’ use of self-made teaching aids, including those specifically for visual/hearing impairment. Indicatively, the proportion of classroom observations that noted teachers to frequently ensure that students were paying attention before beginning a new activity increased from 36% to 68% between baseline and endline. The proportion of observations recording teachers frequently writing lessons objectives on the board increased from 25% at baseline to 66% at endline. The proportion of classroom observations that noted teachers frequently repeating questions increased from 38% at baseline to 57% at endline.

Exhibit 10: Teachers Inclusive Classroom Practices at Baseline and Endline (n = 325)

In addition to changes in instructional practices, classroom observations also indicated changes in classroom arrangements. At baseline, 41% of children screened for a potential vision impairment/hearing impairment were sitting at the front of the room. At endline, this proportion increased to 81%. Similarly, at baseline, 71% of children screened for a potential vision

36%

67% 67%53%

25%38%

16%28%

68%

84% 87%78%

66%57%

31%

72%

0%10%20%30%40%50%60%70%80%90%

100%

Ensuresstudentsare

payingattentionbeforeactivity

Speaksclearlyandloudly

Speaksfacingthe

class

Writesquestionsontheboard

Writeslesson

objectiveontheboard

Repeatsquestionsatleastonce

Usesaself-made

teachingaid

Usesmaterial

specificallyforVI/HI(ofthoseusingself-madeteachingaids)

Percentofobservationsthatfoundteacherfrequently:

Baseline Endline

23

impairment/hearing impairment were sitting where lighting was best, which increased to 91% at endline.

In summary, results from the ATCBI endline data collection indicated notably increased adoption of foundational inclusive practices by participating teachers. Such practices included optimally seating students in the classroom, writing lesson objectives and key questions on the board, repeating questions and responses, and ensuring that students were paying attention before beginning an activity. Identification of students with disabilities most likely played a role in the teachers’ registered changes in inclusive practices. As noted earlier, at baseline teachers were aware of about 2% of their students having a disability. This proportion increased to about 12% at endline. It is very likely that teachers were simply not aware that they had so many students with potential impairments in their classroom and, thus, did not consciously try to accommodate them.

5.4 Changes in Students’ Reading Skills As indicated earlier, students’ reading skills were assessed to add a learning outcome perspective to the study. Few studies on disabilities and inclusion feature learning outcome measures often because sample numbers are not sufficiently large enough to be considered representative of the subpopulations—even at a 5% or more prevalence rate of vision impairment and a prevalence rate of 4% or more for hearing impairment. To obtain a sufficiently large sample, students would need to be sampled from hundreds of classrooms and schools. Similar sample size limitations were encountered with this study and exacerbated by the fact that the sample was subdivided into seven MTs. When data were disaggregated by language, the cell size was very small and confidence intervals were very large. Thus, to more meaningfully report on students’ reading results, this report uses zero scores (i.e., the percentage of students who were unable to read a single letter or word), which can be aggregated across languages. The team compared the incidents of zero scores among the three student population groups of interest (those with a potential visual impairment, those with a potential hearing impairment, and those without a visual/hearing impairment).

Across all languages, at baseline, 13% of participating students without a potential vision impairment or hearing impairment read zero correct letter sounds per minute (CLSPM; see Exhibit 11). This percentage reduced to 4% at endline. At baseline, a higher proportion of children identified with a potential vision impairment or hearing impairment already had zero scores, compared to students who were not identified with a vision impairment or hearing impairment. The proportion of children with zero scores in letter sound knowledge among students with a potential vision impairment was 22% at baseline and 9% at endline and for students with a potential hearing impairment, 16% at baseline and 2% at endline. The net reduction was 9% for children without hearing impairment or vision impairment, 13% for children with a potential vision impairment, and 14% for children with a potential hearing impairment. The reduction in zero scores was greater for children with a potential vision impairment or hearing impairment compared to those without.

24

Exhibit 11: Students’ Zero Scores at Baseline and Endline (n = 624)

Exhibit 12 illustrates the reduction in the percentage of students with zero scores in oral reading fluency (measured in correct words per minute [CWPM]) from baseline to endline across all seven languages. As with the letter knowledge task, at baseline, students identified with a potential vision impairment or hearing impairment had higher proportions of zero scores in oral reading fluency, than peers without hearing impairment or vision impairment. Among those students with a potential impairment, students identified with a vision impairment had a slightly larger proportion of zero scores that those with a potential hearing impairment (e.g., 50% vs. 49%). The net reduction in zeros scores between baseline and endline was 30% for students without hearing impairment or vision impairment, 22% for children with a potential vision impairment, and 23% for children with a potential hearing impairment. The reduction in zero scores was greater for children without vision impairment or hearing impairment compared to those with a potential vision impairment or hearing impairment.

13%

4%

22%

9%

16%

2%0%

5%

10%

15%

20%

25%

Baseline Endline

ZeroScores(CLSPM);alllanguages

NoHI/VI VI HI

25

Exhibit 12: Students’ Zero Scores at Baseline and Endline (n = 624)

In summary, the data on students’ reading outcomes (as measured at baseline and endline using validated early grade reading assessment subtests) indicated that at baseline, after just one year of schooling, there were already measurable differences in reading achievement between students identified with a potential vision impairment or hearing impairment and those that were not (considering zero scores aggregated across the seven languages). A higher proportion of students with a potential vision impairment or hearing impairment were scoring zero correct items, compared to those students without. However, students with a vision impairment or hearing impairment showed measurable progress between baseline and endline. The reduction in zero scores on correct letter sounds per minute was even greater for students screened for a potential vision impairment or hearing impairment compared to those without an impairment. For correct letter sound knowledge, zero scores of students with a potential hearing impairment were even lower at endline compared to those without a vision impairment or hearing impairment. However, for oral reading fluency, the decrease was not sufficient in vision impairment- or hearing impairment-identified students to reach the level of oral reading fluency by those students without vision impairment or hearing impairment.

As outlined earlier, given the very small sample size of participating students in each of the languages and impairment categories, these learning outcome results need to be considered with care. In the absence of a control group of students screened for a potential vision impairment or hearing impairment who were not in an intervention classroom, it is impossible to conclusively determine if these results are a demonstration of the success of the intervention. However, based on existing research on equity and an existing robust body of evidence on reading gaps, the data gives reason for optimism. Additional research with larger sample sizes and control groups is needed to confirm this assumption.

44%

14%

50%

28%49%

26%

0%10%20%30%40%50%60%

Baseline Endline

ZeroScores(CWPM);alllanguages

NoHI/VI VI HI

26

6 Proof of Concept Study Limitations Although results of this study are highly promising across the research questions explored, the following limitations need to be considered:

• This study was a proof of concept. That is, it was an exploratory research effort that was investigating the general feasibility of an approach, rather than a rigorous scientific study aimed at establishing causal relationships.

• A control group, including students with a potential hearing or vision impairment, but who did not participate in the ATCBI (and without letting the teacher know of the child’s potential vision impairment/hearing impairment), would have enriched results. Most notably, it would have allowed the team to better explain whether the reading outcomes of the children identified with a vision impairment or hearing impairment was, in fact, a result of the intervention or illustrative of normal progression.

• A control group that would just have entailed screening the students and letting the teacher know about a student’s potential vision impairment or hearing impairment condition would have deepened the knowledge base on a potential relative contribution of identification versus accommodation on teachers’ behavior change and students’ reading outcomes.

• A larger sample size to disaggregate analysis by levels of disability or impairment (e.g., mild, moderate, and severe) would have enriched knowledge base on outcomes for different groups in the study, especially at the individual language level.

• There is the possibility of false positives particularly for the hearing screening activity due to environmental noise potentially having affected the precision of the tool. It is important to stress again, that the initiative was a screening effort only and did not constitute a medical diagnosis.

• There were teacher re-assignments between baseline and endline data collection, that were not communicated by the school to the study team. This may have led the possibility of changes in the teacher sample between baseline and endline, and thus may have affected teacher-level results.

As outlined earlier, these limitations need to be considered in light of the overall design and purpose of the study, which was a proof of concept. As such, this study presents a first effort towards additional research and practice that should further develop, optimize, and validate the overall initiative and its evaluation.

7 Conclusions and Recommendations In August 2017, READ TA conducted a 2-day consultation workshop with representatives of the regional working group who implemented the ATCBI in their respective regions, representatives from the Ministry of Education, and representatives of several national disabled people’s organizations. During the workshop, READ TA shared preliminary findings from the ATCBI endline data collection but, more importantly, also noted lessons learned and reflections from the

27

participating working groups. ATCBI participants from the working groups, specifically representatives from the RSEBs, shared the following conclusions from the initiative, in consideration of the data collection result and their own experience.

The smartphone, headsets, and selected screening and IMLP apps used in the ATCBI appeared to have been appropriate for the context and target group. Although less than 40% of participating teachers had experience using a smartphone before the ATCBI, and a refresher training was required to enable more teachers in their use, at the end of the 12-week implementation, more than 90% of the participating teachers felt very comfortable with the technology. The inclusive multimedia lesson plan application enjoyed particularly strong adoption. Supporting this argument, expansion of the activity to additional grades and teachers were among the most cited recommendations for enhancing the activity by participating teachers.

In addition to expansion to other grades, a key feedback from participating teachers was that additional practical training, particularly on the hearing screening tool, would benefit future versions of the activity. Teachers also made specific recommendations to simplify the hearing screening app, as it requires login, user validation, and several steps to set up screening for a child. The ATCBI working group members and project teams reflected, however, that rather than just more training, the initial training may benefit from an even stronger focus on role-playing and practice of the screening applications.

The approach to implement training in two parts, although not originally designed that way, with an initial training and a refresher training appeared to have been highly effective in meeting teachers’ needs. Although the total of four training days may be more than what is financially affordable at scale. The ATCBI team saw an opportunity to optimize each of the two training phases to reduce the overall training time, while maximizing time for practice. A future replication or similar activity may explore such a redesign.

Results of the proof of concept study indicated that the ATCBI, as implemented, was very successful in establishing initial hearing impairment and vision impairment prevalence rates in participating schools. ATCBI workgroup members repeatedly mentioned having been “shocked” at realizing how many children may have a vision or hearing impairment in the study schools. Similarly, participating teachers expressed surprise at how many of their students may have a potential vision impairment or hearing impairment, which they did not know about. Several of the 63 ATCBI schools have started screening additional classrooms. This speaks to the efficacy of the ATCBI to meet an explicit demand at the school level. Illustratively, an Amhara RSEB ATCBI working group representative reported on one school in the Amhara region that has initiated a partnership with a closely located education institute to recruit additional support for screening.

Most notably, the ATCBI appeared to have been very successful in improving participating teachers’ attitude towards and self-efficacy in teaching children with vision or hearing impairments in their regular mainstream classrooms. This is a major finding, which is supported not only by analysis of teachers’ responses on the attitude and self-efficacy scales but by teachers’ qualitative responses in interviews, as well as classroom observations. ATCBI data collectors noted significant increases in teachers’ adoption of foundational classroom practices to promote

28

inclusion for students with vision or hearing impairment between baseline and endline. It is likely that identification and awareness played a critical part in this change. The screening activity indicated to teachers that there are several students with a potential vision or hearing impairment in their classroom, which in turn may have led some teachers to more consciously and systematically adopt more inclusive instructional practices.

Based on a comprehensive literature review conducted in 2016 (Wapling, 2016), the ATCBI appears to be one of the first studies in a low- and middle-income country on inclusive education that also considered student learning outcomes. Results from the study indicated that all groups of children, those without vision impairment or hearing impairment, those with a potential vision impairment, and those with a potential hearing impairment, have made measurable progress in reading acquisition—specifically in the reduction of zero scores—between baseline and endline. Most notably, significant gaps already existed at baseline. It is likely that the gap in reading outcomes between children without vision impairment and hearing impairment and those with a potential vision impairment or hearing impairment would have been even wider without the ATCBI; however, this needs to be confirmed through additional research. Given the very small sample size of participating children in each of the languages and severity categories, these learning outcome results need to be considered with care. However, in absence of any empirical evidence comparing student achievement between children with or without disabilities in the same classroom, these results are starting to fill an important gap in the knowledge base on inclusive education in Ethiopia and the East African region.

29

Bibliography Agbenyega, J. (2007). Examining teachers' concerns and attitudes to inclusive education in

Ghana. International Journal of Whole Schooling, 3(1), 41–56.

American Institutes for Research. (2016). Early Grade Reading Assessment (EGRA) in three mother tongues (Aff Somali, Sidamu Affoo and Tigrigna) Data Analytic Report. Washington, DC: USAID. Available at http://pdf.usaid.gov/pdf_docs/pa00m562.pdf

Berhane, Y., Worku, A., Bejiga, A., Adamu, L., Alemayehu, W., Bedri A.,…West, S. (2007). Prevalence and causes of blindness and low vision in Ethiopia. Ethiopian Journal of Health Development, 21(3), 204–210. https://doi.org/10.4314/ejhd.v21i3.10050

Cullen, J. P., Gregory, J. L., & Noto, L. A. (2010). The Teacher Attitudes Toward Inclusion Scale (TATIS). Paper presented at the Annual Meeting of the Eastern Educational Research Association. Available at https://eric.ed.gov/?id=ED509930

Diamond, K. E. (2001). Relationships among young children's ideas, emotional understanding, and social contact with classmates with disabilities. Topics in Early Childhood Special Education, 21(2), 104–13. https://doi.org/10.1177/027112140102100204

Forlin, C., Earle, C., Loreman, T., & Sharma, U. (2011). The Sentiments, Attitudes, and Concerns about Inclusive Education Revised (SACIE-R) scale for measuring pre-service teachers' perceptions about inclusion. Exceptionality Education International, 21(3), 50–65.

Geda, B., Berhane, Y., Assefa, N., & Worku, A. (2016). In rural eastern Ethiopia hearing loss is the most frequent disability during childhood: A community based survey. Plos ONE, 11(5), 1–12. https://do.org/10.1371/journal.pone.0152791

Hofman, R., & Kilimo, J. (2014). Teachers’ attitudes and self-efficacy towards inclusion of pupils with disabilities in Tanzanian schools. Journal of Education and Training, 1, 177–198. https://doi.org/10.5296/jet.v1i2.5760

Krefting, L. (1991). Rigor in qualitative research: The assessment of trustworthiness. The American Journal of Occupational Therapy, 45, 214–222. https://doi.org/10.5014/ajot.45.3.214

Malinen, O.P., Savolainen, H., Engelbrecht, P., Xu, J., Nel, M., Nel, N., & Tlale, D. (2013). Exploring teacher self-efficacy for inclusive practices in three diverse countries. Teaching and Teacher Education, 33, 34–44. https://doi.org/10.1016/j.tate.2013.02.004

Ministry of Education. (2012). Special Needs/Inclusive Education Strategy. Addis Ababa, Ethiopia: The Federal Democratic Republic of Ethiopia.

30

Ministry of Education. (2016a). A master plan for special needs education/inclusive education in Ethiopia. Addis Ababa, Ethiopia: The Federal Democratic Republic of Ethiopia.

Ministry of Education. (2016b). Education statistics 2007 E. C. (2014/15). Addis Ababa, Ethiopia: Education Management Information System and ICT Directorate. Available at http://www.moe.gov.et/statistics

Mitra, S., Posarac, A., & Vick, B. (2013). Disability and poverty in developing countries: A multidimensional study. World Development, 41, 1–18. https://doi.org/10.1016/j.worlddev.2012.05.024

Ojok, P., & Wormnaes, S. (2013). Inclusion of pupils with intellectual disabilities: Primary school teachers' attitudes and willingness in a rural area in Uganda. International Journal of Inclusive Education, 17(9), 1003–1021. https://doi.org/10.1080/13603116.2012.728251

Polat, F. (2011). Inclusion in education: A step towards social justice. International Journal of Educational Development, 31(1), 50–58. https://doi.org/10.1016/j.ijedudev.2010.06.009

RTI International. (2014). Ethiopia 2014. Early Grade Reading Assessment. Report of findings. Research Triangle Park, NC: Author. Available at http://pdf.usaid.gov/pdf_docs/pa00mht3.pdf

Saloviita, T. (2015). Measuring pre-service teachers' attitudes towards inclusive education: Psychometric properties of the TAIS scale. Teaching and Teacher Education, 52, 66–72. https://doi.org/10.1016/j.tate.2015.09.003

Schutt, R. K. (2015). Investigating the social world: The process and practice of research (8th ed.). Thousand Oaks, CA: Sage Publications.

Sharma, U., Loreman, T., & Forlin, C. (2011). Measuring teacher efficacy to implement inclusive practices. Journal of Research in Special Educational Needs, 12(1), 12–21. https://doi.org/10.1111/j.1471-3802.2011.01200.x

Soukakou, E. P., Winton, P. J., West, T. A., Sideris, J. H., & Rucker, L. M. (2014). Measuring the quality of inclusive practices: Findings from the inclusive classroom profile pilot. Journal of Early Intervention, 36(3), 223–240.https://doi.org/10.1177/1053815115569732

Stone-MacDonald, A. (2012). Cultural beliefs about disability in practice: Experiences at a special school in Tanzania. International Journal of Disability, Development and Education, 59(4), 393–407. https://doi.org/10.1080/1034912X.2012.723947

31

United Nations. (2006). United Nations convention on the rights of persons with disabilities. Geneva, Switzerland: Author. Available at http://www.un.org/disabilities/documents/convention/convoptprot-e.pdf

United Nations Educational, Scientific and Cultural Organization (UNESCO). (2003). Overcoming exclusion through inclusive approaches in education. A challenge and a vision. Paris, France: Authors. Available at https://pdfs.semanticscholar.org/73ea/02577de3a23f25e1c08c18ffbb2283695f31.pdf

UNESCO. (2016). Global education monitoring report 2016. Education for people and planet: Creating sustainable futures for all. Retrieved from http://unesdoc.unesco.org/images/0024/002457/245752e.pdf

Wapling, L. (2016). Inclusive education and children with disabilities: Quality education for all in low and middle income countries. Germany: CBM International. Available at http://www.cbm.org/article/downloads/54741/Quality_Education_for_All_LMIC_Evidence_Review_CBM_2016_Full_Report.pdf

World Health Organization (WHO). (2010). International statistical classification of diseases and related health problems 10th revision (ICD-10) version for 2010. Available at http://apps.who.int/classifications/icd10/browse/2010/en#!H54

WHO. (2011). World report on disability. Geneva, Switzerland: Author. Available at http://www.who.int/disabilities/world_report/2011/en/

WHO. (2017). Prevention of blindness and deafness. Grades of hearing impairment. Available at http://www.who.int/pbd/deafness/hearing_impairment_grades/en/

Tangerine

Page 1 of 7

Teacher Questionnaire - English - Endline

Last Updated: 1495120927000

Enumerator Name

Date and time

Date

Time

Record ID

Record ID

Consent

Let me tell you why I am here today. I work with the Ministry of Education on an effort to improve reading andwriting in mother tongue in Ethiopia. The new textbooks and teacher guides you are using were developed underthe project, READ TA, that I am working for.The reason why I am here today is that READ TA is now trying to understand what else may be done to helpchildren that may have difficulty seeing or hearing learn how to read We would like your help in this.We would like to observe one of your reading classes to see how these children are doing in the classroomThen we would like to interview you about your thoughts on inclusive education, your classroom, and your role as ateacher. The intention of the interview is to ask your personal view on several items regarding the inclusion of studentswith disability in regular classrooms and schools in Ethiopia. For this survey, inclusive education is definedas integration of students with mild or moderate vision or hearing disabilities (not fully blind or deaf) into regularclassrooms for 80% or more of the school day. Thus, when providing your answers, please particularly considerthem regarding children with vision impairment and hearing impairment. The information you will provide in this interview will be confidential to the best of our ability. Your real name will notbe used in any publication or report.The interview will take about 30 minutes to complete.If you have any questions about this questionnaire and activity, please contact, Mr. Habtamu Balderas, the READTA project’s Special Needs Advisor, through email or cell phone (hbalderas@read.rti.org or at 0921702241).Do you have any questions?Do you agree to participate in this effort?

Does the teacher consent? ☐

School Information

RegionZone

Appendix A

Tangerine

Page 2 of 7

SchoolName

School information input

Please enter name of school if "other" selected

Please enter name of woreda if "other" selected

Section 1

1. What class section do you teach?

2. What is your sex?

Female Male

3. What is your age range?

under 20 years 20-24 years 25-29 years 30-34 years 35-39 years 40-44 years 45-49 years

50-54 years 55-59 years 60-64 years 65-69 years above 69 years

4. How many students are in your class this year?

5. What is your educational training level?

High school Certificate Diploma Bachelor degree Master degree

6. What is your training level on Inclusive Education or Special Needs Education?

Untrained Attended 1 college common course Attended 2 college courses Certificate Diploma Bachelor Degree

Master degree

7. How many years have you been teaching in Grades 1-4? (Please enter number of years)

8. How many students with disability do you have in your classroom? (Indicate a number)

Tangerine

Page 3 of 7

9. What type(s) of disabilities do they have? ([Select all that apply. Skip if no students have disabilities])

Vision impairments Hearing impairments Mental impairment Physical impairment Behavioural problems

10. How did you identify the children with these impairments? ([Skip if no children have disabilities])

11. Does your headteacher provide any specific support to you about inclusive education?

Yes No No response

12. Is there another teacher at your school with special training or experience in inclusive education/special needs education who provides support to you?

Yes No No response

13. Is there an itinerant teacher who regularly visits your school to provide support with special needs/inclusive education?

Yes No No response

Section 2 (Part 1)

2. I believe that the needs of students with moderate visual or hearing impairment can best be served through special, separate classes.

Strongly disagree Disagree Neutral Agree Strongly agree

4. I think that inclusion promotes more academic growth of students with moderate visual or hearing impairment.

Strongly disagree Disagree Neutral Agree Strongly agree

5. The extra attention that students with moderate visual or hearing impairment require will be to the detriment of the other students.

Strongly disagree Disagree Neutral Agree Strongly agree

6. I believe that inclusion of students with moderate visual or hearing impairment offers mixed group interaction which will foster understanding andacceptance of differences.

Strongly disagree Disagree Neutral Agree Strongly agree

8. I believe that I have sufficient expertise, knowledge, and skills to teach students with moderate visual or hearing impairment in my regular classroom.

Strongly disagree Disagree Neutral Agree Strongly agree

10. I believe that isolation in a special class has a negative effect on the social and emotional development of a student with a moderate visual or hearingimpairment

Strongly disagree Disagree Neutral Agree Strongly agree

Tangerine

Page 4 of 7

11. I believe that a student with a moderate visual or hearing impairment will develop academic skills more rapidly in a special classroom than in a regularclassroom.

Strongly disagree Disagree Neutral Agree Strongly agree

12. I can use a variety of assessment strategies (e.g. modified test, oral instead of written assessment, etc.) to evaluate students with moderate visual orhearing impairment.

Strongly disagree Disagree Neutral Agree Strongly agree

14. In my classroom, if a student with moderate visual or hearing impairment does not understand something that I explained, I will find another way toincrease his/her understanding.

Strongly disagree Disagree Neutral Agree Strongly agree

15. I am sure that I have the ability and skills to motivate students with moderate visual or hearing impairment in my classroom.

Strongly disagree Disagree Neutral Agree Strongly agree

Section 2 (Part 2)

17. If students with disabilities are not disciplined, I cannot accept them in my classrooms.

Strongly disagree Disagree Neutral Agree Strongly agree

18. I can accurately gauge student comprehension of what I have taught.

Strongly disagree Disagree Neutral Agree Strongly agree

21. I feel comfortable designing learning tasks appropriate for children with visual or hearing impairments

Strongly disagree Disagree Neutral Agree Strongly agree

23.I believe that the inclusion of students with visual or hearing impairments can be beneficial for students without disabilities.

Strongly disagree Disagree Neutral Agree Strongly agree

25. I think that inclusion has negative effects on the emotional development of students with visual or hearing impairments.

Strongly disagree Disagree Neutral Agree Strongly agree

27. I believe that students with visual or hearing impairments are socially isolated by students without disabilities.

Strongly disagree Disagree Neutral Agree Strongly agree

Section 3

For each statement below, please rank your level of agreement or disagreement. Choose only one that most represents your level of judgement in the followingproblems or challenges that you face when implementing inclusive education.

Tangerine

Page 5 of 7

Strongly Disagree (D); Disagree; Neutral (N); Agree (A); Strongly Agree

1. Lack of training (is a problem or challenges that you face when implementing inclusive education)

Strongly disagree Disagree Neutral Agree Strongly agree

2. Shortage of teaching and learning materials for visual or hearing impairment

Strongly disagree Disagree Neutral Agree Strongly agree

3. Teacher attitudinal problems

Strongly disagree Disagree Neutral Agree Strongly agree

4. Large number of students in classes

Strongly disagree Disagree Neutral Agree Strongly agree

5. High teaching load for teachers

Strongly disagree Disagree Neutral Agree Strongly agree

6. Insufficient government support

Strongly disagree Disagree Neutral Agree Strongly agree

7. Poor working environment

Strongly disagree Disagree Neutral Agree Strongly agree

8. Severity of disability

Strongly disagree Disagree Neutral Agree Strongly agree

9. Lack of parental support

Strongly disagree Disagree Neutral Agree Strongly agree

10. Lack of support from school leadership in implementing the Ministry of Education inclusive education strategic document

Strongly disagree Disagree Neutral Agree Strongly agree

Respondent understands concept, but mentioned not knowing specific document

11. Lack of clarity on the Ministry of Education inclusive education strategic document

Strongly disagree Disagree Neutral Agree Strongly agree

Respondent understands concept, but mentioned not knowing specific document

Tangerine

Page 6 of 7

12. Lack of practical guidance in implementing the Ministry of Education inclusive education strategy

Strongly disagree Disagree Neutral Agree Strongly agree

Respondent understands concept, but mentioned not knowing specific document

15. Which disabilities are you most comfortable teaching in your classroom?

16. Which disabilities do you believe you cannot teach in your classroom?

17. How do you assess students with disabilities in your classroom?

19. How comfortable did you become in using the READ TA phone for screening children?

Very comfortable Somewhat comfortable Not very comfortable

20. How comfortable did you become in using the READ TA phone and IMLP for teaching the accommodated lesson plan?

Very comfortable Somewhat comfortable Not very comfortable

21. How could we improve the screening tool?

22. How could we improve the IMLP/accommodated lesson plans?

23. How could we improve the training you received for this initiative? ([remind them about the initial training, the two monitoring visits and the refreshertraining; which of these was most helpful; were they too long, too short; what training topics were most important, which were missing?])

24. If we were to do this initiative again with a new set of teachers, what else would you suggest we change? How can we make it better?

25. How has your thinking and practice about inclusive education and teaching children with impairments changed through this activity?

Tangerine

Page 7 of 7

26. How would you rate the effectiveness of this initiative to provide you with skills and instructional tools to better teach children with visual or hearingimpairments reading in mother tongue? ([provide answer options])

Very effective Somewhat effective Somewhat ineffective Very ineffective

Tangerine

Page 1 of 7

Classroom Observation - English - Endline

Last Updated: 1495120231000

Enumerator Name

Date and time

Date

Time

Record Id

Record Id

School Information

RegionZone

SchoolName

School information input

Please enter name of school if "other" selected

Please enter name of woreda if "other" selected

Header Information

E. Teacher name:

F. Grade:

G. Class section:

Appendix B

Tangerine

Page 2 of 7

H. School Shift:

Full day Morning Afternoon

I. Total number of children present ([Count the children in the classroom])

J. Total number of girls present ([How many of the children that are present in the class are girls])

K. Total number of MT reading books present during lesson ([Count the number of mother tongue student books present during lesson])

Child Information

How many children will you observe today?

1 2 3 4

Child 1 ID

Enter Child 1 VI level

Normal Mild Moderate Severe Blind

Enter Child 1 HI level

Normal Slight Mild Moderate Moderate-severe Severe

Child 2 ID

Enter Child 2 VI level

Normal Mild Moderate Severe Blind

Enter Child 2 HI level

Normal Slight Mild Moderate Moderate-severe Severe

Tangerine

Page 3 of 7

Child 3 ID

Enter Child 3 VI level

Normal Mild Moderate Severe Blind

Enter Child 3 HI level

Normal Slight Mild Moderate Moderate-severe Severe

Child 4 ID

Enter Child 4 VI level

Normal Mild Moderate Severe Blind

Enter Child 4 HI level

Normal Slight Mild Moderate Moderate-severe Severe

Student Observation

1. Is the student seated at the front of the room?

Child 1 (Yes) Child 2 (Yes) Child 3 (Yes) Child 4 (Yes) Child 1 (No) Child 2 (No) Child 3 (No)

Child 4 (No)

2. Is the student seated where lighting is best?

Child 1 (Yes) Child 2 (Yes) Child 3 (Yes) Child 4 (Yes) Child 1 (No) Child 2 (No) Child 3 (No)

Child 4 (No)

3. During group work, is the student seated in a circle with peers to facilitate communication

Child 1 (Yes) Child 2 (Yes) Child 3 (Yes) Child 4 (Yes) Child 1 (No) Child 2 (No) Child 3 (No)

Child 4 (No) No group work observed during lesson

4. Are questions repeated or rephrased as needed for this student?

Child 1 (Yes) Child 2 (Yes) Child 3 (Yes) Child 4 (Yes) Child 1 (No) Child 2 (No) Child 3 (No)

Child 4 (No)

Tangerine

Page 4 of 7

5. Is extra time given to this student to respond to questions?

Child 1 (Yes) Child 2 (Yes) Child 3 (Yes) Child 4 (Yes) Child 1 (No) Child 2 (No) Child 3 (No)

Child 4 (No)

6. Is the student permitted to respond verbally if VI or in writing or sign language if HI (note any other response methods used by the student)?

Child 1 (Yes) Child 2 (Yes) Child 3 (Yes) Child 4 (Yes) Child 1 (No) Child 2 (No) Child 3 (No)

Child 4 (No)

6a. Other methods

7. How is the student’s engagement in class?

Child 1 On-task behavior – focused on the teacher, the class, or on the work assigned

Child 1 Off-task behavior – Not focused on the teacher or what the class or group is doing.

Child 1 Disruptive - speaking out of turn, not sitting, making noise, fighting or talking with other children etc.

Child 2 On-task behavior – focused on the teacher, the class, or on the work assigned

Child 2 Off-task behavior – Not focused on the teacher or what the class or group is doing.

Child 2 Disruptive - speaking out of turn, not sitting, making noise, fighting or talking with other children etc.

Child 3 On-task behavior – focused on the teacher, the class, or on the work assigned

Child 3 Off-task behavior – Not focused on the teacher or what the class or group is doing.

Child 3 Disruptive - speaking out of turn, not sitting, making noise, fighting or talking with other children etc.

Child 4 On-task behavior – focused on the teacher, the class, or on the work assigned

Child 4 Off-task behavior – Not focused on the teacher or what the class or group is doing.

Child 4 Disruptive - speaking out of turn, not sitting, making noise, fighting or talking with other children etc.

Tangerine

Page 5 of 7

8. How is the student’s interaction with peers?

Child 1 Interacting positively – talks appropriately with classmates, engages in group discussions, takes turns, plays appropriately with peers outside etc.

Child 1 Interacting negatively –speaks inappropriately, disrupts group work, is aggressive (hits, kicks, insults, etc.),

Child 1 No interaction – sits alone

Child 2 Interacting positively – talks appropriately with classmates, engages in group discussions, takes turns, plays appropriately with peers outside etc.

Child 2 Interacting negatively –speaks inappropriately, disrupts group work, is aggressive (hits, kicks, insults, etc.),

Child 2 No interaction – sits alone

Child 3 Interacting positively – talks appropriately with classmates, engages in group discussions, takes turns, plays appropriately with peers outside etc.

Child 3 Interacting negatively –speaks inappropriately, disrupts group work, is aggressive (hits, kicks, insults, etc.),

Child 3 No interaction – sits alone

Child 4 Interacting positively – talks appropriately with classmates, engages in group discussions, takes turns, plays appropriately with peers outside etc.

Child 4 Interacting negatively –speaks inappropriately, disrupts group work, is aggressive (hits, kicks, insults, etc.),

Child 4 No interaction – sits alone

9. Did any of the observed students provide an individual response during the class? ([response to a question by the teacher, e.g.])

None of the students in the class provided an individual response

No, the observed students never provided an individual response (but other students did)

Yes, one of the observed students provided an individual response Yes, several of the observed students provided an individual response

10. Did any of the observed students ask the teacher a question during the class?

None of the students in the class asked the teacher a question No, the observed students never ask a question (but other students did)

Yes, one of the observed students ask a question Yes, several of the observed students asked a question

Teacher Observation

At the end of the observation, respond to the following questions noting whether teachers carried out these activities and, if so, how frequently they did soduring the lesson.

1. Teacher ensures that student are paying attention before beginning an activity or asking a question

Never Rarely Some times Frequently All the time

Tangerine

Page 6 of 7

2. Teacher speaks clearly and loudly enough to be heard in the back of the class

Never Rarely Some times Frequently All the time

3. Teacher speaks facing the class and his or her mouth is not obstructed

Never Rarely Some times Frequently All the time

4. Teacher writes questions on the board

Never Rarely Some times Frequently All the time

5. Teacher writes lesson objective on the board

Never Rarely Some times Frequently All the time

6. Teacher repeats questions at least once to ensure that VI or HI students have understood the question

Never Rarely Some times Frequently All the time

7. How often did the teacher repeat her or his responses to ensure that the VI or HI students have understood the response?

Never Rarely Some times Frequently All the time

8. How often did the teacher repeat the questions or responses of other students to ensure that the VI or HI students have understood the response?

Never Rarely Some times Frequently All the time

9. How often did the teacher orally describe pictures or illustrations used for the lesson?

Never Rarely Some times Frequently All the time Not applicable for the lesson

10. Did the teacher use a self-made teaching aid to support her instruction?

Yes No

10a. If yes, was this material specifically in support of a child/children with disabilities?

Yes No

10b. If yes, specify what material was used:

11. Did the class use the new student textbooks for reading and writing in Mother Tongue?

Yes No

Tangerine

Page 7 of 7

12.a. Did the teacher use the inclusive multimedia teacher guide on the smartphone for teaching the lessons?

Yes No

12.b. Did the teacher use the new teacher guide for reading and writing in Mother Tongue? ([either in addition to the smartphone or by itself (if the teacherdid not use the smartphone)])

Yes No

12.c. How confident was the teacher in using the IMLP on the smartphone?

Very confident Somewhat confident Not at all confident

13.a. Did the teacher play back any of the audio files from the smartphone for the class or some of the students?

Yes No

13.b. How confident was the teacher in playing back any of the audio files from the smartphone for the class or some of the students?

Very confident Somewhat confident Not at all confident

14.a Did the teacher give the smartphone to a student or groups of students to listen to or access text?

Yes No

14.b. How confident was the teacher in giving the smartphone to a student or groups of students to listen to or access text?

Very confident Somewhat confident Not at all confident

14.c If yes, did the phone distract the other students or cause disruption?

Yes, it distracted the students a lot Yes, it distracted the students a little bit No, it did not distract the students

15. Did the teacher press the ‘Lesson Completed’ button at the end of the lesson?

Yes No Don't know

Tangerine

Page 1 of 6

Student Egra & Questionnaire - English - Endline

Last Updated: 1495120811000

Enumerator Name

Date and Time

Date

Time

Student ID Input

DO NOT press the 'Generate' button below. Enter the learner's code from the list of pre-generatedIDs.BE SURE TO USE ALL CAPS.

If you have pressed 'Generate' by mistake, please delete what appears in the box and then ENTERin the appropriate student code from the list of pre-generated IDs

Student ID Input

Student ID confirmation

Please re-enter the pre-generated student ID

Consent

Good morning. My name is _____ and I live in ______. I’d like to tell you a little bit about myself.

(Number and ages of children; pets; sports; etc)

1. Could you tell me a little about yourself and your family? (Wait for response; if student is reluctant, askquestion 2, but if they seem comfortable continue to verbal consent).2. What do you like to do when you are not in school?

Let me tell you why I am here today. I work with the Ministry of Education and we are trying to understand howchildren that may have difficulty seeing or hearing learn to read. We would like your help in this.Today, we are going to play a reading game. I am going to ask you to read letters, words and a short story out loud.This is NOT a test and it will not affect your grade at school.I will also ask you if you have difficulty seeing or hearing, and a few questions about your classroom.I will NOT write down your name so no one will know these are your answers.Our talk today will take about 20 minutes to complete.

Appendix C

Tangerine

Page 2 of 6

Once again, you do not have to participate if you do not wish to. Once we begin, if you would rather not answer aquestion, that’s all right.Do you have any questions? Are you ready to get started?Does student provide consent? ☐

School Information

RegionZone

SchoolName

School information input

Please enter name of school if "other" selected

Please enter name of woreda if "other" selected

Header Information

F. School Shift

Full day Morning Afternoon

I. Teacher Name

J. Grade

K. Class Section

M. Student Age

6 7 8 9 10 11 12 13 14 15 16 Don't Know

N. Student gender

Boy Girl

Tangerine

Page 3 of 6

O. Screened positive for Vision Impairment

Yes No Don't know

P. Screened positive for Hearing Impairment

Yes No Don't know

Section 1: Letter Sounds

Here is a page full of letters of the English alphabet. Please tell me the SOUNDS of as many letters as youcan—not the NAMES of the letters, but the sounds.For example, the sound of this letter [point to V]: “AH” as in “APPLE”

Let’s practice: tell me the sound of this letter [point to V]:

If the child responds correctly say: Good, the sound of this letter is “VVV.If the child does not respond correctly, say: The sound of this letter is “VVVV.”

Now try another one: tell me the sound of this letter [point to L]:

If the child responds correctly say: Good, the sound of this letter is “LLL.”If the child does not respond correctly, say: The sound of this letter is “LLL.”

Do you understand what you are to do?When I say “Begin,” please sound out the letters as quickly and carefully as you can. Tell me the sound ofthe letters, starting here and continuing this way. [Point to the first letter on the row after the example anddraw your finger across the first line]. If you come to a letter sound you do not know, I will tell it to you. Ifnot, I will keep quiet and listen to you. Ready? Begin.

A I J N S n A t e hI z a V B o h r N Ta C f C S a s O E Ue N t o a e x t o qd L E d G E N o m Oh e K w T i L g y He i r y H l s e T fR y W p U s i l e lR o a E d n D a s lr C n U r T P t m h

Time Remaining

Autostop?

Section 2a. Oral passage reading

Tangerine

Page 4 of 6

My name is Pat. Ilive on a farm withmy mother, father, and brother

Sam. Every year, the landgets very dry before therains come. We watch thesky and wait. One afternoonas I sat outside, I

saw dark clouds. Then somethinghit my head, lightly atfirst and then harder. I

jumped up and ran towardsthe house. The rains had

come at last.

Time Remaining

Autostop?

Section 2b. Oral passage reading

Now I am going to ask you a few questions about the story you just read. Try to answer the questions as well as you can.

1. Where does Pat live?

Correct Incorrect No response

2. Why does the land get dry?

Correct Incorrect No response

3. Why do Pat and his family watch the sky?

Correct Incorrect No response

4. What hit pat on the head?

Correct Incorrect No response

5. How do you think Pat felt when the rains come?

Correct Incorrect No response

Tangerine

Page 5 of 6

Section 3. Pupil context interview. 1st part.

17. Were you absent from school any days last week?

No Yes Do not know/No response

18. [if yes to Question 17] Why were you absent?

I was not absent I was sick There was other work at home I had to take care of a family member

I had no transport, or my transport was late Bad weather Getting to school was too dangerous Being in school is too dangerous

I woke up late I had to take care of siblings I am treated poorly by teachers and other students at school Other (specify)

Don’t know/Refuse

18.a. Other (Specify)

19. Do you have this reading textbook for use at home and classroom? ([enumerator shows color print of cover page of grade 2 student reading textbook])

No Yes Do not know/No response

20. [if yes to Question 19] Did you bring it today?

No Yes Don’t know/No Response

28. Now I am going to ask you about your vision and hearing. When the teacher writes something on the board, do you have any difficulty seeing what s/hewrites?

No Yes Do not know/No response

29. When you look at the textbook do you have any difficulty seeing the letters?

No Yes Do not know/No response

30. When the teacher is speaking do you have any difficulty hearing her or him?

No Yes Do not know/ No response

31. When a classmate responds to the teacher’s question, do you have difficulty hearing their answer?

No Yes Do not know/ No response

32. Now I am going to ask you about your class and your classmates. Do some of your classmates have a disability?

No Yes Do not know/No response

Tangerine

Page 6 of 6

33. Do you have friends who have disabilities?

No Yes Do not know/ No response

34. Last week, how often did your teacher use a phone during the reading and writing lesson?

Every lesson Sometimes Never

35. Last week, did your teaching play back any stories or words from the phone during reading class?

Yes No Don't know

Section 3. Pupil context interview. 2nd part.

36. Do you like school?

No Yes Do not know/ No response

38. Does your teacher make sure that you can see and hear what the teacher and other students say and do?

No Yes Do not know/No response

39. If you can’t see or hear something, can you ask your teacher to repeat it or write it down for you?

No Yes Do not know/No response

41. Last week, during any of your reading lessons, did your teacher ever call on your individually to answer a question?

No Yes Do not know/No response

44. Do you have any special materials to help you in class, such as reading books printed with large letters or devices to help you see or hear better? ([skip ifno hearing or visual impairment])

Yes No

44a. If yes, what?