World Bank Documentdocuments.worldbank.org/curated/en/...ACF Award Confirmation Form ACT Additional Class Teacher AF Additional Financing BANBEIS Bangladesh Bureau of Educational Information

Document of

The World Bank

FOR OFFICIAL USE ONLY

Report No: ICR00004427

IMPLEMENTATION COMPLETION AND RESULTS REPORT CREDIT NUMBER: 4475-BD AND 5325-BD

ON A

CREDIT

IN THE AMOUNT OF SDR 253.1 MILLION (US$395.7 MILLION EQUIVALENT)

TO THE

PEOPLE'S REPUBLIC OF BANGLADESH

FOR THE

BANGLADESH - SECONDARY EDUCATION QUALITY AND ACCESS IMPROVEMENT ( P106161 )

June 11, 2018

Education Global Practice South Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective: December 31, 2017)

Currency Unit = Bangladesh Taka (BDT)

BDT 81.27 = US$1

US$1.42 = SDR 1

FISCAL YEAR July 1 – June 30

ABBREVIATIONS AND ACRONYMS

ADB Asian Development Bank

ACF Award Confirmation Form

ACT Additional Class Teacher

AF Additional Financing

BANBEIS Bangladesh Bureau of Educational Information and Statistics

BSK Bishwo Shahitto Kendro

CPF Country Partnership Framework

DLI Disbursement-Linked Indicator

DLR Disbursement-Linked Result

DPHE Department of Public Heath Engineering

DRH Developing Reading Habit

DSHE Directorate of Secondary and Higher Education

EACM Education Awareness and Community Mobilization

EMF Environmental Management Framework

FSSAP Female Secondary School Assistance Project

FYP Five-Year Plan

LGED Local Government Engineering Department

MOE Ministry of Education

M&E Monitoring and Evaluation

HIES Household Income and Expenditure Survey

ICR Implementation Completion and Results Report

ICT Information and Communication Technology

ISF Improving School Facilities

IRR Internal Rate of Return

ISR Implementation Status and Results Report

KPI Key Performance Indicator

LASI Learning Assessment of Secondary (SEQAEP) Institutions

MEW Monitoring and Evaluation Wing

MIS Management Information System

MMC Madrasah Management Committee

MTR Midterm Review

NPV Net Present Value

OM Operations Manual

PD Project Director

PDO Project Development Objective

PMT Proxy-means Testing

PMTA PMT Administrator

PIU Project Implementation Unit

PRSP Poverty Reduction Strategy Paper

PTA Parents and Teachers Association

RCT Randomized Control Trial

RT Resource Teacher

SDG Sustainable Development Goal

SEPA Sistema de Ejecución de Planes de Adquisiciones (Procurement Plan Execution System)

SEDP Secondary Education Development Program

SEQAEP Secondary Education Quality and Access Improvement Project

SMC School Management Committee

SSC Secondary School Certificate

STEP Systematic Tracking of Exchanges in Procurement

SWAp Sectorwide Approach

TTC Teachers’ Training College

TTL Task Team Leader

USEO Upazila Secondary Education Office

Regional Vice President: Annette Dixon

Country Director: Qimiao Fan

Senior Global Practice Director: Jaime Saavedra Chanduvi

Practice Manager: Keiko Miwa

Task Team Leader(s): Shwetlena Sabarwal

ICR Main Contributor: Shiro Nakata

TABLE OF CONTENTS

DATA SHEET .......................................................................................................................... 1

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5

A. CONTEXT AT APPRAISAL .........................................................................................................5

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 12

II. OUTCOME .................................................................................................................... 14

A. RELEVANCE OF PDOs ............................................................................................................ 14

B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 15

C. EFFICIENCY ........................................................................................................................... 19

D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 21

E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 21

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 22

A. KEY FACTORS DURING PREPARATION ................................................................................... 22

B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 23

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 25

A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 25

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 27

C. BANK PERFORMANCE ........................................................................................................... 29

D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 31

V. LESSONS AND RECOMMENDATIONS ............................................................................. 31

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 35

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 42

ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 45

ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 46

ANNEX 5. BORROWER, CO-FINANCIER, AND OTHER PARTNER/STAKEHOLDER COMMENTS .. 50

ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 51

The World Bank Bangladesh - Secondary Education Quality and Access Improvement ( P106161 )

Page 1 of 84

DATA SHEET

BASIC INFORMATION

Product Information

Project ID Project Name

P106161 Bangladesh - Secondary Education Quality and Access

Improvement

Country Financing Instrument

Bangladesh Investment Project Financing

Original EA Category Revised EA Category

Partial Assessment (B) Partial Assessment (B)

Related Projects

Relationship Project Approval Product Line

Additional Financing P146255-Secondary Education Quality and Access Enhancement Project - AF

03-Dec-2013 IBRD/IDA

Organizations

Borrower Implementing Agency

People's Republic of Bangladesh Directorate of Secondary and Higher Education, MOE

Project Development Objective (PDO) Original PDO

The project development objectives are to improve the quality of secondary education, systematically monitor learning outcomes, and to increase access and equity in project upazilas.


Page 2 of 84

FINANCING

Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$)

World Bank Financing IDA-44750

130,700,000 130,500,446 123,438,835

IDA-53250

265,000,000 265,000,000 243,891,827

Total 395,700,000 395,500,446 367,330,662

Non-World Bank Financing

Borrower 25,000,000 40,000,000 84,600,000

Total 25,000,000 40,000,000 84,600,000

Total Project Cost 420,700,000 435,500,446 451,930,662

KEY DATES

Approval Effectiveness MTR Review Original Closing Actual Closing

31-Jul-2008 04-Sep-2008 12-Jul-2011 30-Jun-2014 31-Dec-2017

RESTRUCTURING AND/OR ADDITIONAL FINANCING

Date(s) Amount Disbursed (US$M) Key Revisions

28-Jul-2011 64.12 Change in Results Framework

26-May-2013 117.17 Reallocation between Disbursement Categories

26-Feb-2014 123.44 Additional Financing

16-May-2016 247.14 Other Change(s)

15-Nov-2017 327.64 Reallocation between Disbursement Categories Other Change(s)

KEY RATINGS

Outcome Bank Performance M&E Quality

Highly Satisfactory Satisfactory High


Page 3 of 84

RATINGS OF PROJECT PERFORMANCE IN ISRs

No. Date ISR Archived DO Rating IP Rating Actual

Disbursements (US$M)

01 19-Sep-2008 Satisfactory Satisfactory 0

02 11-Mar-2009 Satisfactory Moderately Satisfactory 10.07

03 09-Sep-2009 Satisfactory Moderately Satisfactory 14.65

04 16-Mar-2010 Satisfactory Satisfactory 22.52

05 14-Nov-2010 Satisfactory Moderately Satisfactory 47.67

06 06-Jun-2011 Satisfactory Moderately Satisfactory 62.65

07 12-Oct-2011 Satisfactory Moderately Unsatisfactory 70.78

08 06-May-2012 Satisfactory Moderately Satisfactory 82.56

09 06-Nov-2012 Satisfactory Satisfactory 104.57

10 05-Jun-2013 Satisfactory Satisfactory 117.17

11 12-Jan-2014 Satisfactory Satisfactory 123.54

12 05-Aug-2014 Moderately Satisfactory Moderately Satisfactory 164.94

13 18-Mar-2015 Satisfactory Moderately Satisfactory 192.13

14 07-Oct-2015 Satisfactory Satisfactory 222.78

15 12-May-2016 Satisfactory Satisfactory 247.14

16 29-Nov-2016 Satisfactory Satisfactory 292.05

17 15-Jun-2017 Satisfactory Satisfactory 313.70

18 29-Dec-2017 Satisfactory Satisfactory 348.88


Page 4 of 84

SECTORS AND THEMES

Sectors

Major Sector/Sector (%)

Education 100

Public Administration - Education 8

Secondary Education 92

Themes

Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Human Development and Gender 100

Education 100

Access to Education 50

Education Financing 50

ADM STAFF

Role At Approval At ICR

Regional Vice President: Praful C. Patel Annette Dixon

Country Director: Xian Zhu Qimiao Fan

Senior Global Practice Director: Michal J. Rutkowski Jaime Saavedra Chanduvi

Practice Manager: Michelle Riboud Keiko Miwa

Task Team Leader(s): Nazmul Chaudhury, Irajen Appasamy

Shwetlena Sabarwal

ICR Contributing Author: Shiro Nakata


Page 5 of 84

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES

A. CONTEXT AT APPRAISAL

Context

1. The Secondary Education Quality and Access Enhancement Project (SEQAEP) was conceived in the background of mounting pressure on the Government to expand and improve postprimary education to enhance labor productivity and accelerate economic growth. Since the 1990s, Bangladesh has achieved remarkable economic growth and impressive gains in poverty reduction. The gross domestic product per capita grew above 5 percent annually since 2000, while poverty headcount had gone down from 59 percent in 1991 to 40 percent in 2005. Human development outcomes had been equally impressive. Gross enrollment in primary education reached 90 percent by 2006, generating considerable pressure for expansion of secondary education. Student enrollment in secondary education expanded rapidly and reached the gross enrollment ratio of 47 percent by 2007. Much of the increase in enrollment of secondary education was achieved through successful expansion of private provision of secondary education based on a unique public and private partnership model where the Government provides supply-side fiscal incentive to private institutions. Currently, around 98 percent of secondary schools are privately owned and operated. At the same time, to address the demand-side constraint of secondary education, the female secondary stipends program was initiated by the Government. Female enrollment increased from 1.1 million in 1991 to 4 million by 2006. It was a significant achievement that gender parity had been achieved at both primary and secondary education.

2. Despite these noteworthy achievements, significant challenges remained, especially in the following areas:

(a) Low internal efficiency. Student dropout in secondary education was a serious concern. Students after Grade 8 were particularly vulnerable to dropping out due to higher opportunity costs and national examinations. As a result, completion rate of secondary education was only 20 percent.

(b) Poor quality of secondary education. Basic literacy and numeracy competencies were low among secondary students, and there had been very few internationally comparable assessments on secondary student learning.

(c) Out-of-school secondary school-age children especially among the poor. The gross enrollment rate for students in the poorer 50 percent of income group was 38 percent against 75 percent among the richer 50 percent in 2005.

(d) Inadequate institutional capacity at central, regional, and school levels. Management, coordination, and implementation capacity needed to be strengthened at every level.

(e) Inadequate monitoring and evaluation (M&E) capacity at project and ministry levels. Capacity for effective monitoring was weak.

3. Rationale for the World Bank’s support was strong at appraisal and closing. The project had high complementarity with the World Bank’s previous engagement in the sector and consistent alignment with


Page 6 of 84

the country’s and the World Bank’s development priorities. The World Bank had supported the pioneering female secondary stipends program, Female Secondary School Assistance Project (FSSAP), for 15 years since 1993, which laid a strong foundation for the project. The World Bank had also supported the Education Sector Development Support Credit Project in 2004 to promote a range of policy reform agenda. SEQAEP took measures to continue the support for key policy reforms from the predecessor projects. SEQAEP was also best positioned to consolidate and deepen the quality and access gains in primary education generated under the primary education development program, for which the World Bank was one of the major contributors. At the time of appraisal, Bangladesh Country Assistance Strategy 2006–2009 prioritized enhanced social services that reached the poor as well as improvement of quality and relevance of secondary education. Meanwhile, the Government’s Poverty Reduction Strategy Paper (PRSP) 2005–2008 called for a paradigm shift in education from focus on access to emphasis with quality while retaining the focus on equity. SEQAEP continues to be fully aligned with the World Bank’s priorities. Improved equity in access and quality of education is one of the Country Partnership Framework (CPF) objectives which is identified as essential for high productivity-led growth. The CPF also commits to closing the gender gap and enhancing the quality and relevance of education. The seventh five-year plan (7th FYP) of the Government highlights priorities in the education sector, including access, retention, and equity and gender discrimination and particularly emphasizes on improving quality and relevance of education to contribute to skills development and labor productivity enhancement in the economy.

Theory of Change (Results Chain)

4. Theory of change of SEQAEP is illustrated in figure 1. SEQAEP’s development outcomes will ultimately contribute to achievement of national development priorities. The project aimed to achieve three main objectives, namely, improved quality of secondary education, establishment of a student learning assessment system, and improved equitable access to secondary education. The project was designed to address the five key challenges of the sector discussed above through a range of interventions with innovative features that would eventually be mainstreamed in the secondary education sector in Bangladesh. First, SEQAEP approached quality challenges through three main initiatives including provision of incentives to students and teachers, organizing additional classes in priority subjects, and expanding book-reading programs. Quality initiatives were underpinned by an ambitious SEQAEP/nationwide learning assessment program which measured students’ competency levels every two years in priority subjects and by strengthening of M&E capacity of secondary education subsector, which would contribute to greater accountability in the sector. To increase equitable access, the project introduced a selection mechanism based on proxy-means testing (PMT) in its pro-poor stipend program, a drastic shift away from gender and geography-based targeting of the FSSAP, and addressed the long-standing issue of substandard school facilities through improving the school facility program. At the same time, the project also addressed institutional capacity challenges and governance risks at local education office and school levels through strengthening their management capacity and community mobilization. These were also expected to contribute to improving access and quality.


Page 7 of 84

Figure 1. Theory of Change: Secondary Education Quality Enhancement Project

Note: ACT = Additional Class Teachers; DRH = Developing Reading Habit; EACM = Education Awareness and Community Mobilization; LASI = Learning Assessment of Secondary (SEQAEP) Institutions; MEW = Monitoring and Evaluation Wing; PTA = Parent Teacher Association; SMC = School Management Committee; SSC = Secondary School Certificate; USEO = Upazila Secondary Education Office.

Project Development Objectives (PDOs)

5. The PDO is “to assist the Government of Bangladesh in improving the quality of its secondary education and monitoring of learning outcomes systematically, and in increasing access and equity in education in Project Upazilas.”

Key Expected Outcomes and Outcome Indicators

6. The PDO has three key objectives with the following outcome key performance indicators (KPIs):

Activity Outputs PDOs/Outcomes Long Term Outcomes

Improved quality of education

• Increased completion rate

• Higher number of students appearing for SSC exam

• Increased SSC pass rate

Established learning assessment system

• Monitoring the learning level

Improved equitable access

• Increased enrollment of poor students

• Higher number of poor students appearing for SSC exam

• More balanced gender ratio in enrollment

Quality and relevant

secondary education

Enhanced foundational

skills

Improved labor productivity

Students/teachers incentivized for better teaching and learning

Contact hours and quality of teaching enhanced for mathematics, English, and science

Reading habit developed, and reading and writing skills improved

School-level management capacity and community participation enhanced

School environment improved especially for girls

Parents incentivized for continued schooling, and financial constraints alleviated

Learning performance assessed regularly

Incentive program

ACT program

DRH program

SMC/PTA/EACM

LASI program

MEW strengthening

PMT-based stipend

USEO strengthening Upazila-level management capacity improved

Improving school facility program


Page 8 of 84

Objective 1: To improve the quality of secondary education

• KPI 1: Completion rate in Grade 10 in project upazila1

(a) Proportion of 16–20 year-old primary completers who have completed Grade 10

(b) Percentage of Grade 6 entrants who have passed SSC exam

• KPI 3: SSC pass rate, by poor/non-poor (discontinued at Additional Financing [AF])

Objective 2: To establish the monitoring system of learning outcomes

• KPI 4: Monitoring learning levels in secondary schools

Objective 3: To increase access and equity in secondary education in upazilas targeted by the project

• KPI 2: Number of students appearing in SSC exams in project areas

(a) Total students

(b) Poor

• KPI 5: Gender parity ratio (male/female) in Grade 6–10 enrollment in project areas

• KPI 6: Share of poor children in total secondary enrollment in project areas

• KPI 7: Total number of direct project beneficiary (core indicator requirement)

7. Quality and relevance of outcome indicators. All aspects of the PDOs are accounted for by one or more KPIs and supported by relevant intermediate indicators (see annex 1-B). All the KPIs are relevant and reliable indicators for measuring the achievement of the objectives. Determining the outcome-level indicator for education quality in the absence of standardized assessment is always a challenge; however, the completion rate captures the aggregate impact of improved education quality through improved retention and academic performance up to the final grade. Furthermore, to ensure attributability, the project conducted rigorous impact evaluations to establish causal links of project activities to observed improvements in academic performance and retention in schools. One minor caveat would be that KPI 1 (a) and KPI 6 were calculated using the datasets of nationally representative household surveys, which occurs only intermittently, and that made regular tracking of the indicator somewhat challenging. Based on these assessments, quality and relevance of outcome indicators are deemed satisfactory.

1 The definition of completion is passing the SSC exam which is equivalent to completing Grade 10. The project used two methods to calculate the completion rate. KPI 1 (a) calculates the rate as the percentage of primary school leavers (16–20 years old) who have completed Grade 10, in the SEQAEP targeted areas, by using the nationally representative household surveys. KPI 1 (b), on the other hand, basically traces the same cohorts of secondary students from the first year of secondary education up to the SSC examination at Grade 10. By using two indicators, the project was able to triangulate the results, account for calculation difficulties such as internal migration of students, and assess the overall impact in the targeted areas.


Page 9 of 84

Components

8. SEQAEP has four components and the Disbursement-Linked Indicator (DLI) program that contributed to PDO achievement. The description in the following paragraphs is derived from the AF Project Paper.

Component 1: Improving Education Quality and Monitoring Learning Levels (Estimated Cost: US$109.7 million; Actual Cost: US$93.8 million)

Subcomponent 1.1: Incentive Awards to Students, Teachers, and Institutions

9. This subcomponent provided monetary incentives to students, teachers, and institutions to improve and retain higher levels of student learning achievement and retention. Incentives provided are (a) Best Student Awards, which included Achievement Awards to best-performing students in each beneficiary institution, (b) SSC Pass Award to poor students (that is, stipend beneficiaries) who passed the SSC/equivalent examination, and (c) SSC Institutional Awards for achieving excellent pass rates for SSC/equivalent examination and achieving a high retention rate (90 percent) from Grades 6 to 9.

Subcomponent 1.2: Support for English Language, Mathematics and Science Subject Learning and Teaching

10. This subcomponent aims to enhance classroom teaching and learning in English, mathematics, and science. Science was also added at the AF. English, mathematics, and science were the subjects with the poorest outcomes in SSC/equivalent examinations. The subcomponent financed (a) provision of additional class grants for hiring ACTs, (b) organization of additional classes, and (c) provision of management. This subcomponent evolved over the project period in design and coverage to maximize effectiveness of the intervention. Under the AF, it was expanded to support provision of ACTs and targeted 2,000 institutions by the project end. SMCs/Madrasa Management Committees (MMCs) were tasked with selection and contracting of ACTs based on enhanced selection criteria and monitoring. The ACTs were responsible to take not only additional classes but also to teach regular subject classes where there’s no subject teacher. It was envisioned that ACTs may possibly be regularized (that is, brought under the Monthly Pay Order (MPO)) in the Ministry of Education (MOE) system; however, that decision is yet to be reached at the time of the Implementation Completion and Results Report (ICR).

Subcomponent 1.3: Developing the Reading Habit (DRH)

11. This subcomponent aimed to develop reading habits among students through school-based reading programs. Activities financed under this subcomponent include (a) setting up of a reading program facility at school for Grades 6–10, (b) provision of age- and ability-appropriate books in Bangla and English, (c) training of librarians for management of the reading program, (d) annual reading habit test, (e) annual Book Award incentive system based on the test result, and (f) grant to finance incentive for librarians and coordinator teachers (added under the AF). SEQAEP partnered with a specialized agency, Bishwo Shahitto Kendro (BSK), to establish and manage developing the reading habit program. The partner agency reported to SEQAEP monthly, and SEQAEP had an oversight responsibility for the program. Each institution was expected to have at least 100 student readers.


Page 10 of 84

Subcomponent 1.4: Assessment of Education Quality

12. The aim of this subcomponent was to systematically measure the student learning outcome through internationally comparable and competency-based numeracy and literacy test on a sample basis. It was also envisioned that in the long run the appropriate capacity would be built to mainstream the learning assessment system. The subcomponent financed the development of instruments for English, mathematics, and Bangla (added under the AF), undertaking of the learning assessments, and dissemination of the assessment results to stakeholders at every level. The MEW is the focal agency to carry out LASI. Instrument design and implementation of the assessments were outsourced to external specialized agencies under the overall supervision and coordination by the MEW. During the project, LASI was conducted four times, once in every two years (2011, 2013, 2015, and 2017). LASI was expanded to be nationally representative since 2015. Implementation and dissemination of LASI were also supported as one of the DLIs.

Component 2: Improving Equity and Access (Expected Cost: US$286.3 million; Actual Cost: US$285.8

million)

Subcomponent 2.1: PMT-based Stipends and Tuition to the Poor

13. The objective of this subcomponent was to increase access and retention of poor boys and girls in secondary education through provision of stipends and tuition support. Selection of beneficiaries was pro-poor based on PMT with enhanced coverage for girls. Poor students, belonging to the poorest 40 percent of household in an upazila, received stipends and tuition support. In addition, female students who belong to the 50th to 80th percentile received only tuition support. The Local Government Engineering Department (LGED) was preselected as the PMT administrator (PMTA). Agrani Bank, which was the disbursing agency in the FSSAP, was also preselected to be responsible for the disbursement of stipends and tuition support for SEQAEP. To be eligible for stipends and tuition support, beneficiaries are expected to meet three basic conditions: (a) maintaining 75 percent attendance rate, (b) maintaining at least 33 percent pass marks in final examinations (replaced with successful promotion to the next grade), and (c) remaining unmarried. It was also envisioned that SEQAEP and the cash transfer program of social protection would work together to develop a national targeting system; however, this did not materialize due to delayed establishment of the poverty database2. PMT stipend beneficiaries are selected through rigorous PMT-based procedure: (a) invitation of PMT application, (b) PMT booth operation to collect filled applications, (c) sample-based validation through household visit surveys, (d) processing of application data, (e) publication of beneficiary lists, (f) enrollment confirmation, (g) processing of appeals, and (h) Award Confirmation Form (ACF) preparation.

Subcomponent 2.2: General Stipends and Tuition Program

14. This subcomponent was to ensure smooth transition from the existing gender-based program to a PMT-based program.

2 Possible harmonization of poverty targeting mechanism continues to be explored under the new program.


Page 11 of 84

Subcomponent 2.3: Improving School Facilities (ISF)

15. The objective of this subcomponent was to supply safe water and sanitation facilities to selected number of institutions, based on needs assessment, to improve student retention and health outcomes. The project provided grants for water and sanitation facility construction and testing of tube wells for arsenic, salinity, and manganese contamination. Planning and construction of facilities were implemented by SMCs with school-level community participation. The ISF was implemented with a cost-sharing mechanism where the schools contribute 20 percent of the cost. The Department of Public Health Engineering (DPHE) provided technical services for safe water testing, undertaking awareness raising programs, and advisory for construction works for water and sanitation facilities.

Component 3: Institutional Capacity Strengthening (Expected Cost: US$31.3 million; Actual Cost:

US$30.5 million)

Subcomponent 3.1: Project Management

16. This subcomponent was to establish the SEQAEP Unit for project management and the MEW under the Directorate of Secondary and Higher Education (DSHE) for M&E. Establishment of the MEW as a permanent wing of the DSHE would strengthen the M&E capacity to monitor and evaluate both the SEQAEP activities and DSHE’s other programs.

Subcomponent 3.2: Institutional Capacity Building

17. The capacity of the MOE at central, upazila, and community levels was to be strengthened through staff trainings and workshops. A number of in-country and overseas trainings were conducted targeting government officials at different levels.

Subcomponent 3.3: School Management Accountability

18. This subcomponent supported the strengthening of the SMC, MMC, and PTA to increase accountability and transparency at the school level. The project supported the establishment and capacity building of SMCs and PTAs. The SMCs and PTAs were responsible for conducting social audits to enhance accountability. The project provided (a) information and communication technology (ICT) support grant of BDT 800 per month for use of Internet in monitoring and communication and (b) social audit grant of BDT 5,000 per year to all PTAs to organize parent meetings and disseminate school performance information through school report card.

Subcomponent 3.4: Education Awareness and Community Mobilization

19. This sub-component aimed to increase awareness among all key stakeholders, especially school and community beneficiaries. Two types of activities (awareness raising program and community mobilization) were supported through technical assistance and multimedia campaigns. In addition, a dedicated grant of taka 5,000 per year to all PTAs to carry out community mobilization activities including gender awareness campaigns and dropout prevention.


Page 12 of 84

Component 4: Monitoring and Evaluation (Expected Cost: US$8.7 million; Actual Cost: US$4.2 million)3

Subcomponent 4.1: Monitoring

20. This subcomponent would strengthen the capacity to monitor through systematically managing, integrating, and using various data streams and databases. Supported activities include (a) results monitoring (KPI and other indicators), (b) annual census of secondary institutions, (c) PMT validation, and (d) compliance verifications on institutions and stipend beneficiaries.

Subcomponent 4.2: Evaluation

21. This subcomponent aimed to undertake rigorous impact evaluation studies on the project’s innovative interventions such as incentive program, additional classes, reading programs, and PMT-based stipend program.

Disbursement-Linked Indicator (DLI) program (Expected Cost: US$50.0 million; Actual Cost: US$17.56 million)

• DLI 1: Learning Assessment. This DLI aimed to enhance the implementation and dissemination of LASI.

• DLI 2: Institutionalization of MEW. This DLI aimed to facilitate regularization of the MEW under revenue budget.

• DLI 3: M&E activities. This DLI aimed to ensure implementation of key monitoring activities including impact evaluation, PMT verification surveys, and compliance surveys.

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE)

Revised PDOs and Outcome Targets

22. PDOs were not revised during implementation. Outcome indicators and targets were revised during the midterm review (MTR) and the AF for refinement (see Key Expected Outcomes and Outcome Indicators section and to account for expanded scope and project period.

Revised PDO Indicators

23. Refinements in KPIs during implementation. At the MTR (July 12––28, 2011), several refinements were introduced to finetune the indicators with regard to definition of indicators and sources of data, including (a) refinement of KPI 1 (completion rate) with the addition of a sub-indicator: percent of Grade 6 entrants who passed SSC exam, (b) introduction of sub-indicators for ‘Poor’ to KPI 2 (students appearing SSC), and (c) clarification of data source for KPI 6 (share of poor children). At the time of the AF (February

3 Budget was underspent chiefly due to overbudgeting as some of the monitoring activities were novel in the sector. The MEW completed all assigned tasks satisfactorily. However, the budget provision for monitoring activities was overbudgeted in every activity except LASI. Specially, impact evaluation studies, PMT validation and compliance verification, MIS Cell operation costs, local training and foreign training, individual consultants, and operating cost were executed at lower costs.


Page 13 of 84

26, 2014), KPI 3 (SSC pass rate) was discontinued and shifted to the intermediate indicator because SSC examinations are carried out by different regional boards without the standardization across boards and years, and KPI 7 (number of beneficiaries) was added as a core indicator.

Revised Components

24. Enhanced project components under the AF. The AF extended the project coverage to additional 90 upazilas and made substantial revisions in the project components, including

(a) Component 1. Expansion of institutional achievement award to nationwide coverage, addition of science to English and mathematics under the ACT, expansion of the DRH program to all project institutions, and extension of LASI with national coverage;

(b) Component 2. Improvement in the PMT stipend program (by updating of the PMT scoring formula, modified booth operations to minimize exclusion and inclusion errors, and use of student cash card for more efficient stipends disbursement);

(c) Component 3. Strengthening of SMC and PTAs through the provision of SMC and PTA grants for school-level ICT monitoring, social audits, education awareness, and community mobilization activities

25. Introduction of the DLI program. At the AF, DLIs worth a total of US$50 million (later revised to US$17.56) were introduced exclusively focusing on M&E. The DLIs were aimed to provide greater incentives for the MOE to strengthen and institutionalize the M&E system which was crucial for capacity development of the ministry. Specifically, three areas of DLI program were (a) periodic undertaking of nationally representative student learning assessments, (b) institutionalization of the MEW under revenue budget, and (c) implementation of M&E surveys.

Other Changes

26. Other noteworthy changes in the project design include the following:

• Additional Financing. In February 2014, Financing Agreement (FA) for an AF of US$265 million was signed, with an extension of project period up to December 2017. The AF was expected to (a) extend the project activities to more upazilas and more students; (b) scale up successful interventions toward nationwide rollout; and, importantly, (c) allow adequate time for mainstreaming of successful interventions under a harmonized sectorwide program that the ministry intended to launch by January 2018. At the time of the AF, most of the outcome indicators were met (five out of six), and 100 percent of the original IDA credit had been disbursed. As such, the project could have been successfully wrapped up as per the original schedule (that is, Aug 15, 2008–June 30, 2014); however, the rationale for the AF was compelling, especially in the light of synchronization with and preparation for the foreseen transition toward a harmonized sectorwide secondary education development program.


Page 14 of 84

• Addition of 35 upazilas. At the AF, the project was expanded to cover a total of 215 upazilas (out of the 484 rural upazilas in Bangladesh). In 2015, the MOE requested to include additional 35 upazilas without cost implication due to the smaller-than-expected number of institutions in 90 new upazilas that were added in the AF. As a result, the total number of target upazilas of SEQAEP became 250 upazilas.

• Restructuring. There were a total of five restructurings of the project, including the AF. Two of them involved reallocations of loan proceeds across expenditure categories which contributed to enhanced flexibility and effectiveness of project operation by diverting more fund to fast-moving high-performing activities. Other restructuring elements include revision in the Results Framework and introduction of retroactive financing clause.

Table 1. Changes in the Project Design

No. Approved Date Description of Changes Rationale for Changes

1 July 28, 2011 Change in Results Framework KPIs and intermediate indicators revised based on MTR recommendations

2 May 26, 2013 Reallocation between disbursement categories

Increased financing need for the PMT stipend program due to increased beneficiaries

3 February 26, 2014

Additional financing To extend successful intervention to more areas and prepare for the harmonized sectorwide program

4 May 16, 2016 Other change(s) Amendment in the FA to allow retroactive financing under Category (5)

5 November 15, 2017

Reallocation between disbursement categories

Adjustment to fill the financing gap for the stipend and grant programs

Rationale for Changes and Their Implication on the Original Theory of Change

27. The changes discussed above do not affect the project’s theory of change as the changes were mainly to scale up the project activities, refine indicators, and shift fund across subcomponents according to actual expenditures and did not involve changes in the composition of project activities.

II. OUTCOME

A. RELEVANCE OF PDOs

Assessment of Relevance of PDOs and Rating

28. Relevance of PDOs is rated as High. The PDOs are fully aligned with the current Government of Bangladesh (GoB) strategy and CPF priorities. The PDOs are fully in alignment with the 7th FYP of the GoB. As the Government shifts more attention to higher levels of education and skills acquisition, expansion of access and improvement of quality in secondary education is one of the educational priority areas of the FYP. The project addressed several key concerns of the MOE as stated in the National Education Policy, including low student learning achievement, deteriorating education quality in the face of booming enrollment, inadequate pedagogical skills of teachers, and access gaps for the poor and girls. Furthermore, the current CPF FY16–FY20 of Bangladesh identifies human development as one of the foundational


Page 15 of 84

priorities and prerequisite for faster job creation and growth. Under Focus Area 2, the CPF promotes improved equity in access and quality of education (Objective 2.1). SEQAEP's development objectives—quality and access enhancement in secondary education with increased enrollment especially among disadvantaged and improved student learning outcomes—are fully in support of the CPF objective.

29. In addition, reading skills and basic mathematics proficiency have been increasingly seen as core foundational skills that support quality student learning experience and outcomes. The Sustainable Development Goals (SDGs) 4 specifically targets achievement of minimum proficiency in reading and mathematics for all basic education students (SDG Indicator 4.1.1). The project’s innovative interventions, ACT and DRH programs, are directly relevant to and have made significant contribution to development of reading and mathematics proficiencies among secondary school children.

B. ACHIEVEMENT OF PDOs (EFFICACY)

Assessment of Achievement of Each Objective/Outcome

30. As discussed above, there are three main outcomes that the project set out to achieve. Ratings of achievements of each objective and their justifications are discussed in the following paragraphs.

31. Improvement of quality of secondary education (Objective 1). Achievement of quality objective is rated as High. KPIs relevant to quality performance of the project are KPI 1 (completion rate). All the sub-indicators for KPI 1 have exceeded the end of project targets. Particularly impressive is improvement in completion rate. Before the project, less than 30 percent of Grade 6 entrants were expected to complete Grade 10. The ratio has more than doubled to 60 percent by the end of the project, making a significant dent in tackling the low internal efficiency of secondary education, which was a major challenge in Bangladesh. As discussed earlier, the project has conducted rigorous impact evaluations that established causalities and attributability of the project interventions to the positive changes in academic scores and student retention in school. In addition to the improvement in completion rate, overall student learning outcomes have also been improving. The results from LASI suggest that secondary students in the project-supported areas have improved their learning performance especially in English and Bangla subjects.4

32. The intermediate indicators indicate that most of the quality-related activities have consistently met or exceeded their targets. Three main project activities that supported quality improvement include (a) incentive program, (b) AC program, and (c) DRH program (see figure 1 for the Theory of Change). In 2016, the incentive program awarded SSC Pass Awards to 148,000 students annually against the target of 105,000 students and provided Institution Awards to 1,402 institutions annually against the target of 1,440 institutions. The DRH program enrolled 2,030,000 student members in reading clubs annually against the target of 1,310,000 student members. The AC program, on the other hand, has experienced uneven performance over the years. The number of additional classes conducted in mathematics, English, and science has fluctuated widely from one year to the next. This fluctuation was mainly due to

4 Summary of the comparison between LASI 2013 and 2015 in Grade 6: The mean achievement for Bangla for 2015 was 370 while in 2013, it was 352. The learning growth between years was Moderate. The mean achievement for English for 2015 was 319 while in 2013, it was 290. There exists a significant learning growth between years for English reading. The mean achievement for mathematics in 2015 was 374 in comparison to 369 in 2013.


Page 16 of 84

operational challenges deriving from the stringent selection criteria applied to recruitment of ACTs5 to ensure placement of competent and stable ACTs, as well as capacity constraints of schools for hiring and deploying resource teachers (RTs)/ACTs to schools. Despite the challenges, a significant number of additional classes were implemented, amounting to a total of around 3.2 million additional classes since project start, at 2,032 disadvantaged institutions.

33. Quality impacts of these interventions have also been adequately substantiated by evidences from evaluation studies. Two of the quality-oriented interventions (AC and DRH programs) were evaluated rigorously using Randomized Control Trial (RCT) for their impacts on students learning performance in English, mathematics, and Bangla6 (see annex 7-5 for more information).7 The impact evaluation studies concluded that both the AC and DRH programs have a positive impact on student learning. In case of the AC program, there is an overall significant positive impact in English and mathematics. The impact evaluation shows that the AC program improved the assessment score by as much as about 9 percent points for mathematics and 10 percent points for English. By gender, the impact of AC program is mainly for girls in both subjects. In case of the DRH, there is a significant positive impact for both boys and girls in Bangla. The IE shows that the DRH program improved the assessment score by about 5.5 percent points in Bangla. In English, the positive impact was found only for girls. In the DRH program, over 75 percent of books for reading were provided in Bangla. This could explain the larger impact observed in Bangla compared to English. The incentive program was not evaluated at the same level of rigorousness; however, the project undertook an ex post assessment in 2013 that evaluated the effectiveness of the incentive programs more qualitatively. The assessment identified that the incentive program reached the intended beneficiaries, was managed properly though with delays in disbursement, and created a significant positive impact on repetition and dropout of students as well as on the community’s perception.

34. Moving forward, both the ACT and DRH programs will be continued under the new SEDP. The MOE plans to retain the existing ACTs and recruit new ACTs based on needs and merit under the new program.

35. It should also be noted that secondary education sector had been supported by other development partners. Asian Development Bank (ADB), another major financer of the sector, supported a project with a demarcation of responsibility with SEQAEP in terms of coverage of upazila. Other development partners of the sector include Korean International Cooperation Agency and United Nations Educational, Scientific, and Cultural Organization which mostly provided more focused technical and financial assistance.

5 Strict eligibility criteria for recruitment were enforced in the ACT program, which were (a) having demonstrated meritorious academic performance in Bachelor’s Degree with Honor’s (50 percent marks or above in all three subjects) and (b) being a permanent resident of the upazila or adjoining upazila where institutions were located. 6 Science is differentiated into multiple specialized subjects depending on selection of stream in secondary education. As such it was not targeted in LASI and IEs. 7 The impact evaluation design was an RCT. In 20 randomly selected upazilas, 200 schools were randomly selected, from which 100 schools were randomly assigned to treatment and the other 100 were assigned to the control group. Treatment schools started receiving the intervention in 2015 while the control schools were not to receive intervention before 2017. In each sample school, 15 students from Grades 6 and 15 students from Grade 8 were randomly selected. To assess impact, a baseline survey was conducted in 2014 in sample schools before the intervention started. A follow-up survey was conducted two years later in 2016 covering the baseline sample.


Page 17 of 84

36. Establishment of monitoring system of learning performance (Objective 2). Achievement of this objective is rated as High. The KPI for this objective is KPI 3 (Monitoring learning levels) and was fully achieved. Over the project period, four rounds of LASI were carried out by the project (led by the MEW) and disseminated nationwide; two earlier LASIs derived their samples from SEQAEP-supported upazilas (in 2011 and 2013) only while two more recent LASI rounds covered the whole country (in 2015 and 2017), progressively expanding the scope and refining the methodologies. The project achieved all the Disbursement-Linked Results (DLRs) of the DLI for the learning assessment system. Each round of LASI produced a comprehensive report which is publicly available on the website for public consumption and policy discussion. Furthermore, LASI has been successfully institutionalized within the institutional framework of the DSHE. The MEW—to be one of the permanent wings of the DSHE—is officially tasked with undertaking of LASI as one of its institutional mandates. It should be noted here that in the next sectorwide program, LASI will be built more deeply into the system. LASI (to be renamed as National Secondary Student Assessment) is expected to be a data source for the program KPI for student learning performance for mathematics, English, Bangla, and science, which the MOE will constantly monitor and hold itself accountable for through DLIs.

37. Increasing access and equity in secondary education in targeted upazilas (Objective 3). Achievement of this objective is rated as High. The KPIs for this objective include KPI 2 (Students appearing SSC), KPI 4 (Gender parity), KPI 5 (Percent of poor students), and KPI 6 (Number of beneficiaries). All the KPIs, except for KPI 4 (Gender parity), were achieved or exceeded the targets substantially. Gender parity narrowly missed the target (90 boys as opposed to targeted 92 boys for every 100 girls). However, the project also worked intensively to improve access and retention of girls through its interventions; thus, girls’ enrollment was enhanced successfully along with increased boys’ enrollment. As such, this minor shortfall in the gender parity KPI would not be deemed to compromise the PDO achievement.

38. This objective was supported mainly by two project interventions: (a) PMT-based stipend program and (b) ISF program. The PMT stipend program is the best performing activity of the project and has exceeded the targets by a considerable margin. By 2017, the stipend program benefited a cumulative total of around 10.3 million student years against the target of 5.4 million student years—an overachievement of 190 percent. This was partly driven by its own success of the project in attracting more students into secondary education and by assigning stipend to a larger-than-expected share of students (that is, around 50 percent of students received stipends while the expectation was to be around 40 percent). For water and sanitation facilities under the ISF, nearly all the SEQAEP schools have safe drinking water facilities and separate latrine for boys and girls (96 percent for water facilities and 98 percent for separate latrine). Maintenance of school facilities is a constant challenge in Bangladesh. To address the issue, the project initiated user groups at institutions which are expected to take care of the facilities.

39. A rigorous impact evaluation was conducted to assess the impact of the project’s stipend program.8 Based on analysis of baseline and follow-up survey data, the evaluation revealed a significant

8 Detailed household and school-level baseline surveys were carried out in late 2008 before any project intervention under SEQAEP. The sampling process and survey administration include the following steps. First, 10 upazilas were randomly selected from 61 upzilas of SEQAEP Phase-1 intervention and 10 upazilas from 61 upazilas of SEQAEP Phase-2 intervention. The probability proportional to size method of random sampling was used. Thus, the first 20 upazilas were selected in total from 121 SEQAEP intervention areas. Similarly, 10 upazilas were randomly selected from non-SEQAEP intervention area. Propensity score matching was used to carefully find comparable treated and control groups by matching on key household/individual characteristics observed at baseline so that both treatment and control households are made equally likely to be selected to participate in the pro-poor stipends program.


Page 18 of 84

impact of the stipend scheme on secondary school enrollment. The effects are significant for both genders; however, the impact is more pronounced for poor boys. Eligible boys are estimated to be 21 percentage points more likely to be enrolled in secondary schools, while boys from the poorest two income quintiles are 28 percentage points more likely to be enrolled. For girls, the average effect of the stipend is estimated to be around 19 percentage points increase in school enrollment. The stipend program was not found to have a significant effect on student learning performance, which is consistent with findings from cash transfer programs in other countries.

40. It should also be highlighted that the achievement of quality and access/equity objectives were accelerated by the project intervention for strengthening institutional capacity at the local level. One of the major achievements of the project in institutional capacity building was establishment of functional PTA at almost all the targeted institutions (98 percent). PTAs meet at least nine times a year to discuss school management issues and actively participate in community awareness campaign for promoting schooling and PMT stipend program. The project provided training to PTA members (a total of around 67,000 PTA members trained) and grants to support PTA meetings and awareness activities through a social audit grant (provided to around 10,500 institutions), ICT grants (around 7,600 institutions), and EACM grants (around 10,500 institutions). This PTA system will be continued under the new program.

Justification of Overall Efficacy Rating

41. Overall achievement of the PDOs is rated as High based on the satisfactory achievement of all the objectives discussed above and well-documented quantitative and qualitative evidences of improvements accomplished under each subcomponent. In terms of achievement of outcome indicators, all the end of project targets of the KPIs have been achieved (see figure 2) except for one indicator (KPI 4) which missed the target narrowly. This unmet gap is deemed insignificant as discussed above and do not substantially undercut the project’s outcome performance because increase of boys’ enrollment was well on track while at the same time the project has successfully undertaken many activities to promote girls’ enrollment—both of which are highly positive development from the PDOs’ perspective.

Table 2. Achievements of Key Performance Indicators

Outcome Indicators Baseline End-of-Project Target

Achieved Status

1 Completion rate in Grade 10 in project upazilas (%) (a) Proportion of 16–20 year-old primary completers who have completed Grade 10 (b) Percent of Grade 6 entrants who have passed the SSC exam

20

28

33

50

36

60

Achieved Exceeded

2 No. of students appearing in SSC (in thousands) (a) Total students (b) Poor

187 —

485 160

578 228

Exceeded Exceeded

3 Monitoring learning levels in secondary schools Initial stage

Grade 6 and 8 nationally in Bangla, mathematics, and English

Grade 6 and 8 Nationally in Bangla, mathematics, and English

Achieved


Page 19 of 84

Outcome Indicators Baseline End-of-Project Target

Achieved Status

4 Gender parity (male/female) in enrollment in Grade 6

0.82 0.92 0.90 Nearly achieved

5 Percentage of poor children enrolled in secondary school (%)

30 41 42 Achieved

6 Total number of direct project beneficiary students: (a) Total number (‘000) (b) Share of females (%)

4,242 53

4,991 53

Exceeded Achieved

42. In addition, all the three DLIs have been fully achieved (see figure 3) to support three key priority actions in learning assessment and strengthening of M&E. Under three DLIs, nationally representative learning assessment was carried out and disseminated, the MEW has been placed under the revenue budget, and the MEW undertook impact evaluations and validation and compliance verification surveys. These achievements are directly linked with the project objective of establishment the monitoring system of learning outcomes.

Table 3. Achievements of DLIs

Area Indicator DLI Values in

US$ equivalent

Status and Summary of Achievements

Learning Assessment

DLI1: Learning assessment carried out nationally

US$1.19 million (3 DLRs)

Fully achieved. Learning assessment on nationally representative sample carried out and disseminated.

Institutionalization of the MEW

DLI2: MEW institutionalized and financed through revenue budget


Fully achieved. The MEW established as a permanent wing of the DSHE, fully staffed as per the organogram, and placed under the revenue budget.

Monitoring and Evaluation Activities

DLI3: Monitoring and evaluation surveys conducted as scheduled


Fully achieved. Impact evaluation baseline and follow-up conducted; PMT validation and compliance verification surveys conducted.

Total US$17.56 million (12 DLRs)

Note: The DLI value is after the restructuring in November 2017, reduced from the original allocation of US$50 million. This restructuring was needed due to the need to fill the financing gaps in other expenditure categories. It took place after the DLIs have been achieved; as such, it did not hamper the incentive for achievement of DLIs.

C. EFFICIENCY

Assessment of Efficiency and Rating

43. Efficiency of the project investment is rated as Substantial. Economic analysis at the appraisal and AF both indicated healthy high returns on investment for SEQAEP. The appraisal mission calculated the internal rate of return (IRR) to be around 22 percent based on expected benefits from increases in secondary education completers, employment probability, and future wages. At the AF, the IRR was again estimated to be around 22 percent by using a similar assumption about project benefits. Based on these cost-benefit assessment, the project was expected to be a very healthy investment for the country. New


Page 20 of 84

investment from the AF has mostly gone to finance expansion of coverage, rather than filling the financing gap, of selected high-performing interventions such as PMT to achieve project outcomes at a higher level; therefore, the AF did not compromise the cost efficiency of the project. Finally, this ICR conducted a cost-benefit analysis using the latest available information about project cost and benefits. The IRR with the base scenario was calculated to be around 22.6 percent, equivalent to the IRRs of the previous cost-benefit analysis.

44. The project was managed efficiently. There has been no cost escalation in management budget for the Project Implementation Unit (PIU) other than a reasonable increase to support the AF operation. The project disbursed funding efficiently. By the time of AF effectiveness, the original credit had been disbursed 100 percent. The partnership with specialized agencies greatly boosted efficiency of implementation. Disbursement and reconciliation of a large number of PMT stipend and other grants were expedited by use of adaptable financial system such as mobile banking with the support of Agrani Bank. The LGED established a massive and integrated management information system (MIS) for processing of award certification forms and payment authorization. Without support from these powerful agencies with substantial grassroots level presence, handling of millions of small transactions would have severely hampered the project implementation. In fiduciary aspects, use of off-the-shelf accounting software from the early stage contributed to regular and coherent unaudited interim financial reports. In addition, flexible use of the school grant system and central procurement allowed the project to efficiently undertake procurement and financial management activities such as procurement of books for DRH program and salary payment for ACTs.

45. On the side of the World Bank team, one significant advantage of SEQAEP was minimal turnover of task team leaders (TTLs) and retention of institutional memory. TTLs were changed only three times, and the transitions were supported by the periods of overlapping through co-TTL-ship and fully supported by the country-based team. The longest-serving TTL had been a team member from the onset of the project. This, along with the dedicated team efforts over the years by all the team members, has contributed to consistency in project leadership and retention of relevant expertise and institutional knowledge about the project, which likely has contributed to efficient and effective implementation support.

46. The only reason for lowering to the “Efficiency” rating to Substantial, is due to some inclusion errors that occurred in the Subcomponent 2.1—PMT-based stipend program. By the end of the project, the stipend program exceeded the cumulative intake target by 90 percent (10.2 million student years against the target of 5.4 million student year). Originally, the stipend was expected to be provided to students with PMT score below the bottom 40th income percentile, following the share of people living below the national poverty line. However, in practice the share of stipend beneficiary was above 50 percent. Such a higher-than-expected intake possibly led to more frequent inclusion errors. The sample-based PMT validation survey conducted in 2016 by the MIS Cell of Bangladesh Bureau of Educational Information and Statistics (BANBEIS) found an operational inclusion error of 21 percent.9 Such cases of

9 According to the verification survey analysis, operational inclusion errors during the implementation of PMT-based targeting often occur for intentional and unintentional underreporting of students’ background provided in the application form for stipend. Exclusion errors often occur because of PMT proxy variables missing out specific household characteristics related to poverty in the locality. Assuming an inclusion error of 20 percent through the project period, the amount that went to beneficiaries in inclusion error can be roughly calculated to be around US$40 million (10.2 million × 0.2 × US$20) which is equivalent to around 9 percent of the total project cost.


Page 21 of 84

inclusion errors and overshooting of stipend beneficiary warrant important lessons that regular revision of PMT formula and strengthening of selection procedure would be needed in the next program to ensure best efficiency of PMT-based stipend operation. It should also be noted, however, that the inclusion and exclusion error of SEQAEP is likely to be in line with, if not better than, the experience of other PMT-based stipend programs in other countries. Occurrence of actual inclusion error was, for instance, assessed to be above 50 percent in Mongolia. 10 As such, the inclusion and exclusion errors of this magnitude is expected of any PMT-based stipend program, and the PMT stipend program of SEQAEP has maintained an efficient operation in the global comparison.

D. JUSTIFICATION OF OVERALL OUTCOME RATING

47. SEQAEP is rated as Highly Satisfactory for its overall outcome rating based on high relevance, high efficacy, and substantial efficiency ratings. Split ratings for the project are not necessary as (i) changes made through restructuring in July 2011 did not lower the scope of the project; and (ii) the AF was to expand the scope of the project.

48. Furthermore, the project’s interventions achieved high sustainability. Sustainability is ensured through the following arrangements: (a) most of the SEQAEP interventions are being carried over to the next sectorwide secondary education development program, including the PMT-based stipend, ACT program, DRH program, student assessment program, and school-based grant program; (b) institutional capacities of central and local agencies developed under the project for M&E will be fully integrated and used in the next program; and (c) school-based institution, PTA, established under the project will continue to be the cornerstone of school-based improvement activities in the secondary education.

49. The project was also successful in bringing about innovations to the sector at the national-level, including (a) PMT-based targeting, (b) new transparent disbursement methods, (c) enhanced teacher hiring method, and (d) student- and institution-level performance incentives. These innovations were proven possible and scalable by the project and brought about qualitative changes in development effort of the secondary education sector.

50. Feedback from the project’s local stakeholders were overwhelmingly positive. Project beneficiaries who took part in the ICR beneficiary workshop session iterated a number of concrete examples of the benefits of SEQAEP interventions to their localities and schools. Excerpts of comments from beneficiaries on specific SEQAEP activities are included in annex 7-4.

E. OTHER OUTCOMES AND IMPACTS (IF ANY)

Gender

51. Gender equity has been one of the main focuses of the project. It should be noted that the predecessor project dealt primarily with promotion of girls’ enrollment in secondary education through provision of stipend. In fact, unlike many other countries, student enrollment in secondary education in Bangladesh has been in favor of girls. This was largely attributable to the FSSAP which contributed to rapid increase of female enrollment while neglecting males. The analysis showed that poor male students were particularly at risk of dropping out. To address this challenge, SEQAEP’s stipend program shifted gender-

10 According to AusAID. 2011. “Targeting the Poorest: An Assessment of the PMT Methodology.”


Page 22 of 84

based targeting toward poverty-based targeting using the PMT system to expand the stipend benefit to poor students, male and female alike. Boys’ enrollment recovered steadily during the SEQAEP implementation.

52. The project continued to address female-specific challenges. In Bangladeshi communities, girls are particularly vulnerable to poor conditions of school sanitation and water facilities. Poor toilet and lack of access to clean water are often huge nuisance for girls and discouraged them from attending schools regularly. The ISF program of SEQAEP contributed to improving school environment including sanitation facilities and tube wells at most disadvantaged schools. The EACM program also conducted awareness raising campaign to prevent girls’ early marriage, eve teasing prevention, and dropouts. The project also conducted awareness media campaigns on gender issues.

Institutional Strengthening

53. Contribution to strengthening of local education management at the upazila level. Through SEQAEP, the USEOs have been instituted and strengthened, through provision of USEO grants and technical support, as an administrative body at the lowest level that directly interact with schools and madrasahs. Their role has been crucial in supervision and monitoring of implementation of the project activity at school level. USEOs are expected to continue to play a key role in the next sectorwide program as local supervisory entities to support school-level activities.

Mobilizing Private Sector Financing

54. The ISF program obligated participating institutions—private institutions—to contribute at least 20 percent of the cost for school facility construction and renovation. Anecdotal evidence shows that some institutions performing well took advantage of this opportunity and contributed much more than 20 percent.

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME

A. KEY FACTORS DURING PREPARATION

55. Appropriate identification of priority issues and well-informed choice of main interventions. Design of SEQAEP was informed by lessons learned from the previous engagement in the sector. Placing a strong focus on the quality of education in the face of quality deterioration due to rapid access expansion was a timely and appropriate identification of priority issues. For quality-side interventions, the project was designed to tackle very fundamental issues, such as learning incentives for students, availability of quality teachers at schools, particularly weak academic performance in a few challenging subjects, low reading proficiencies, and lack of reliable assessment. Appropriate identification of priority issues and right choice of interventions have led to positive buy-in from stakeholders and effective implementations.

56. Realistic and appropriate selection of implementation mechanism. Multiple partnerships with other public agencies—some are outside of the education arena—to support the implementation of key project activities were built into the project design from the beginning to enable efficient project operations and the implementing agency designed with a modest number of staff. Strong partnership with capable agencies, namely the LGED as PMTA, Agrani Bank as a stipend disbursement agency, the BSK


Page 23 of 84

as a DRH administrator, BANBEIS as a survey agency for school census, and the DPHE for arsenic testing, has contributed significantly to the smooth implementation of key SEQAEP activities. Partnership agreement and Memorandum of Understanding with these agencies were prepared and finalized at an early stage during the appraisal mission to ensure smooth start-up of the delegated activities.

57. Appropriate design for monitoring and rigorous impact evaluation. SEQAEP holds a high standard for monitoring and impact evaluation. Establishment of comprehensive monitoring system for the whole secondary education sector was integrated in the project design through strengthening of the MEW, which has successfully become a regular entity of the DSHE. The plan for monitoring subcomponents was ambitious, including various validations, surveys, and census, and made remarkable contributions to building up the knowledge base in the sector. Impact evaluations were also ambitiously designed. A total of three RCT-based rigorous impact evaluations have been carried out to assess and prove significant positive impacts of those innovative interventions and paved the way toward nationwide scale-up.

B. KEY FACTORS DURING IMPLEMENTATION

58. Extensive and consistent use of Operations Manuals (OMs) for innovative interventions. In total, there were 12 OMs to support implementation of project activities (listed below). The OMs were prepared by the project with intensive technical assistance from the World Bank team and other experts and clearly specify all the procedures and criteria for relevant interventions. These OMs were consistently utilized and followed through by the PIU and USEOs and were revised if and when any revisions were necessary. Consistent use of well-articulated OMs has been highly instrumental in ensuring efficient and transparent implementation of complex project activities.

(a) Incentive Award Manuals

(b) Proxy-Means Testing Manual

(c) Disbursement Manual

(d) Improved Institutional Facilities Manual

(e) Disbursement Linked Indicator Manual

(f) Additional Class Teacher Manual

(g) MIS Manual

(h) Financial Management Manual

(i) ICT Grant Manual.

(j) EACM Manual

(k) Tuition Grant Management Manual

(l) Social Assessment Manual

59. Adequate monitoring and supervision on local level implementation. Given the data-intensive design of the project and diverse nature of the project components, robust data collection and results monitoring systems were established. The cornerstones of the monitoring system in SEQAEP include the


Page 24 of 84

following: (a) with support from PMTA (LGED), SEQAEP established an integrated data system that brings both PMT-based stipends and all non-PMT based data (incentives, additional classes, DRH, as well as school grants) into one single platform and (b) surveys and census monitoring data streams (on PMT validation, compliance verification, and annual school census) from the MEW and MIS Cell at BANBEIS to merge with the ACF database. More importantly, under the guidance of the project director (PD), program officers of SEQAEP constantly engaged in field monitoring visits to receive first-hand feedback from beneficiaries and supervise proper implementation of relevant activities. Data from the robust monitoring system coupled with in-person field monitoring has contributed to ensuring smooth implementation and early detection of implementation challenges.

60. Use of DLIs to specific high-stakes project activities. In the AF, three DLIs with a total pricing value of US$50 million were introduced, specifically focusing on M&E and LASI to incentivize the client for achieving results in these priority areas. This was one of the education projects in Bangladesh that pioneered the use of DLI-based financing. Despite the initial struggle in meeting the targets and producing evidences, all the agreed results of the DLI program were accomplished. The PD acknowledges the importance of compulsion created by DLIs in pushing the politically complex reform agenda such as regularization of the MEW and completing technically challenging tasks such as verification surveys and LASI on time.

61. Flexible adjustments during implementation. In consultation with the World Bank team, the project has been proactively making timely and appropriate adjustments in implementation and fiduciary designs of key project activities to overcome bottlenecks and reflect lessons learned. For instance, as discussed above, Subcomponent 1.2 (additional class) has gone through redesigning during the implementation, including introduction of mobile RTs, shift to RTs as interim arrangement, and finally introduction of more refined and sustainable ACTs in 2015. Subcomponent 1.1 (incentive program) has also been modified substantially to streamline a large number of incentive programs to only three incentive schemes. Subcomponent 1.3 (DRH) experienced difficulties in central procurement of books through suppliers; however, the procurement method was revised to allow direct purchase from publishers, which contributed to expedited procurement and distribution of millions of books. For Subcomponent 2.1 (PMT stipend), the project continuously searched for most efficient disbursement system for rural schools. Among different payment methods, mobile banking was the most preferred option for schools. In 2016, the project added new 35 upazilas into its targeted upazilas with no additional costs to fully leverage its financing capacity.

62. Maximizing the strength of specialized partner agencies. As discussed, the design of SEQAEP included extensive and multiple partnership with external specialized agencies such as the LGED and BSK for undertaking specific tasks. During implementation, the project continued to engage the partner agencies effectively and hold them accountable for results by requiring submission of quarterly and annual progress reports and by engaging them directly in supervision missions. Because of such empowerment of partner agencies, performance of some of the partner agencies exceeded their original terms of reference and contributed to deepening development impacts and improving efficiency of SEQAEP. The BSK, for instance, has managed to expand, without additional costs to the project, coverage of reading program to 12,117 institutions enrolling around 2,166,780 student readers by 2017 which were above the original target of 11,700 institutions and 1,310,000 student readers. The LGED’s role has also expanded beyond the role of PMTA as originally envisioned. The LGED now manages an integrated database, as mentioned above, that handles transactions and ACFs for other subcomponents with


Page 25 of 84

decentralized disbursement processes (that is, incentive, ACT, school grants, and ISF). Continuous high performance and expanded scope of work of partner agencies have been a significant factor of successful implementation of SEQAEP.

63. Effective project management and leadership. SEQAEP has gone through several rounds of replacement in project management. Due to the MOE’s investigation into procurement irregularities based on the World Bank’s Procurement Post Review of August 2010, nine project staff, including the then PD and deputy project director, were released from project duty. Implementation progress went down to Moderately Unsatisfactory in October 2011 for failing to meet a range of agreed actions in management and fiduciary issues. In April 2012, a new PD came on board and made significant improvements in project management to enable a quick turnaround of the project, especially in terms of positive relationships with key implementing partners, greater efficiency of project implementation, and enhanced efficacy of monitoring activities. The rating quickly recovered to Satisfactory. The current PD, who joined SEQAPE around mid-2015, has also been highly proactive and demonstrated stable project management and trustful relationship with the partner agencies. Implementation progress ratings under the leadership of the capable PD since 2012 have been mostly Satisfactory.

64. Initiation of discussion at an early stage with the Government for a next generation sector-wide approach (SWAp) program. By the time of the AF, the project had in its vision a road map toward a sectorwide SEDP and initiated a preliminary discussion led by the Government involving major development agencies active in the sector. This excellent foresight and early planning has allowed a long-term coordination and planning for scaling up of well-performing subcomponents and capacity building of the DSHE and local education offices.

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME

A. QUALITY OF MONITORING AND EVALUATION (M&E)

M&E Design

65. Monitoring. The design of the project for monitoring revolves around the SEQAEP PIU, MEW, and key implementing partners, particularly the LGED (PMTA) and BANBEIS, and has been an integral component to support data intensive operations of SEQAEP. From the beginning, the MEW was envisioned to be established as a regular wing of the DSHE to be responsible for monitoring all the DSHE’s activities as well as for results monitoring of SEQAEP.

Table 4. Main Responsibility in Monitoring

Agency Main Responsibility in Monitoring

SEQAEP PIU Monitoring of project results indicators and progress monitoring of project activities

MEW Monitoring of development activities of the DSHE including SEQAEP; Coordination for undertaking learning assessment, PMT validation, and compliance verification surveys

LGED Providing monitoring data for PMT stipend and other subcomponents based on transactional data in the integrated database

BANBEIS Undertaking secondary school census, PMT validation surveys, and compliance verification surveys, in collaboration with the MEW.


Page 26 of 84

66. Evaluation. The project had an ambitious design for evaluation. It has successfully carried out qualitative and quantitative impact evaluation studies for the main project interventions, including PMT stipend, additional classes, and DRH.

M&E Implementation

67. The M&E activities under SEQAEP has been satisfactorily implemented. Key achievements include the following: (a) in total, 17 semiannual monitoring reports were produced by the MEW throughout the project life cycle with results indicators updated consistently and regularly; (b) the MEW has been regularized as a permanent unit of the DSHE to be responsible for M&E activities for all the development activities of the DSHE; (c) the PMTA maintained a comprehensive database for and provided monitoring data on the PMT stipend, ACT, ISF, and school grants; (d) schools census, PMT verification surveys, and compliance monitoring surveys were undertaken by BANBEIS MIS in collaboration with the MEW; (e) rigorous quantitative impact evaluation was conducted for PMT stipend, ACT, and DRH programs with technical support from the World Bank team; and (f) qualitative ex post effectiveness assessment studies were carried out for additional class program, incentive program, and DRH.

68. The capacity development and regularization of the MEW had a slow start due to frequent staff turnovers and inadequate implementation guidance and coordination between the MEW, DSHE, BANBEIS, and SEQAEP. These challenges led to a long delay in achievement of DLI-tagged M&E activities, particularly compliance verification and PMT validation surveys—both are time-sensitive activities. Deteriorated implementation performance has once led to downgrading of the M&E rating to Moderately Unsatisfactory. However, SEQAEP managed to quickly turn around the situation and deliver a satisfactory performance. By the next supervision mission, the project took satisfactory corrective actions by appointing a capable director, processing critical M&E activities, and achieving DLI backlogs.

M&E Utilization

69. Utilization of M&E data was actively pursued during the implementation. Major achievements in terms of utilization of M&E include (a) dissemination of findings from LASI to policy makers and other stakeholders, (b) dissemination of secondary education institutions census, (c) dissemination of PMT stipend verification surveys and compliance verification surveys, (d) use of arsenic-level monitoring information for tracking and addressing environmental condition of tube wells, and (e) distribution of semiannual monitoring reports to the project management and World Bank missions. It merits special attention that strong validation and verification process have proven extremely helpful to achieve data reliability and minimize financial leakages.

70. Although M&E data were actively used for policy dialogue and project management, there is little evidence of utilization of M&E data, especially LASI results, for review of teaching content and methods and improvement of teaching and learning practices at schools. To use learning assessment results to improve quality of teaching and learning, dedicated capacity development would be useful in future targeting relevant central and local stakeholders.


Page 27 of 84

Justification of Overall Rating of Quality of M&E

71. Overall quality of M&E is rated as High based on (a) innovative and realistic M&E design which considers the local context and attempts to address crucial capacity constraints of the implementing agency; (b) solid implementation performance by all the agencies involved in M&E, though with some challenges; and (c) substantial utilization of M&E, especially for management purposes. It is particularly noteworthy that the project aimed for and achieved strengthening of the M&E capacity of the implementing agency through establishment, strengthening, and regularization of the MEW. In addition, the introduction of the DLI-based incentive significantly helped raise motivation for the ministry to implement the reform, as discussed above. Based on the capacity strengthening under SEQAEP, though it is still facing many capacity constraints, the MEW is set to play a crucial role in the next sectorwide program in which it will be coordinating and undertaking all the M&E activities of the sector program in collaboration with Education Management Information System Cell of the DSHE, which was strengthened under the ADB-funded SESDP. The MEW is now a full-fledged wing of the DSHE with manpower of 13 regular staff positions.

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE

72. The project has put in place adequate due diligence to safeguard against social and environmental risks.

73. Social safeguards. Because the project’s target areas include some upazilas with significant tribal population, OP 4.10 (Indigenous Peoples) was triggered. A Tribal Development Plan/Social Management Framework was formulated and adhered to as the PMTA continued to maintain database on tribal students’ enrollment and tribal students were automatically eligible for stipend. By 2017, more than 93,000 students from tribal communities were registered in the stipend program cumulatively. SEQAEP also had significant gender initiatives. The ISF program provided gender-separated sanitation facilities, as discussed earlier. Furthermore, the safety of adolescent girls has been a deep concern for parents and communities in Bangladesh. In recognizing these challenges, the EACM program has implemented awareness raising to prevent harassment and raise communities’ awareness about protection of adolescent girls. In addition, the stipend program benefited students with disability by providing stipends to all students with disability in the targeted upazilas.

74. Environmental safeguards. OP 4.01 (Environmental Assessment) was triggered for SEQAEP. The project was classified as Category B. The ISF subcomponent, which provided safe drinking water and sanitation facilities to targeted schools, was the main area in the project with environmental implications. The Environmental Management Framework (EMF) was prepared and incorporated in the ISF program. The DPHE provided technical services for undertaking periodical safe water testing of tube wells and other facilities, and yearly environmental monitoring reports on ISF were produced and appropriate corrective measures have been taken. For instance, water quality testing in 2017 revealed that among 8,456 tube wells tested at targeted schools, 467 (5 percent) have been identified to have high arsenic contamination. The project ensured all the contaminated wells were sealed, though none of them was constructed under the project. Subject to fund availability, the project is taking actions as per the EMF, to replace contaminated tube wells.


Page 28 of 84

75. Procurement. The project was identified as having Substantial risk in procurement operation based on the DSHE’s operational weakness in procurement. Measures taken include (a) formation of a Procurement Core Team, (b) placement of Procurement Focal Point in the DSHE, (c) placement of a procurement consultant, (d) introduction of e-GP; (e) procurement training for SEQAEP staff, and (e) introduction of Procurement Risk Mitigation Plan which defined standard actions for various procurement stages. The project also used a Procurement Plan Execution System (Sistema de Ejecución de Planes de Adquisiciones, SEPA) and the Systematic Tracking of Exchanges in Procurement (STEP) system which yielded good results in terms of procurement monitoring and contract management.

76. In the early stage of the project, ratings for both procurement and financial management were downgraded to Moderately Unsatisfactory based on accounts of mismanagement of accounts and procurement packages as well as vacancies of essential fiduciary positions in the project. Through the project implementation experience, the client’s capabilities for fiduciary management have been upgraded significantly. Satisfactory ratings for procurement and financial management in the final Implementation Status and Results Report (ISR) demonstrate significant improvements made in fiduciary capacity of the client.

77. Inappropriate bidding practice which was reported in the August 2010 Procurement Post Review report and the MOE conducted an investigation in 2011 on inappropriate bidding practices reported in the Procurement Post Review report. Because of findings of the investigation report, nine key project staff, including the then PD and Deputy Director, were removed from the project. Firms involved in the fraudulent practices were issued show cause notices. An action plan for improving fiduciary oversight was shared with IDA in December 2011 and as part of the action plan, procurement capacity was enhanced through (a) training of relevant officials of the DSHE and SEQAEP Unit and (b) engaging of a full-time procurement specialist.

78. Financial management. Financial management capacity and practices of SEQAEP were adequate and satisfactory. Financial management function of SEQAEP was mainly carried out by project specific financial management specialists who were hired for the project. Throughout the project life cycle, there was no significant financial management-related bottlenecks within the SEQAEP Unit. Reconciliation of accounts were made on time, and unaudited interim financial reports were mostly submitted by the agreed time line with quality acceptable to the World Bank. The project used a commercial accounting software to record financial transactions and produce financial information to prepare unaudited interim financial report acceptable to the World Bank. External audits on the annual consolidated financial statement of the project were undertaken by the Foreign Aided Project Audit Department (FAPAD), and the audited financial statements were submitted to the World Bank within six months of the end of fiscal year. There were no delays in submission of audited annual financial statements to the World Bank.

79. There have been several cases of audit observations. At the time of the ICR, there were six audit observations that remained pending from the FY18 audit report. The project submitted a copy of clarification and memo on resolution for all the observations in December 2017. All of them are expected to be resolved within a short span of time. Expenditures declared ineligible and worth a total of BDT 258,160.80 were refunded by the Government to the World Bank as of April 21, 2016 (as per the IDA letter dated June 9, 2015).


Page 29 of 84

C. BANK PERFORMANCE

Quality at Entry

80. World Bank performance at entry is based on how effectively the World Bank team identified and facilitated preparation of the operation to achieve development outcomes.

(a) The PDO of the project was highly relevant to the development agenda of the country and priorities of the World Bank and remained so throughout the project life cycle. The latest FYP of the Government highlights criticality of high-skilled manpower for sustained economic growth and calls for improved quality in secondary education with continued attention to equity. Design of the project greatly benefited from lessons learned from the previous World Bank-supported operations in the sector, such as the FSSAP, and appropriately focused upon key demand- and supply-side constraints in secondary education in Bangladesh.

(b) The implementation arrangement in partnership with specialized partner agencies is undoubtedly one of the most important factors behind the successful implementation of SEQAEP’s key interventions. All the five partners were carefully handpicked and draft terms of reference were negotiated by appraisal. Choice of partner agencies was appropriate in terms of expertise, institutional capacity, and local networks of the agencies which enabled efficient and effective implementation as well as close monitoring of the large-scale, complex, and data-intensive operations.

(c) The M&E arrangement was appropriate and ambitious. Multiple annual census and surveys were planned to provide comprehensive and updated statistical database for monitoring and supervision. Several rigorous impact evaluations were included in the design, which assess and establish the impacts of innovative interventions on student learning achievements and equity. Regularization of the MEW was envisioned to enhance the institutional capacity of the DSHE to monitor all the development works under its jurisdiction, which is now expected to be essential for results monitoring and program management in the next operation. The Results Framework was comprehensive and adequately captured development outcomes, though indicators for quality outcomes faced some difficulties as one of them had to be dropped. Intermediate outcome indicators were streamlined appropriately and captured relevant outputs.

(d) Fiduciary and environmental issues and risks were analyzed in detail at the appraisal, drawing on the experience of the previous operation in the sector and concrete measures to address fiduciary and environmental risks were included in the project design. For instance, it was identified that the implementing agency, DSHE, did not have a dedicated unit for fiduciary management, and most of fiduciary manpower from the previous operation would be lost due to staff turnover. To address this, additional arrangements were made to enhance fiduciary capacity of the project by retaining core fiduciary members of the previous operation and establishing a Procurement Core Team at the DSHE. As for environmental risk, a then emerging environmental hazard of arsenic contamination in groundwater was properly identified by the Environmental Assessment, and careful risk


Page 30 of 84

mitigation measures such as arsenic testing were included in the project’s Environmental Management Plan design to minimize potential adverse impacts.

Quality of Supervision

81. The quality of supervision by the World Bank team has been effective, and the World Bank team proactively supported the project to find solutions to implementation issues. The current PD has noted in the interview that “the Bank team’s implementation support has been good and effective, although we occasionally had disagreements.”

(a) Effective MTR. The MTR was carried out in July 2011, and a thorough review of the project progress was conducted and several key constraints and corrective measures were identified. The MTR resulted in several major changes that would streamline and strengthen some of the subcomponents, such as simplification of incentive program, redesigning of additional class modalities, rationalizing roles of the MEW, and introduction of DLIs. The Results Matrix was also updated during the MTR with supplemental indicators, streamlining of intermediate indicators, and fine-tuning of the definitions.

(b) Adequate supervision mission and continuity in supervision. The World Bank team regularly fielded implementation support missions in Dhaka. Agreed actions for next six-month period formulated and included in the Aide Memoires are well structured and clear to indicate time-bound milestones for each subcomponent. As discussed, some of the World Bank team members were retained from the initial stage of the project, including the longest serving TTL and a key consultant, which appears advantageous in terms of consistent and seamless guidance at the time of TTL handover and new PD onboarding. Program officers noted that technical advice and innovative suggestions from the World Bank team were helpful in improving operational procedures and solving problems for their components.

(c) Continuous follow-up on fiduciary and safeguard risks. The World Bank team provided effective fiduciary supervision. At the MTR in July 2011, major lapses were identified in procurement practices of the project, and the World Bank team worked with the project and client to prepare a number of agreed actions by December 2011 to enhance the procurement capacity and ensure adequate procurement oversight. As a result, procurement performance improved substantially in 2012. The World Bank team also supported the project to use digital procurement management platforms of the World Bank, SEPA, and STEP systems, which yielded good results in terms of procurement monitoring and contract implementation. The World Bank supervision missions consistently followed up on the implementation of social and environmental safeguards. When the arsenic test results confirmed the incidence of arsenic contamination in schools, the World Bank team agreed with the project to hire an environmental specialist consultant to develop an arsenic mitigation measure action plan and advised on additional mitigation measures to be taken by schools.

(d) Detailed and candid Aide Memoires and ISRs. Reflecting effective monitoring during supervision, Aide Memoires and ISRs detail out key implementation challenges every 6 months and consistently follow up on the issues raised. Changes in ratings were frequent


Page 31 of 84

and used effectively to flag important issues that require management attention and urgent actions.

(e) Promoting dialogue for sectorwide harmonization. The World Bank team played a leading role in promoting the dialogue and coordination with the client and relevant development partners, in particular ADB, toward the formation of the SWAp in the secondary education sector. One of the reasons for having an AF in 2013 was to allow sufficient time for the client to prepare for the SWAp based on the successful intervention of SEQAEP. The collaboration with the ADB toward SWAp included (a) overlapping missions, (b) preliminary joint discussion on the basic SWAp modality and sector review, and (c) undertaking studies for harmonizing interventions.

Justification of Overall Rating of Bank Performance

82. Based on the above assessment, overall World Bank performance is rated as Satisfactory.

D. RISK TO DEVELOPMENT OUTCOME

83. The risk to development outcomes is deemed as Low to Moderate as the transition to the next sectorwide program will give reasonable assurance about the continuity of successful key interventions of the project but at the same time, inevitably entail significant challenges. Most of the activities initiated under SEQAEP are expected to be carried forward, expanded, and mainstreamed under the next sectorwide SEDP, for which the World Bank is one of the financial and technical contributors along with ADB and others.11 For instance, the PMT stipend program is expected to be harmonized with other stipend programs to create a PMT-based nationwide stipend program. LASI will be mainstreamed. The DRH program will be implemented nationwide and so will the Incentive Award programs. The SEDP will be fully owned and operated by the Government with financial and technical support from the multidonor coordination consortium. At the time of project completion, transition plan, first-year operation plan, and technical design for key components were being prepared. In addition, the new program will strengthen the Government’s capacity further to take over complex interventions of SEQAEP.

V. LESSONS AND RECOMMENDATIONS

84. Incentives and motivation do matter in enhancing results for teaching and learning quality as well as retention through behavioral changes among beneficiaries. Changing people’s behavior requires both skills and motivation. Often, education projects assume motivation as given, and focus on providing skills through interventions such as teacher training and curriculum revisions. SEQAEP’s initiatives, including the incentive program and DRH, are incentive driven. SEQAEP’s experience suggests that carefully designed incentive schemes result in motivating students and teachers to improve their performance in teaching and learning, as well as in their retention. Feedback from stakeholders suggests that SSC Pass Awards give strong motivation especially among poor students who have fewer prospects of moving up to higher levels of education. Best Student Award, Institutional Award, and Award Books of DRH create a competitive environment where students and schools aspire to be high performers. The

11 The total cost of the SEDP is estimated at US$17.2 billion, for which the World Bank supports US$500 million for five years.


Page 32 of 84

SEDP should attempt to build in incentive elements in different components, where possible, to maximize their impact.

85. PMT is a good alternative to household income information; however, the experience suggests that it needs to be supplemented with other means of screening to improve targeting accuracy. Providing stipend to non-poor makes no difference in retention as it was shown that, dropping non-poor female students from FSSAP beneficiaries did not harm their enrollment and retention. SEQAEP’s experience shows that PMT is a viable, though costlier, approach for systematic pro-poor targeting when household income data are unavailable. Screening by PMT can be reasonably reliable and accurate so long as information provided accurately reflect the reality. Stakeholders identified two main sources of errors, which are (a) mismatch between PMT formula and economic conditions (mostly responsible for exclusion errors) and (b) falsified information in PMT application (mostly responsible for inclusion errors). These led to significant inclusion and exclusion errors as identified in PMT validation surveys. Mismatches can be minimized by constantly reviewing the formula. However, preventing falsified information from applicants would be a challenge as applicants quickly ‘outsmart’ the system no matter how formula designs are tweaked to prevent falsified information. Additional screening by multiple community stakeholders who would be held accountable for accuracy of information may alleviate the problem. The SEDP should continue use of PMT as the main selection protocol and verification survey as assurance mechanism and improve targeting accuracy by adding a supplementary screening process and holding local stakeholders to be accountable for data accuracy. In addition, the project tested several methods for cash disbursement. The experience suggests that direct payment would work most efficiently and transparently. Use of technologies such as cash card and mobile banking would significantly help alleviate logistical constraints. Mobile banking was the preferred method under SEQAEP and may well be used in the next program. As technologies continuously evolve, efforts to identify the most efficient methods should be continuous.

86. Continuous assessment of student learning performance is a difficult task but a big step toward quality improvement, and the effort should be strengthened through establishing a central agency for student assessment. SEQAEP initially struggled to start up LASI; however, over the four rounds of LASI, it has accomplished to develop the capacity of the DSHE to undertake nationally representative students learning assessment regularly. As the SEQAEP experience of having to drop SSC pass rate from KPIs shows, it is often challenging to identify reliable indicators to track students learning performance across regions and over time. Periodic nationwide student learning assessments such as LASI would provide just that which would allow policy makers to assess the gaps and have more evidence-based discussions for quality improvement. The SEDP will have a KPI that derives from results of national secondary student assessments—an enhanced version of LASI. The SEDP should continue strengthening the methodology of student assessment and improve feedback mechanism and follow-up on recommended actions for teaching quality enhancement. Establishing a central assessment agency which is responsible for planning and implementation of student assessment and promoting feedback to the system would be a promising strategy to strengthen student learning assessment program.

87. Integrating impact evaluations on innovative interventions in project design from the beginning would be an effective approach; the experience suggests that ensuring adequate technical expertise would be a must. SEQAEP was forward-looking in terms of designing impact evaluations. As the impact evaluation studies had been conceived before any implementation began, they could exploit project features effectively to plan quasi-experimental rigorous evaluations. It should also be noted that expertise


Page 33 of 84

needed to undertake and analyze rigorous experimental/quasi-experimental impact evaluations may not be easily available in the Project Management Unit. As such, including an evaluation specialist in the implementation support team would be essential.

88. Leveraging the capacity of external specialized agencies through partnership agreements would be a good strategy to overcome capacity constraints of the implementing agency; adequate capacity development opportunities for the implementing agency leveraging expertise of specialized agencies must be ensured. SEQAEP’s successful implementation would have not been possible without relying on external partner agencies. Similar to many other projects, the main implementing agency of SEQAEP has technical capacity constraints and is constantly overstretched for its human and financial resources. Under such circumstances, leveraging the capacity of other agencies would be a quick fix to get around the capacity constraints and develop capacity of the central agency. Measures should be taken to ensure a good capacity development plan in place for the main agency with the aim that the main agency would gradually become capable of taking over the responsibility from external partner agencies, should that be appropriate. In the SEDP, the Government should consider capitalizing on existing strength and experience of the partner agencies and leverage their expertise to transfer knowledge to develop the MOE’s central agencies.

89. Additional supply of quality teaching staff on needs basis improves quality of teaching and learning and reduces the need for private coaching. Like in many other countries, in rural Bangladesh, teacher shortage poses a serious challenge in the effort to improve the quality of teaching and learning in secondary education. At the same time, educationalists are increasingly concerned about negative effects of private coaching services where school teachers themselves are often providers of such services. SEQAEP had addressed these issues head-on by providing additional class teachers who are hired on contract basis with mandate to teach regular classes as well as additional supplementary classes outside of regular class hours. ACTs were provided based on rigorous needs assessment favoring underserved and low-performing areas. ACT program turned out to be one of the most demanded interventions of SEQAEP. Impact evaluation demonstrates positive impacts of the ACT program on student learning performance, and anecdotal evidence suggests significant reduction in prevalence of private coaching practices at schools where ACTs are operating. Quality of ACTs tend to be higher than regular teachers because of rigorous selection procedures, pedagogical training, and additional incentive pays. Provision of contract teachers sometimes raises questions about sustainability; however, in the absence of viable long-term solutions, providing trained and qualified additional teaching staff on needs-basis could be an effective midterm solution with large positive impacts on learning. Furthermore, for a long-term solution, the Government should look into the ACT program for possibility of adjusting the Government’s hiring and deployment formula of teachers.

90. Power of digital tools needs to be harnessed for efficient operation and management of numerous small transactions. SEQAEP had several highly transaction-intensive operations. Hundreds of thousands of small transactions had to be processed in a short span of time. SEQAEP, with the support of the LGED, established an integrated database that support data collection, verification, and processing of transaction information from schools and beneficiaries. In decentralized and transaction-heavy operations, establishment of such integrated database would yield significant positive return in terms of efficient implementation and effective monitoring. The SEDP would benefit significantly from utilizing or replicating those databases for upcoming program interventions.


Page 34 of 84

91. Use of DLIs gives strong compulsion to the Government for achieving results. SEQAEP used the DLI program selectively on three areas where concentrated attention and institutional commitment need to be built up. After initial struggle, all DLIs were achieved including one that requires organizational changes and revenue budget allocation. Feedback from the central stakeholders indicate that having DLIs did help in significant ways to build up everyone’s consensus on expected results and provide compulsion to the government officers to achieve the agreed results. SEQAEP’s experience clearly shows that result-based financing is highly effective in advancing challenging reform agenda forward.

92. Shared vision and extensive dialogue for preparation for the transition to SWAp. The project could offer an excellent case study on preparation for the transition from the project-based approach to SWAp. The project was extended with an AF in 2013 with an assumption that the AF would buy time for the Government to be ready for the transition to SWAp. In fact, the Government successfully initiated the sectorwide program from January 2018, after the completion of SEQAEP in December 2017, with participation of all the interested development partners. As discussed, since as early as 2014, the World Bank team had maintained close collaboration with the client and ADB to carry out various groundwork for the transition. Some of the lessons that could be drawn from the experience of SEQAEP for a successful transition to SWAp include the following: (a) start the dialogue early to allow adequate time for the preparation, (b) involve all the major partners of the sector in open discussion on sector analysis and SWAp modality to create shared vision and consensus, and (c) undertake analytical works on critical interventions to address technical complications for harmonization.

.


Page 35 of 84

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS

A. RESULTS INDICATORS A.1 PDO Indicators

Objective/Outcome: Objective 1: To improve the quality of secondary education

Indicator Name Unit of Measure Baseline Original Target Formally Revised

Target

Actual Achieved at Completion

Completion Rate in Grade 10 Percentage 20.00 27.00 33.00 36.00

30-Jun-2007 30-Jun-2013 31-Dec-2017 15-Nov-2017

Percent of Grade 6 entrants who have passed SSC exam

Percentage 28.00 38.00 50.00 60.00

30-Jun-2007 30-Jun-2013 31-Dec-2017 15-Nov-2017

Comments (achievements against targets): Achieved

Objective/Outcome: Objective 2: To establish the monitoring system of learning outcomes


Target


Monitor Learning Levels in Yes/No N Y Y Y


Page 36 of 84

Secondary Schools 30-Jun-2007 30-Jun-2013 31-Dec-2017 30-Sep-2016


Objective/Outcome: Objective 3: To increase access and equity in secondary education in upazillas targeted by the project


Target


Number of Students appearing in SSC exams in project areas ('000)

Number 187.00 220.00 485.00 578.00

30-Jun-2007 30-Jun-2013 31-Dec-2017 07-Dec-2017

Number of Poor students appearing in SSC exams in project areas ('000)

Number 66.00 70.00 160.00 228.00

01-Jul-2010 30-Jun-2013 31-Dec-2017 23-May-2017



Target


Gender Parity Ratio (male-female) in Grade 6-10 enrolment in project areas

Number 0.82 0.92 0.92 0.90

30-Jun-2007 30-Jun-2013 31-Dec-2017 23-May-2017

Comments (achievements against targets): Nearly Achieved


Page 37 of 84


Target


Share of poor children in total secondary enrolment in project areas

Percentage 30.00 39.00 41.00 42.00

30-Nov-2007 30-Jun-2013 31-Dec-2017 07-Dec-2017



Target


Direct project beneficiaries Number 2200000.00 4242000.00 4242000.00 5254000.00

31-Dec-2012 31-Dec-2017 31-Dec-2017 07-Dec-2017

Female beneficiaries Percentage 54.00 53.00 53.00 54.00

31-Dec-2012 31-Dec-2017 31-Dec-2017 07-Dec-2017


A.2 Intermediate Results Indicators

Component: Component 1: Improving Education Quality and Monitoring Learning Levels


Target


Number of SSC Pass Number 0.00 130000.00 500000.00 631759.00


Page 38 of 84

awardees among PMT Beneficiaries

30-Jun-2007 30-Jun-2013 31-Dec-2017 07-Dec-2017

Comments (achievements against targets): Cumulative figures.


Target


Number of additional classes in English and Mathematics

Number 0.00 2980000.00 3730000.00 3183000.00

30-Jun-2007 30-Jun-2013 31-Dec-2017 07-Dec-2017

Comments (achievements against targets): Cumulative figures

Component: Component 2: Improving Equity and Access


Target


Number of eligible poor girls and boys receiving PMT stipends

Number 0.00 2830000.00 5400000.00 10271264.00

30-Jun-2007 30-Jun-2013 31-Dec-2017 07-Dec-2017

Comments (achievements against targets): Cumulative figures

Component: Component 3: Institutional Capacity Strengthening


Target



Page 39 of 84

Proportion of project schools with functional PTA

Percentage 1.00 20.00 50.00 98.00

31-Dec-2012 30-Jun-2013 31-Dec-2017 07-Dec-2017

Comments (achievements against targets):

Component: Component 4: Monitoring and Evaluation


Page 40 of 84

B. KEY OUTPUTS BY COMPONENT

Objective/Outcome 1: To improve the quality of secondary education in targeted upazilas

Outcome Indicators 1. Completion rate in Grade 10 in Project Upazilas

Intermediate Results Indicators

1. Number of SSC pass awardees among PMT beneficiaries 2. Number of institutions receiving SSC institutional award 3. Number of additional classes in English and mathematics 4. Number of student members of reading habit program 5. SSC pass rate in project areas 6. Monitoring and evaluation surveys conducted as scheduled (DLI 3)

Key Outputs by Component (linked to the achievement of the Objective/Outcome 1)

1. Students and teachers incentivized for better teaching and learning 2. Contact hours and quality of teaching enhanced for mathematics, English and science 3. Reading habit developed and reading skills improved among student members

Objective/Outcome 2: To establish the monitoring system of student learning outcomes

Outcome Indicators 1. Monitor learning levels in secondary schools (in Bangla, mathematics, and English) (KPI and DLI 1) 2. MEW institutionalized and financed through revenue budget (DLI 2)

Intermediate Results Indicators None


1. Learning performance of students assessed regularly with a nationally representative sample

Objective/Outcome 3: To improve access and equity in secondary education in targeted upazilas

Outcome Indicators 1. Number of poor students appearing in SSC in project areas 2. Gender parity (male-female) in enrollment in Grade 6–10 increases in Project Upazilas


Page 41 of 84

3. Percentage share of poor children in total enrollment in Grade 6 and 10 increases in Project Upazilas

Intermediate Results Indicators

1. Number of eligible poor girls receiving stipend 2. Number of eligible boys receiving stipend 3. % of schools with safe drinking water in Project Upazilas 4. % of schools with separate latrines for boys and girls in Project Upazilas 5. Monitoring and evaluation surveys conducted as scheduled (DLI 3)


1. Parents incentivized for continued schooling 2. Financial constraints of poor households alleviated 3. School environment improved especially for females


Page 42 of 84

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION

A. TASK TEAM MEMBERS

Name Role

Preparation

Irajen Appasamy Task Team Leader(s)

Nazmul Chaudhury Task Team Leader(s)

Helen J. Craig Sr. Education Specialist

Subrata Dhar Sr. Operations Officer

Dilip Parajuli Education Economist

Leopold R. Sarr Economist

Syed Rashed Al Zayed Research Analyst

Shakil Ferdausi Environmental Specialist

Fabio Pittaluga Sr. Social Development Specialist

Burhanuddin Ahmed Sr. Financial Management Specialist

Tanvir Hossain Procurement Analyst

Hena Mukherjee Consultant – Education

Shiva Raj Lohani Consultant – Education Economist

Mokhlesur Rahman Consultant – Management Specialist

Nazma Sultana Program Assistant

Bertha Mburugu Program Assistant

Supervision/ICR

Shwetlena Sabarwal Task Team Leader(s)

Dilip Parajuli Team Member / Former Task Team Leader(s)

Ayesha Y. Vawda Former Task Team Leader(s)

Irajen Appasamy Former Task Team Leader(s)

Nazmul Chaudhury Former Task Team Leader(s)

T. M. Asaduzzaman Team Member


Page 43 of 84

Md Kamruzzaman Procurement Specialist(s)

Mohammad Reaz Uddin Chowdhury Financial Management Specialist

Sabah Moyeen Social Safeguards Specialist

Nadia Sharmin Environmental Safeguards Specialist

Nazma Sultana Team Member

Shiva Raj Lohani Team Member

Mosharraf Hossain Team Member

Golam Faruque Khan Team Member

Uttam Sharma Team Member

Sandra X. Alborta Team Member

Mokhlesur Rahman Team Member

Hena G. Mukherjee Team Member

Helen J. Craig Team Member

Shinsaku Nomura Team Member

Bertha M. Mburugu Team Member

Leopold Remi Sarr Team Member

Mostafiz Kazi Team Member

Nicole Andrea Maywah Team Member

Tanvir Hossain Team Member

Subrata S. Dhar Team Member

Leopold R. Sarr Team Member

Seo Yeon Hong Team Member

Juan Prawda Witenberg Team Member

Asif Ali Procurement Specialist

Tanvir Hossain Procurement Specialist

Mohammed Atikuzzaman Financial Management Specialist

Burhanuddin Ahmed Financial Management Specialist

Shakil Ahmed Ferdausi Environmental Safeguards Specialist

Shakil Ferdausi Environmental Safeguards Specialist


Page 44 of 84

Fabio Pittaluga Social Safeguards Specialist

Khawaja M. Minnatullah Social Safeguards Specialist

Faiyaz Talkudar ICR Associate Writer

Shiro Nakata ICR Main Writer

B. STAFF TIME AND COST

Stage of Project Cycle Staff Time and Cost

No. of staff weeks US$ (including travel and consultant costs)

Preparation

FY08 101.061 537,168.36

FY09 15.710 68,653.61

Total 116.77 605,821.97

Supervision/ICR

FY09 67.358 332,961.03

FY10 65.220 316,686.93

FY11 64.013 279,588.40

FY12 95.365 641,010.17

FY13 88.166 477,082.72

FY14 75.720 337,900.07

FY15 22.817 125,886.12

FY16 14.327 98,184.10

FY17 10.072 73,040.23

FY18 13.453 84,891.87

Total 516.51 2,767,231.64


Page 45 of 84

ANNEX 3. PROJECT COST BY COMPONENT

Components Amount at Approval

(US$, millions)

Amount at Additional Financing

(US$, millions)

Actual at Project Closing

(US$, millions)

Percentage of Amount at AF (US$, millions)

Improving Education Quality and Monitoring Learning Levels

37.40 109.70 93.78 85

Improving Equity and Access 106.70 286.30 285.82 100

Institutional Capacity Strengthening

9.90 31.30 30.46 97

Monitoring and Evaluation 1.70 8.20 4.16 50

Total 155.70 435.50 414.22 95


Page 46 of 84

ANNEX 4. EFFICIENCY ANALYSIS

Economic Analysis

1. The economic analysis presented in this annex is an assessment of future benefits and costs associated with the Bangladesh SEQAEP project activities during the complete project period of 2009–2017. Costs include (a) program costs and private costs that comprise (i) direct household outlays and (ii) opportunity costs for schooling. Similarly, the benefits are taken to be those changes in the quantity, quality, and reduction in internal inefficiency of education produced over the period because of SEQAEP. The main sources of data used in the analysis come from the SEQAEP MIS, Household Income and Expenditure Survey (HIES) 2010, and the cost tables for SEQAEP. The economic analysis presented here focuses on the complete project period, with benefit stream extended to 20 years beyond the project horizon.

I. Benefits of SEQAEP

2. For this analysis, benefits from the following two sources are quantified:

• Improved access to and retention in secondary education of the poor by providing them stipends, resulting in increased number of secondary school (Grade 10) completers who earn higher wages (relative to non-completers)

• Increased quality and relevance of education through improved education quality resulting in reduced time taken to complete secondary schooling as well as consequent higher productivity and earnings for all Grade 10 completers (relative to Grade 10 completers from non-SEQAEP institutions).

3. Underlying the analysis of access and retention is the estimation of poor and non-poor students completing Grade 10 in SEQAEP institutions, based on improved intake and internal efficiency due to SEQAEP. The latest SEQAEP PMT Validation Report (2017) reports an inclusion error of 21.1 percent, indicating that over one-fifth of all PMT recipients were from non-poor families. Increase in Grade 6 intake is assumed to be 20 percent according to the finding of the impact evaluation study. Increased retention rate (averaged over Grades 6 to 10) through the pro-poor stipend program is estimated as 6 percent (based on counter-factual analytical comparison with non-SEQAEP enrollment), and it is assumed that the increased internal efficiency for non-poor students due to education quality interventions to be 3 percent. These estimates are used to calculate the number of Grade 10 completers during the project periods. The projections of Grade 10 completers and impacts of SEQAEP on number of Grade 10 completers are shown in table 4.1. The number of students completing Grade 10 over SEQAEP life cycle is estimated at roughly 256,000; these students would not have completed the secondary cycle without SEQAEP intervention.


Page 47 of 84

Table 4.1. SEQAEP Grade 10 Completers and its Impact on the Number of Grade 10 Completers

2009 2010 2011 2012 2013 2014 2015 2016 2017

Non-poor Grade 10 completers

7,513 18,034 17,916 13,358 17,562 13,214 27,367 26,771 33,961

Poor Grade 10 completers

28,093 67,435 66,993 49,949 65,670 49,411 102,334 100,106 126,993

Increased (cumulative) number of completers due to SEQAEP

2,136 48,809 88,790 100,121 123,291 136,588 166,365 201,228 256,401

4. Grade 10 completers will earn the wage premium compared to the primary completers who have not completed Grade 10. Wage level of Grade 10 completers is estimated from HIES 2010 and then projected to account for inflation for future years (the earning differential in 2010 between earners with and without a complete secondary education is estimated to be about taka 20,100 or US$248 per person annually). Around 6 percent of the wage of Grade 10 completers for quality premium due to increased relevance through SEQAEP quality interventions applies to all Grade 10 completers. The benefit stream accruing from lifetime earnings for all cohorts under SEQAEP during the project life cycle is assumed to continue for 20 years beyond project completion.

II. Costs of SEQAEP

5. SEQAEP costs are estimated based on the projected stipend beneficiaries and number of institutions expected to be supported for quality interventions and M&E cost estimates. Cost estimates for the project life cycle, as shown in table 4.2, sum up to roughly US$1,459 million. Per student private household expenditure of secondary education is about US$101 in 2010 (constant prices) as estimated using HIES 2010 data and projected for future years. From the same data, opportunity cost (annual forgone earnings) per student per year while attending school is estimated to be US$413 per student for Grade 5 completers of ages 11 to 17 in 2010; HIES 2010 indicates that almost 50 percent of non-schooling children would work for wages. Schooling cost and opportunity cost are accounted for the students who would have not enrolled in school in the absence of SEQAP interventions.

Table 4.2. SEQAEP Costs and Household Costs (US$, millions)

2009 2010 2011 2012 2013 2014 2015 2016 2017 Total

SEQAEP Project Cost 42 24 29 21 38 39 62 48 19 320

Private (Household) Schooling Cost

47 41 32 22 10 9 32 15 46 253

Opportunity Cost 164 144 112 78 34 31 111 53 160 889

Total Cost 253 209 173 121 81 76 205 117 226 1,459


Page 48 of 84

III. Net Present Values of Benefits and Costs and Internal rate of Return

6. The discounted present value benefit and cost streams are presented in table 4.3. Projections are made for two scenarios based on a low, medium (baseline), and high discount rate of 10 percent, 12 percent, and 14 percent, respectively.12 Based on a baseline discount rate of 12 percent for the benefit and cost streams described above, the present discounted value of benefits for the base-case scenario is estimated to be US$2,906 million while the present discounted value of costs is estimated to be US$1,400 million. The net present value (NPV) of program benefits is US$1,195 million. The IRR associated with this NPV is 22.56 percent. DPVs of each source of benefit and cost streams in the base-case scenario are shown in table 4.3. The largest part of the benefits accrue from quantity premium for SEQAEP secondary school completers. In terms of costs, opportunity cost to forgoing income earning opportunities constitutes the largest category.

Table 4.3: DPVs of Benefits and Costs Streams, SEQAEP

Category US$, millions US$, millions US$, millions

Discount Rate 10% 12% 14%

Quantity Benefits (additional Grade 10 completers) 2,570 1,960 1,524

Relevance/Quality Benefits 1,104 830 636

Efficiency Benefits 130 116 105

Total Benefit (PV) 3,804 2,906 2,265

Project Costs 200 185 171

Household Education Costs 324 270 231

Opportunity Costs 1,133 945 806

Total Cost (PV) 1,657 1,400 1,208

Net Benefits (PV) 1,801 1,195 778

IRR 22.56% 22.56% 22.56%

IV. Sensitivity Analysis

7. In table 4.4, a sensitivity analysis of IRR and NPV (with a medium discount rate of 12 percent) under different scenarios is presented—with allowance for varying employment wage rate (external efficiency) and progress in internal efficiency. The IRR is 22.56 percent under the base case under the assumptions (justified by HIES 2010 data) that SEQAEP secondary school completers would have earned 6 percent higher wages than non-SEQAPE secondary school completers, and retention rate of poor students is 6 percent higher with PMT stipend program (base-case scenario). As seen in table 4.4, IRR varies between 20.04 percent and 22.56 percent under varying scenarios.

12 The discount rate consists of the national inflation rate and project risk premium. The average inflation rate between 2009 and 2017 is assumed to be 6 percent, and two risk premia scenarios of 4 percent, 6 percent, and 8 percent are presented in table 4.3.


Page 49 of 84

Table 4.4: Sensitivity Analysis of IRR Under Multiple Scenarios (Discount Rate = 12%)

External Efficiency

6% Quality Premium (Base Case)

3% Quality Premium (Low Case)

Internal Efficiency

6 % increased retention rate, poor student (base case) 22.56% 20.53%

3 % increased retention rate, poor student (low case) 22.08% 20.04%

8. With the IRR ranging from 20.04 percent to 22.56 percent, the above results (including the robustness analysis) suggest that SEQAEP is expected to be a fairly sound investment. In fact, these are conservative lower-bound estimates, given that they do not yet account for externality benefits arising from a healthier, better educated, and equitable/inclusive society.


Page 50 of 84

ANNEX 5. BORROWER, CO-FINANCIER, AND OTHER PARTNER/STAKEHOLDER COMMENTS

Dear TTL,

Greetings.

This is tom inform that I, the Project Director of SEQAEP and Additional Secretary of MOE, have gone

through the ICR report although it is a bit late because of my visit to the USA. The facts and figures that

have been mentioned in the ICR are found to be consistent to the MOE’s Project Completion Report

(PCR). The percentile (20% completion rate) was in the baseline information at the appraisal period. The

achievements are mentioned in the ICR, and those are also in line with the findings of the PCR. So, you

can go ahead with the ICR report and if there is any additional information needed I will be happy to provide

upon your request.

Thank you.

Best regards,

Dr. Mahmud

Dr. Md. Mahamud-Ul-Hoque

Additional Secretary

Ministry of Education, Government of Bangladesh


Page 51 of 84

ANNEX 6. SUPPORTING DOCUMENTS (IF ANY)

Key Bank Documents

Strategies

World Bank. 2006. Bangladesh Country Assistance Strategy 2006–2009.

World Bank. 2016. Country Partnership Framework for Bangladesh for the period of FY16–FY20.

Project Documents:

World Bank. 2007. Project Concept Note - Secondary Education Quality and Access Enhancement Project.

World Bank. 2008. Project Appraisal Document on a Proposed Credit for the Secondary Education Quality

and Access Enhancement Project.

World Bank. 2013. Project Paper on a Proposed Additional Credit for the Secondary Education Quality

and Access Enhancement Project.

Aide Memoire

Aide Memoire for Preparation Mission (October 29–November 14, 2007)

Aide Memoire for Pre-appraisal Mission (January 24–February 13, 2008)

Aide Memoire for Appraisal Mission (March 31–April 16, 2008)

Aide Memoire for First Supervision (November 24–December 3, 2008)

Aide Memoire for Second Supervision (May 11–20, 2009)

Aide Memoire for Third Supervision (November 8–19, 2009)

Aide Memoire for Fourth Supervision (April 11–20, 2010)

Aide Memoire for Fifth Implementation Support Mission (October 4–14, 2010)

Aide Memoire for MIDTERM REVIEW (July 12–28, 2011)

Aide Memoire for Seventh Implementation Support Mission (March 7–12, 2012)

Aide Memoire for Additional Financing–Identification (February 19–28, 2013)

Aide Memoire for Implementation Support Mission (February 23–March 6, 2014)

Aide Memoire for Implementation Support Mission (August 24–September 4, 2014)

Aide Memoire for Implementation Support Mission (February 15–26, 2015)

Aide Memoire for Implementation Support Mission (September 13–21, 2015)

Aide Memoire for Implementation Support Mission (May 2–9, 2016)

Aide Memoire for Implementation Support Mission (May 16–23, 2017)

Aide Memoire for Implementation Support Mission and ICR Mission (November 5–8, 2017)

Implementation Status and Results (Archived Date)

ISR #5 (October 19, 2010)

ISR #6 (Apr 27, 2011)

ISR #7 (September 28, 2011)


Page 52 of 84

ISR #8 (April 12, 2012)

ISR #9 (November 6, 2012)

ISR #10 (June 5, 2013)

ISR #11 (January 12, 2014)

ISR #12 (August 5, 2014)

ISR #13 (March 18, 2015)

ISR #14 (October 7, 2015)

ISR #15 (May 12, 2016)

ISR #16 (November 29, 2016)

ISR #17 (June 15, 2017)

Borrower ICR

Ministry of Education. 2017. Project Completion Report - Secondary Education Quality and Access

Enhancement Project.

Data sets and reports

Strategies

Government of Bangladesh. 2005. Bangladesh Poverty Reduction Strategy Paper: Unlocking the

Potential.

Government of Bangladesh. 2015. The 7th Five Year Plan FY2016–FY2020.

Safeguard Assessments

DSHE. 2008. Environmental Assessment and Environmental Management Framework of Secondary

Education Quality and Access Enhancement Project (SEQAEP).

DSHE. 2013. Social Management Framework for Secondary Education Quality and Access Enhancement

Project (SEQAEP).

DSHE. 2013. Environment Assessment and Environmental Management Framework of Secondary

Education Quality and Access Enhancement Project (SEQAEP), Updated.

Monitoring Reports

DSHE. 2014. Semi-Annual Monitoring Report, 10th (July–December 2013).

DSHE. 2014. Semi-Annual Monitoring Report, 11th (January–June 2014).







Studies


Page 53 of 84

SEQAEP. 2015. Annual Report on Tube-well Water Quality Monitoring of SEQAEP Covered Educational

Institutes, Year 2015.

SEQAEP and Australian Council for Education Research. 2014. A Report on Learning Assessment of

SEQAEP Institutions (LASI 2013).

SEQAEP and Australian Council for Education Research. 2016. A Report on Learning Assessment of

Secondary Institutions (LASI 2015).

Census and Surveys

SEQAEP and BANBEIS. 2015. Annual Education Institution Census 2014.

SEQAEP and BANBEIS. 2015. Report on Validation Survey 2015 of PMT Applicant Students.


SEQAEP and BANBEIS. 2016. Report on Validation Survey 2016 of PMT Applicant Students.


Analytical studies

National Academy for Planning and Development. 2017. Effects of Developing Reading Habits (DRH) on

Academic Performance: A Study on the Students at Secondary Level.

SEQAEP. 2013. Effectiveness Study on Additional Class.

SEQAEP. 2013. Impact and Effectiveness Study on Incentive Sub Component.

World Bank. 2009. Baseline Report.

Impact evaluations

World Bank. 2012. Interim Impact Evaluation Report on PMT Stipend.

World Bank. n.a. Impacts of Additional Class Teacher and Developing Reading Habit Interventions in

Secondary Schools in Bangladesh.


Page 54 of 84

Annex 6-2: Summary of Borrower’s ICR (Project Completion Report)

Government of the People’s Republic of Bangladesh Ministry of Planning

Implementation Monitoring and Evaluation Division

PROJECT COMPLETION REPORT: IMED 04/2003 (Revised)

A. PROJECT DESCRIPTION

01. Name of the Project : Secondary Education Quality and Access Enhancement Project

02. Administrative Ministry/Division : Ministry of Education/ Higher and Secondary Education

03. Executing Agency : Directorate of Secondary and Higher Education

04. Location of the Project : 250 Selected Upazila of Bangladesh

05. Objective of the Project

The overall objective of the project was to improve quality of schooling, strengthen capacity to systematically assess and monitor learning outcomes and to improve equitable access in 250 Project Upazilas. The specific objectives of the project were: SEQAEP Unit-

(a) Improving the quality of education, particularly, the teaching-learning process through arranging additional classes at the secondary level (grades 6-10), development of reading habits and providing incentives based on academic performance;

(b) Ensuring equitable access by providing stipends and incentives to the poor boys and girls students, subvention against tuition fees to institutions at the rural and semi-urban areas;

(c) Attracting new and retaining existing students especially girl students in schools through the provision of safe drinking water and sanitation facilities etc.

(d) Reduction of the dropout rates at the secondary level by providing incentives and simultaneously creating awareness among the parents and community;

(e) Strengthening the capacity of school management, monitoring and assessing the teaching-learning outcomes at the secondary level; and

(f) Improving and strengthening the capacity of DSHE through institutionalization of monitoring and evaluation activities.


Page 55 of 84

B. IMPLEMENTATION POSITION

Implementation Period SEQAEP Unit

Implementation Period as per PP

Actual Implementation

Time Over-run (% of original

Remarks

Original Latest Revised period implementation period)

1 2 3 4 5

1 July 2008–30 June 2014

1 July 2008–31 December 2017

1 July 2008–31 December 2017

1.58%

Project Personnel

SEQAEP Unit

Sanctioned Strength as per

PP

Manpower Status of the Existing Manpower

Manpower Employed

Employed During

Execution

Manpower Requirement for O&M as per pp

Existing Manpower for

O&M Others

1 2 3 4 5 Male Female

Officer(s) 24 15 9

Staff(s) 23 21 2

Total : 47 36 11

MEW Unit

Sanctioned Strength as Per

PP

Manpower Status of the existing manpower Manpower

Employed During

Execution

Manpower Requirement for O&M as per pp

Existing Manpower for O&M

Others Employed

1 2 3 4 5 Male Female

Officer(s) 7 5 2

Staff(s) 4 4

Total : 11 9 2

6. Information regarding PS (s):

SEQAEP Unit

Name and Designation with Pay

Scale.

Full Time

Part Time

Responsible for more

than One Project

Date of Remarks

Joining Transfer

1 2 3 4 5 6 7

Mr. Sirajul Islam (16,800– Yes No No 18/08/08 02/07/09


Page 56 of 84

20,700)

Mr. Badiul Alam (29,000–35,600)

Yes No No 02/07/09 23/11/10

Sirajul Islam (29,000–11x6-35,600)

Yes No No 23/11/10 12/04/12

Mr. Md. Shahid Bakhtiar Alam (29,000–35,600)

Yes No No 12/04/12 29/03/15

Dr. Md. Mahamud-ul-Hoque (66,000–3,490)

Yes No No 29/03/15 Till date

D. ACHIEVEMENT OF OBJECTIVES OF THE PROJECT:

SEQAEP UNIT

Objectives as per PP Activities

Actual achievement Reasons for shortfall, if

any Project Target

Achievement (%)

(a) Improving the quality of education, particularly, the teaching-learning process through arranging additional classes at the secondary level (Grades 6–10), development of reading habits and providing incentives based on academic performance

Improving Education Quality and Capacity to Monitoring Teaching-Learning Levels:

Incentive to student, teachers and institutions

1,041,532 1,212,552 (116)

Best Students Achievement Award

496,706 565,842 (113)

PMT SSC/Equivalent Award 528,114 631,759 (119)

Institutional Achievement Award

5,784 8,815 (152)

Support for English Language and Mathematics Learning and Teaching:

Mobile Training Resource Team

Inst = 3,000; ACs = 1,125,625; Trainers = 616.

Inst = 3,000 (100); ACs = 1,125,625 (100); Trainers = 616 (100).

Resource Teacher Program Inst. = 400 RTs = 960 ACs = 10,50,000 Training = 960

Inst = 402 (100) RTs = 1,328 (138) ACs = 954,525 (91) Training = 1,328 (138)

Additional Class Teacher Scheme

Inst = 2,000 ACTs = 6,000 ACs = 1,604,988 Orientation = 10,000 Training = 6,000

Inst = 2,018 (101) ACTs = 9,723 (162) ACs = 1,621,552 (101) Orientation = 9,832 (98) Training = 5,384 (90)

Developing Reading Habits:

Program Upazilas 250 250 (100)

Institutions covered 11,000 12,100 (110)

Organizer Training

Readers enrollment 4,348,100 8,333,703 (192)

Award Book Distribution 2,977,391 3,438,537 (115)

Supply of Appropriate Program Books

2,816,645 3,563,327 (127)


Page 57 of 84

Assessment of Education Quality

Learning Assessment in SEQAEP Institutions (LASI)

4 4 (100)

Conduct Compliance Verification Survey (Rounds)

5 5 (100)

(b) (c)

Ensuring equitable access by providing stipends and incentives to the poor boys and girls students, subvention against tuition fees to institutions at the rural and semi-urban areas Attracting new and retaining existing students especially girl students in schools through the provision of safe drinking water and sanitation facilities etc.

Improving Equitable Access for the poor and School Environment:

PMT Based Stipends and Tuition to the Poor under Cat-1

PMT Students from 2009 to 2013 = 4,359,548 AF PMT Students from 2014 to 2017= 5,764,206

PMT Students from 2009 to 2013 = 4,152,197 (95) AF PMT Students from 2014 to 2017= 10,271,264 (178)

Tuition for PMT Beneficiaries

PMT Students from 2009 to 2013 = 4,359,548 AF PMT Students from 2014 to 2017 = 5,764,206

PMT Students from 2009 to 2013 = 4,152,197 (95) AF PMT Students from 2014 to 2017= 10,271,264 (178)

General Stipends General Stipend from 2009 to 2017 to 862,261 Students

General Stipend from 2009 to 2017 to 841,963 Students (98)

Improving School Facilities:

Wash Block 420 418 (100)

Low Cost Wash Block 814 814 (100)

Class room renovation 1,197 1,180 (99)

Shallow Tube well 1,020 980 (96)

Deep Tube well 1,620 1,587 (98)

Twin Latrine 1,865 1,826 (98)

Rainwater Harvesting 115 69 (60)

Water Pump and Water Tank

2,400 1,783 (74)

Solar Water Treatment 400 260 (65)

Manganese test 460 460 (100)

Arsenic Test (Kits) 905 905 (100)

Water Purifier 1,760 1,365 (78)

Awareness Materials LS LS

(d) Reduction of the dropout rates at the secondary level by providing incentives and simultaneously creating awareness among the parents and

Institutional Capacity strengthening

Project Management

Transport and Vehicle

252 nos. Motor Cycle for USEOs

244 nos. Motor Cycle for USEOs

Computer and Accessories

Laptop 262 nos. Scanner 217 nos. Fax/Printer 216 nos. Photo Coppier 222 nos.Digital Class Solution

Laptop 262 nos. Scanner 217 nos. Fax/Printer 216 nos. Photo Coppier 222 nos. Digital Class Solution 380


Page 58 of 84

community

380 nos. nos.

Institutional capacity building

120 mm 109 mm (91)

Undertaking Overseas Training/Study Tour

272 269 (99)

Education Awareness and Community Mobilization

Awareness Materials (Poster, Leaflet, Brochure, Booklet, Newsletter)

Poster-95,000, Leaflet-55,000, Brochure-60,000, Stcker-105,000, Booklet-11,000, Newsletter-84,000

Poster-95,000, Leaflet-55,000, Brochure-60,000, Stcker-105,000, Booklet-11,000, Newsletter-84,000

Upazila PTA Member Secretary Orientation, Institution level orientation for PTA, SMC/MMC members, Planning Meeting with Institutional Heads, Cluster wise cultural programs, Folk Song, Street Drama, Documentary, TVC

1. Upazila PTA Member Secretary Orientation-4,000; 2. Institution level orientation for PTA, SMC/MMC members-95,544; 3. Planning Meeting with Institution Heads-1,300; 4. Cluster wise cultural programs-1,300; 5. Folk Song-2,400; 6. Street Drama-242; 7. Documentary-2 and TVC-2.

1. Upazila PTA Member Secretary Orientation-4,000; 2. Institution level orientation for PTA, SMC/MMC members-95,544; 3. Planning Meeting with Institutional Heads-1,300; 4. Cluster wise cultural programs-1,300; 5. Folk Song-1,836; 6. Street Drama-194; 7. Documentary-2 and TVC-2.

(e) Strengthening the capacity of school management, monitoring and assessing teaching- learning outcomes at the secondary level;

School Management accountability (SMA)

1. FY 2012–13 = 6,700 inst. 2. PTA Training 67,000 persons; 3. January–December 2014 (2 semesters): SA 6,700 inst., 6,700 ICT 6700 Inst., 6,700 4. January–December 2015 (2 semesters) SA 9,600, 9,600 inst. ICT 9,600, 9,600 inst. 5. January–December 2016 (2 semesters) SA 11,750, 11,750 inst. ICT 11,750, 11,750 inst. 6.. January–December 2017 (1 semester) SA 10,500, ICT 9600

1. FY 2012–13 = 6,381 Inst. (95) 2. PTA Training 67,000 persons 3. January–December 2014 (2 semesters): SA 6264, 6285 inst., ICT 1115, 1567 Inst. 4. January–December 2015 (2 semesters) SA 6,018, 8,676 Inst., ICT 1,779, 1,113 Inst. 5. January–December 2016 (2 semesters) SA 8,425, 10,479 Inst. ICT 3,339, 6,334 Inst. 6. January–December 2017 (1 semester) SA 10,477, ICT 7558


Page 59 of 84

PTA Formation (Institutions)

12,117 12,117 (100)

Social Audit Grant 12,117 10,409 (86)

ICT Grant 12,117 7,358 (81)

EACM Grant 12,117 10,409 (86)

(f) Improving and strengthening the capacity of DSHE through institutionalization of monitoring and evaluation activities.

Monitoring and Evaluation

Monitoring

Semi-Annual Monitoring Report Published

16 16 (100)

Evaluation Need based 4

G. DESCRIPTIVE REPORT

1. General Observations/Remarks of the Project on:

1.1 Background

Bangladesh has recorded impressive progress in poverty reduction, and certain human development outcomes. Gross enrollment rate specifically in primary and secondary education significantly increased. In primary level it is around 90 percent, while secondary enrollment has more than doubled since independence. The gender parity target set in the Millenium Development Goals (MDG) has already been achieved at both primary and secondary level of education. At the secondary level, the MOE has introduced a unique system of public-private partnership. Through this partnership more than 98 percent of the secondary schools are managed and operated privately with 100 percent salary support from the government. Besides these, the GoB undertakes projects for constructing buildings in private schools. The result of these initiatives reflects in the threefold increase in overall secondary enrollments and sevenfold increase in female enrollments since 1980 (Reference ). All these initiatives have resulted in the successful expansion of the private sector via supply-side incentives and the gender-targeted stipend program since the early 1990s.

The GoB, with the support of IDA initiated the first FSSAP during 1993–2001. It played a vital role in fostering gender parity in access to secondary schooling in rural areas. IDA funded the implementation of female stipend program in 121 upazilas, while the remaining upazilas were covered by the GoB, ADB and NORAD. FSSAP 2 helped sustain the previous interventions through a series of innovations and pilot measures to improve quality, equity and accountability. However, key challenges remain: (a) low systemic quality reflected in low completion rates and declining number of SSC Examination appearing students; (b) many poor children still left out of the secondary school system; and (c) institutional capacity remained weak. The management, coordination and implementation capacity needs to be strengthened from the central to the school level; (d) M&E both at central and local level remained weak. These common challenges need to be addressed adequately so that the landmark achievement could be sustained. Otherwise the dropout rate would increase, retention rate would reduce, pass rate would continue to fall, access of the poor and girl students would decline and child marriage would increase.


Page 60 of 84

At this crucial stage, the MOE of the GoB initiated the launch of the SEQAEP to improve quality of secondary education, facilitate equitable access, and monitor teaching-learning outcomes. To build upon the successes of the past decade and to address challenges described above, SEQAEP retained many of the previous interventions and galvanized its focus on improving quality and expand the reach of education to both poor girls and boys. The project was implemented in remote 250 upazilas of Bangladesh.

1.2 Justification/Adequacy

SEQAEP was implemented primarily in 125 upazilas which was scaled up to 250 including AF SEQAEP Upazilas (90 + 35). The upazilas are from the remotest, poverty stricken, and hard-to-reach Char, Hills, Haor, and Bour areas. The focus of the project was to improve quality and access enhancement of both poor girls and boys. So the undertaking of the project was justified, adequate, and appropriate. Thus, the cost of not undertaking such a project would be manifold. Secondary education is strongly correlated with economic development and increased productivity and is one of the key solutions to breaking the vicious cycle of poverty.

The interventions of the project were in line of the aspiration and target of MDGs, SDGs, The National Education Policy 2010, PRSP, and other important documents. The purposes of the project interventions were to provide quality education and increase access in secondary education. The project also built institutional arrangement to ensure School Management Accountability. On the other hand, the project introduced internationally accepted PMT Based Stipend and competency-based learning assessment. This project addressed a large group of disadvantaged and ensured community participation.

1.3 Objectives

The key project objectives were to improve the quality of secondary education, facilitate equitable access, and monitor teaching-learning outcomes of the secondary education in Project Upazilas. The specific objectives of the project were

(a) Improving the quality of education particularly, the teaching-learning process by arranging additional classes at the secondary level (grades 6-10), developing reading habits and providing academic performance based incentives;

(b) Ensuring equitable access by providing stipends and incentives to the poor boys and girls, subvention against tuition fees to institutions at the rural and semi-urban areas;

(c) Reducing the rate of dropouts at the secondary level by providing incentives and simultaneously creating awareness among the parents and community people;

(d) Strengthening the capacity of school management, monitoring and assessing the teaching-learning outcomes at the secondary levels;

(e) Attracting the new and retaining existing students especially the girl students in schools through the provision of safe drinking water and sanitation facilities; and

(f) Improving and strengthening the capacity of DSHE through institutionalization of M&E


Page 61 of 84

activities.

1.4 Project revision with Reasons

The original SEQAEP was approved by ECNEC on May 26, 2008. Its implementation period was July 1, 2008, to June 30, 2014. But due to obvious reasons the project was revised for 4h times. The main features of the revision are given below:

1st Revision

A Midterm IDA Mission, carried out during May 12–28, 2011, identified certain critical issues and constraints and recommended to revise SEQAEP DPP to remove the impediments and bottlenecks in implementation. The main features of the 1st revision with justification are given below:

(a) Revised Project Description: Based on the MTR recommendation some adjustments were made to projected description.

(b) Enhancement of PMT Based Stipend: As per the decision of the Steering Committee Meeting the beneficiary falls under the coverage of 50th percentile would continue to be provided PMT Based Stipend.

(c) Revised Result Framework and Monitoring: The KPIs and IOs needed to be fine-tuned by elaborating data sources and formulae, updating baseline and target values to reflect the revision of data sources and data formulae to improve efficiency of monitoring.

(d) Restructuring of SEQAEP Manpower: SEQAEP has two DD posts that hinder the efforts toward a more articulated management style. The MTR recommended eliminating two DD positions and creating one Assistant Project Director position. Moreover, the project would deploy a Management Consultant, Procurement Consultant, and Quality Consultant to facilitate the project authority to implement the project.

(e) USEOs would perform a series of tasks under the project and act as a focal point for timely and reliable data and information for project management to undertake corrective and preventive measures. Besides they performed various tasks; however, they had not been able to fully perform the expected role. In SEQAEP, upazilas were not sufficiently equipped with existing operational budget as well as transportation and other logistics. To strengthen the USEOs and facilitate as well to accomplish the tasks, the project would finance under the cost category.

(f) Discrepancies between the FA and the Development Project Proforma: There were several inconsistencies between the FA and DPP. According to the IDA General Conditions for Credit and Grants, dated July 1, 2005, SEQAEP’s FA was the only binding document underpinning the implementation of SEQAEP. In case of possible contradictions with existing local laws or regulation that might impact project implementation, the FA would prevail.

Meanwhile the FA had been amended to allow IDA financing of books and materials under additional classes and reading habit program. Moreover, it included operating costs that were retrospectively


Page 62 of 84

effective going back to the project effectiveness. Therefore, the DPP needed to be revised to remove the inconsistencies and reflect the sharing of the GoB and IDA in the light of categories of disbursement appended in the amended FA.

(a) In the DPP the rates of honorarium, remuneration, and other expenditures for orientation, workshop training, meetings which creates obstacle in implementing various programs. Again, DPP lacked detailed cost breakdown and description for different components and subcomponents that were given in the DPP;

(b) In addition to the above, the revision proposed to procure a cross-country vehicle for SEQAEP; procure furniture for SEQAEP, MEW, and USEOs; and provide equipment, computers, and accessories for SEQAEP and MEW.

2nd Revised DPP

Initially, the implementation period of SEQAEP was July 2008 to June 2014. In 2011, the World Bank conducted a an MTR Mission which identified the overall implementation progress of the project as moderately unsatisfactory, because there were a bundle of critical issues impacting implementation. But immediate after the 1st revision, the project gained momentum and pace due to changed implementation modality, refinements of project design and inclusion of innovative intervention. As of December 2013 it was found that the project either achieved or surpassed the project end targets compared to baseline of 2008 in most of the cases. This progress was recorded well ahead of project end at June 2014. Most of the project targets had already been achieved and in some cases surpassed the original targets well ahead of remaining one and half years of the project life. However, the fund for PMT Based Stipend and Tuition also had been exhausted and it was difficult to continue the Stipend Program beyond June 2013 until the terminal year of the project. Accordingly, the GoB and IDA conducted a series of missions and agreed to provide AF. The mission noted that the impact of PMT Stipend program would be maximized if the 2012 cohort beneficiaries of the PMT Stipend would be provided with an opportunity to complete the secondary cycle in December 2016. The IDA Mission recommended that the GoB consider an AF approach that extend the project to December 2017 and allow a more harmonized SWAp that the GoB was planning to initiate by early 2018. Finally, IDA committed toward providing US$ 265 million with increased coverage of new 90 upazilas in addition to existing 125 SEQAEP upazilas. In view of the above, the 1st revised DPP was required to revise for the 2nd time. The following were the important reasons for the 2nd revision:

(a) Creation of provision for AF by the GoB and IDA. IDA committed to provide US$265 million, while the counterpart (GoB) would be equivalent to US$65 million, of which IDA would reimburse US$50 million to the GoB upon meeting the agreed DLIs.

(b) Create provision to extend project duration and inclusion of additional new 90 upazilas beyond existing 125 upazilas. The duration of AF implementation would be over the period of four years, January 2014 to December 2018. The SEQAEP activities would be continued in existing 125 upazilas and rolled out in new 90 upazilas. To accommodate the extended duration and inclusion of 90 new upazilas, the DPP had to revised for the 2nd time.

(c) It was proposed that IDA credit funding would be a combination of transaction based (US$ 215 million) and result based (US$ 50 million). The DPP was required to revise to reflect the


Page 63 of 84

financing modality.

(d) It was proposed to enhance the Incentive Awards Rates. The incentive for BSAA would propose BDT 1,000.00 instead of BDT 500.00 and PMT SSC Pass Award was proposed to BDT 1,500.00 per SSC or equivalent pass students. On the other hand, three best performing institutes including one Madrasha of each upazila would receive at the rate of BDT 1 lakh per institute.

(e) The use of updated PMT Formula, refine booth operation and introduction of Cash Card for disbursement of stipend required revision of DPP.

(f) The DPP was revised for the 2nd time to create provision of three School Grants (EACM, Social Audit, and ICT) under School Management and Accountability.

(g) To reflect change in Management Structure, transfer of MEW Manpower to revenue budget, to create provision of ACTs for English, Mathematics and Science subjects, Engagement of Technical Assistance support for ACTs the DPP was revised for the 2nd time.

3rd revised DPP:

The salient features of the 3rd RDPP reflected in the project description and the reasons for revision were as follows:

• The third revised DPP extended to 35 additional remote, backward, and deserving upazilas;

• No additional funding was required since the project authorities were committed to utilize the available resources through inter component and subcomponent budget adjustment in a more efficient and effective manner.

• The required cost of rolling out SEQAEP activities into 35 new upazilas would be met through reallocation of component and subcomponent-wise budget allocation.

2. Rationale of the Project in Respect of Concept, Design, Location, and Timing.

The initial concept of this project was based on FSSAP 1 and FSSAP 2 and as per Baseline Survey 2007 including Project Appraisal Documents, in June 2008. IDA also carried out a feasibility study in 2008 to find the rationale of the project with respect to concept, design, location, and timing. The rationale of SEQAEP was to contribute to the policy of Bangladesh Government to poverty reduction through improved and sustainable education, create a sustainable environment to enhance access to education, and reach the Millennium Development Targets for secondary education. The project was designed in such a way that all the stakeholders from central to community level could participate and contribute to achieve the goals. Based on the concept, the design and location of the project were determined and the project was implemented in 250 remote and backward upazilas where the demand for quality education was high and equitable access was poor. Considering the implementation of interventions, areas, and beneficiary, the timing was set. In view of the above, the project was rational in respect to concept, design, location, and timing.


Page 64 of 84

3. Brief Description on Planning and Financing of the Project and its Applicability

• Project identification. Education, particularly secondary education, is the thrust sector of Bangladesh economy. Compared to investment in secondary education, the return is manifold, while gender disparity is a priority area that needs to be addressed to improve quality of education and increase access to education. The interventions of SEQAEP contributed to reduce gender disparity in access that resulted in women empowerment. A joint IDA and Bangladesh Team carried out a Project Identification and Feasibility Study in 2008 which ascertained that quality education and increased access could reduce dropout rate, increase retention rate, and contribute to secondary cycle completion. It also identified the key areas of intervention; suggested institutional arrangements for service delivery; and recommended a priority list of upazilas considering poverty, backwardness, remoteness and hard to reach.

• Project preparation. SEQAEP was designed by the MOE with the support of IDA. As the executing agency DSHE Education coordinated all the activities related to project preparation. The project Steering Committee headed by the Secretary, Secondary and Higher Education Division, provided overall guidance in relation to the policy formulation and setting strategy and budget.

• Appraisal. A joint team of IDA and Bangladesh carried out project appraisal on May 8 2008, and recommended to implement SEQAEP.

• Credit Negotiation. Credit Negotiation was held between the Delegation of the GoB and the delegation of International Development Association (IDA) at the World Bank in Washington D.C. during June 23–24, 2008.

4. Analysis of the Post-Implementation Situation and Result of the Project

4.1 Whether the beneficiaries of the project have clear knowledge about the Target/Objectives of the project

In each of the Project Upazilas and institutions several Workshops, Orientations, Trainings, Awareness Campaigns, Social Mobilization, Community level Meetings were held with the beneficiaries and stakeholders to provide a clear idea about the activities and the project, its objectives, implementation modalities, targets and benefits. The SMC/MMC, PTA functionaries, local educationist and LGI Functionaries were also oriented. Later, the project went on, the bottom and field level beneficiaries got the knowledge on the objectives of the project and targets with regard to their areas.

4.2 Programme for use of created facilities of the project

Although SEQAEP provided benefits to the rural poor students basically, it was a female targeted stipend program. The interventions highly contributed to reduce gender disparity in access to secondary education. Besides, SEQAEP installed about 12,000 water and sanitation options including Wash Block, Twin Latrine, Class Room Renovation, Deep Tube Well, etc. To ensure operation and maintenance a user committee was formed in each institution. The Parent Teachers Association which is an organization of


Page 65 of 84

mutual cooperation of parents and teachers work together to ensure congenial school environment. SEQAEP established a school library in each Project Upazila to create Readers and deployed a Librarian cum Organizer to continue the library after the phase out of the project. Training, hands-on orientation, and follow-up motivations were provided to the user’s committee, Librarian-cum-organizers and the PTA Members. Follow-up motivations were also given to communities to retain their learnings on SEQAEP activities and benefits of using the services delivered and the institutions created.

4.3 O & M Programme of the Project

Basically SEQAEP was a female and poor targeted stipend program and does not require any sort of O&M. In each upazila there is a well-equipped office of USEO that could address any problem of O&M, in case of necessity.

4.4 Impact of the Project

4.4.1. Direct

Direct impact of the project interventions will be measured through an impact study at a later stage. However, to measure immediate impact or result achieved the Semi-Annual Monitoring Reports and Annual Education Institution Census Reports are relevant. Since inception, significant progress was made by the project based on different key indicators. The direct impacts are given below as per the Revised Project Result Framework:

• The completion rate in grade 10 (primary school completers who complete grade 10) was 36% in 2016 compared to 20% in baseline in 2008.

• In base year the number of students appeared in SSC Exam were 187,214. As of 2016 actual achievement was 578,000 compared to target 485,000.

• 3 rounds of LASI were completed of which one round was carried out nationwide. Moreover, Annual Education Institution Census was conducted times.

• The gender parity in enrollment was calculated 0.85% in 2016 compared to 0.82 in baseline. The project target was 0.90.

• The percentage of poor children enrolled in secondary school was estimated at 30% percent in baseline, the achievement was 42% in 2016, while the project end target was 39%.

• The targeted total number of beneficiaries and the share of female were estimated to 41,50,000 and 54% respectively in AY 2016. The achievement was 52,54,000 in 2016, of which the share of female students was 54%.

4.4.2 Indirect

Out of 13 projects implemented by DSHE in secondary education SEQAEP was the most successful pro-poor project. It had some indirect impacts described in the following:


Page 66 of 84

• Reduced the dropout rates significantly.

• Secondary education cycle completion rate increased.

• Woman empowerment increased.

• Readers among non DRH students increased.

• Competitiveness among SEQAEP and non SEQAEP students was prevailing and gradually increasing.

• Early marriage the ever great challenge had reduced.

• One of the major threats Eve Teasing reduced so that the school as well as the road to school had been secured and safe specifically for the girl students.

• Community participation had increased and strengthened.

• Reduced incidences of sufferings from water, excreta and arsenic related diseases that minimized the rate of absence in school.

4.5 Transfer of Technology and Institutional Building through the project

SEQAEP did not introduce any technology and building institution that can be transferred to the beneficiaries, stakeholders and communities. However, the project successfully implemented different interventions, such as ACT Scheme, Development of Reading Habit, PMT Based Stipend, Processing and Screening of Appeal Settlement, Social Audit and ICT Grant. Most of the institutions are now well conversant to the process of recruitment (On Line), Library operation, PMT Booth Operation, PMT implementation modality, processing and screening application including appeal settlement, conduction of Social Audit and e-mail correspondence to receive ICT Grant. By now most of the institutions have gained technical and managerial capacity to implement SEQAEP.

4.6 Impact on environment

This was a social sector project. Therefore, it had no scope to create any negative impact on environment like land, water, air, biodiversity etc. However, recommendations of the EMF that was disclosed for public information followed during implementation of school facilities under the project.

4.7 Sustainability of the Project

While a significant number of secondary institutions have been operated both in public and private sector in the project areas the actual enrollment is yet to be universal especially when it comes to the national definition of access to sustainable education for all. However, SEQAEP trained, oriented, mobilized, motivated and aware most of the beneficiaries and stakeholders so that they can carry on many of the project interventions. It also formed various committees including PTA and enhanced the capacity of SMC. It is expected that after the phase out of SEQAEP the community will phase in to continue to implement


Page 67 of 84

the best practices.

4.8 Contribution to Poverty Alleviation/Reduction

Education has been found instrumental to reduce poverty level substantially in Bangladesh. SEQAEP was one of the most successful pro-poor safety net projects, which adequately contribute to the GoB’s commitment toward poverty reduction. Most of its participants and beneficiaries were from poorer segment of population of which 58% are girl students. An educated man can support himself through receiving quality education which is one of the important factors that contribute to poverty reduction. The project developed systems for institution-based training mainstreamed into SEQAEP for the poor students, increased access of working/drop out children and females to SEQAEP activities and employability, strengthened capacity of key agencies, under privileged and people with disabilities.

4.9 Opinion of the public representatives, local elite, local administration, teachers, religious leaders, women's representatives etc.

There was no formal assessment or impact study in this project, but the project authorities and IDA jointly organized experience and learning sharing workshop at BSK Auditorium in Dhaka. Different level of stakeholders, institution heads, religious leaders, women's representatives, Upazila Administrations, USEOs, LGI Functionaries, District Administrations, Honorable Member of Parliament and Development Partners were attended the workshops. The stakeholders expressed their deep satisfaction over the achievement of SEQAEP. The participants recommended to continue the best practices of phased out SEQAEP through implementing the proposed and planned SEDP. Simultaneously they expressed their grave concern regarding the negative impacts of non-undertaking the SEDP. From the workshop the following lesson learnt and recommendations were drawn: …………………

4.10 Problems Encountered During Implementation (with duration and steps taken to remove those)

5.1 Project Management No problem arose with the project management.

5.2 Project Director No problem arose.

5.3 Land Acquisition There was no provision for land acquisition.

5.4 Procurement No major problem arose with procurement.

5.5 Consultancy There was no problem with Consultancy.

5.6 Contractor No major problem arose with implementing partner and Contractors.

5.7 Manpower During execution of the project there was no problem in recruiting manpower.

5.8 Law & Order Law & Order was congenial and favourable.

5.9 Natural calamity There was no major natural calamity during project implementation period.

5.10 Project financing, allocation and

Release

Most of the things were in order.

5.11 Design formulation/approval

There was no problem with the formulation/approval.

5.12 Project aid disbursement and reimbursement

No problem arose in project aid disbursement and reimbursement.


Page 68 of 84

5.13 Mission of the development

Through the project period several Review Missions were conducted, but many of their recommendations could not be executed.

5.14 Time & Cost Over-run The project had 4 years time over-run to implement AF SEQAEP and 80 crore Taka over-run from GoB contribution.

5.15 Project Supervision/Inspection

The project was supervised by Ministry, Directorate, Project Staff, Project officials, World Bank Representatives, etc.

5.16 Delay in Decision None

5.17 Transport Transport procured on time

5.18 Training There was no problem arose in providing training.

5.19 Approval No problem in approval

5.20 Others None

6. Remarks and Recommendations of the PD

(a) National Education Policy 2010 adopted by the GoB attaches the highest priority to secondary education. The target was to improve the quality of education through teaching-learning process, increase equitable access, reduce the dropout rates and retain the existing students. To realize the over-arching objective of the policy the government implemented ‘SEQAEP.’ This was IDA supported one of the largest pro-poor projects in secondary education subsector. The project implemented a good number of innovative interventions that brought a silent revolutionary change both in improving quality and increasing equitable access to ensure Education for All in Project Upazilas. To realize the benefits of the project it is recommended to roll out the innovations and best practices through the proposed SEDP covering all the upazilas.

(b) It is worth-mentioning that SEQAEP was a follow on project of FSSAP I & II. They were basically stipend providing projects for female students that contributed highly to reduce gender disparity in access to secondary education. SEQAEP introduced internationally comparable PMT with the support from the GoB and IDA. PMT Based Stipend and Tuition subvention had been successfully implemented by SEQAEP since 2009. It played a great role to increase the access of girl students and reduce the drop out rates. Although the stipend rate of SEQAEP was considerably encouraging the rate should be sustainably and adequately enhanced in future planned program.

(c) PMT Stipend and Tuition for the poor students, ACT support and DRH program are the most important, significant, effective and very popular program. The Government, particularly the MOE and DSHE is committed to roll out these programs in all secondary institutions across the country. It is essential to continue these activities after phase out of SEQAEP, so that the benefits received by all deserving beneficiaries could be secured and continued. If required and the planned sector development program is delayed in any case, there should have appropriate alternative arrangement.

(d) Poverty and economic condition compel many of the poor students to engage themselves in income generating activities. Accordingly they could not be able to complete the secondary school cycle. To reduce the dropout rate a family compensatory support is instrumental and could contribute significantly. So, along with increased PMT based stipend


Page 69 of 84

family compensatory support will be crucial to reduce dropout rates as a sustainable initiative. This is a potential area to strengthen the cooperation of parents and extend the scope of community participation. The government and the development partners should give special focus on it and may take further initiative.

(e) DRH was a creative program that inspired the students to read other books and gather knowledge. It provided them the opportunity to avoid spending time leisurely and in immoral practices. They possess high moral and social values that would create deep impact in life. It helped to create competitive attitude and behavioral change among them. Generally the students were involved in reading curriculum based books and in SEQAEP institutes they used to read other books under DRH program. In many of the schools the students did not have adequate recreational facilities including playground. So, they suffer from class fatigue and fully or partially fail to develop their creativeness. In many of the SEQAEP institutes where ACTs were working the beneficiary students became inspired to engage in participating debate, cultural events, education fair or other extra and co-curricular activities. It helped to build confidence, self-respect and competitive attitude which in turn motivated them to translate their dreams into reality. The students of all secondary level institutes should be encouraged to be involved in extra-curricular activities like developing reading habits.

(f) It was observed that most of the secondary institutions in Project Upazilas were suffering from adequate safe drinking water, improved sanitation, sufficient class room, science lab and other required facilities. Thus there was a huge demand for small scale structure. Due to fund scarcity the institutions were not able to provide the required facilities. As a result, the students did not feel safe to attend school regularly and contribute to increased dropout rate. SEQAEP introduced ISF intended to to attract and retain the girls and boys students in school through the provision of safe drinking water, sanitation and other small scale infrastructures to selective schools based on need assessment using community contribution. The facilities were likely led to better health outcomes for children and increase their aptitude to learn. It is recommended to continue the support for ISF facilities to ensure healthy and congenial environment in schools.

(g) The MEW of SEQAEP was mandated to prepare and publish Semi Annual Monitoring Reports since inception of SEQAEP. The objective of publishing the report was to assess the status of implementation and achievements of all interventions of the project. In addition to that the MEW was also responsible to monitor the projects of DSHE. During the implementation period of SEQAEP MEW monitored 13 projects of DSHE including SEQAEP. The MEW published 17 reports up to June 2017. The reports were very useful, effective and important to project managers and decision-making authorities. The reports also provided inputs to measure the achievement, impacts of the projects, KPI and IOIs. The report helped the project management to keep track of the interventions and activities during implementation. It is recommended to continue to prepare and publish the Semi Annual Monitoring Report through independent assessment in future program since it played a vital role to document and make available relevant information, such as KPI, IOIs, physical progress.


Page 70 of 84

7. Remarks/Comments of Agency Head

ACT Scheme of SEQAEP was a challenging and highly demand-driven program. At institution level the Head Teacher, Subject Teacher, Student and the parents were happy and satisfied with the Teaching-learning process of ACTs. The implementation of ACT Scheme was challenged by a couple of constraints.

• The huge task was to implement by the TA Agencies was done by the ACT Team comprised with 4 Officials only.

• In first phase ACTs in few positions for EL and Mathematics subject was not possible to deploy due to non-matching with home upazila, neighboring upazilas, and districts.

• Due to non-availability of applicants some positions remained vacant.

• Dropout rate gradually increased compared to total deployment and the working ACTs.

• No IT support and field level monitoring were available, since the TA had to provide support.

• Availability of applicants from hill districts, Sylhet Division were low.

In future program there should have provision to fill up vacant positions from national merit, deployment of TAs without delay, non-demanding institutions should be replaced by demanding ones and provision of mutual transfer in same upazila and district.

SEQAEP introduced three Incentive Awards, namely, Best Student Achievement Award (BSAA), PMT SSC Pass Award and Institutional Achievement Award since inception. BSAA encouraged the students of grade 6-9 including JSC/JDC to engage them in a healthy competitive attitude. The Award was given in the form of money and a certificate. It was observed that the students became attentive to their lessons and thus they reach the desired objectives. PMT SSC Pass Award created a good opportunity for the poor students of SEQAEP institution who got admission in higher secondary level. The award was given in the form of Tk. 1500/- as financial support to secure admission. As a result dropout of a large number of both boys and girls students reduced. The IAA was given to 3 secondary institutions including one Madrasha of each upazila in the form of Tk. 1,00,000.00 to each awardees’ institution. It made the teacher and education managers responsible to their duties toward the students and to institution. It created a healthy competition among the secondary institutions as well. It was observed that the competition inspired the policy makers to support institutions to provide quality education. Thus the provision of providing Incentive Awards should be retained in the future program.SEQAEP introduced LASI to assess the quality level of education in the secondary institutions. The objective of the survey was to measure the quality of learning by administering internationally comparable and national curriculum based numeracy and literacy testing on a sample basis which will be considered representative at national level. Four rounds of learning assessment were conducted in 2012, 2013, 2015, and 2017. Reports of the assessment were prepared, published and disseminated through National Workshops. The findings of the reports were mainly confined to: (a) difference of quality levels between madrasa and schools, (b) difference of quality levels between boys and girls, (c) difference in English, Mathematics and Bangla between, boys, girls, schools and madrasa between urban & rural area (d) difference of quality between division to division etc. The findings of this independent survey provided feedback to policy makers and stakeholders to raise


Page 71 of 84

their awareness to adjust quality-related policies and interventions accordingly. Considering the importance of assessment LASI should be continued in future program to build the appropriate capacity to mainstream the assessment system.

8. Remarks/Comments of the Officer in Charge of the Ministry/Division

(a) ACT Scheme was one of the important and popular interventions introduced by SEQAEP. It was expected that the ACTs as a young and dedicated change maker would take teaching as a dignified profession and would continue as a teacher to support the lagging behind and weak students in difficult subjects. But the real scenario was far away from that of desired one. It was observed that many of the deployed ACTs left the job to avail better options and the rate of turnout was excessively high. If it was continued to happen then the objective of introducing ACT Scheme would face a serious setback. Since the ACTs had been facing several problems, such as, job security, sustainability, remuneration, etc. need to be addressed on priority basis.

(b) The secondary level school and madrasha start at 10 am and close at 4 pm in most of the upazilas. Some institutes start even at 7 am. Due to school hour many of the students of the poorer families usually rush to school without having breakfast or tiffin. Sometimes the students including the AC participants starve for a longtime and suffer from malnutrition. Besides, the beneficiary students could not be able to participate normal classes with full attention due to class fatigue that leads to malnutrition related complicacies. Sometimes they leave the school after leisure and do not return. On the other hand, because of poverty or lack of provision most of the students could not arrange snacks/tiffin. The future program should have the provision for mid-day meal for all secondary level students.

(c) School Management Accountability (SMA) and EACM were two important and challenging innovative sub-components of SEQAEP. The purpose of SMA was to support the MOE’s efforts to increase management capacity, accountability and transparency through SMCs/MMCs, PTAs and concerned stakeholders. SEQAEP introduced conduction of social assessment at school level through PTAs which was an innovative initiative. The objective of EACM was to build and increase awareness among all key stakeholders, Communities and beneficiaries. Under the coverage of these subcomponents three school grants provided namely, Social Audit Grant, ICT Grant and EACM Grant. The grants were provided at institution level to facilitate accountability and transparency, motivate parents to send their children to school, promote good understanding between parents and school managers, raise awareness and orient parents to realize the role to ensure quality education. Through the EACM activities campaigns against Eve Teasing, Early Marriage, violence against women and children and addiction to narcotics by adolescence were organized. As a result the drop outs were brought back to school and retained to complete secondary cycle. In the project area out of 12036 secondary institutions 11,000 were brought under the coverage of grants. The AF SEQAEP initiated to provide ICT Grant to all institutions and SMC/MMCs in project area to use internet in monitoring and communication. The institutions that sent at least 1 e-mail each month were eligible to receive the grant. The first installment of grant was disbursed to ….. institutions in 2nd semester of 2014 which was gradually scaled up to 11,500 institutions in first semester of 2017. The grant developed enthusiasm, skill and knowledge


Page 72 of 84

of target group to “Increase relevance of secondary education to the workforce.” It was observed without ICT knowledge and enhanced institutional capacity the goal of building Digitalized Bangladesh would be far reaching. For sustainability and continuity reasons three school grants program should be included in the future program as priority area.

(d) On completion of SEQAEP, the MEW will be institutionalize through transfer into revenue budget to carrying out M&E of ongoing development and non-development programs/projects of DSHE. In institutionalization process, adequate manpower including expertise should be taken into consideration. Monitoring is the crucial part of the project implementation. It helps the project management to keep track on the project activities and can control the deviations. It is the most critical path to realize project objectives. It was observed that both at field and central level SEQAEP lacked proper monitoring due to absence of skilled, expertise and appropriate process. So, effective and result oriented monitoring of the project should be established. With the insufficient manpower and organizational structure it will be very difficult to carry out M&E of 32 thousand education institutions, 512 upazilas, 64 District and 9 Zonal education Offices and all development projects of the DSHE. Considering the said volume of tasks, additional manpower for the MEW needs to be provided.


Page 73 of 84

ANNEX 7. ADDITIONAL ANNEXES

Annex 7-1: Summary of Achievements - Developing Reading Habit (DRH) Program

Objectives

1. The overall objective of this program is to improve the quality of education in secondary level educational institutions by developing reading habits and skills among students through providing (a) access to a range of selected Bangla and English books in the school library and (b) incentives to enhance use of library services and practice more readings. The goal of the DRH component is to encourage the students to use their leisure time for reading books, both in schools and at home, so that students will develop reading skills which are foundational skills for learning in all other subjects and for lifelong learning.

Implementation Arrangement and Process

2. BSK has been implementing DRH in partnership with SEQAEP to introduce the program across all the Project Upazilas. Target institutions were selected to participate in the program. By 2017, the number of institutions participating in the program was 12,117. The participating institutions provided books, developed strategies for readers groups, and provided orientation to the stakeholders. The project, through BSK, offered 175 books which were selected by a panel of educationists. To manage the library, a teacher was assigned from each institution and was trained on basic library management to carry out duties for their library. Within clusters of institutions, the most efficient librarian teachers were selected as Library Coordinators. The coordinators, along with officials of SEQAEP and BSK, systematically monitored the DRH activities. Moreover, to encourage students and demonstrate importance of reading books, the project gave Award Books to best student readers based on book-reading habit tests. Each year, reading skills of student readers were assessed by BSK to give students feedback on improvements of their reading skills.

Key Outputs

3. Tables 7.1 to 7.3 summarize the achievements up to October 2017 of the DRH program.

Table 7.1. Number of Students Enrolled in DRH by Year and Gender

Phase/Academic Year (AY)

Upazila Number of Institutions

Number of Enrolled Readers Comments

Boys Girls Total

Phase I, AY 2010 26 1,000 43,534 51,106 94,640

Phase II, AY 2011 70 2,500 131,926 152,177 284,103

Phase III, AY 2012 122 4,500 258,596 303,568 562,164

Phase IV, AY 2013 125 6,669 339,117 455,068 794,185

Phase V, AY 2014 125 6,680 341,674 470,288 811,962

Phase VI, AY 2015 215 9,734 640,074 905,371 1,545,445

Phase VII, AY 2016 250 11,982 846,512 1,227,912 2,074,424

Phase VIII, AY 250 11,902 862,849 1,303,931 2,166,780 215


Page 74 of 84

2017 institutions were closed down

Total 3,464,282 4,869,421 8,333,703

Table 7.2. Number of Program and Award Books Distributed by Year

Phase/AY Program Books Award Books Students Received Award

Books

Phase I, AY 2010 170,000 53,283 34,388

Phase II, AY 2011 255,000 216,698 99,041

Phase III, AY 2012 336,000 427,081 207,014

Phase IV, AY 2013 406,831 477,811 295,010

Phase V, AY 2014 29,607 487,770 297,780

Phase VI, AY 2015 1,634,136 743,031 490,528

Phase VII, AY 2016 604,482 1,032,863 651,414

Phase VIII, AY 2017 127,271 Pending Pending

Total 3,563,327 3,438,537 2,075,175

7,001,864

Table 7.3. Number of Teachers Trained for Library Operation

Phase/AY Number of Teachers

Phase I, AY 2010 1,000

Phase II, AY 2011 1,500

Phase III, AY 2012 2,000

Phase IV, AY 2013 2,169

Phase V, AY 2014 11

Phase VI, AY 2015 3,054

Phase VII, AY 2016 2,248

Phase VIII, AY 2017 135

Total 12,117

Other Outputs by Institutions’ Own Initiative

• A total of 4,156 institutions allocated new rooms for the DRH library.

• Many institutions purchased additional books for the DRH library and collected library furniture.


Page 75 of 84

Annex 7-2: Summary of Achievement - Additional Class Teacher Program

Objective

1. SEQAEP’s ACT program is dedicated to provide quality education particularly to develop the English language skills, mathematics, and science knowledge of the students at the secondary level including in madrasahs. It also aimed at reducing the high rate of failures and increasing the pass rate in the secondary-level examinations. In the original DPP of SEQAEP (2008–13), it introduced additional class opportunity through RTs in English and mathematics in 400 institutions selected based on needs. Based on the reality, the AF Project (2014–2017) included science in addition to English and mathematics and expanded this scheme to 2,000 selected institutions in the project intervention upazilas. Under the AF, the RTs were renamed as ACTs.

Implementation Process

2. The schedules of additional class were either before or after the regular school hours. RTs/ACTs were trained by SEQAEP on using teaching aids and multimedia to make to make the class interesting and effective to the students. The training program for the RTs/ACTs also aimed at strengthening capacity of other teachers of the selected schools. The additional class scheme also contributed to stop the need for private tuition and reduced the phobia of English, mathematics, and science among students.

Mobilization and Capacity Building of RTs and ACTs

Resource Teachers (First Phase, Original DPP)

• Under the original DPP, 400 RTs in English and mathematics were mobilized in 55 upazilas, who continued up to 2013.

• A total of 556 and 612 RTs were trained by the project in English and mathematics (due to dropout and replacement of RTs, the number of teachers under training program has increased).

• The RTs were also provided with one-day refresher training by the project.

• The RTs conducted 9,54,525 additional classes up to December 2013.

• In addition to the RTs, the project extended support to build capacity of regular teachers of 3,000 institutions in English and mathematics. These teachers were provided 6-day-long basic and one-day refresher training on two subjects by the master trainers of the project.

• These teachers conducted 1,125,625 additional classes between 2008 and 2011.

Additional Class Teachers in 64 Upazilas (Second Phase, SEQAEP AF)

• Under the AF (2014–17), 9,723 ACTs for English, mathematics, and science have been mobilized in 2013 institutions between 2014 and 2017, following the eligibility criteria. Out of this, 5,200 have been continued up to September 2017.


Page 76 of 84

• To develop knowledge on the SEQAEP activities, one-day orientation session was organized by the project for ACTs.

• For capacity building of ACTs, 6-day-long training session was organized by the project. Under a Memorandum of Understanding with 9 teachers’ training colleges (TTCs), ACTs have been trained. The module of training was prepared by the Training Wing of the DG, DSHE, with technical support from a panel of experts. Up to September, nearly 4,500 ACTs have been provided with training in the TTCs. This basic training for three subjects was organized separately by the TTCs.

• A total of 13, 19, 444 additional classes have been conducted by the ACTs up to September 2017, in addition to regular classes.

Challenges

• Due to the absence of a monitoring firm, regular monitoring of ACTs in terms of competency of quality lesson delivery in the classroom, regular attendance of ACTs, and their attitude could not be carried out. However, junior consultants carried out academic supervision for continuous professional development of ACTs but far less than the requirement.

• Dropout rate was found to be high due to uncertainty of jobs.

• Due to eligibility criteria for recruitment and mobilization, terms and conditions became a challenge to recruit ACTs, especially for English.


Page 77 of 84

SSC/Dakhil Result of ACT’s Institutions under SEQAEP

JSC/JDC Result of ACT’s Institutions under SEQAEP

Year ACT

Institutions JSC/JDC

Appeared JSC/JDC Passed Passed%

Drop Out/Failed

Total %

2014

2005

88188 78179 88.65 10009 11.35

2015 150626 138518 91.96 12108 8.04

2016 148580 137640 92.64 10940 7.36

Total= 387394 354337 91.47 33057 8.53

Year ACT

Institutions SSC/DAKHIL Appeared

SSC/Dakhil Passed

Passed% Drop Out/Failed

Total %

2015

2005

86480 74286 85.90 12194 14.10

2016 101874 93608 91.89 8266 8.11

2017 112626 91207 80.98 21419 19.02

Total = 300980 259101 86.09 41879 13.91


Page 78 of 84

Annex 7-3: Summary of Achievements - Improving School Facilities (ISF)

Objectives

1. The aim of the ISF component is to ensure availability of safe drinking water, provide sanitation facilities for boys and girls separately, and provide arsenic free water in the project-supported institutions.

Implementation Process

2. These facilities were part of the previous FSSAP- Phase II. SEQAEP has been continuing the activities with some modifications. In the implementation process for this component, the local community contributed 20 percent while SEQAEP invested on the remaining 80 percent of the cost. The School and Madrasa Management implemented ISF through community stakeholders. Through this component, the following facilities were provided:

(a) Construction of twin latrines

(b) Setting up of shallow and deep tube well

(c) Rainwater harvesting

(d) Water pump and water tank

(e) Distribution of arsenic testing kits

(f) Classroom improvisation

(g) Solar water treatment

(h) Low-cost wash block

3. Besides, head teachers, teachers, and SMC/MMC/PTA members were provided orientation on how to keep the school clean with better participation of SMC and PTA members, hygiene education for students, cleanliness of classrooms, and development of 12-member student user groups and the process of using them in ISF activities.

Table 7.4. ISF Work Summary

Item 2009 2010 2011 2012 2013 2014 2015 2016 2017 Total

Deep tube well 206 224 300 190 86 245 236 100 6 1,593

Twin latrine 195 570 672 389 0 0 0 0 0 1,826

Shallow tube well

110 266 343 261 0 0 0 0 0 980

Rainwater harvesting

0 15 25 29 0 0 0 0 0 69

Classroom improvisation

0 0 0 0 50 250 570 269 46 1,185

Low-cost wash 0 0 0 0 200 200 200 200 14 814


Page 79 of 84

Item 2009 2010 2011 2012 2013 2014 2015 2016 2017 Total

block

Solar water treatment

0 0 0 0 100 30 20 110 0 260

Wash block 0 0 0 0 0 100 112 2 206 420

Water pump and water tank

0 0 1,125 190 184 179 105 0 0 1,783

Water purifier 0 0 1,365 0 0 0 0 0 0 1,365


Page 80 of 84

Annex 7-4: Excerpts of Feedback at SEQAEP Stakeholders Consultation Workshop for ICR

Venue: BSK

Date: November 5, 2017

Beneficiary Participants:

• District/Upazila Education Officers - Six representatives

• School/Madrasa Head Masters and Teachers - Four representatives

• PTA Chairpersons/Members - Eight representatives

Comments from participants on SEQAEP interventions:

• How did the incentive program help students and schools to achieve higher academic performance?

Answer (Head Master, Lalore High School, Natore): Incentive awards help students continue their education and pay full attention to their studies. Incentive awards enhance their security and thus reduce their chance to drop out, contribute to their good performance and smooth transition to college. Such awards are particularly useful for the poor who may not progress to higher secondary education. Even if students are generally aware of the importance of passing SSC, some students do still need incentives to complete secondary education.

• How are PTAs helping school?

Answer (PTA Chairman, Keshobpur Pilot High School): Yes, the PTA is useful and necessary. It helps to reduce dropout and increase awareness of various issues. It is particularly useful in raising girls’ awareness. Sometimes the PTA can help poor students with books and other educational materials and provide counseling on certain problems. The money they get is used for such purposes. In fact, more money should be given to them to enable them to play a more meaningful role.

• Social audit is a tool for the PTA to promote changes. How effectively is social audit being used?

Answer (Head Master, Choroikul High School, Pabna): The social audit tool is being used by the PTA with good effects on school attendance, particularly on girl students’ attendance, teachers’ attendance as well as on school accountability. The social audit format is used to collect information and then corrective measures are taken. Students’ performance is also monitored and measures are suggested for the improvement of their performance. However, more motivational programs are necessary to make social audit more effective. Local government may also be involved in the social audit process in future.


Page 81 of 84

• How are ISF activities helping students and schools?

Answer (USEO, Gazaria, Munshiganj): ISF can go a long way in attracting and retaining students in schools by providing safe drinking water and sanitation facilities. Small repairs can also be done through ISF. There is also a provision for the participation of the local community and users in the program. However, better planning is needed to make ISF more effective. More orientation and training would be helpful.

• Is LASI needed and successfully implemented and utilized?

Answer (USEO, Gazaria, Munshiganj): Yes, LASI is very much needed because it helps to have an objective understanding of students’ learning levels. Implementation of LASI has also been successfully done. However, it would be good to do LASI after the syllabus is completed so children can be more prepared by then. The piloting should be done in December. Currently, the recommendations put forward by LASI are not properly implemented. There should be more initiatives at the policy level to implement the LASI recommendations.

• How effective is the USEO strengthening program?

Answer (DEO, Mymensingh): Yes, it is effective and through this USEOs are getting good support. However, frequent transfers of USEOs is an obstacle to their strengthening and smooth functioning. Lack of adequate manpower also constrains their capacity, particularly the capacity of supervision and monitoring. Their operating cost is also insufficient.

• How can the M&E data and LASI data be utilized at the local level?

Answer (USEO, Narayanganj Sadar): Some local level initiatives can be made based on the M&E data and LASI data. Special classes can be arranged to meet students’ deficiencies identified in the assessment. Other measures can also be taken but Head Teachers, SMCs, and other stakeholders have to be properly motivated for the purpose.

* The full list of the participants in the stakeholder consultation workshop is available with SEQAEP and

the World Bank mission team.


Page 82 of 84

Annex 7-5: Summary of Findings from the Impact Evaluation Study on Additional Class Program and Developing Reading Habit Program

1. SEQAEP. Bangladesh launched SEQAEP in 2008 with the objective of improving quality of education. SEQAEP has supported innovations toward improving quality education in secondary schools. This short note reports findings from evaluation of two such interventions, ACT and DRH programs. An evaluation was designed and implemented to assess the impact of these programs before rolling out coverage to a wider set of schools in the country. The design of each intervention, impact evaluation design, and findings are described in the following paragraphs.

2. ACT intervention. This intervention was to provide ACTs in targeted schools. The goal was to improve learning levels of students by providing remedial classes (called additional classes) in Grades 6 through 10 in English, mathematics, and science. The number of additional teachers recruited depends on the availability and needs of existing teachers. The ACT program supported both regular classes and an additional 10 remedial classes per grade per subject per month (with additional sections depending on the number of students).

3. Impact evaluation design of ACTs. The impact evaluation design for ACT intervention was an RCT. In 20 randomly selected upazilas, 200 schools were randomly selected, from which 100 schools were randomly assigned to treatment and the other 100 were assigned to control group. Treatment schools started receiving the intervention in 2015 while the control schools were not to receive intervention before 2017. Although the assignment of sample schools to ACT treatment and control was random, some treatment schools did not receive the intended treatment.

4. Sampling, questionnaire, and testing. In each sample school, 15 students from Grade 6 and 15 students from Grade 8 were randomly selected. The total sample size is 2,978 students in Grade 6 and 2,936 students in Grade 8 in the 200 schools. Additionally, head teachers and all English and mathematics teachers in Grades 6–10 were surveyed. Sample students were given LASI in mathematics and English for their respective grades. Students were interviewed to gather information on parental education, occupation, and household assets along with a series of questions on psychological well-being. Subject teachers took the same assessment grade-wise as students. To assess impact, a baseline survey was conducted in 2014 in sample schools before the ACT intervention started in academic year 2015. A follow-up survey was conducted two years later in 2016 covering the baseline sample. In the follow-up survey, Grade 8 students (Grade 6 at baseline) were given LASI assessment for Grade 8. Because LASI assessment for Grade 10 did not exist, Grade 10 students were given secondary school examination in mathematics and LASI Grade 8 assessment in English. In analyzing results, we focus on the Grade 8 sample because learning assessments at follow-up and baseline are strictly comparable here, which is not the case for Grade 10.

5. DRH intervention. The second intervention was to develop reading habit in school children through supporting a library system and a reading program. The program targets about 100–150 students in each school to participate and borrow books for reading. Participation is voluntary and if more than 100 students are interested to participate, priority is given to poor students identified by the poverty-targeted stipend scheme under SEQAEP. Students from non-poor backgrounds can join the scheme by paying a nominal participation fee. About 76 percent of books that students could check out were in


Page 83 of 84

Bangla, and the remaining 24 percent were in English. A student receives an Award Book after reading a certain number of books. All the Award Books were in Bangla.

6. Impact evaluation design of DRH. Similar to the evaluation for ACT, the evaluation design was an RCT. In 20 randomly selected upazilas, 200 schools were randomly selected, from which 100 schools were randomly assigned to treatment and the other 100 were assigned to control group. Treatment schools started receiving the intervention in 2015 while the control schools were not to receive intervention before 2017.

7. Sampling, questionnaire, and testing. Only Grade 6 students were included in the sample. In treatment schools, 30 students were randomly selected, of which 20 would actually participate in DRH and 10 would be nonparticipating control students.13 In control schools, 15 students were randomly selected. In total, the sample covered 4,456 students in Grade 6 in 200 schools. Head teachers and Grade 6 teachers for Bangla and English were surveyed. Information collected from students and teachers was the same as in the ACT sample except students and subject teachers took the LASI Grade 6 assessments in Bangla and English. A baseline survey was conducted in late 2014 before the DRH intervention started in academic year 2015. The end-line survey was conducted after 2 years in 2016 covering the baseline sample. In both the ACT and DRH samples, the main outcome of interest in analyzing the impact of the interventions is student test score. Test score is constructed as percentage of correct score.

8. Findings of impact evaluation. To measure impact, difference in student test score between follow-up and baseline is regressed on a treatment dummy variable for the school. Results are in table 7.5.

9. ACT results. Because some treatment schools did not receive the intended treatment, intent has been used to treat instrumental variable regression to measure impact. The difference in test score between follow-up and baseline is regressed on actual treatment status variable. The instrument for actual treatment status is the treatment assignment variable (whether the school was assigned to treatment or control). In Grade 8, there is a positive and significant impact on test score of 8.75 percent points (19 percent) in mathematics and 9.93 percent points (21 percent of baseline) in English. Splitting the sample by gender, the impact is significant and large for girls and not significant for boys in both mathematics and English.

10. DRH results. Because all treatment schools received the intended treatment, Ordinary Least Squares is used to regress the difference in test score between follow-up and baseline on treatment dummy variable. In Bangla, there is a significant impact of 5.51 percentage points (9 percent) on student test score. On separating the sample by gender, the impact is significant for both boys and girls. In English, there is no overall impact, but on splitting the sample by gender, a significant impact for girls of 5.13 percentage points (9 percent) exists and none for boys.

13 100 beneficiary students (20 from each grade) were randomly selected for participation in the program.


Page 84 of 84

Table 7. 5 Difference in Difference Regression: Dependent Variable Is Difference in Percentage Correct Score between Baseline and Follow-up

Intent to Treat (IV)

ACT Al l (1)

Boys (2)

Girls (3)

Grade 8 mathematics Treatment dummy

8.75*

(0.013) 5.39

(0.17) 11.06** (0.006)

N 2,948 1,267 1,681

Grade 8 English Treatment dummy

9.93** (0.007)

4.74 (0.26)

13.54** (0.001)

N 2,947 1,267 1,680

OLS

DRH All (1)

Boys (2)

Girls (3)

Grade 8 Bangla Treatment dummy

5.51** (0.01)

5.37* (0.02)

5.59* (0.02)

N 4,436 1,800 2,636

Grade 8 English Treatment dummy

4.19 (0.15)

2.82 (0.38)

5.13+ (0.103)

N 4,437 1,799 2,638 Note: p value in parentheses is based on standard errors clustered at school level. ** represents significance level at 1 percent, * at 5 percent, + at 10 percent.

11. Conclusion. Results show that both the ACT and DRH programs have a positive impact on student learning. In case of ACT, there is an overall impact in English and mathematics. By gender, the impact is mainly for girls in both subjects. In case of DRH, there is a positive overall impact as well as impact for both boys and girls in Bangla. In English, the positive impact is only for girls. In the DRH program, over 75 percent of books for reading and all Award Books were in Bangla. This could explain the larger impact observed in Bangla compared to English.