Independent Assessment Report of the Solomon Islands ... · Independent Assessor Report 2018: Solomon Islands Education Sector Support Program v Acronyms and Abbreviations ACER Australian

www.metis-analytics.com

Independent Assessment Report of the

Solomon Islands Education Sector

Program 2 Performance-Linked Aid

Payment

July 2018

Independent Assessor Report 2018: Solomon Islands Education Sector Support Program

i

This report has been prepared by Belynda McNaughton under the management of the Government of Australia’s (GoA) Department of Foreign Affairs and Trade (DFAT), through its consultancy arrangements with METIS Analytics. The report was commissioned through the Solomon Islands Resource Facility, managed by Cardno.

The views expressed in the report are those of the independent assessor and do not necessarily represent the views and opinions of the Government of Australia, the Solomon Islands Government (SIG) or their agencies and personnel.

The support provided by the Solomon Island’s Australian High Commission (AHC) personnel and personnel in the SIG Ministry of Education and Human Resources Development (MEHRD) is acknowledged. The support of the MEHRD Strategic Support Unit in facilitating and managing the review process was invaluable.


ii

Table of Contents

Acronyms and Abbreviations .................................................................................................................. v

Executive Summary ................................................................................................................................. 1

1. Introduction .................................................................................................................................... 5

2. Background and Context ................................................................................................................. 5

3. Assessment Methodology ............................................................................................................... 6

4. Review of performance against 2017 Education Performance Matrix ........................................... 7

4.1 Education Quality .................................................................................................................... 8

4.1.1 Target 1: SISTA 2017 results are analysed to assess improvements in literacy and

numeracy and design .......................................................................................................... 8

4.1.2 Target 2: SISTA 2017 analysis is used to develop strategies to improve the quality of

literacyand numeracy teaching in schools .......................................................................... 9

4.1.3 Target 3: The quality of teaching in primary and junior secondary schools is improved . 11

4.1.4 Target 4: Curriculum support materials support the implementation of the

basic education curriculum ............................................................................................... 12

4.2 Education Access ................................................................................................................... 14

4.2.1 Target 5: Access to basic education for all children and young people is improved ........ 14

4.3 Education Management ........................................................................................................ 16

4.3.1. Target 6: Management capability of Education Authorities (EA) is improved ................. 16

4.3.2 Targets 7 and 8: NEAP M&E Framework is developed and implemented, and data are

analysed; NEAP M&E data are analysed and reported, and used for planning and policy

development purposes ..................................................................................................... 17

4.4 Education Resourcing............................................................................................................ 19

4.4.1 Target 9: The Solomon Islands Government maintains its recurrent budget allocation

for education in 2017 ........................................................................................................ 19

4.4.2 Target 10: The 2017 recurrent expenditure for Basic Education is maintained or

improved ........................................................................................................................... 20

4.4.3 Target 11: Acquittal percentage of 2017 School Grants and distribution percentage of

2018 first tranche grants improve .................................................................................... 21

4.4.4 Target 12: External Audits of MEHRD finances are conducted ........................................ 22

4.4.5 Target 13: The economic and educational benefits of the SIG tertiary scholarship

program

are reviewed and assessed ............................................................................................... 23

4.4.6 Target 14: Solomon Islands Government scholarships budgeting and expenditure

is sustainable ..................................................................................................................... 23

4.5 Public Financial Management ............................................................................................... 24

4.5.1 Target 15: Public Financial Management Action Plan implementation ongoing ............. 24

5. Payment Recommendation .......................................................................................................... 25


iii

6. Strengths and Weaknesses of 2017 Performance Indicators ....................................................... 26

7. Recommendations Going Forward and Proposed Education Targets and Indicators for 2018 ... 27


iv

APPENDICES .......................................................................................................................................... 29

Appendix 1: Education Performance Matrix 2017 ................................................................................ 30

Appendix 2: Terms of Reference ........................................................................................................... 33

Appendix 3: Key Documents Reviewed ................................................................................................ 35

Appendix 4: Key Informant Interviews ................................................................................................. 36

Appendix 5: Curriculum Mapping ......................................................................................................... 38

Appendix 6: 2017 Performance Summary ............................................................................................ 40

Appendix 7: Proposed 2018 Education Performance Matrix ............................................................... 41

List of tables

Table 1: Appropriated recurrent and development budget 2017 ........................................................ 19

Table 2: 2017 First biannual grant disbursement information ............................................................. 21

Table 3: First half year 2018 grant disbursement summary ................................................................. 22

List of figures

Figure 1: Number of teacher appraisals for probationary teachers completed ................................... 11

Figure 2: Number of certified teachers 2016 and 2017 ........................................................................ 12

Figure 3: Transition rates years 6-7 and 9-10 ....................................................................................... 15

Figure 4: Linking outcomes to education plans .................................................................................... 18

Figure 5: The percentage of SIG recurrent budget allocated to MEHRD .............................................. 20

Figure 6: Per unit costs for primary and secondary school students.................................................... 21

Figure 7: Increase in basic and senior education grants ....................................................................... 21

Figure 8: Total scholarship expenditure 2016 and 2017 ...................................................................... 24

Figure 9: Percentage of indicators achieved 2015, 2016 and 2017 ...................................................... 25

Figure 10: Summary of indicators achieved by priority area ................................................................ 25


v

Acronyms and Abbreviations

ACER Australian Council for Education Research

AHC Australian High Commission

AWP Annual Work Plan

DFA Direct Funding Agreement

DFAT Department of Foreign Affairs and Trade (GoA)

EA Education Authority

EACI Education Authority Coordination and Improvement

EO End Outcome

EPM Education Performance Matrix

ESF Education Sector Framework

EQAP Educational Quality and Assessment Programme

EYP Early Years Program

GoA Government of Australia

GPI Gender Parity Index

IA Independent Assessment

KO Key Output

LB Learner Book

LEAP Leadership for Education Authorities Program

M&E Monitoring and Evaluation

MEHRD Ministry of Education and Human Resource Development

MEL Monitoring, Evaluation and Learning

MOFT Ministry of Finance and Treasury

NDS National Development Strategy

NEAP National Education Action Plan

NEAD National Examinations and Assessment Division

NEB National Education Board

NER Net Enrolment Rate

NTU National Training Unit

PAR Performance Assessment Report

PEA Provincial Education Authority

PFM Public Financial Management

PPY Pre-Primary Year

SIEMIS Solomon Islands Education Management Information System

SIMS Scholarship Information Management System

SISE Solomon Islands Secondary Entrance (Examination)

SIG Solomon Islands Government


vi

SINU Solomon Islands National University

SISTA Solomon Islands Standardised Tests of Achievement

SITESA Solomon Islands Tertiary Education and Skills Authority

SMT Senior Management Team

SPC The Pacific Community

SSU Strategic Support Unit – MEHRD

TG Teacher Guide


Page 1

Executive Summary

Under the Solomon Islands Education Sector Program 2, a performance linked aid payment is provided yearly to the Ministry of Education and Human Resource Development (MEHRD) on the achievement of agreed targets set out in the Education Performance Matrix (EPM). For 2017, based on the evidence received and consultations with relevant officers on the EPM, the independent assessor has recommended 64 per cent1 of the performance-linked aid payment be disbursed to MEHRD, which equates to AUD1,024,000.

The 2017 EPM included 15 targets (and 31 indicators) across five priority areas: access, quality, management, education resourcing and public financial management. Out of 31 indicators, MEHRD achieved 100 per cent of 13 indicators, partly achieved 13 indicators and did not meet five of the indicators. For 2017, there was an increased focus on quality in the EPM and an increase in the number of indicators from 13 in 2016 to 31 in 2017.

Summary of Performance against Quality Indicators

Most of the quality indicators related to the 2017 Solomon Islands Standardised Test of Achievement (SISTA) and curriculum reform. Delays carrying out the 2017 SISTA analysis made it difficult for other dependent indicators to be achieved. The timeframes for using the findings of SISTA 2017 in professional development programs were not realistic and, for this reason, the assessor reviewed the programs that had taken place since SISTA 2015 to assess progress against this indicator. While professional development has taken place in recent years in relation to improving literacy, there were no evaluations (of teacher practice) to assess how effective these programs have been in changing teacher behaviour in the classroom. Significant progress was made in relation to curriculum development, however, there is still further work to be done in relation to professional development (both pre- and in-service) to support the successful implementation of the new curriculum. Contextual challenges and realistic timeframes need to be considered when setting targets and measuring quality indicators, particularly regarding improvements in learning outcomes2.

1 The actual per cent was 63.9% and it was agreed with AHC for this to be rounded to 64%. 2 DFAT, Office of Development Effectiveness, (2016) Investing in Teachers Evaluation


Page 2

Summary of Performance against Access Indicators

Various analysis and reports have been done in relation to identifying blockages to student participation. However, there was not a clear strategy on how this work will be taken forward and implemented. Transition rates have declined and, without interventions to improve transition, particularly for Year 6 and Year 9 students, these rates are likely to continue to drop. Although the decline is small, data from other sources such as examination placements suggests it may be worse than what is currently showing in SIEMIS. Both the National Disability and Inclusive Education Plan and Gender Equality in Education Policy are in draft form, requiring further work on their implementation strategies and costing. There have been no improvements to the collection of disability disaggregated data in the last 12 months.

MEHRD M&Eref Performance Indicators IA Payment AUD

EO2 1.1.1 SISTA 2017 results are analysed by MEHRD and the findings communicated to schools and

Education Authorities

80% 41,290$

EO2.1 1.1.2 SISTA 2017 results indicate an improvement in student numeracy and literacy at each year level

assessed.

0% -$

KO4.2 1.2.1 Findings from SISTA 2015 are used to develop professional development programs or teacher

support packages for teachers and principals.

100% 51,613$

KO4.6 1.2.2 Professional development programs for teachers and principals are implemented and their

effectiveness evaluated.

60% 30,968$

KO8.2 1.2.3 Teacher and principal SISTA support materials are distributed to relevant schools. 100% 51,613$

KO10.1 1.3.1 MEHRD analyses teacher appraisal data, or establishes new data collection methods, to assess

the percentage of teachers who meet the Teacher Standard and establish a baseline against

which future improvement can be measured.

20% 10,323$

EO2.3/

KO4.4

1.3.2 Increase from 2016 in % of certified teachers in primary and junior secondary schools 100% 51,613$

KO5.1 1.4.1 Revised basic education curriculum is completed for all learning areas. 86% 44,301$

KO8.1 1.4.2 Teacher support materials are developed and distributed to all primary and junior secondary

schools

41% 20,968$

KO4.5 1.4.3 Teacher professional development programs are designed to support the implementation of the

revised curriculum.

10% 5,161$


KO2.1 2.1.1 MEHRD develops and publishes a strategy that identifies and plans the progressive removal of

blockages to student participation in school education.

70% 36,129$

EO1.2 2.1.2 2017/2018 transition rates, for boys and girls, from Year 6 to Year 7, and from Year 9 to Year

10, improve from 2016/2017 rates.

0% -$

KO2.4 2.1.3 The draft Solomon Islands National Disability Inclusive Education Plan is approved and

disseminated.

40% 20,645$

KO2.3 2.1.4 More structured and formalised processes are implemented to collect and analyse data about

the participation rates (including data on what % of school-age children with disabilities attend

school) of students with disabilities in basic education.

0% -$

2.1.5 The Gender in Education Policy is launched. 40% 20,645$


Page 3

Summary of Performance against Management Indicators (includes Education Resourcing and Public Financial Management)

MEHRD has made significant progress in the past 12 months in relation to planning, monitoring and evaluation. The annual work planning process is being replicated among Education Authorities (EAs) and performance management standards for EAs are soon to be launched. The school grants disbursements are working well, however, the retirement target was not met this year. There is increased interest in understanding how school grants are being used at the school level, particularly as Solomon Islands Government (SIG) funding to grants continues to increase. There are ongoing challenges with the allocation of scholarships, causing unbudgeted expenditure blow outs. MEHRD has begun improving data collection and reporting on scholarships to provide a better evidence base for decision makers including Cabinet and the public. Recurrent funding to MEHRD increased to a new high at 38% of the total SIG recurrent budget, however, 44 per cent of these funds go towards scholarships. Public financial management reforms to streamline processes are continuing across MEHRD and also on advice from Ministry of Finance and Treasury (MOFT).

The Office of the Auditor General has a huge backlog of external audits to be conducted across 24 ministries. Progress has, however, been made in the past few months in relation to the 2015 External Audit of MEHRD.


KO7.1 3.1.1 Standards for Education Authorities are developed in partnership with EAs. 70% 36,129$

KO11.1 3.1.2 Baseline data to determine strengths and weaknesses of EAs is collected and analysed. 100% 51,613$

KO11.2 3.1.3 MEHRD implements programs or activities to improve the management capability of Education

Authorities.

100% 51,613$

KO13.2 3.2.1 M&E Framework is approved by Permanent Secretary. 100% 51,613$

KO13.3 3.2.2 M&E Data are collected and analysed 100% 51,613$

KO13.4 3.3.1 NEAP M&E data are reported in the 2017 Performance Assessment Report 100% 51,613$

KO13.5 3.3.2 M&E findings are used to develop 2018 NEAP Annual Work Plan. 100% 51,613$

EO3.3 4.1.1 At least 23% of SIG national recurrent budget was allocated to education. 100%51,613$

EO3.6 4.2.1 Allowing for inflation the 2017 SIG recurrent expenditure for the per unit cost of primary

education is at least equal to the 2016 per unit cost of primary education.

100% 51,613$

EO3.6 4.2.2 Allowing for inflation the 2017 SIG recurrent expenditure for the per unit cost of junior

secondary education is at least equal to the 2016 per unit cost of junior secondary education.

100% 51,613$

KO12.3 4.3.1 At least 80 % of 2017 School Grants are retired on time 0% -$

KO12.6 4.3.2 At least 85% of schools receive the first tranche of their 2018 School Grant by the end of March

2018.

100% 51,613$

KO12.4 4.4.1 At least one external audit is conducted by the Auditor-General’s Department or a suitably

qualified outside provider.

60% 30,968$

4.5.1 A cost-benefit analysis of the scholarships program is designed and approved for

implementation.

40% 20,645$

4.6.1 2017 scholarships expenditure is not higher than 2016 scholarships expenditure. 0% -$

5.1.1 % of activities scheduled in the PFM Action Plan for 2017 completed. 64% 32,975$


Page 4

Achievements, Opportunities and Challenges

The development of a Monitoring and Evaluation Learning Plan has improved the structure around MEHRD’s reporting processes in the past 12 months. The culture of planning, reporting and using evidence to inform decision making is gradually increasing across many areas within MEHRD. The routine collection of data and reporting will improve MEHRD’s ability to demonstrate progress against the National Education Action Plan (NEAP) and identify areas for improvement or changes in approach. The EPM provides MEHRD and development partners an opportunity to highlight performance against specific education outcomes and incentivise progress towards more ‘sticky’ challenges impacting learning outcomes. While some indicators are not necessarily within the control of MEHRD officers, having them in the EPM provides the opportunity for MEHRD and development partners to dialogue and engage strategically across government on issues that impact the education sector.

Given MEHRD’s progress in monitoring and evaluation, there is room to further align the indicators for the independent assessment with MEHRD systems and processes. The proposed recommended changes in the timing of the Annual Joint Review discussed at the Mid-Year Review would further support this alignment and allow for more consistent reporting against the EPM based on the calendar year. The EPM should continue to focus on a standard group of indicators selected from MEHRD’s results framework, with a smaller number of indicators changing annually to reflect emerging policy priorities. The selection of process indicators should be reflective of progress towards tasks put forward by divisions in the Annual Work Plan. To uphold the Statement of Partnership Principles between MEHRD and development partners, the potential inclusion of indicators for development partners within the EPM was discussed at the Mid-Year Review. This, along with a consultative approach to the development of indicators, has the potential to empower MEHRD in this process and provide a more holistic picture of the education partnership and performance.


Page 5

1. Introduction

Under the Solomon Islands Education Sector Program 2, a performance-linked aid payment is disbursed yearly to the MEHRD on the achievement of annually agreed targets set out in the EPM. An independent assessor reviews the performance against the EPM and recommends a percentage for the performance-linked aid payment, as well as indicators for the following year.

This report presents the findings and recommendations of an independent, external assessment of the extent to which the MEHRD and the Solomon Islands Government (SIG) have achieved the targets and performance indicators articulated in the 2017 EPM (see Appendix 1).

The purposes of the independent review, detailed in the Terms of Reference (see Appendix 2), were to:

a) review SIG performance against the agreed milestones for 2017 in the attached performance matrix;

b) provide a recommendation of what performance-linked aid payment should be made based on the SIG performance;

c) provide advice on the strengths and weaknesses of 2017 performance indicators;

d) include MEHRD officers working on M&E in as much of the in-country visit as possible, as a learning experience, while maintaining the assessment’s independence; and

e) provide advice on indicators for 2018, informed by priority policy reforms; the Education Sector Program 2 design; and the MEHRD M&E Framework.

The preliminary findings from the review process were reported at the Education Sector Mid-Year Review conducted in Honiara from 19–20 June 2017.

2. Background and Context

The Government of Australia (GoA) and Solomon Islands Government (SIG) signed a four-year Direct Funding Agreement (DFA) in 2015, which gave effect to the implementation of the Solomon Islands Education Sector Program 2. The DFA, and the related Statement of Partnership Principles between SIG and Development Partners 2016–2020, established the conditions for the provision of up to AUD32 million over four years through direct budget support for basic education. An additional AUD32 million over four years was to be provided for technical assistance and support for Australia Awards scholarships and (through a managing contractor) the TVET sector.

Of the AUD32 million in direct budget support, up to 20 per cent was to be provided as annual bonus performance-linked aid payments, subject to the MEHRD meeting the targets and indicators detailed in the annual Education Performance Matrix (EPM). This equates to AUD1.6 million per annum.

The targets in the 2017 EPM were derived from the SIG Education Strategic Framework 2016–2030, the MEHRD National Education Action Plan (NEAP) 2013–2015 and NEAP 2016–2020, and Development Partners’ priority areas. The priorities for these NEAPs mainly relate to:

the expansion of quality, equitable access and completion of Basic Secondary Education,


Page 6

the design and development of a quality Early Childhood Care and Education system; and

substantial institutional transformations to govern and manage Tertiary Education and

achieve future improvements, developing a more efficient, demand-based and equitable

system3.

This is the sixth year that an independent assessment against the targets in the EPM has been

conducted. For 2017, there was an increased focus on quality in the EPM and an increase in the

number of indicators from 13 in 2016 to 31 in 2017 (See Section 6: Strengths and Weaknesses of

2017 Performance Indicators for more detail).

3. The Assessment Methodology

Performance Cycle

According to the DFA between GoA and SIG, the performance cycle starts at the Annual Joint Review where the Performance Targets for the following year are tabled, and it ends at the following Annual Joint Review where the results of the Independent Performance Appraisal are presented. However, over time, the MEHRD timelines/meeting names have changed somewhat, so now the performance cycle starts at the Mid-Year Review (around June each year). For 2017, this performance cycle period was applied for most output level indicators. However, for some outputs and outcome level reporting, the assessor (with agreement from AHC and MEHRD) reported on the 2017 calendar year as this aligns with MEHRD’s own reporting period and, therefore, the assessor can demonstrate trends over time and use published data available in the recently produced Performance Assessment Report (PAR) 2017, along with the Annual Plan 2017.

Data Collection

Background and contextual information was collected through the review of key documents (Appendix 3) and interviews with key informants at MEHRD, Office of the Auditor General and AHC including relevant technical advisers (Appendix 4). A new Monitoring, Evaluation and Learning Plan (MEL Plan) was developed in 2017 to report against the NEAP 2016–2020 and the Education Strategic Framework 2016–2030, as well as international goals such as the Pacific Education Development Framework (PEDF) 2009-2015 and the Sustainable Development Goals (SDGs).

The 2017 Performance Assessment Report (PAR) is the first to be produced according to the new MEL Plan. The 2017 PAR establishes a baseline and reports on agreed intermediate and end-of-program outcomes. The Annual Report for 2017 was also produced for the first time in many years, reporting on outputs. There is also an Annual Work Plan process that produces quarterly reports for the Senior Management Team. These documents formed a useful basis from which the assessor could verify progress, identify gaps and seek supporting documentary evidence as required. Such evidence included various documents such as policies, meeting minutes, Solomon Islands Education Management Information System (SIEMIS) data and expenditure reports.

The assessor worked closely with the Strategic Support Unit (SSU) to validate findings and where

possible, supplement MEHRD’s existing data collection processes.

3 MEHRD, 2016, NEAP 2016–2020


Page 7

The assessor presented the preliminary findings of the independent assessment at the Mid-Year Review Workshop, held on 19–20 June 2018 in Honiara, and received progress reports and inputs from personnel from MEHRD Divisions on progress to date. The workshop also identified challenges and considered future priorities for the education sector and donor partner support. These recommendations provided important input for the development of the 2018 EPM which was also discussed by participants on day two.

Measuring Performance

The 2017 EPM consisted of five priority areas: quality, access, management, resourcing and public

financial management, with 15 targets and a corresponding 31 indicators. As per the DFA, the

performance-linked aid payment was calculated based on the percentage of performance indicators

that were achieved, multiplied by the total performance-linked aid payment available, which for

2017 is AUD1.6 million. For 2017, as per the DFA, each indicator will have equal weight (1/31). In

some cases, where performance indicators are only partly achieved, a pro-rata performance-linked

aid payment will be made based on evidence of progress. The 2017 EPM provided guidance on what

kind of evidence (documents) would support the indicators, and the assessor weighted the evidence

or the stage at which the evidence was at, to show partial achievement of an indicator.

For the 2017 EPM, the assessor identified several indicators4 where the timeframe for completion

was not realistic given the available human and financial resources within MEHRD, the Solomon

Islands context and other competing priorities. Where this was the case, with the agreement of

MEHRD and AHC, the assessor used workplans or historical trends to provide a more realistic target

for what could be achieved within a 12-month period and assessed progress against this notional

target.

4. Review of performance against 2017 Education Performance

Matrix

This section of the report provides information about the extent to which the 2017 EPM targets were achieved based on MEHRD reporting and evidence provided to the assessor. It also provides comment on challenges, and suggestions about how MEHRD could improve, where necessary, the processes and procedures it uses to collect data or achieve targets.

The publication of the 2017 PAR and 2017 Annual Report provided an excellent foundation from which the assessor could verify progress. Many divisions also prepared a report for the assessor, and discussions and presentations at the Mid-Year Review confirmed much of the evidence on indicator achievement.

4 This related to curriculum development and SISTA 2017 results being analysed and reflected in professional development and learning materials within an eight-month period.


Page 8

4.1 Education Quality

4.1.1 Target 1: SISTA 2017 results are analysed to assess improvements in literacy and

numeracy

Evidence: Preliminary Report on SISTA 2017 and Technical Notes

The finalised SISTA5 results and analysis for 2017 were not available as at June 2018. Part of the SISTA implementation was contracted out to Australian Council for Educational Research (ACER) in 2017 with the support of AHC. A representative sample of students across Solomon Islands sat the assessment in November 2017. MEHRD outsourced the analysis of the data using their own procurement processes (to the SPC’s Educational Quality and Assessment Programme (EQAP)). At the time of the assessment, the data had been cleaned and MEHRD had recently validated the data. A preliminary report including technical notes had been provided to the National Examination and Assessment Division (NEAD) and SSU were in the process of providing feedback to EQAP. It was the view of the assessor that there was still significant work to be done to make the report user friendly for national, provincial and school level reporting. As per the contract (with EQAP), the first draft report of results and analysis are scheduled for release in early July 2018.

Some of the delays to receiving the analysis can be attributed to having two separate contracting processes for the implementation and analysis of SISTA 2017. Previously, the SISTA implementation and analysis was contracted out by AHC, outside SIG systems. It was agreed between Development Partners and MEHRD that for 2017 SISTA, MEHRD would take over this responsibility. However, the planning for and implementation of the SISTA in 2017 was delayed due the absence of key personnel in the NEAD. Due to these delays and the risk of the assessment not going ahead, AHC agreed to take the responsibility for contracting technical support for the implementation (only) of SISTA 2017 and MEHRD took on the procurement of technical support for the analysis of the results.

While using MEHRD procurement systems for the analysis has taken longer than if done by a donor, it is hoped that by using MEHRD systems for this process, its systems will be gradually strengthened and MEHRD capacity to undertake this assessment in future years will be improved.

The timeframes for implementation and completion of the analysis were tight, given the SISTA is always conducted late in the year (November) to coincide with national examinations and save costs. However, given the 2015 SISTA Report and Analysis were completed by March the following year, it was not unreasonable to expect the Ministry to have received the report and commenced dissemination. Given a preliminary report was received and recognition of MEHRD using its own contracting processes for the analysis is a step towards greater sustainability of the process, indicator 1.1.1 is assessed at 80 per cent achieved. Dissemination of the results to national, provincial and school level is scheduled for the second half of 2018.

5 The Solomon Islands Standardised Tests of Achievement (SISTA) is a large scale national assessment, first introduced in 2003, and has been implemented every two to three years since then to a sample of students in Years 4 and 6. Funding for SISTA comes from internal sources as well as development partners.

MEHRD M&Eref Performance Indicators IA

EO2 1.1.1 SISTA 2017 results are analysed by MEHRD and the findings communicated to schools and


80%

EO2.1 1.1.2 SISTA 2017 results indicate an improvement in student numeracy and literacy at each year level

assessed.

0%


Page 9

As the analysis of the results is not yet publicly available6, indicator 1.1.2 is not achieved.

In future years, to improve coordination and timeliness, it is recommended that implementation and analysis of results are contracted together. It is also important that there are enough similarities (linking) in the assessments to ensure they are comparable between years. The analysis of the results should be presented in several user-friendly formats tailored to a variety of audiences7 including headline messages for the Senior Management Team (SMT), reports by province and individual school level reports. The different user-friendly reporting formats should be the responsibility of the contractor and outlined in the contract with MEHRD.

4.1.2 Target 2: SISTA 2017 analysis is used to develop strategies to improve the

quality of literacy and numeracy teaching in schools

Evidence: 2017 Annual Report, Monitoring Visit reports, Professional Development Reports

As per the above target, the analysis of the SISTA 2017 results are yet to be finalised and, therefore, this target could not be achieved. However, it is the view of the assessor (supported by international practice) that the time taken to translate national-level assessment analysis into professional development materials and implement strategies and professional development programs in a systemic way takes time. It was not realistic to expect this to happen within eight months of the assessment being undertaken.

For this reason, the assessor measured progress against this indicator based on the 2015 SISTA analysis8, as it is more reasonable to expect strategies to be developed, changes to professional development courses and then implementation to take several years. Against this analysis, indicator 1.2.1 is assessed at 100 per cent achieved.

The 2015 SISTA analysis9 (and 2013 SISTA) showed poor literacy rates, in particular for writing. For Year 4 students, 60 per cent of students were critically below the required standard in writing. MEHRD, with support from development partners, responded in several ways to develop strategies and professional development courses to target the issues highlighted in the 2015 SISTA. These strategies focused on professional development, particularly untrained teachers, and the importance of strengthening MEHRD’s approach to early childhood care and education (ECCE) and literacy programs in the first three years of primary school.

Specific activities that have sought to improve literacy rates include:

the development of a Pre-Primary Year (PPY) for pre-school age children (aged 5 years);

6 The preliminary report states the 2015 and 2017 SISTA results were not comparable due to insufficient linking items. This was being discussed further with EQAP at the time of the assessment. 7 As a result of additional work by the SSU, the 2017 PAR presented findings from the 2015 SISTA in a much more user-friendly way than the actual report itself. 8 This was agreed with Development Partners and MEHRD at the commencement of the 2017 assessment. 9 The 2017 Performance Assessment Report has reported in detail on the 2015 SISTA results

KO4.2 1.2.1 Findings from SISTA 2015 are used to develop professional development programs or teacher


100%

KO4.6 1.2.2 Professional development programs for teachers and principals are implemented and their


60%

KO8.2 1.2.3 Teacher and principal SISTA support materials are distributed to relevant schools. 100%


Page 10

an increased focus on the use of the vernacular in some schools, including teacher training

workshops, benchmarking and assessment for Grades 2 and 3 and new story books (for Sa’a

and Arosi program);

the Literacy Program Management Unit (LPMU) was set up in 2015, with a specific focus on

literacy training for teachers; the Early Years Learning (EYL) program (two-day workshop)

was carried out between 2015 and 2017 in all provinces;

in the past 12 months, a merit-based selection process recruited eight (out of a proposed

10) Provincial Literacy Trainers (PLTs) to support clusters of schools at the provincial level;

induction has been completed for the PLTs and SINU staff and will complement a revised EYL

training that focuses on improving writing skills;

the Nguzu Nguzu resource books (Year 1 to 3 English books) were revised in light of the

SISTA findings;

SISTA Results were used to prepare classroom-based assessment rubrics to support

assessment practices by teachers in the classrooms;

an on-going Certificate in Primary Teaching for untrained teachers (eight modules in flexi-

mode);

recognising the important role school leaders have in improving learning outcomes, MEHRD

implemented the University of the South Pacific (USP) School Leaders Certificate, with 1,136

school leaders having completed four out of five units by the end of December 2017; and

LEAP is using SISTA to inform support for EAs, school leaders and teachers in literacy.

These courses have all included support materials for principals and teachers. Therefore, indicator

1.2.3 is assessed at 100 per cent achieved10.

The added complexity for MEHRD is that the ongoing curriculum reform agenda means that in-service professional development will need to focus on supporting the introduction of the new curriculum. NEAD have planned for the 2017 SISTA results in literacy and numeracy to be used to inform the curriculum professional development (in-service) to be started in 2018.

While there is a significant amount of professional development occurring, there is little evidence yet to show the effectiveness of these trainings over time11. Monitoring visits by MEHRD officials have occurred, but the focus has been on the training course rather than teacher practice in the classroom. Developing systems and processes to measure what programs are changing behaviours and improving teacher practices at the school level need to be part of a longer-term monitoring plan and will be crucial in determining the value of these investments. Due to there being no data on the effectiveness of the professional development, indicator 1.2.2 is assessed at 60 per cent achieved.

The SSU is currently in discussions with Teacher Professional Development Division (TPDD) about improving monitoring to capture positive changes in teacher practices. The LPMU’s Early Years Program recently introduced a locally-developed Literacy Learning Observation Tool that provides a simple checklist trainers can use to see what teachers (and students) are doing in the classroom. Given the many different professional development activities occurring within MEHRD, coordination among different units and a systemised approach to monitoring teacher practice will avoid

10 This indicator is possibly a duplication of 1.2.1 which also refers to teacher and principal support packages. 11 This is consistent with the AHC Office of Development Effectiveness, Investing in Teachers Evaluation (2016), which found across 27 bilateral aid investments almost no data on outcomes that could be attributed to AHC’s teacher development investments. It was impossible to judge whether teacher development had led to improved teaching practices.


Page 11

duplication of effort and improve efficiencies. Also, as highlighted in the Mid-Year Review, greater emphasis should be placed on a system of ongoing professional development starting with pre-service training that is responsive to teacher needs and student performance rather than just short workshops.

4.1.3 Target 3: The quality of teaching in primary and junior secondary schools

is improved

Evidence: 2017 Performance Assessment Report, SIEMIS data, Teacher Appraisal Report

Teacher appraisals

Teacher appraisals (for probationary teachers) were conducted, however, in the 2017 MEHRD reporting period, the number of completed appraisals referred to the TSC was lower than previous years (see Figure 1). There is also a significant backlog of teacher appraisals to be conducted (over 3,000 outstanding since 2009) and this was noted at the 2017 Annual Joint Review. Since that time, a streamlined process has been put in place to process the backlog. This has involved suspending the time consuming Whole School Inspection process and recruiting a team of MEHRD and provincial inspectors to focus solely on appraising probationary teachers against the standards. A plan has been developed for training school leaders in appraisals, which has commenced, and radio awareness conducted in 2017. The inspectorate aims to process these 3,000 teachers by the end of the year.

In recognition of the work that has commenced since the Annual Joint Review, indicator 1.3.1 is assessed as 20 per cent achieved. A system for school leaders to do appraisals for teachers on an ongoing basis is being introduced in 2018.

Figure 1: Number of teacher appraisals for probationary teachers completed

Source: Independent Assessment 2016, 2017 Annual Report

KO10.1 1.3.1 MEHRD analyses teacher appraisal data, or establishes new data collection methods, to assess



20%

EO2.3/

KO4.4

1.3.2 Increase from 2016 in % of certified teachers in primary and junior secondary schools 100%

143

216

161

0

50

100

150

200

250

2015 2016 2017


Page 12

Certified Teachers

SIEMIS data has shown the number of certified teachers12 has increased in 2017 (see Figure 2). This could be attributed to improvements in reporting and approximately 500 teachers completing the Certificate in Primary Teaching for untrained teachers in 2016 (through SINU13 and USP), and then being recorded as certified in the SIEMIS census in 2017. Some of this increase may also be attributed to new SINU teacher graduates. The assessor was unable to verify the exact numbers of new graduates being placed in schools. Anecdotal evidence from key informants suggests it is quite low. Note that SIEMIS data is unable to distinguish between Junior Secondary and Secondary teachers, many of whom work across the years. Indicator 1.3.2 is assessed as 100 per cent achieved.

Figure 2: Number of certified teachers from 2016 to 2017

Source: SIEMIS, Performance Assessment Report 2017

Planning is underway for the Teaching Services Division to undertake a Teacher Supply and Demand Study. This has the potential to improve data on teacher qualifications and inform planning for pre-service cohorts. There is currently a USP Certificate of Primary Training for 79 untrained teachers underway, with teachers likely to complete the remaining modules by the end of 2019.

4.1.4 Target 4: Curriculum support materials support the implementation of the

basic education curriculum

Evidence: Curriculum Mapping (MEHRD & Cognition, June 2018)

Curriculum reform has been underway for several years at MEHRD. A broad Curriculum Statement was developed in 2011 for basic education (Years 1 to 9). While progress has been slow, in the past

12 A certified teacher is a teacher who has completed a teaching qualification. 13 The SINU course has been suspended due to concerns over the length time taken for teachers to complete the eight modules. Through flexible learning this course can take up to five years to complete.

1.30%

65.6%

82.2%

4.1%

72.2%

85.2%

0%

20%

40%

60%

80%

100%

ECE Primary Secondary

2016 2017

KO5.1 1.4.1 Revised basic education curriculum is completed for all learning areas. 86%

KO8.1 1.4.2 Teacher support materials are developed and distributed to all primary and junior secondary

schools

41%

KO4.5 1.4.3 Teacher professional development programs are designed to support the implementation of the

revised curriculum.

10%


Page 13

12 months there has been a concerted effort by MEHRD, along with the support of a contractor (Cognition), to identify gaps and progress the development of outstanding syllabi, teachers guides and learning materials.

As of June 2018, progress is as follows:

PPY syllabus is approved and Teacher Guides and resource books are undergoing a final edit

before being printed;

Maths and English syllabi for Years 1 to 6 are complete and awaiting endorsement from the

National Curriculum Advisory Board (NCAB);

Science and Social Science syllabi, Teacher Guides and Learner Books for Years 1 to 6 were

completed under an earlier contract (by Pearson);

for Junior Secondary School (JSS) curriculum, the Year 9 Teacher Guides and Learner Books

have been combined for English and Social Studies. This, along with the Year 9 Maths

Teacher Guide, are in progress and should be completed by the end of June 2018; and

the development of materials for non-core subjects is at various stages.

Overall progress of curriculum development by subject is summarised in Appendix 5. This matrix was developed by MEHRD and Cognition and was used to calculate progress to date for indicators 1.4.1 and 1.4.2. For the purposes of assessing progress against indicator 1.4.1, basic education curriculum was defined by the assessor (and agreed with MEHRD and AHC) as completion of syllabi (primary and JSS). Out of a total of 10 primary syllabi, eight were completed (80 per cent) and for JSS, 11 out of 12 syllabi were completed (92 per cent). For indicator 1.4.2, teacher support materials were defined by the assessor (and agreed with MEHRD and AHC) as completion of Teacher Guides (TG) and Learner Books (LB). For primary, 31 per cent of TG and LB were complete and 50 per cent of TG and LB were complete for JSS14. Using the average of primary and JSS percentages, indicator 1.4.1 is assessed as 86 per cent achieved and indicator 1.4.2 is assessed as 41 per cent achieved. Distribution of materials is yet to occur in most subjects (indicator 1.4.2 refers) and while some planning has taken place, no professional development has taken place with teachers to support the introduction of the new curriculum. As a result, the score for indicator 1.4.3 is low. Indicator 1.4.3 is assessed as 10 per cent achieved. Professional development workshops for the new PPY curriculum were starting the week of the Mid-Year Review.

As highlighted at the Mid-Year Review, curriculum reform is a long and iterative process. Efforts to improve the sustainability of curriculum development were also discussed. Along with development of curriculum materials, MEHRD requires a robust and systemic plan for teacher professional development to ensure that improved learning outcomes can be achieved through investment in curriculum. MEHRD is planning to use in-service training to upskill teachers in the new curriculum. To maximise efficiencies, pre-service (SINU) and other professional development training will also need to adapt to support the new curriculum.

14 Note that where some LB and TG were in progress and close to finalisation, 0.5 point for progress was included in the calculation.


Page 14

4.2 Education Access

4.2.1 Target 5: Access to basic education for all children and young people is improved

Evidence: SIEMIS data, draft policies, MEHRD and SNU MOU, reports and proposals.

A Strategy for the Removal of Blockages to Student Participation

The NEAP has outlined MEHRD’s overarching plan for addressing blockages to participation by 2020. There has been a lot of reporting on barriers to access since 2009. In the past 12 months, the SSU has confirmed that most of the findings from the 2010 Barriers to Access Report still hold true today. Several pieces of work were carried out including research on school fees and a second chance education survey. As a result, a taskforce has been established to progress the Solomon Islands Secondary Entrance (SISE) phase-out plan15 along with a Second Chance Education Proposal. This includes an MoU with SINU (in draft) to provide Education Pathways. Work is also underway to cost the infrastructure needs that are hampering automatic transition due to the limited places available in secondary school. The review of the secondary curriculum, particularly the inclusion of non-core subjects, will also enable greater pathways to vocational education opportunities. The rollout of professional development for teachers on classroom-based assessment16 starting in 2018 will also shift the focus from using national examinations to inform schools and teachers of student progress.

While a published strategy is not in place, significant work has been done to set the foundation for implementation of activities to address the blockages to participation. Work is needed to identify a starting point for implementation and ensure blockages to both primary and secondary are considered. Given the implementation strategy for progressing this work is unclear, the indicator was not considered fully achieved. Indicator 2.1.1. is assessed as 70 per cent achieved.

Transition Rates

Based on the SIEMIS data, transition rates declined slightly in 2017 and, therefore indicator 2.1.2 is assessed as not achieved (see Figure 3). Although the decline is small, data from other sources17 such as examination placements suggests it may be worse than what is currently showing in SIEMIS. Analysis of 2017 examinations18 shows for the Year 6 exam (SISE), all students who sat the exam

15 It was announced at the Mid-Year Review that, although it the removal of the Year 6 examination has been announced, it will continue to go ahead for 2018. 16 The Classroom Assessment Policy Framework (CAPF) has been completed and the Year 1 to 3 handbook for Science and Social Studies was due for piloting from July to November 2018. 17 Transition rates to Year 7 also do not reflect the large number of drop outs in the earlier grades. (See 2017 PAR, p.21) 18 Review of 2017 examination data presented by SSU at the Mid-Year Review June 2018.

KO2.1 2.1.1 MEHRD develops and publishes a strategy that identifies and plans the progressive removal of


70%

EO1.2 2.1.2 2017/2018 transition rates, for boys and girls, from Year 6 to Year 7, and from Year 9 to Year


0%

KO2.4 2.1.3 The draft Solomon Islands National Disability Inclusive Education Plan is approved and

disseminated.

40%

KO2.3 2.1.4 More structured and formalised processes are implemented to collect and analyse data about



0%

2.1.5 The Gender in Education Policy is launched. 40%


Page 15

were placed, however, seven per cent of Year 6 students did not show up for examination19 and, therefore, did not transition in to Year 7. Similarly, lack of school infrastructure for students in Year 10 and above is resulting in 29 per cent of students not being placed in Year 10.

Figure 3: Transition rates

National Disability Inclusive Education Policy and Data Collection on Children with Disabilities

The draft National Inclusive Education Policy includes an implementation plan and has been consulted on widely. During consultations, particularly with the previous Minister, it was agreed a few technical areas required revisiting. Furthermore, the policy is yet to be costed. It is anticipated that the policy will go to the National Education Board (NEB) in December 2018, followed by endorsement by Cabinet as it is a cross-sectoral strategy. Some awareness raising on the policy has already commenced. A draft of the policy was sighted by the assessor. Therefore, indicator 2.1.3 is assessed as 40 per cent achieved. The Policy Unit would benefit from further technical assistance to ensure the policy is feasible within the Solomon Islands context, consistent with the new Education Bill and other reforms taking place within MEHRD.

Since 2016 there have been no improvements to the collection of disability disaggregated data. Therefore, indicator 2.1.4 is assessed as not achieved. The current process involves teachers who have received no guidance, listing the number of children with disabilities. The data is not reliable as evidenced by the fact that between 2016 and 2017 there has (supposedly) been a 21.3 per cent drop in the number of children reported by teachers as having a disability on the SIEMIS census20. Disaggregating data by disability is a high priority for AHC and there are potential examples of approaches from other Pacific islands such as Fiji, that could be adapted for Solomon Islands21.

The Gender in Education Policy is Launched

The Gender in Education Policy is still in draft form. Consultations including with NEB in November 2017 have finished and the Policy area is making the necessary changes, finalising and costing the

19 This may be because of the fees charged to sit the Year 6 examination. 20 2017 MEHRD Performance Assessment Report 21 See example of Fiji Student Learner Profile which uses the Washington Questions to identify students with disabilities http://www.education.gov.fj/images/FEMISdisabilitydisaggregationpackageFinal.pdf

91%

78%

89%

77%

50%

60%

70%

80%

90%

100%

Year 6 to 7 Year 9 to 10

2016 2017

http://www.education.gov.fj/images/FEMISdisabilitydisaggregationpackageFinal.pdf


Page 16

implementation strategy. It is anticipated the policy will also go to the NEB in December 2018. Awareness activities have commenced including an event with 95 girls on ICT Day, encouraging girls to take up careers in STEM. Also, in preparation for implementation, five MEHRD officers are completing a Postgraduate Certificate in Gender Studies. Similar to the National Inclusive Education Policy, the assessor recommends the policy and implementation strategy are reviewed by a gender expert before final endorsement by NEB, including ensuring it is consistent with other reforms going on within MEHRD. Indicator 2.1.5 is assessed as 40 per cent achieved.

4.3 Education Management

4.3.1. Target 6: Management capability of Education Authorities (EA) is improved

Evidence: Draft EA Standards, EA reporting template, EA Workplans, EA Buala Conference

Recommendations.

EA Standards

As per the NEAP, an EA Management Strengthening Program is underway involving several activities to improve management performance and provide capacity development. Management Performance Standards for EAs are currently in draft. They are due to be approved in June/July by the SMT and launched at the EA Annual Conference in August. Consultations on the standards took place during the Annual EA National Conference held in Buala in 2017 where all the EAs in Solomon Islands were represented and the 2017 joint MEHRD division/EA meeting. These meetings provided the avenue for MEHRD and EAs to come together to reflect and discuss ways to improve the management capacity of EAs to deliver education services. A communique from the Buala conference was sighted by the assessor including recommendations and proposed resolutions.

There is still further work to be done on the standards to align them with other work happening in MEHRD. The draft standards were sighted by the assessor and indicator 3.1.1 is assessed as 70 per cent achieved.

Baseline of Strengths and Weaknesses of EAs

To inform ongoing capacity development activities, a capacity evaluation was conducted with eight Provincial Education Authorities in 2017 and the summary was reported in the 2017 Performance Assessment Report. As EA standards were yet to be finalised, the EA functions as per the Education Act were used to evaluate current capacity. While these functions are not based on the draft standards, there are strong similarities and this process has informed several initiatives including: the development of the new standards and forming the baseline for the LEAP project and the EA Management Strengthening Program which is currently under development. Indicators 3.1.2 is assessed as 100 per cent achieved.

Management capability of EAs

KO7.1 3.1.1 Standards for Education Authorities are developed in partnership with EAs. 70%

KO11.1 3.1.2 Baseline data to determine strengths and weaknesses of EAs is collected and analysed. 100%

KO11.2 3.1.3 MEHRD implements programs or activities to improve the management capability of Education

Authorities.

100%


Page 17

The Education Authorities Division in MEHRD undertook several other activities to help improve EA capacity in delivering its services. To improve the timely collection and analysis of data, the division developed and implemented a planning and reporting cycle for EAs. A standard reporting template based on the national level annual work planning process has been designed and all provincial education authorities have now developed a plan. This training has been ongoing through the past year. The assessor sighted workshop minutes where the template was introduced to EAs. To improve Whole School Development Plans, the EACI have provided a user-friendly quick guide for EAs adapted from the lengthy manual, that will be used in future workshops. This guide is still in draft. Indicator 3.1.3 is assessed as 100 per cent achieved.

4.3.2 Targets 7 and 8: NEAP M&E Framework is developed and implemented, and data

are analysed; NEAP M&E data are analysed and reported, and used for planning and

policy development purposes

Evidence: MEHRD MEL Plan 2016–2020, 2017 Performance Assessment Report, 2017 Annual Report,

2017 Annual Work Plan, 2017 Quarterly Reports

Based on an evaluability assessment in 2017 of the NEAP 2016–2020 and the associated Implementation framework, along with consultations with MEHRD officers, an M&E Framework for the NEAP was developed by SSU. It includes a results framework that measures progress against the NEAP agreed education outcomes. The framework was encapsulated within a MEHRD Monitoring, Evaluation and Learning Plan 2017–2020 developed in 2017 and approved by the SMT on 28 August 2017. Indicator 3.2.1 is assessed as 100 per cent achieved.

The following principles guided the development of the MEL Plan22:

support iterative information flows and utilise a range of communication and reporting tools

to make information more accessible, timely and useful;

value qualitative and narrative data as much as quantitative data to assist in describing the

MEHRD progress and performance; and

provide support to MEHRD data collection systems as they develop and align to overall

M&E.

The M&E work of MEHRD reflects internationally recognised ethics and standards for evaluation practice set out by the Australasian Evaluation Society23, the OECD Development Assistance Committee24 and the AHC M&E standards.25

22 MEHRD Monitoring and Evaluation Learning Plan, 2017–2020 23 Australasian Evaluation Society Guidelines for Ethical Conduct of Evaluation (Revised 2013) 24 www.oecd.org/development/evaluation/dcdndep/47069197.pdf (Accessed June 2018) 25 http://dfat.gov.au/about-us/publications/Pages/dfat-monitoring-and-evaluation-standards.aspx (Accessed June 2018)

KO13.2 3.2.1 M&E Framework is approved by Permanent Secretary. 100%

KO13.3 3.2.2 M&E Data are collected and analysed 100%

KO13.4 3.3.1 NEAP M&E data are reported in the 2017 Performance Assessment Report 100%

KO13.5 3.3.2 M&E findings are used to develop 2018 NEAP Annual Work Plan. 100%

http://www.oecd.org/development/evaluation/dcdndep/47069197.pdf

http://dfat.gov.au/about-us/publications/Pages/dfat-monitoring-and-evaluation-standards.aspx


Page 18

The MEL Plan includes a Theory of Change and results framework that links to all relevant SIG and MEHRD education documents and plans and provides a range of M&E processes and activities to support output and outcome data collection, analysis and reporting (see Figure 4). The three end outcomes in the results framework are:

1. Access: Girls and boys have safe and equitable access to complete education irrespective of social, economic or other status;

2. Quality: Girls and boys receive quality education with relevant and effective outcomes; and

3. Management: Management systems and practices are embedded and sustained at school, Education Authorities and MEHRD to enable education outcomes to be achieved.

Figure 4: Linking outcomes to education plans

Source: MEHRD Monitoring Evaluation and Learning Plan 2016–2020

As part of quarterly progress reports in 2017, the SSU collected data from all divisions in order to report at an output and outcome level against the agreed indicators. The data collection process also included information from specific reviews, adviser reports and key informant interviews. The 2017 PAR included reporting against the results framework, while the Annual Report provided details of MEHRD’s overall progress at the output level. Indicator 3.2.2 is assessed as 100 per cent achieved.

The 2017 PAR and 2017 Annual Report were finalised in May 2018 and published in June 2018 in both hard copy and on MEHRD’s website. Indicator 3.3.1 is assessed as 100 per cent achieved. The SSU received praise from across government for the quality of the documents, with the former Permanent Secretary, who recently moved to the Prime Minister’s Office, stating they provided a model for the other ministries.

Based on the findings from the 2017 reporting period and discussions at the Annual Joint Review, MEHRD divisions developed an Annual Work Plan that covers the tasks for all MEHRD divisions. It shows timing for implementation, a budget tracking tool and a breakdown of how donor funds were allocated. While the plan contains over 800 line items, it can be easily filtered by themes and divisions, to provide targeted information for specific audiences. Indicator 3.3.2 is assessed as 100 per cent achieved.


Page 19

4.4 Education Resourcing

4.4.1 Target 9: The Solomon Islands Government maintains its recurrent budget

allocation for education in 2017

Evidence: SIG Budget statements, 2017 Performance Assessment Report

Evidence presented to the assessor by the Technical Adviser, Finance, and MOFT Budget Statements confirmed that the recurrent education budget appropriation (272) was SBD1,198,288,453. This represented 38 per cent of the SIG Recurrent Government budget26 (SBD3,142,500,000) (see Table 1 and Figure 5).

While this is extremely high, 44 per cent of this expenditure is for scholarships, which benefits less than 0.5 per cent of the population27.

Table 1: Appropriated recurrent and development budget 2017

272 Education and Human Resource Development

Appropriated Funds (2017)

272 Education and Health Resource Development

Statutory Expenditure (2017)

272 Education and Human Resource

Development Funds Appropriated through Development Budget

(2017) (472)

Total MEHRD (2017)

272 1,198,288,453.00 1,198,288,453

372 70,373,837.00 70,373,837.00

472 77,000,000.00 77,000,000.00

Total 1,268,662,290 - 77,000,000.00 1,345,662,290

Total SIG Appropriated Funds Total SIG Recurrent Budget including Statutory Expenditure

Total Funds Appropriated through the Development budget (2017)

Total SIG (2017)

3,142,500,000.00 1,489,500,000.00 4,632,000,000.00

Source: Solomon Islands Government, Year 2017, Approved Recurrent Estimates Budget Paper: Vol 2 (p 14) and sig info

16.04.18

It should also be noted be noted that SBD200,000,000 of that total was supplementation for scholarships and was not initiated by MEHRD.

Indicator 4.1.1 is assessed as 100 per cent achieved.

26 In future reporting, including actual expenditure would be a good comparison. Unfortunately, actual expenditure figures were not available at the time of the 2017 assessment. 27 Based on the number of 2017 scholarships allocated (not including those on award) and latest population estimates.

EO3.3 4.1.1 At least 23% of SIG national recurrent budget was allocated to education. 100%


Page 20

Figure 5: The percentage of SIG recurrent budget allocated to MEHRD

Source: 2017 Annual Report

4.4.2 Target 10: The 2017 recurrent expenditure for Basic Education is maintained

or improved

Evidence: SIG Budget statements, MEHRD Expenditure Reports (June 2018)

Both per-unit costs for primary and secondary increased in real terms28 between 2016 and 2017, therefore achieving both indicators 4.2.1 and 4.2.2 (see Figure 6). The MEHRD finance system is unable to separate Junior Secondary costs from that of Senior Secondary so a combined total is shown. This increase in per-unit costs is attributed to MEHRD increasing funding for grants, 29 per cent for Basic education and 6 per cent for Senior education (see Figure 7).

Prior to 2017, basic education grants were partly funded by development partners. MEHRD has now taken full responsibility for grant funding for schools. Teacher payroll is currently not included in the per-unit calculation, as payroll coding for teachers was previously miscoded, with the majority allocated to the primary code. This is gradually being rectified.

Indicators 4.2.1 and 4.2.2 are assessed as 100 per cent achieved.

28 Inflation was 1.8 per cent for 2017 according to the 2017 Central Bank of Solomon Island’s Annual Report.

38%

62%

Other SIG Recurrent Expenditure

MEHRD Recurrent Expenditure

EO3.6 4.2.1 Allowing for inflation the 2017 SIG recurrent expenditure for the per unit cost of primary


100%

EO3.6 4.2.2 Allowing for inflation the 2017 SIG recurrent expenditure for the per unit cost of junior


100%


Page 21

Figure 6: Per-unit costs for primary and

secondary school students

Figure 7: Increase in basic and senior

education grants

Source: MEHRD Expenditure Reports, June 2018

4.4.3 Target 11: Acquittal percentage of 2017 School Grants and distribution

percentage of 2018 first tranche grants improve

Evidence: MEHRD Finance Acquittal Report 21 June 2018 (live database)

The target for school grant disbursements was increased from 70 per cent in 2016 to 80 per cent in 2017. MEHRD fell short of this target, with retirement rates averaging 76 per cent across all sectors (see Table 2). While the target was not met, it was a 2 per cent increase in performance from last year’s average of 74 per cent. Indicator 4.3.1 is assessed as not achieved.

In the past 12 months, there were significant staff changes within the grants team, which is now relocated within the Finance Division. It is also worth noting that there is no enforced policy within MEHRD on what is considered “on time” and the database for retirements is live, i.e. continuously updating as retirements come in. For the purposes of this assessment and scoring, retired “on time” equates to all retirements entered into the system when the acquittal report is run. This indicator had the potential to score higher, as there was a significant amount of retirements that had come in from schools but were yet to be entered into the system at the time of the assessment.

Table 2: 2017 First biannual disbursement information

Sector Disbursement Retirement Retirement Rate

ECE 182 127 70%

Primary 685 589 86%

Secondary 266 142 53%

TVET 40 28 70%

Total 1173 886 76% Source: MEHRD Acquittal Report, 21 June 2018

$6

77

$7

33

$7

33

$9

08

P R I M A R Y S E C O N D A R Y

2016 2017

37

61 53

64

-

10

20

30

40

50

60

70

Basic Education Grant Senior Education GrantM

illio

ns

KO12.3 4.3.1 At least 80 % of 2017 School Grants are retired on time 0%

KO12.6 4.3.2 At least 85% of schools receive the first tranche of their 2018 School Grant by the end of March

2018.

100%


Page 22

In 2018, the four sub-sectors received 87 per cent of disbursement in their first grant prior to March 2018 (see Table 3). It is, however, worth noting, that due to the change of government in late 2017, the handing down of the budget was delayed in 2018. For this reason, the schools received only half of their first grant in February 2018. The remainder of the first grant was paid as part of the second biannual grant payment. Indicator 4.3.2 is assessed as 100 per cent achieved.

Table 3: First half year 2018 grant disbursement summary

Sector Registered Schools

1st Biannual % of disbursement

ECE 303 193 64%

Primary 747 696 93%

Secondary 273 263 96%

TVET 44 43 98%

Total 1367 1195 87% Source: MEHRD Acquittal Report, 21 June 2018

It is recommended that for future years, these school grants indicators are defined with more detail, i.e. clarify whether it is an average across all sectors or total number, first or second biannual.

4.4.4 Target 12: External Audits of MEHRD finances are conducted

Evidence: Audit plan, Draft MEHRD Audit Report 2015 and management letter

The SIG is facing a huge backlog in relation to external audits across the 24 Ministries. As a result, the Office of the Auditor General introduced a streamlined audit process to undertake the 2015 and 2016 external audits. This is being done in two parts. Part 1 includes the audit of four specific cycles within MEHRD. The four cycles included are:

1. Accountable Officer Reports 2. Assets/ Stores 3. Bank Accounts 4. Commitment Cards.

Part 2 of the audit is on the financial statements and focuses on transactions across the 24 Ministries that exceed SBD18 million and/or are from high risk areas (e.g. NTU). At the time of the assessment, the draft report of Part 1 and relevant management letter had been sent to MEHRD, and the Office of the Auditor General was awaiting a management response from MEHRD. The draft report noted that three out of four areas were rated as high risk. Part 2 of the audit was underway and scheduled to be finalised by the end of June. The Office of the Auditor General stated that staff shortages and a lack of qualified staff delayed the auditing process.

For the 2016 Audit, the Office of the Auditor General is still waiting on the completion of the 2016 SIG Financial Statement by MOFT.

Indicator 4.4.1 is assessed as 60 per cent achieved.

KO12.4 4.4.1 At least one external audit is conducted by the Auditor-General’s Department or a suitably


60%


Page 23

4.4.5 Target 13: The economic and educational benefits of the SIG tertiary scholarship

program are reviewed and assessed

Evidence: 2017 Scholarship Analysis report (O’Meara)

While not a cost-benefit analysis, a detailed Scholarship Analysis Report29 involving a desk top analysis of scholarships data for the period of 2009 to 2017 was conducted. Data came from the NTU and the SIMS database. Among other things, the report highlights the peaks in scholarship allocations for 2014 and 2017 (44 per cent are domestic and the rest overseas), fields of study (the largest is Education at 25 per cent, followed by Business, Administration and Law) and geographical spread, which aligned with the population, apart from Malaita, which had more awards per capita than other provinces. The report did not include an analysis of termination, suspension, extension and completion rates. Indicator 4.5.1 is assessed as 40 per cent achieved.

While this report may seem routine for many countries, this was the first time any such reporting was undertaken by the SIG. Furthermore, the findings were discussed at the 2017 Annual Joint Review, starting the conversation and representing a good first step in providing a more solid evidence base to inform decision making around scholarship allocations. Increased data and analysis around scholarship figures, in particular, around completion rates and employment of new graduates, has the potential to better equip MEHRD officers and Development Partners to dialogue with political leaders on the often-sensitive issue of scholarship allocations and expenditure.

Currently, MEHRD staff have issues accessing the Scholarship Information Management System (SIMS) database and running reports. As a result, they are running a separate, unsecure system in Excel spreadsheets. The SSU is working to secure a local contractor that can support improved data collection and reporting through the SIMS database.

The Mid-Year Review noted the ongoing importance of using data to inform the Prime Minister’s Office reporting to Cabinet, in particular showing the trade-off with educating primary students versus tertiary scholarships and other concerns related to value for money.

4.4.6 Target 14: Solomon Islands Government scholarships budgeting and

expenditure is sustainable

Evidence: MEHRD Finance Report on actuals for 2016 and 2017

The allocation of new scholarship awards increased significantly in 2017, causing expenditure to exceed that of 2016 (see Figure 7). Indicator 4.6.1 is assessed as not achieved. NTU reported 2,756 students30 were allocated a scholarship in 2017, adding to the increasing cohort still on award. In 2016, there were 1,721 records of new awardees as per the scholarship analysis report.

29 This was carried out by Fiona O’Meara under the management of MEHRD, firstly as a volunteer and later as a local TA for MEHRD. 30 For 2017, only 2,283 student records were located in NTU during the Scholarship Analysis Report.

4.5.1 A cost-benefit analysis of the scholarships program is designed and approved for

implementation.

40%

4.6.1 2017 scholarships expenditure is not higher than 2016 scholarships expenditure. 0%


Page 24

In terms of total expenditure between 2016 and 2017, there was a 40 per cent increase to scholarship expenditure from SBD243 million to SBD402 million31.

Figure 8: Total scholarship expenditure from 2016 to 2017

Source: MEHRD Expenditure Reports, June 2018

4.5 Public Financial Management

4.5.1 Target 15: Public Financial Management Action Plan implementation ongoing

Evidence: Various reports across relevant divisions

The PFM Action Plan was developed in 2015. There are 47 activities that are the responsibility of several different divisions across MEHRD. The assessor discussed progress with each of the teams responsible and verified which activities had been completed and which ones were no longer relevant. Despite there being limited ownership over the plan as such, there was significant progress in the past 12 months on the following areas:

Ministry of Finance and Treasury (MOFT) introduction of payroll recovery for retirements reduced the outstanding retirements from 100+ to only 4 last year in MEHRD

Asset Register for school infrastructure in place (SIEMIS)

School Financial Management Handbook revised

Electronic payment vouchers.

Based on the 36 activities in the PFM Action Plan still relevant in 2017-2018, indicator 5.1.1 is

assessed as 64 per cent achieved.

One of the biggest concerns across divisions is the time taken to process funding requests, particularly for provincial travel. Most service providers in provincial areas will not accept Local Purchase Orders and require cash payment. Delays are generally caused by incomplete paperwork from MEHRD requesting officers and delays in MOFT processing times. The Finance Adviser established a payment tracker that can track payments and identify where the delays are occurring.

The Mid-Year Review highlighted the need for more training for officers in preparing the necessary paperwork to help expedite the payment process from the MEHRD side.

31 Note the 2016 figure is based on actual expenditure and the 2017 figure is based on a revised budget estimate as actuals for the 2017 reporting period were not available at the time of the assessment.

2017

2016

$402,655,149

$242,309,678

5.1.1 % of activities scheduled in the PFM Action Plan for 2017 completed. 64%


Page 25

5. Payment Recommendation

The payment recommendation was calculated by giving equal weighting to each of the 31 performance indicators under the 15 targets. A summary of performance against the indicators is at Appendix 6. In previous years, the payment was divided equally across targets or objectives. The methodology for the weighting was adjusted this year to account for a late addition of one target related to public financial management (PFM) and an indicator for PFM and the Gender Equality Strategy. The approach used for 2017 places less weighting on the achievement of the PFM and Gender Equality Strategy targets, which would have equated to a weighting of 1/15 each as opposed to 1/31 with this methodology.

At the Mid-Term Review Workshop, the assessor recommended that 64 per cent of the donor performance linked aid payment be made based on the level of achievement of and progress towards performance indicators. During the workshop, the Curriculum Unit revised its progress in relation to the number of teacher guides and learner books required, and is now reflected accurately in this report. Upon further reflection by the assessor and agreed by DFAT and MEHRD, the per cent for the achievement of indicators related to the National Disability Inclusive Education Policy and the Gender in Education Policy was reduced to 40 per cent for both. These changes balance each other out and do not affect the overall recommended performance-linked aid payment of 64 per cent. Figure 9 shows how 64 per cent compares with previous years. It was noted at the Mid-Year Review that the number of indicators had increased from 13 in 2016 to 31 in 2017 and the focus on quality indicators had also increased from one to 10. Progress towards the access, quality and management indicators is shown in Figure 10.

Figure 9: Percentage of indicators achieved

between 2015 and 2017

Figure 10: Summary of indicators achieved

by priority area

Source: Independent Assessment Reports 2015, 2016 and 2017 (preliminary)

Source: 2017 Independent Assessment

0%

20%

40%

60%

80%

100%

2015 2016 2017

Achieved Partly Achieved Not Achieved


Page 26

6. Strengths and Weaknesses of 2017 Performance Indicators

AHC usually engages a different independent assessor annually to review the performance of MEHRD. The assessor is also responsible for developing a proposed EPM for the following year. While this approach ensures new thoughts and consideration of different approaches year-on-year, the ‘goal posts’ as such are potentially able to change quite significantly year-on-year. Where there is a fixed pool of funding allocated as the performance-linked aid payment, this then has the potential for SIG to feel discouraged if the same or similar level of disbursement is not reached each year. It was the view of the assessor that the degree of difficulty in relation to the indicators had increased from 2016 to 2017 and thus contributed to a lower performance-linked aid payment being achieved, i.e. a drop from 91 per cent to 64 per cent. The assessor compared the 2017 EPM with the 2015 and 2016 indicators to better understand the degree of change, difficulty and any trends. The assessor also used the criteria developed by Holzapfel and Janus (2015)32 as a framework for assessing the strengths and weaknesses and also for making recommendations for the 2018 EPM.

The focus of the 2017 performance indicators was very closely aligned with the priorities being pursued by MEHRD. In fact, as a result of the NEAP M&E indicators being developed after the last assessment and drafting of the EPM, only four of the 31 performance indicators did not correlate with either a MEHRD output or outcome level indicator as per the MEHRD MEL plan. The MEHRD reference has also been used in this report to show this link and allow for easy identification of the indicator (see Appendix 7).

The number of indicators has increased significantly from 10 in 2015, to 13 in 2016, and 31 in 2017. Of the 31 indicators in the 2017 EPM, only eight (or 26%) were the same (or similar to) indicators used in 2015. There has been a much higher number of consistent indicators in years prior to 2017. In line with the NEAP’s focus on quality, the EPM was revised in 2017 to have an increased number of ‘quality’ education indicators. This included a large focus on the 2017 SISTA results and curriculum development. While the focus on quality is welcomed, some of the timeframes for these indicators were unrealistic, particularly those related to development of professional development programs in response to the SISTA 2017. Other indicators lacked specificity, such as curriculum, making it difficult for the assessor to determine what progress was to be measured against. Ten of the indicators were also dependent indicators, meaning that if one was not achieved, then it was impossible to achieve the others. The performance cycle, as mentioned under Section 3, is different for some indicators and for others, it was unclear. Going forward, it is important that timeframes for reporting against indicators are clear and realistic based on current levels of resourcing, benchmarked to previous performance wherever possible. Future proposed changes to the performance cycle are highlighted below, as a result of MEHRD realigning its meetings with the reporting period. Finally, some indicators were added late to the matrix, as mentioned above.

With the development of the MEL Plan in 2017, identifying the indicators for measuring progress against the NEAP, it was easier for the current assessor (and future assessors) to be more consistent with the selection of the indicators between years. Using the existing framework and reporting processes will also ensure there is a high level of understanding and ownership within divisions on which indicators are their responsibility.

32 Holzapfel, S. and Janus, H. (2015), Improving Education Outcomes by Linking Payments to Results: An Assessment of Disbursement-linked Indicators in five Results-based Approaches


Page 27

7. Recommendations Going Forward and Proposed Education

Targets and Indicators for 2018

The assessor made several recommendations going forward related to the development of the 2018 EPM. These were also presented and discussed at the Mid-Year Review with relevant divisions.

As highlighted in other sections, the current timing of the Annual Joint Review does not align with the MEHRD reporting cycle; this was raised with Development Partners and the SMT at the Mid-Year Review. Given MEHRD now has a strong reporting framework for the calendar year, it was proposed33 at the Mid-Year Review, that from 2019, the Annual Joint Review will be held in the second quarter of the following year. This means for the 2018 reporting period, progress will be reviewed around April–May in 2019 to ensure all data on the 2018 period has been collected and PARs and Annual Reports are published. These changes will also ensure better alignment with SIG budget process and MEHRD’s annual work planning process.

This change has implications for the timing of the independent assessment. The performance cycle for the assessment will need to change to be based on the calendar year for all indicators (rather than from meeting to meeting). Newly-developed indicators in the 2018 EPM will need to be achieved by 31 December 2018. For 2018 only, there will only be six months from the drafting of indicators until the end of the performance period, but thereafter, the cycle will revert to the calendar year, allowing a full 12 months between agreement on the indicators and the time when performance against them is measured. It was proposed at the Mid-Year Review that the Annual Joint Review for 2018 will go ahead as planned, but this will be the last time it will be held before the end of the year. In 2019, it will be replaced by an Annual Progress Update (or similar). This meeting prior to the end of the year will be used to identify and agree to performance indicators for the following calendar year. The 2019 EPM will need to be agreed at the Annual Joint Review scheduled for November 2018, to allow MEHRD officers a full 12-month period to achieve the indicators.

Further to the changes highlighted above and in relation to the indicators for 2018, the assessor made the following recommendations.

1. Further align the future EPM with the NEAP MEL Plan. This includes using NEAP end outcomes as per the MEL Plan as targets for the EPM and other indicators from the MEL Plan as much as possible to measure progress. For process indicators, it will be important to align expected progress with tasks in the Annual Work Plan. This will ensure consistent reporting, as MEHRD will be reporting against these indicators and outputs annually.

2. While the EPM will continue to have a mix of outcome34 and process indicators, weight these differently (specifying the evidence required) to reflect the level of effort required to achieve a process indicator, and also recognise that outcomes will not change year-on-year unless there is a specific intervention introduced or progressed.

3. Ensure timeframes for the achievement of indicators are clear and realistic, based on contextual challenges, the human and financial resources available, and aligned with expected progress in the Annual Work Plan.

33 A new MEHRD Permanent Secretary has commenced since the Mid-Year Review so these changes to timing will need to be agreed with him. 34 AHC use the EPM assessment to highlight progress towards outcome indicators, particularly those related to literacy and numeracy rates, transitions rates and number of certified teachers.


Page 28

4. Use measurable and verifiable indicators (e.g., within a 12-month period), with the EPM to include a ‘rubric’ of evidence required to verify progress, and the weighting for this evidence35 agreed as part of the finalisation of the EPM.

5. Agree which indicators will be measured every year, and then allow approximately only 20 per cent of those to vary depending on MEHRD’s policy priorities.

6. To reflect the partnership principles agreed between Development Partners and MEHRD36, consider the inclusion of indicators for MEHRD to measure the performance of development partners, similar to what is done in the health sector. Drawing on the Paris Principles, these indicators might relate to Development Partners providing predictability of funding, timely and relevant technical assistance and using MEHRD systems and processes37.

7. Measure progress at the target level (or NEAP end outcome level stated in the MEL Plan Results Framework), rather than at the indicator level38.

For 2018, although the indicators are to be agreed mid-year39, most indicators will be measured as per the 2018 calendar year, in line with the 2018 Annual Report and 2018 Performance Assessment Report. An exception to this is the indicator related to school grant disbursements in 2019 (3.2) where data will not be available until after the school year starts. To standardise reporting, a date of 31 March (2019), has been proposed as the date to run the report for both school grant acquittals for the second biannual in 2018 and the disbursement of the 2019 school grants (first biannual). Given the database is live, in order to compare trends over time, it will important for MEHRD to run the report for these indicators at the same time every year.

Based on these recommendations and consultations with MEHRD officers and Development Partners prior to and during the Mid-Year Review, Appendix 7 provides a draft 2018 EPM for agreement between Development Partners and MEHRD. These outcome areas and indicators reflect the priorities of MEHRD as per the NEAP 2016–2020, including those areas identified by Development Partners as pre-conditions for the DFA and strategic outcome areas that require ongoing dialogue across SIG to improve learning outcomes for Solomon Islands.

35 For example, consultation on a report is worth 20% and an SMT-approved report is worth 50%. 36 Statement of Partnership Principles between SIG and Development Partners 2016–2020. 37 The Health sector currently has Development Partner indicators. See previous Health assessments for samples of indicators used. http://dfat.gov.au/about-us/publications/Documents/solomon-islands-health-sector-support-independent-review.pdf 38 The assessor also discussed the possibility of weighting the indicators according to degree of difficulty. For 2018, it was agreed there was insufficient time to undertake this process as it would involve lengthy consultation. It may be considered an option in future EPMs. 39 Indicators still in draft as at 13 July 2018.


Page 29

Appendices


Page 30

Appendix 1: Education Performance Matrix 2017

Solomon Islands Education Sector Program 2 2017 Education Performance Matrix

Goal: To improve the quality of education in Solomon Islands by improving:

1. the quality of teachers and teaching;

2. equitable access to education;

3. the quality and effectiveness of educational leadership and management at all levels of the

system; and

4. the resourcing and utilization of available education resources.

TARGETS PERFORMANCE INDICATORS EVIDENCE

MEANS OF VERIFICATION

1. Education Quality

1.1

SISTA 2017 results are analysed to assess improvements in literacy and numeracy and design.

1.1.1 SISTA 2017 results are analysed by MEHRD and the findings communicated to schools and Education Authorities.

Copy of SISTA analysis conducted by MEHRD

Copy of communique sent to schools and EAs

1.1.2 SISTA 2017 results indicate an improvement in student numeracy and literacy at each year level assessed.

SISTA results 2017 and 2015

Copy of MEHRD analysis of results

1.2

SISTA 2017 analysis is used to develop strategies to improve the quality of literacy and numeracy teaching in schools.

1.2.1 Findings from SISTA 2017 are used to develop professional development programs or teacher support packages for teachers and principals.

Copy of professional development programs or support packages

1.2.2 Professional development programs for teachers and principals are implemented and their effectiveness evaluated.

Copy of professional development program agendas

Record of participation in any professional development programs

Copy of PD evaluation report(s)

1.2.3 Teacher and principal SISTA support materials are distributed to relevant schools.

Record of distribution of any teacher support package

1.3

The quality of teaching in primary and junior secondary schools is improved.

1.3.1 MEHRD analyses teacher appraisal data, or establishes new data collection methods, to assess the percentage of teachers who meet the Teacher Standard and establish a baseline against which future improvement can be measured.

Copy of analysis and reports of findings

Baseline rates of achievement for each section of the Teacher Standards

1.3.2 Increase from 2016 in the percentage of certified teachers in primary and junior secondary schools.

Data from 2016 and 2017 showing percentage of certified teachers in schools by province

Training records


Page 31

1.4

Curriculum support materials support the implementation of the basic education curriculum.

1.4.1 Revised basic education curriculum is completed for all learning areas.

Copy of curricula completed by end of May 2018

1.4.2 Teacher support materials are developed and distributed to all primary and junior secondary schools.

Copies of teacher support materials

Distribution records

1.4.3 Teacher professional development programs are designed to support the implementation of the revised curriculum.

Copy of professional development materials/ programs

2. Education Access

2.1

Access to basic education for all children and young people is improved.

2.1.1 MEHRD develops and publishes a strategy that identifies and plans the progressive removal of blockages to student participation in school education.

Copy of strategy document

2.1.2 2017/2018 transition rates, for boys and girls, from Year 6 to Year 7, and from Year 9 to Year 10, improve from 2016/2017 rates.

Transition rates as reported in 2016 and 2017 Performance Assessment Reports

2.1.3 The draft Solomon Islands National Disability Inclusive Education Plan is approved and disseminated.

Minute of Ministerial approval

Distribution record

2.1.4 More structured and formalised processes are implemented to collect and analyse data about the participation rates (including data on what % of school-age children with disabilities attend school) of students with disabilities in basic education.

Data collection plan

Documentation of definitions of various disabilities

Data from data collection process

Record of data analysis

2.1.5 The Gender in Education Policy is launched.

Finalised Policy document.

3. Education Management

3.1

Management capability of Education Authorities (EA) is improved.

3.1.1 Standards for Education Authorities are developed in partnership with EAs.

Copy of approved standards for EAs

3.1.2 Baseline data to determine strengths and weaknesses of EAs is collected and analysed.

Data and subsequent analysis

Baseline figures for each component of standards

3.1.3 MEHRD implements programs or activities to improve the management capability of Education Authorities.

Copy of programs

Program implementation records

Participation records

3.2

NEAP M&E Framework is developed and implemented, and data are analysed.

3.2.1 M&E Framework is approved by Permanent Secretary.

Copy of M&E Framework

Minute of PS approval

3.2.2 M&E data are collected and analysed.

Record of data

Report of data analysis

3.3 NEAP M&E data are analysed and reported

3.3.3 NEAP M&E data are reported in the 2017 Performance Assessment

Copy of 2017 Performance


Page 32

and used for planning and policy development purposes.

Report. Assessment Report

3.2.4 M&E findings are used to develop 2018 NEAP Annual Work Plan.

Analysis of 2018 Work Plan against M&E Report and 2017 PAR.

4. Education Resourcing

4.1

The Solomon Islands Government maintains its recurrent budget allocation for education in 2017

4.1.1 At least 23% of SIG national recurrent budget was allocated to education.

SIG and Education budgets for 2017

4.2

The 2017 recurrent expenditure for Basic Education is maintained or improved.

4.2.1 Allowing for inflation, the 2017 SIG recurrent expenditure for the per unit cost of primary education is at least equal to the 2016 per unit cost of primary education.

Analysis of per unit expenditure

Evidence of 2017 inflation rate

4.2.2 Allowing for inflation, the 2017 SIG recurrent expenditure for the per-unit cost of junior secondary education is at least equal to the 2016 per-unit cost of junior secondary education.

Analysis of per-unit expenditure

Evidence of 2017 inflation rate

4.3 Acquittal percentage of 2017 School Grants and distribution percentage of 2018 first tranche grants improve.

4.3.1 At least 80 % of 2017 School Grants are retired on time.

4.3.2 At least 85% of schools receive the first tranche of their 2018 School Grant by the end of March 2018.

Record of acquittals

Record of distribution of 2018 Tranche 1

4.4 External Audits of MEHRD finances are conducted.

4.4.1 At least one external audit is conducted by the Auditor-General’s Department or a suitably qualified outside provider.

Audit Plan

Audit Report

4.5 The economic and educational benefits of the SIG tertiary scholarship program are reviewed and assessed.

4.5.1 A cost-benefit analysis of the scholarships program is designed and approved for implementation.

Study proposal

Minute of PS approval

4.6 Solomon Islands Government scholarships budgeting and expenditure is sustainable.

4.6.1 2017 scholarships expenditure is not higher than 2016 scholarships expenditure.

Analysis of budget and expenditure

5. Public Financial Management

5.1 PFM Action Plan implementation ongoing.

5.1.1 % of activities scheduled in the PFM Action Plan for 2017 completed.

Analysis of PFM Action Plan and 2017 activity reporting


Page 33

Appendix 2: Terms of Reference

Australia and the Solomon Islands Government agreed to set annual targets in an Education Performance Matrix attached to the Direct Funding Agreement (DFA) that links performance-linked aid payments to achievement of the agreed targets (see Schedule F). The aim of these payments is to incentivise performance of the education sector.

An Independent Assessor is required to conduct a review of the performance against the agreed 2017 Education Performance Matrix (attached), recommend a percentage for the performance-linked aid payment and recommend targets for 2018.

An incentive payment is provided yearly to the Ministry of Education and Human Resource Development’s (MEHRD) on the achievement of annually agreed targets set out in the Education Performance Matrix.

This year will be the sixth year that an Independent Assessor has assessed the SIG’s performance against a performance matrix.

The assessor’s report will contribute to the broader sector review undertaken by SIG and development partners at the Annual Joint Review (AJR).

The assessor should have the following qualifications and experience:

- Experience in Sector-Wide Approaches particularly in the service delivery sectors. - Experience of performance incentive programs/mechanisms. - Experience in the education sector important and knowledge of education in the Pacific and

Solomon Islands an advantage. - Academic qualifications, i.e. a master’s degree or higher in a relevant field.

The assessor will provide advice to all parties on the payment of the performance component of the education sector budget support, specifically:

(a) Review SIG performance against the agreed milestones in the 2017 Education Performance Matrix.

(b) Provide a recommendation of what performance-linked aid payment should be made based on the SIG’s performance.

(c) Provide advice on the strengths and weaknesses of the 2017 performance indicators. (d) Include MEHRD officers working on M&E in as much of the in-country visit as possible as a

learning experience while maintaining the assessment’s independence. (e) Provide advice on indicators for 2018 informed by: priority policy reforms; the Education

Sector Program 2 design; and the MEHRD M&E framework.


Page 34

The assessor will implement the assessment as follows:

(a) Document review of data sources as included in the 2017 Education Performance Matrix (up to four days, indicatively 21–24 May 2018).

(b) In-country visit (indicatively from 28 May to 8 June 2018) to conduct consultations with key stakeholders including the MEHRD Senior Management Team, key MEHRD division heads, and Donor Partners – Australia and New Zealand (up to 12 working days + up to four days travel):

Data collection, analysis, consultations, drafting.

Separate briefings to the MEHRD Senior Management Team and AHC on preliminary findings.

Presentation of findings and recommendations to the AJR (indicatively on 8 June 2018). This presentation should include: tabling of preliminary findings; outlining of areas of strength and weakness; outlining 2017 payment recommendation; outlining recommended 2018 performance indicators; recommendations on how the independent assessment process can be improved.

(c) The assessor will provide the following reports:

Initial findings from document review two weeks before the AJR (indicatively by 25 May 2018).

Preliminary findings report provided at the time of the AJR.

Final report (up to four days preparation) submitted no later than two weeks after the AJR (indicatively by 22 June 2018). As part of preparing this report the assessor will, as required, follow-up with AHC and the MEHRD on issues raised during the in-country mission and AJR to confirm findings and data.


Page 35

Appendix 3: Key Documents Reviewed

Document Source

Direct Funding Agreement between the Government of Australia and Solomon Islands Government

AHC

Solomon Islands Education Sector Statement of Partnership Principles between Solomon Islands Government and Development Partners 2016–2020

AHC

Solomon Islands Education Sector Support Program Independent Performance Assessment 2015

AHC

Solomon Islands Education Sector Support Program Independent Annual Performance Assessment 2016

AHC

Solomon Islands Education Sector Statement of Partnership Principles 2016-2020 AHC

National Education Action Plan 2016–2020 MEHRD

National Education Action Plan 2016–2020 Implementation Framework MEHRD

Performance Assessment Report 2015 & 2016, 2017 MEHRD

MEHRD 2018 Quarter 1 Report MEHRD

MEHRD Monitoring, Evaluation and Learning Plan 2017 to 2020 MEHRD

MEHRD Annual Work Plan 2017 and 2018 MEHRD

Public Financial Management Plan AHC

Solomon Islands Government Recurrent Estimates 2018 MOFT

Solomon Islands Budget 2018: Budget Strategy and Outlook, Volume 1 MOFT

Central Bank of Solomon Islands, 2017 Annual Report MOFT

School Financial Management Handbook MEHRD

Solomon Islands National Disability Inclusive Education Policy 2016–2020 (Draft) MEHRD

Whole-School Inspection Guidelines for Inspectors in Solomon Islands 2017 MEHRD

Solomon Islands Gender Equality in Education Policy 2018-2022 (Draft) MEHRD

Solomon Islands SISTA Report 2016 MEHRD

O’Meara, F. (2017) Solomon Islands Scholarship Analysis Report MEHRD

Tavola, H. (2010) Barriers to Education in Solomon Islands MEHRD

Holzapfel, S. and Janus, H. (2015) Improving Education Outcomes by Linking Payments to Results: An Assessment of Disbursement-linked Indicators in five Results-based Approaches.

Internet


Page 36

Appendix 4: Key Informant Interviews

Name Position Organisation

Leah Horsfall First Secretary Education AHC

Moses Tongare Senior Program Manager AHC

Alice Eric Program Manager AHC

Dana Avram Second Secretary, Development New Zealand High Commission

Helen Rose Education Sector Management Adviser MEHRD – SSU

Jimmy Rodgers Permanent Secretary (former) MEHRD

James Bosamata Deputy Secretary MEHRD

Joe Ririmae Manager, Monitoring and Evaluation Manager MEHRD –SSU

Richard Ardomana Manager, Planning MEHRD – SSU

Emily Siriki Manager, Inspectorate MEHRD – National Education Services

Steven Rose Monitoring and Evaluation Adviser MEHRD - SSU

McGreggor Richards Technical Adviser, Payments and Finance MEHRD – Finance

Martin Marairi Chief Accountant - Grants MEHRD – Finance

Ben Namo Manager, Internal Audit MEHRD – Internal Audit

Linda Wate Director, Teaching and Learning MEHRD – National Education Services

Tabe Tongoua Chief, Classroom Assessment, NEAD MEHRD – National Education Services

Rose Paia Manager, Curriculum Development MEHRD – National Education Services

Charles Rouikera Manager, Teacher Professional Development MEHRD – National Education Services

Joe Takeli Team Leader, Literacy and Numeracy Unit MEHRD – National Education Services

Alison Davis Literacy Adviser NZ MFAT

Clement Tito Manager, National Training Unit MEHRD - NTU

Mathias Kutai Manager, Teaching Services MEHRD - TSD

Robert Peupelu Chief, Learning Resources MEHRD - CESS

John Solodia Store Supervisor MEHRD Warehouse MEHRD - CESS

Silas Sitai Policy Analyst MEHRD - SSU

Christina Bakolo Coordinator – International Education Coordination and NEB

MEHRD - SSU

Ali Kiko Manager, EACI MEHRD – EACI

Godfrey Sautehi Chief, EACI MEHRD - Procurement


Page 37

Deborah Wargent Procurement Technical Adviser MEHRD

Johnson Lobo Supervising Audit Manager Office of the Auditor General

Constance Nasi Under Secretary, National Education Services (NES)

MEHRD – NES


Page 38

Appendix 5: Curriculum Mapping

Green = completed; Red = not started; Orange = in progress; Purple: not needed

YEARS 1–6 Syllabus Learner

Book (LB)

Teacher

Guide

(TG)

Comment

English Nguzu Nguzu 1–3 almost complete;

Nguzu Nguzu 4–6 to be completed

Mathematics Nguzu Nguzu 1–6 to be re-developed

under a new contract

Science

Social Studies

Health

Education

Year 1 and 3 LB and TG to complete

(completed by Pearson but not delivered

as contract terminated);

Year 4–6 complete

PE Syllabus in draft – all resources yet to be

developed

Arts and

Culture

Syllabus approved; TGs and LB to be

developed

Christian Life

ICT Syllabus in draft – nothing else

Vernacular

languages

Mentioned in NCS statement – status

unclear


Page 39

Junior Secondary (as at June 2018)

YEARS 7–9 Syllabus Learner

Book

(LB)

Teacher

Guide

(TG)

Comment

English Y 7–8 LB/TGs complete; Y9 LB/TG in

development

Mathematics Y 7–9 LBs & Y7 TG complete; Y 8–9 TG being

developed

Science

Social

Studies

Y 7–8 LB/TGs complete; Y9

Learning/Teaching Guide in development

Health

Education

Y8–9 TG and Y9 LB need developing (all in

draft with Pearson)

PE LB not needed; Y7 TG complete; TGs

needed for Years 8–9

Arts and

Culture

Syllabus complete; TG to be developed; LB

not needed

Christian Life Syllabus complete; TG to be developed; LB

not needed

ICT Syllabus approved; TG to be developed; LB

not needed

Technology Y7 LB complete; Other TGs and LBs needs

development

Home

economics

Business

Studies

Y 7–8 complete; Y9 LB and TG needs

completion


Page 40

Appendix 6: 2017 Performance Summary

Priority MEHRD M&Eref Performance Indicators IA Payment AUD

Quality EO2 1.1.1 SISTA 2017 results are analysed by MEHRD and the findings communicated to schools and


80% 41,290$

Quality EO2.1 1.1.2 SISTA 2017 results indicate an improvement in student numeracy and literacy at each year level

assessed.

0% -$

Quality KO4.2 1.2.1 Findings from SISTA 2015 are used to develop professional development programs or teacher


100% 51,613$

Quality KO4.6 1.2.2 Professional development programs for teachers and principals are implemented and their


60% 30,968$

Quality KO8.2 1.2.3 Teacher and principal SISTA support materials are distributed to relevant schools. 100% 51,613$

Quality KO10.1 1.3.1 MEHRD analyses teacher appraisal data, or establishes new data collection methods, to assess



20% 10,323$

Quality EO2.3/

KO4.4

1.3.2 Increase from 2016 in % of certified teachers in primary and junior secondary schools 100% 51,613$

Quality KO5.1 1.4.1 Revised basic education curriculum is completed for all learning areas. 86% 44,301$

Quality KO8.1 1.4.2 Teacher support materials are developed and distributed to all primary and junior secondary

schools

41% 20,968$

Quality KO4.5 1.4.3 Teacher professional development programs are designed to support the implementation of the

revised curriculum.

10% 5,161$

Access KO2.1 2.1.1 MEHRD develops and publishes a strategy that identifies and plans the progressive removal of


70% 36,129$

Access EO1.2 2.1.2 2017/2018 transition rates, for boys and girls, from Year 6 to Year 7, and from Year 9 to Year


0% -$

Access KO2.4 2.1.3 The draft Solomon Islands National Disability Inclusive Education Plan is approved and

disseminated.

40% 20,645$

Access KO2.3 2.1.4 More structured and formalised processes are implemented to collect and analyse data about



0% -$

Access 2.1.5 The Gender in Education Policy is launched. 40% 20,645$

Mgt KO7.1 3.1.1 Standards for Education Authorities are developed in partnership with EAs. 70% 36,129$

Mgt KO11.1 3.1.2 Baseline data to determine strengths and weaknesses of EAs is collected and analysed. 100% 51,613$

Mgt KO11.2 3.1.3 MEHRD implements programs or activities to improve the management capability of Education

Authorities.

100% 51,613$

Mgt KO13.2 3.2.1 M&E Framework is approved by Permanent Secretary. 100% 51,613$

Mgt KO13.3 3.2.2 M&E Data are collected and analysed 100% 51,613$

Mgt KO13.4 3.3.1 NEAP M&E data are reported in the 2017 Performance Assessment Report 100% 51,613$

Mgt KO13.5 3.3.2 M&E findings are used to develop 2018 NEAP Annual Work Plan. 100% 51,613$

MgtEO3.3 4.1.1 At least 23% of SIG national recurrent budget was allocated to education. 100%

51,613$

Mgt EO3.6 4.2.1 Allowing for inflation the 2017 SIG recurrent expenditure for the per unit cost of primary


100% 51,613$

Mgt EO3.6 4.2.2 Allowing for inflation the 2017 SIG recurrent expenditure for the per unit cost of junior


100% 51,613$

Mgt KO12.3 4.3.1 At least 80 % of 2017 School Grants are retired on time 0% -$

Mgt KO12.6 4.3.2 At least 85% of schools receive the first tranche of their 2018 School Grant by the end of March

2018.

100% 51,613$

Mgt KO12.4 4.4.1 At least one external audit is conducted by the Auditor-General’s Department or a suitably


60% 30,968$

Mgt 4.5.1 A cost-benefit analysis of the scholarships program is designed and approved for

implementation.

40% 20,645$

Mgt 4.6.1 2017 scholarships expenditure is not higher than 2016 scholarships expenditure. 0% -$

Mgt 5.1.1 % of activities scheduled in the PFM Action Plan for 2017 completed. 64% 32,975$

Recommended Peformance Payment by Independent Assessor 64% 1,024,000$

2017 PERFORMANCE LINKED AID ASSESSMENT SUMMARY


Page 41

Appendix 7: Draft 2018 Education Performance Matrix

NEAP Priority

NEAP/MEHRD AWP40

MELP Outcome areas

2018 Indicators/AWP outputs for Performance Payment

Responsible Evidence/rubric for scoring (%)

Access EO1.2 (TBA)

1.1 Transition rates by level and gender

1.1.1 The SISE Phase-out Committee has costed plan in place to cater for increased enrolments in years 7 and 10 and alternative placement method and plan socialised with EAs 1.1.2 Transition Rates for year 6 to 7 and year 9 to 10 from 2017 to 2018 show an increase over 2016-2017

NEAD Quality costed plan (30%)

SMT Minute endorsing plan (50%)

EAs socialisationcompleted(10%)

Transition Rates increase (10%) (as per Annual Report 2018)

Access EO1 (65)

1.2 Safe and equitable access

1.2.1 Gender Equality in Education Policy – implementation strategy costed and approved by NEB 1.2.2 Child Protection Policy consulted on and revised in light of consultations

PA A quality Gender Equality in Education Policy – implementation strategy costed (30%) and NEB endorsement minute (30%)

A quality Child Protection Policy consulted with selected EAs and schools (30%) and revisions made based on consultations (10%)

Quality IO2.1 (11& 13)

2.1 More teachers using new improved curriculum

2.1.1 PPY Pilot in Honiara, Guadalcanal and Malaita completed and evaluated 2.1.2 Cognition contract (as at June 2018) deliverables completed and accepted

CDD PPY Monitoring Report received by MEHRD and UNICEF (35%)

Cognition deliverables (65%) including:

Year 9 English Teaching Learning Guide ready for desktop publishing and printing

Year 9 Social Studies Teaching Learning Guide for desktop publishing and printing

Year 8 Maths Teacher Guide ready for printing

Year 9 Maths Teacher Guide ready for desktop publishing and printing

Senior secondary curriculum framework complete

Syllabi developed in priority subject(s)

40 These references relate to either a MELP End or Intermediate outcome, Annual Work Plan 2018 Task number or whether it is a DFA (Direct Financing Agreement) condition. TBA (to be advised) refers to indicators that currently have no Annual Work Plan reference.


Page 42

NEAP Priority

NEAP/MEHRD

MELP Outcome areas



Quality IO2.3.1

(137)

2.2 Teachers meeting agreed standards

2.2.1 Revised National professional standards for teachers (within Teacher Service Handbook)

TPDD Standards

Revised quality National professional standards draft consulted with selected provinces (50%)

Approval minute from SMT (50%)

Quality EO 2 (93)

2.3 Number of students achieving at or above the expected literacy and numeracy level

2.3.1 2017 SISTA results analysed and disseminated in user-friendly way to national, EAs and school level 2.3.2 SISTA 2017 results show an improvement in both literacy and numeracy for both grades 4 and 6

NEAD SMT endorsement minute of SISTA results including national, provincial and school level reports (40%)

Evidence of dissemination to national, provincial and school level (e.g. meeting minutes, back to office reports) (30%)

An average of all literacy domains has improved and numeracy has improved (30%)41

Quality EO2.3 (140)

2.4 Percentage of certified teachers by gender

2.4.1 By the end of 2018, delivery of USP Certificate in Teaching Primary (courses ED152 and ED100) for 75 untrained teachers completed 2.4.2 A Teacher Supply and Demand study endorsed by SMT

TPDD TSD

Untrained teachers (target of 75) complete both courses as reported in Annual Report 2018 (30%)

A quality Teacher Supply and Demand study endorsed by SMT (70%)

Quality (48)

TBA

2.5 School grant expenditure used for learning materials

2.5.1 By the end of 2018, school grants (Secondary, Primary and TVET) training completed in three provinces 2.5.2. Analysis of retirements to show breakdown of expenditure

Grants Division/ Finance

Training that promotes the use of school grants for learning materials carried out. Training reports or records for three provinces in 2018 (50%)

Report on analysis of acquittals showing currently how much of school grants is used for learning materials (50%)

41 As per main body of report, the 2018 Preliminary Report on SISTA 2017, states SISTA 2015 & 2017 are not statistically comparable, further discussions with EQAP pending. There is a possibility that EQAP can make a judgement on this.


Page 43

Management

EO 3.1 (55-56)

3.1 Retirement percentage of 2018 Second Biannual School Grants improved and 2019 first tranche disbursement on time

3.1.1 At least 65% of 2018 School grants are retired by 31 March 2019 (2018 Second Biannual) 3.1.2. At least 85% received the first tranche of their 2019 grant by 31 March 2019

Grants Division/ Finance

MEHRD Acquittal report for Primary, Secondary, TVET as at 31 March 2019 – retirements (50%)

MERHD Acquittal report for Primary, Secondary, TVET as at 31 March 2019 - disbursement of first tranche for 2019 (50%)

(NB: ECCE to be phased out)

NEAP Priority

NEAP/MEHRD

MELP Outcome areas



Management systems

TBA 3.2 New education bill is passed

3.2.1 Draft Education Bill is endorsed by NEB in October

SMT Bill in final format approved by SMT (50%)

NEB memorandum to Minister of EHRD (50%)

Management systems

IO3.2 (40)

3.3 Education Authorities operate to agreed standards

3.3.1 Standards approved, 27 EAs receive training including initial self-assessments completed and analysed

EACI Quality EA Standards drafted (20%)

SMT approval minute of EA Standards (50%)

Training in initial self-assessments completed by EACI (30%)

Management systems

E03.6 3.4 The 2018 recurrent expenditure for Basic Education is maintained or improved

3.4.1 Recurrent expenditure for per-unit cost per student at Primary and Secondary School in 2018 is equal or greater than 2017

Finance Calculate using real prices, recurrent allocation includes primary (50%) and secondary (50%) grant component. (2018 Annual Report, expenditure report (actuals)42

Management systems

DFA 3.5 Solomon Islands Government scholarships budgeting and expenditure is sustainable

3.5.1 The total of new SIG scholarship awards allocated for 2018 is 700 or fewer including for local and international institutions 3.5.2 2018 MEHRD Scholarship expenditure is not higher than 2017 MEHRD Scholarship expenditure

NTU, Finance SIEMIS Expenditure Report as per 2018 Annual Report

(50%)

NTU SIMS data report (50%)

42 It is recommended that this calculation use only grant expenditure as payroll expenditure for secondary and primary teachers is often wrongly coded. In future years, as the data is cleaned, it would be more accurate to include payroll.


Page 44

Management systems

EO3.3 DFA

3.6 The Solomon Islands Government maintains its recurrent budget allocation for education in 2017

3.6.1 Overall MEHRD recurrent expenditure above 22% of total SIG recurrent expenditure

Finance, SMT SIEMIS Expenditure Report as per 2018 Annual Report (100%)

Management systems

DFA 3.7 External Audits of MEHRD finances are conducted

3.7.1 2015 External Audit of MEHRD complete (four cycles and transactions), including MEHRD management response

OAG SMT

MEHRD Management Response for External Audit Part 1 completed (70%)

MEHRD Management Response for External Audit Part 2 completed (30%)