MONTHLY TECHNICAL REPORT EVIDENCE S R M N MORTALITY …es.dipi.id/wp-content/uploads/2017/07/6.Jan-2017-Monthly-Report.pdf · PONEK Pelayanan Obstetri Neonatal Emergensi Komprehensif

This report is made possible by the support of the American People through the Unites States Agency for International Development (USAID). The contents of this report are the sole responsibility of the Indonesian Academy of Sciences (AIPI) and do not necessarily reflect the views of USAID or the United States Government.

MONTHLY TECHNICAL REPORT EVIDENCE SUMMIT FOR REDUCING MATERNAL AND NEONATAL MORTALITY IN

INDONESIA

JANUARY 2017

Indonesian Academy of Sciences

Jalan Hang Lekiu I no. 6A, Kel. Gunung, Jakarta Selatan

DKI Jakarta, Indonesia 12120

Technical Monthly Report: January

i

CONTENTS Contents ........................................................................................................................................... i

Abbreviations .................................................................................................................................. ii

1. Executive Summary .................................................................................................................. 1

2. Objective & Output Assessment .............................................................................................. 1

3. Activities completed .................................................................................................................. 2 3.1 SUMMARY OF ACTIVITIES .............................................................................................................................. 2 3.2 DESCRIPTION OF ACTIVITIES ....................................................................................................................... 2

Reflective analysis ........................................................................................................................... 8

Next steps ....................................................................................................................................... 9

Annexes ......................................................................................................................................... 10 Annex 1. Recap of Systematic Review Process ................................................................................................. 10 Annex 2. List of local teams based on the location of institutions ............................................................... 11 Annex 3. Stakeholder Engagement Strategy ....................................................................................................... 13


ii

ABBREVIATIONS

AER Associate Evidence Reviewer

AIPI Akademi Ilmu Pengetahuan Indonesia (Indonesian Academy of Sciences)

CTG Core Technical Group

EC Executive Committee

ERT Evidence Review Team

ES Evidence Summit

FQ Focal Question

IDAI Ikatan Dokter Anak Indonesia / Indonesian Paediatric Society

IT Information Technology

MCSP Maternal Child Survival Program

PIC Person in Charge

POGI Perkumpulan Obstertri & Ginekologi Indonesia / Indonesian Association of Obstetrics and Gynecology

PONED Pelayanan Obstetri Neonatal Emergensi Dasar/ Basic Emergency Obstetry Neonatal Care

PONEK Pelayanan Obstetri Neonatal Emergensi Komprehensif/ Comprehensive Emergency Obstetry Neonatal Care

SC Steering Committee

SOP Standard Operating Procedures

TA Topic Area

URC University Research Co., LLC

USAID United States Agency for International Development


1

1. EXECUTIVE SUMMARY In overall, the activities during January 2017 focused on accelerating the review process. The major accomplishment is the parallel supervision meeting between the CTG and AER which resolved several review process issues. The secretariat also accelerated of the evidence searching process by appointing local young health professionals (persons in charge, or PICs) to do hand searching in 30 universities. The main challenge in the administrative issues is the inadequate communication pathway, while the main challenge in the technical issues of review process is insufficient searching strategy. Therefore, the recommendation had been proposed to address the challenge by developing a SOP of communication and stakeholders engagement strategy, an SOP for monitoring and evaluation for ES program (conducted by ES secretariat); and an SOP for protocol assessments which will be conducted by CTG prior to the quality assessment.

2. OBJECTIVE & OUTPUT ASSESSMENT

Objective Timeline Activities Outputs

1. Accomplishment of literature searching within the web-based systematic searching, local journals and grey area literatures

November - December 2016

1. AER Group Working

2. Hand-searching process to universities

1. The glossaries for all focal questions are developed to improve the keywords for literature searching

2. Selected 24 local journals that have been accredited by LIPI and Ministry of Research, Technology, and Higher Education

3. 6,357 articles gathered from web-based systematic searching and local journals as of 31st January 2017

4. 36 literatures obtained from 12 institutions from the Call for Evidence

5. 776 grey literatures gathered through hand searching from 30 universities (518 bachelor degree theses, 235 master degree theses; and 23 doctoral dissertations)

2. Accomplishment of evidence selection and evidence review

End of December 2016

1. CTG-ERT Workshop I

2. AER-CTG Supervision Meeting

1. 1,087 articles screened as of 31st January 2017

2. 945 articles sorted as of 31st January 2017

3. 18 eligible evidences as of 31st January 2017


2

3. Stakeholder engagement and communication plan developed

January 2017 1. AIPI-ES Consolidation Meeting

2. EC-SC Consolidation Meeting IV

3. Workshop Stakeholder Engagement Strategy

1. The agreement between AIPI and ES on harmonizing the procedure of product publication was finalized

2. The SC approved the proposed strategy for stakeholders’ engagement

3. Preliminary analysis of communication strategy

4. Preliminary stakeholders’ analysis

3. ACTIVITIES COMPLETED

3.1 SUMMARY OF ACTIVITIES

No Objective Activity January 2017 I II III IV

1 Accomplishment of literature searching

AER Group Working 5-11 Jan

5-11 Jan

2 Hand Searching 16 – 20 Jan

3 Accomplishment of evidence selection

and evidence review

CTG-AER Supervision Meeting

5 – 9 Jan

4 CTG-ERT Workshop I 11 Jan

5

Stakeholder analysis

AIPI-ES Consolidation Meeting 9 Jan

6 EC-SC Consolidation Meeting IV

9 Jan

7 Workshop Stakeholder

Engagement Strategy with JHPIEGO

26 Jan

3.2 DESCRIPTION OF ACTIVITIES

3.2.1 OBJECTIVE I: ACCOMPLISHMENT OF LITERATURE SEARCHING WITHIN THE WEB-BASED SYSTEMATIC

SEARCHING, LOCAL JOURNALS AND GREY AREA LITERATURES 3.2.1.1 AER group working (January 5-11, 2017) Objectives a. Provide the supported working environment to finish the delayed literature searching b. Provide an opportunity for AER and ERT members which are staying in Jakarta to have direct

meeting and coordination about the literature searching.

Results


3

The participation rate is 100%. Twenty-four journals were selected to be addressed by the AER in expanding the literature searching. The AERs have not yet finalized the literature searching from non-database bibliographies. AERs finalized the matrices and glossaries. Challenges 1) Administrative issues

a) Due to the ERT’s tight schedule, not all AER members were able to work closely with their ERT.

b) As a result of the CTG’s decision to expand the number of sources and extend the process timeline, the administrative procedures for the review process are unclear, which has made the literature searching process more challenging.

2) Technical issues of evidence gathering process a) The AER could not upload the full text article into Covidence because a high number of

articles from particular journals require a payment to access the full text. b) The lack of understanding about the importance of method rigor among AER members

effected the quality of the result of literature searching.

Recommendations 1) The group work model less likely to be conducted for the AER members in the future due to the

lack of efficiency of the activity in achieving the outputs. 2) The performance of AER members need to be improved by intensifying the monitoring and

supervision. 3) The development of the SOP of administrative procedures and technical literature searching process

is essential and should be disseminated to all AER members as soon as it is finalized.

3.2.1.2 Hand searching by local team (16-20 January 2017) Objectives 1) Inform the higher education institutions about Call for Evidence 2) Gather the grey literatures to support the review process

Results 1) There were 30 higher education institutions which received the information and invitation to the

“Call for Evidence”. The list of responsible persons in each institution is provided in the annex. 2) There were 22 local PIC who participated in hand searching activities.


4

5

211

4832

167

16611

6841

9

4

10

0 20 40 60 80 100 120

UniversitasYarsiUniversitasIslamSultanAgung

UniversitasAtmajayaUniversitasMuhammadiyahSurakarta

UniversitasPelitaHarapanUniversitasCenderawasihUniversitasIslamIndonesia

UniversitasMuhammadiyahMalangUPNVeteran

UINSyarifHidayatullahUniversitasUdayana

UniversitasMaranathaUniversitasIslamBandung

UniversitasBrawijayaUniversitasSebelasMaret

UniversitasBengkuluUniversitasMuslimIndonesia

UniversitasDiponegoroUniversitasMuhammadiyahYogyakarta

UHAMKAUniversitasPadjajaran

UniversitasAndalasUniversitasLampungUniversitasIndonesia

UniversitasMuhammadiyahJakartaUniversitasAirlangga

UniversitasSumateraUtaraUniversitasWijayaKusuma

UniversitasGadjahMadaUniversitasHassanudin

#ofSourcesofEvidence

ResultsofhandsearchingbasedontargetedinAtuAons

UndergraduateThesis

PostgraduateThesis

DoctoralDissertaRon

3) The total number of gathered grey literatures is 776 articles which contained of 518 undergraduate theses, 235 postgraduate theses and 23 doctoral dissertations.

Challenges 1) Some of the local PIC took longer to conduct the hand searching due to local institutions’

regulation requiring a formal letter to retrieve evidence 2) Some universities still have no digitalized library hence the local PIC performed the literature

searching manually. This poses the risk of potentially missing some intended literatures and prolonged the length of searching process.

Recommendations 1) More time is needed to perform hand searching in the future. 2) The local personnel for hand searching are suitable from young health professionals or

students which have organizational experience as they have a good soft skills and familiarity with the university regulation/procedures.

3.2.2 ACCOMPLISHMENT OF EVIDENCE SELECTION AND EVIDENCE REVIEW

3.2.2.1 CTG-AER Supervision Meetings (January 5-9, 2017)


5

Objective Provide support and consultation from substantial and methodological aspects for literature searching process.

Result The participation rate of the supervision meeting is 100% (two AER members joined the meeting by videoconference and teleconference). The meeting resulted in the establishment several priority areas for improvement and next steps: develop a glossary to standardize terminology, improve the protocols and matrices, approve protocols once final review of PICO variables is complete, and correct search terminologies to improve literature searching results. Challenges

1) Administrative issue • Not all focal questions were discussed due to lack of preparation for the video conference

2) Technical Issue

• The literatures for some Focal Questions are very difficult to find due to limited search and keywords, such as Focal Question 1i, 2C and 3B (FQ related to PONEK and PONED). Therefore, the ES team will use hand searching and qualitative data analysis in coordination with the next implementing partner, Jhpiego.

Recommendations 1) For the next meeting, the secretariat should consider using a small group working style to improve

the coordination among CTG, ERT and AER and thus accelerate the review process 2) The secretariat should purchase a Journal subscription which is essential to provide the full text

articles.

3.2.2.2 CTG-ERT Workshop I (January 11, 2017)

Objectives 1. Consensus among CTG members regarding the evaluation of the review process 2. Progress report of evidence review from ERT to CTG 3. Preparation for quality assessment � 4. Finalization of evidence review

Results Participation rate of the workshop is 67.2% (41 out of 81 invited participants). There were some emerged issues related to the review process and administrative issues, in details below:

1) The progress of review process is described as bellow • Total gathered literatures (result of literature searching by AER) = 5,007 • Total sorted literatures (the number of relevant literatures after title and abstract

screening) = 2,189 • Total eligible literatures (the number of included literatures after abstract review and ready

for full text screening) = 656


6

2) There is no protocol that is sufficient to support the literature searching 3) The CTG members discussed the performance of AER which is perceived to be unsatisfactory and

its effects on the result of literature searching 4) The evidence review is not finalized yet due to the need to revise the searching method and

protocol 5) While the ERT is working with revision of systematic review and bibliography searching, the grey

area literatures are essential to enrich the provision of literatures

Challenges 1) Administrative issue

• Actual activities strayed from the established agenda, caused inefficiency of the workshop due to lack of monitoring of ERT and AER by the secretariat and CTG during the workshop

2) Technical issue • The current protocol and matrix are not sufficient to perform a quality assessment, no is the

instrument ready • Poor internet connection in the venue due to too many people accessing it at the same time.

This resulted in a delay in the searching and reviewing processes, including uploading the literatures to Covidence

Recommendations 1) According to the importance of internet connection for supporting the literature searching and

review, the venue is not recommended for the next workshop because of poor internet connection 2) The secretariat needs to designate PICs for hand searching in higher education institutions to gather

the grey area literatures 3) Rearranging the time commitment of AER and ERT is essential as well as more adequate monitoring

and evaluation system from secretariat 4) A protocol assessment needs to be conducted to ensure the protocols and matrices drafted follow

the guidelines.

3.2.3 OBJECTIVE III: IDENTIFY THE INVOLVEMENT OF VARIOUS ELEMENTS OF STAKEHOLDERS IN THE ES PROJECT

WITHIN THE INTERNAL AND EXTERNAL OF THE PROJECT

3.2.3.1 AIPI-ES Consolidation Meeting (January 9, 2017) Objectives 1) Discuss the standard operating procedures (SOP) for AIPI products 2) Discuss the recommended stakeholders engagement and communication strategy for the Evidence

Summit project

Results The meeting was attended by 12 participants out of 14 invited participants (85.7%). The meeting met

the two proposed objectives and achieved the following results:

1) Standard operating procedures for AIPI products • In order to assure the quality of the Evidence Summit products, it is essential to comply with the

AIPI SOP for assessing and reviewing products which are published under AIPI’s name.


7

2) Stakeholders Engagement Strategy for Evidence Summit project • The low response rate of the “Call for Evidence” indicates the insufficiency of stakeholder

engagement. • ES secretariat conducted a preliminary stakeholders’ analysis to identify the existing involvement

of various internal and external stakeholders in the ES project. The result (Annex 3) requires further development and analysis.

3) Communication Strategy for Evidence Summit project • Communication strategy becomes the main concern of AIPI which should be in line with SOP of

AIPI. Therefore, the ES secretariat developed communication and branding strategies (Attachment 1) according to the lessons learned during ES implementation.

• The model of communication among internal project team members will be a lessons learned for Indonesia or other countries in conducting systematic review in national scale.

Challenges 1) The SOP for communication and product development of AIPI are not yet finalized 2) The advisory committee has not been established yet to ensure the quality assurance of AIPI’s

products

Recommendations 1) The ES is the pilot project for AIPI’s communication strategy. Therefore, the ES will involve AIPI’s

communication officer during the assessment and development of the SOP. 2) Once AIPI’s SOP for communication and product development is implemented, further discussion

will be required to harmonize the ES production with AIPI’s SOP. 3) Although the SOP of quality assurance of AIPI’s products is still under development, the ES team

will perform the mechanism which is proposed by AIPI. Currently, the products of ES are proposed to follow four formats: Consensus Report, Convening Activity Report, White Paper/Policy Brief, and Public Outreach.

4) ES and AIPI need to work together in developing the standards of communication, branding, stakeholder engagement and product development.

3.2.3.2 EC-SC Consolidation Meeting IV (January 9, 2017)

Objectives 1) Present the progress report of Evidence Summit Project by Executive Committee � 2) Discuss the proposed extension of grant for Evidence Summit Project � 3) Discuss the preparation to disseminate project outputs in Pre-Summit (Rakerkesnas)

Result The meeting was attended by 14 participants out of 19 invited participants (73.7%). The following issues emerged during the meeting:

1) The need for support from international experts to translate the evidence into policy 2) It is essential to highlight the local government contributions and roles on reducing maternal and

neonatal mortality, particularly the dynamic implementation of local government regulations 3) The preparation of Pre-Summit materials for the Rakerkesnas (national meeting of Ministry of

Health) will be focused only on focal questions 1, 2, and 3


8

4) Steering committee members approved the proposed timeline and transitional period activities between URC and Jhpiego (please see Annex 4). Some activities need to be readjusted due to the extended schedule, as below: • The deadline of bibliography finalization needs to move from June to July 2017 • Summit is expected to be held in early August 2017. • The National Stakeholder consultations remains in the first week of Sept 2017.

Challenges a. The scheduled date of the Rakerkesnas was changed by the MoH.

Recommendations

a. Readjustment of timeline requires intensive coordination among the Secretariat, SC, EC, CTG, USAID, URC, and Jhpiego.

REFLECTIVE ANALYSIS 1. The new structure of project team

The new structures are established for the secretariat team and the review team. In the secretariat, the project officer tasks are divided into two officers: (1) administration and stakeholder engagement; and (2) data and literature management. The new structure is expected to decrease the burden of work in one officer hence it is expected to improve the coverage and productivity of works.

On the other hand, the new structure of the reviewer teams focuses on team-based working groups instead of vertical coordination among CTG, ERT and AER. The team-based working groups aim to improve the communication among CTG, ERT and AER by allowing them to monitor each other. Therefore, involvement of CTG is expected to improve the participation of ERT and reduce the power imbalance between AER and ERT.

2. The delay of review process

Based on the project timeline, the review process is delayed particularly for quality assessment and data extraction. There are three potential causes of delayed review process, such as: Inadequate communication pathways and monitoring system; lack of commitment of ERT and unfamiliar of ERT to the virtual working style; and unequal performance of AER.

In order to mitigate these challenges, the CTG will be more involved in the coordination and communication between ERT and AER; the Secretariat will facilitate supervision meetings between the CTG, ERT, and AER to ensure the process review is on track; and the Secretariat will conduct more intensive monitoring through Covidence and the WhatsApp group.

3. The importance of improving stakeholders’ engagement The low response rate of the call for evidence is a result of a lack of stakeholder engagement.

The low participation of external stakeholders is caused by inadequate information and exposure to the project which contributes to a lack of awareness among stakeholders. Therefore, significant efforts need to be made to improve the stakeholder engagement in order to ensure the acceptability and the sustainability of the formularized recommendations from the ES project.


9

NEXT STEPS As for the plan of actions to be commenced in the next stage, in order to ensure the quality of the protocols, which affects the appropriateness of literature searching, the protocol assessment will be conducted on 2nd February 2017. Moreover, developing a monitoring and supervision strategy for the review process is essential to improve the performance of CTG, ERT and AER. Better performance and appropriate recognition of that performance will make the review process more effective.

A Stakeholder Communication Forum is scheduled for 2nd February 2017 in order to strengthen the engagement among external and internal stakeholders. The forum is expected to give the mapping of stakeholders’ expectations and their articulated roles in the ES project. As a consequence, the communication strategy can be proposed effectively and tailored to the culture and condition of expected stakeholders, in order to ensure the sustainability of ES program.

Documents

MONTHLY TECHNICAL REPORT EVIDENCE S R M N MORTALITY …es.dipi.id/wp-content/uploads/2017/07/6.Jan-2017-Monthly-Report.pdf · PONEK Pelayanan Obstetri Neonatal Emergensi Komprehensif