12/04/2014
STRENGTHENING INDEPENDENT
IMMUNIZATION AND VACCINE
ADVISORY COMMITTEES (NITAG)
TECHNICAL CAPACITIES
Report of the East and Southern
African NITAG training workshop,
Naivasha (Kenya) 3-7 August 2014
2
Table of contents 1 Background 3
2 Objectives of the workshop 4
3 Method of work 4
3.1 Format 4
3.2 Topics on the agenda 4
4 Introduction 5
5 Training roll-out 5
5.1 Session1: roles and mode of operations of NITAGs 7
5.2 Session 2: Guidelines and tools for NITAGs functioning 9
5.3 Session 3: Monitoring the committee’s progress and achievements 10
5.4 Session 4: Development of an evidence-based recommendation- 1) defining
the recommendation framework 11
5.5 Session 5: Development of an evidence-based recommendation - 2) how to
search evidence 12
5.6 Session 6: Development of an evidence-based recommendation – 3) how to
evaluate data relevance and quality 13
5.7 Session 7: Development of an evidence-based recommendation – 4) how to
structure a recommendation note 14
5.8 Session 8: Communicating NITAG recommendations 15
5.9 Session 9: Networking 16
6 Conclusions 16
6.1 Next steps 16
6.2 Overall evaluation of the training 17
6.3 Considerations for future trainings 18
3
1 Background
Following on the WHO and its Strategic Advisory Group of Experts (SAGE) recommendations to
countries to establish National Immunization Technical Advisory Groups (NITAGs), the World
Health Assembly and other international immunization stakeholders have reiterated the
importance of establishing NITAGs that would be responsible for advising policy-makers on all
immunization-related aspects, including new vaccine adoption, strategies to increase use of
under-utilized vaccines and routine immunization programme performance, and evaluating
effectiveness of current immunization schedules. Establishment of NITAGs will ensure national
immunization policy decision process is more transparent, evidence-based, taking into account
local context therefore reinforcing the credibility of immunization policy, its sustainability and its
acceptance by the population.
The Supporting Independent Immunization and Vaccine Advisory Committees (SIVAC) Initiative
supports the creation and strengthening of sustainable NITAGs. The SIVAC Initiative is
implemented by the Health Policy and Institutional Development Unit of the Agence de
Médecine Préventive (AMP), a WHO collaborating Centre for the use of evidence-informed
immunization policy-making. The core of the initiative’s business is to build capacity at the
national level to establish and run a NITAG in accordance with WHO guidelines.
Between 2008 and 2013 the SIVAC Initiative has supported the creation or strengthening of 13
NITAGs and organized over 20 training workshops in collaboration with partners such as the
World Health Organization (HQ; Regional Offices; Evidence-Informed Policy Network (EVIPNet)
project), the International Vaccine Institute (IVI), US Centers for Disease Control (US-CDC), and
other units within AMP.
In East and Southern Africa, the SIVAC Initiative has supported Kenya and Uganda in the set-up
of their NITAG and has initiated collaboration the Republic of South Sudan and Zambia who
have expressed interest in setting-up/strengthening their NITAG. The initiative has also started
engaging Ethiopia in the discussion of the possible role of a NITAG in the country.
The SIVAC Initiative collaborated with WHO and CDC-Kenya to organize on August 4-7, 2014 in
Naivasha (Kenya) a regional training for Eastern and Southern African NITAGs. The workshop
brought together members of the Kenya, Uganda and South Sudan NITAGs, secretariats of
future NITAGs in Ethiopia and Zambia, and EPI focal persons in WHO and UNICEF country
offices. A total of 35 people participated in the workshop. The list of attendance is in annex 1 of
this report.
4
2 Objectives of the workshop
++++ To orient participants on NITAGs functioning: roles, responsibilities, mode of operations and evaluation
++++ To familiarize participants on the principles and methodology for issuing evidence-based recommendations, including tools adapted from EVIPNet, using case studies on flu vaccine and rotavirus vaccine
++++ To initiate mechanism for regional collaboration and experience sharing between NITAGs
3 Method of work
3.1 Format
The 3 ½ day interactive training workshop combined plenary sessions alternating with group
work. Reference materials (templates, check lists, articles and reports) were provided to facilitate
the group work. The training started with a pre-workshop assessment of participants’ skills in
developing evidence based recommendations and use of evidence search strategies. An
evaluation was conducted at the end of each day: participants were asked to rate each session
(using the scale : below average, average or above average) in terms of its comprehensiveness,
clarity and interest. A final evaluation was also conducted at the end of the training.
3.2 Topics on the agenda
The modules were selected from the 3 training units of the SIVAC Initiative training catalogue
and are listed below:
1. Training 1: Establishment and mode of operations of an effective NITAG
• Module 1: Role, scope and functions of a NITAG;
• Module 2: NITAG processes (tools);
• Module 3 : Communication of recommendations and involvement of stakeholders;
• Module 4: Evaluation of NITAG functionality, performance and impact.
2. Training 2: Analysis of Health Systems, Immunization and Policy-decision process
5
• Module 1: Landscape analysis of the health system and immunization context and
how it can impact proposed recommendation
3. Training 3: Technical and scientific capacities of NITAGs
• Module 1: Evidence assessment methodologies;
• Module 2: Development of an evidence-informed recommendation
The topics were covered in 8 sessions. In addition, a session was devoted to discussion around
networking and exchanges between NITAGs. The agenda of the workshop is shown in annex 2.
4 Introduction to the workshop
In his welcome note, the head of Kenya Unit of Vaccine and Immunization Services shared
Kenya experience in the process of establishing the advisory group ( the KENITAG) highlighting
some of the challenges faced along the way.
The Chair of the Kenya NITAG provided a brief introduction and the rationale for establishing
NITAGs; he presented the proposed structure and functioning of the KENITAG. The KENITAG
will be supported by a secretariat (UVIS); within the committee focal points are designated for
each of the following areas: VPD epidemiology, health systems and immunization, vaccinology,
monitoring and evaluation, policy.
The key note delivered by WHO Inter-country Support team for East and Southern Africa
highlighted the role NITAG may have in supporting the implementation of the Global Vaccine
Action Plan.
5 Training roll-out
The quick pre-training assessment of participants’ skills and knowledge in issuing evidence-based recommendation showed that the majority of them do not have experience in using a systematic process to identify and evaluate the quality of the evidence. While most of them have used scientific databases, the majority have not used information search strategies.
6
7
5.1 Session1: roles and mode of operations of NITAGs
The session started with a group exercise: participants were asked to list on a flip chart key
features they attribute to the various immunization committees existing in their countries in terms
of membership, structure, scope and mandate.
8
The aim of the exercise was to assess participants understanding of the differences between
NITAGs and other immunization committees.
Each group reported back to plenary. Bridging from the exercise, a presentation was made to
further clarify NITAGs role and mandate as recommended by the WHO. Participants from Kenya
and Uganda then shared the terms of reference of their respective NITAG, highlighting how
these compare to the WHO guidelines.
Take-away messages from the
session
Participants evaluation of the session
• Authorities and all key stakeholders should
understand and buy into the idea, this can
make the process of NITAG establishment
quite long ( 2-3 years from Kenya experience)
• Participants realized the need to have clear
terms of reference for NITAGs so as to avoid
duplication with other existing committees
especially with the ICC.
• Linkages with existing ad-hoc committees
should be established to create synergies
• The systematic, transparent and independent
process of decision-making used by the NITAG
and the wide range of technical expertise in the
committee justify its place in countries health
systems
• Lack of national experts in certain areas may
limit the NITAG work (South Sudan)
• Unfailing commitment of volunteer experts
throughout their tenure is not guaranteed as
they are busy with other activities
• Learning from other countries will help those
considering setting-up NITAGs speed up
process and learn about how it actually works
in “real life”
9
5.2 Session 2: Guidelines and tools for NITAGs functioning
The objective of the session was to familiarize participants with available tools and guidelines
developed by SIVAC Initiative to assist in an efficient functioning of NITAGs.
Guidelines on NITAGs internal procedures manual development and work plan templates were
distributed to participants. Participants were guided in the reading of the documents before going
into group work. The assignment was to do a review of the draft Kenya and Uganda internal
procedures manuals and workplan, i-e assessing their compliance with the guidelines and
templates presented earlier. This peer- review enabled Kenya and Uganda secretariats to take
note of areas where the documents need improvement; it was also an opportunity for the other
participants to grasp with the guidelines and templates.
Take-away messages from the
session
Participants evaluation of the session
• Role of the secretariat, core members
and no-core members needs to be spelt-
out with great details, and paying
attention to the principle of independence
of the committee
• Independence should be reflected in the
procedures for members nomination
• Clarity in all procedures is key
• Principle of conflict of interest and bias
declaration should be well highlighted in
the manual
• Process for setting-up working group
should follow same guiding principles of
transparency; responsibilities for its
functioning should be clarified
10
5.3 Session 3: Monitoring the committee’s progress and achievements
NITAG draft assessment protocol developed by the SIVAC initiative was presented to
participants. The tool intends to provide guidance for monitoring NITAGs activities as well as
assessing its performance and to a lesser extend its impact. The protocol refers to the 17
standard indicators proposed by WHO and propose an analysis framework for evaluation of
NITAGs with regards to their relevance, effectiveness, management efficiency and sustainability.
Participants provided comments on the assessment tool which will be finalized after additional
field testing. Uganda ACVI shared the lessons learnt from their evaluation exercise conducted in
February 2014
Take-away messages Participants evaluation of the session
• Monitoring and evaluation of
NIATG is made against what
is indicated in the manual
procedures and the workplan
which should reflect MoH
priorities
• Reasons for no
implementation of NITAGs
recommendations should be
investigated even though they
may not lead to the
conclusion of NIATG
inefficiency or failure
• Regular monitoring of
NITAGs activities as well as
scheduled independent
evaluation are means to
identify areas that require
corrective action or capacities
strengthening
05
101520253035404550
Below
Average
Average Above
Average
Comprehensiveness 16 11
Interest 12 15
Clarity 19 8
198
12
15
16
11
Session 3: Monitoring NITAGs progress and
achievements
11
The day ended with the feedback of the Kenya team who attended the US NITAG (ACIP)
meeting in April 2014 (the invitation was facilitated by CDC-Kenya ) followed by the projection of
an edited excerpt of the ACIP proceedings video. The projection illustrated some of the concepts
presented during the sessions: declaration of conflict of interest; framing the question of a
recommendation and organization of working groups.
5.4 Session 4: Development of an evidence-based recommendation - 1) defining the recommendation framework
In this session, participants learnt the 1ststep for developing a recommendation note e-g how to
define the key elements/criteria which will inform the recommendation. The session started with
a presentation of the principles for issuing evidence-based recommendation and a generic
immunization recommendation framework which builds on WHO guidelines for new vaccine
introduction and SAGE criteria. Participants were taken through the concept of framing a
question using the PICO approach (Population/Problem; Intervention; Comparator; Outcome).
Elements which will inform a recommendation should include vaccine-related criteria as well as
health systems, socio-cultural, equity and other issues. The 2nd presentation talked about health
system building-blocks, analysis of the immunization context and how they impact the decision-
making process.
Brainstorming on criteria a
NITAG would consider for
issuing a recommendation
on Rotavirus vaccine :
results from a group work
12
Take-away messages Participants evaluation of the session
• Defining clear recommendation
framework ensure transparency
about the ground for decision
• The outcomes/effects expected
from the vaccine under
consideration should be defined
• Specific information needed to
measure efficacy and
effectiveness should be clarified
• Local data (health systems,
economic, socio-cultural
environment) form an important
part of the recommendation
framework
• Literature review and
consultations with experts and
relevant stakeholders is used to
categorize the importance of
each factor for decision-making
• Different options should be
considered in the
recommendation articulated
according to different elements
that need to be in place, timing,
conditions, and with the aim of
adapting international evidence
to the country context
5.5 Session 5: Development of an evidence-based recommendation - 2) how to search evidence
This session focused on the 2nd step of the development of recommendation note e-g how to
find evidence on the elements of the recommendation framework. The session was organized as
a practice session during which participants had to retrieve evidence on some of the elements
(2-3 elements deemed critical) they have included in the framework of recommendation. The
13
assignment for each group was to register to 2 of the major data bases presented, define
appropriate criteria for literature search, refine search and select relevant articles.
Take-away
messages from the
session
Participants evaluation of the session
• For each critical and
important criteria
listed in the
recommendation
framework, search
should be done 1st for
relevant systematic
reviews and then for
other studies of
acceptable design
0
5
10
15
20
25
30
35
40
45
Below
Average
Average Above
Average
Comprehensiveness 2 7 13
Interest 1 4 17
Clarity 1 8 13
18
13
1
4
17
2
7
13
Session 5: Development of evidence-based
recommendation: 2) evidence search
5.6 Session 6: Development of an evidence-based recommendation – 3) how to evaluate data relevance and quality
The plenary session consisted in the review of the tools proposed for assessing the relevance
and the quality of the data reported in the articles retrieved. The check list and tools presented to
the participants included the GRADE methodology (when doing a systematic review), AMSTAR,
CASP, SIGN and CAQR checklists (for assessing systematic reviews and other types of studies)
The session continued with the groups work: each group was to assess a number of articles on
the topic under study (seasonal flu or rotavirus) using the appropriate check list according to the
study design. Each group presented their report on the data quality assessment exercise.
14
Take-away messages
from the session
Participants evaluation of the session
• The quality of the studies
found needs to be
assessed before they can
be included in the final list
of studies that will be used
to make the
recommendation
• For each criteria a
summary of the quality of
the evidence should be
provided in a table
(Evidence profile)
5.7 Session 7: Development of an evidence-based recommendation – 4) how to structure a recommendation note
The structure of a recommendation note was presented in plenary; this was followed by a
working group session where participants were tasked to review a recommendation prepared by
the Mozambique NITAG in light of the guidelines presented and give their feedback.
15
Take-away messages
from the session
Participants evaluation of the session
• The systematic and
transparent process
used to identify, collect
and assess the
evidence should be
reflected in the write-
up of the
recommendation note
• The recommendation
should include: the
context of the question;
general information on
the question; the
methodology used; the
analysis/judgement of
the evidence; the
proposed
recommendations or
options
0
5
10
15
20
25
30
35
40
45
Below
Average
Average Above
Average
Comprehensiveness 1 10 14
Interest 1 7 17
Clarity 1 12 12
1
12 121
7
17
1
10
14
Session 7: development of evidence-based
recommendation: 4) recommendation structure
5.8 Session 8: Communicating NITAG recommendations
The session included a presentation on principles of effective communication with policy-makers
and an exercise in the form of a role play. For the role play participants were provided with
recommendations on rotavirus and flu vaccination published by WHO and the CDC and asked to
summarize their key points to be communicated to a group supposedly made of a minister of
health, a minister of finance and a key immunisation stakeholder.
Take-away messages from the session
Key principles about communicating with policy-makers: the message should be clear with regards
to:
• The issue addressed by the recommendation ( referring to the PICO question and criteria
for the recommendation
16
• The action required from the audience
• The approaches suggested for undertaking the action
5.9 Session 9: Networking
The format and method of work during the workshop intended to also provide a platform for
initiating collaboration and exchanges between NITAGs. For example the topic on flu vaccine
was selected because 2 participating countries (Uganda and Kenya) are conducting active
surveillance activities on flu vaccine. The working groups were organized to allow the 2 countries
to share perspectives on the issue of flu vaccination and lay down the ground for possible future
collaboration and concerted action on flu recommendation.
In plenary, participants shared their perspectives on a possible mechanism for networking,
information sharing, and peer-to-peer support between NITAGs. Points arising from the
discussion included:
• The network should include all Africa NITAGs; the main goal would be to share
resources and get assistance from each other
• 2-3 days annual meetings could be organized to discuss common issues
• Online forum could be a useful mean to have discussion on technical issues and share
experience
• Role of partners such as SIVAC: assist in the coordination and resources mobilization
6 Conclusions
6.1 Next steps
Each participating country presented the activities planned after the workshop in order to keep
the momentum in starting their NITAGs operations.
17
6.2 Overall evaluation of the training
The graphic below shows the level of satisfaction of participants on some aspects of the training. It is to be noted that the majority expressed high satisfaction in getting new knowledge
++++ General comments from participants:
o Good training, informative and need to be done for others potential members so that a bigger group is trained
o Provide opportunity to get inputs from others countries and review what structures are already in place
++++ Participants suggestions to improve the training:
o Provide the materials to the participants before the workshop and include slides for literature search
++++ Gaps filled through the training:
o Understanding of NITAGs ToR, requirements for establishment of a functional NITAG
o Understanding of NITAGs members role
Very low Very high
18
o Understanding of the need for a strong secretariat
o Knowledge in methods for reviewing studies, PICO approach and formulating recommendation
6.3 Considerations for future trainings
The regional training was also organized with the intention to field test some NITAG capacity
building modules developed by the SIVAC Initiative. The workshop has helped identify revisions
needed in the content of the modules as well as in the roll-out of the training:
++++ Module on evidence quality assessment: will go into greater details on the GRADE methodology which will be taught as a sub-module ( of at least 4 hours)
++++ Analysis of participants evaluations have shown that groups work sessions were the most useful and interesting; future trainings will include more practical exercises; pre-reading materials will be distributed prior to the group work session
++++ Plans for distance support and follow-up in partnership with WHO EVIPNet will be clarified during the workshop
12/04/2014