Annual Review - Summary Sheet - WHO · 1 Annual Review - Summary Sheet This Summary Sheet captures the headlines on programme performance, agreed actions and learning over the course

1

Annual Review - Summary Sheet This Summary Sheet captures the headlines on programme performance, agreed actions and learning over the course of the review period. It should be attached to all subsequent reviews to build a complete picture of actions and learning throughout the life of the programme.

Summary of Programme Performance

Summary of progress and lessons learnt since last review The Medicines Transparency Alliance (MeTA) has been fully implemented in all MeTA countries, despite slow progress at the start of the programme. Information collected through MeTA and the MeTA dialogue has led to improved transparency, policy recommendations, practice recommendations and empowerment of citizens through informed awareness on and access to better quality and more affordable medicines. During Year 4 of the programme, there has been more concrete evidence of the value of MeTA. Countries have clearly recognised MeTA as a credible source to be consulted for analysis and recommendations, as a platform for dialogue, for supporting government, for strengthening the voice of Civil Society Organisations (CSOs) and prominent in creating more effective policy implementation through multiple stakeholder co-operation in the policy development process (see Medicines Transparency Alliance Global Meeting 2014 report:

www.who.int/medicines/areas/governance/tansparency_global_meet). The MeTA approach has changed relationships between stakeholders, who now work differently and together on policy intervention based on evidence-based knowledge that MeTA has collated and disseminated. Key factors behind the programme’s success are: dedicated individuals, who drive the MeTA platform in countries; support from governments and good communication between all stakeholders involved in access to medicines issues.

It is the poor who are penalised by compromised provision of medicines in the public sector and by the high cost or poor quality of medicines. MeTA Councils in all countries work to address these barriers to access to medicines for the poor. Key highlights in MeTA countries include: policy revisions including approval of the national medicines policy in Jordan and Kyrgyzstan and revisions more recently in Uganda with approval likely in the next month; practice recommendations on rational use of medicines in Ghana, Uganda, and the Philippines and widespread dissemination and mobilisation activities in Zambia. Changes to medicines’ policies and practices have led to improved market conditions and improved availability of medicines which has in turn increased confidence in public sector services and improved overall health service utilisation. Improved access to affordable medicines can in turn contribute to poverty reduction, by reducing potentially impoverishing household expenditures on medicines and by contributing to better health outcomes further averting the impoverishing impacts of ill-health.

Title: Medicines Transparency Alliance (META)

Programme Value: £6m Review Date: June 2015

Programme Code: 202779

Start Date: July 2011 End Date: December 2015

Year 2012 2013 2014 2015

Programme Score B A A A Risk Rating M M M M

http://www.who.int/medicines/areas/governance/tansparency_global_meet

2

Pilot countries are developing workplans to ensure the sustainability of the programme. The recognition of the need and success of this type of platform is motivating the MeTA countries to continue to implement MeTA principles. MeTA’s sustainability is dependent on a combination of the country context and the international and domestic donor landscape and is most likely where there are strong individuals driving the MeTA principles, alternative funding sources or the ability to raise additional funds, and/or strong political support both locally and globally. For example, MeTA is already semi-institutionalized in Jordan and Peru and the platform could continue with little external funding. Where governments have funding, MeTA may be able to offer services through subcontracts, such as is being considered in the Philippines and Kyrgyzstan. Of all the stakeholders, civil society funding is the most dependent on external donors. Risk to funding may result in exclusion of civil society. In order to address this, Health Action International (HAI) has worked with CSOs and secretariats in Uganda, Zambia, Philippines and Kyrgyzstan Republic and included them in funding proposals which will use elements of MeTA (e.g. multi-stakeholder engagement and councils) in other programmes with specific targets (Sexual & Reproductive Health (SRH) and non-communicable diseases (NCDs) in older people in Middle Income Countries (MICs)).

Unstable political systems and turnover amongst high level politicians and government staff is a common challenge in the MeTA countries, particularly in the face of long policy processes. The MeTA countries have mostly overcome this issue, by having strong champions in the secretariats or councils and dedicated stakeholders that continue the engagement with new governments. The independent external review scheduled for August 2015 will provide further data on the impact (or lack thereof) of unpredictable political support. As the programme predates the Equality Act, MeTA activities have not been specifically designed to address gender considerations, however, sexual and reproductive health (SRH) commodities form a significant proportion of the essential medicines list. Moreover, Health Action International (HAI) incorporate specific focus on SRH and MeTA Zambia and Uganda have included gender as a core issue in this phase, ensuring male involvement in sexual and reproductive health-focused commodities issues. In addition, MeTA councils try to ensure balanced representation of men and women in their overall membership. This is not an easy task which needs attention as six out of seven MeTA councils have a male chair, whilst Uganda rotates between two women and a man and Civil Society Coordinators are mostly men with the exception of Kyrgyzstan and Philippines. MeTA has two main types of beneficiaries. The primary beneficiaries are the patients who should have more access to better medicines as a result of MeTA interventions. However, impacts for patients are not likely to be felt until the end of the programme, as it can take some time to realise the effects of policy change. Some insight into this is expected in the evaluation which will be shared in August. Intermediary beneficiaries are the policy makers and MeTA councils who will have improved access to better data and transparency allowing more informed and evidence-based dialogue towards altering medicines policies and practices. All MeTA countries continue to work well with IMS and have benefitted from capacity building and support strengthening their role and importance. Further feedback will also be shared in the evaluation.

3

Summary of recommendations for the next period

ACTION by whom by when

Consolidate country programmes, measuring the impact indicators, collecting, analysing and sharing the lessons learnt.

WHO/HAI December 2015

International MeTA Secretariat (IMS) to continue to support countries as needed and with sustainability plans.

IMS ongoing

Taking into account that private sector engage more directly with governments, ensure continued inclusion of civil society organisations (CSO), which are involved in public health, to mitigate the threat of unbalanced policy and regulatory capture.

HAI ongoing

Finalise the country case studies as the programme nears completion in order to document achievements, lessons and best practices.

WHO December 2015

Continue to share practices and strategies that lead to MeTA outcomes (see output 6).

IMS ongoing

Incorporate recommendations from the evaluation into the remainder of phase 2 and any further phases (non-DFID funded).

DFID and WHO by October 2015

Consider using innovative approaches to measure response to different dissemination methods, to inform future phases and methodologies.

WHO ongoing

Measure changes in knowledge in future interventions, using learning from Ghana.

WHO by September 2015

Ensure united CSO organisations like ‘Coalition of NGO’s in Health’ (Ghana) have an independent sustainability plan similar to Coalition for Health Advocacy and Transparency (CHAT - Philippines).

HAI by September 2015

Utilise an adapted version of Ghana’s Knowledge, Attitude, Behaviour and Practice assessment tool in other countries to support strengthening of CSOs.

HAI by August 2015

Continue to address gender equality in structure of MeTA councils to ensure balanced representation and insight on access to medicines needs.

HAI ongoing

Assess the effectiveness of the working partnership through beneficiary feedback from country stakeholders.

HAI ongoing

DFID to ensure evaluation is on-track to ensure timely utilisation of findings to feed into project close-out and future phases funded by other partners. Use the evaluation findings to define lessons in supporting policy making processes.

DFID by August 2015

A. Introduction and Context (1 page)

Outline of the programme Support from UK The Department for International Development (DFID) funded the pilot phase (May 2008 – Dec 2010) of the MeTA in seven countries: Ghana, Jordan, Kyrgyzstan, Peru, Philippines Uganda and Zambia. DFID is now providing £6 million between 2011 and 2015 to MeTA Phase 2 (henceforth stated as MeTA) to support these countries in:

i. Bringing civil society, private sector and government together to collect, share and analyse robust data on medicine price, availability, quality and promotion

DevTracker Link to Business Case:

http://iati.DfiD.gov.uk/iati_documents/3718929.odt

DevTracker Link to Log frame:

http://iati.DfiD.gov.uk/iati_documents/4589676.xls

http://iati.dfid.gov.uk/iati_documents/3718929.odthttp://iati.dfid.gov.uk/iati_documents/4589676.xls

4

ii. Using evidence to inform better policies and to improve practice/systems in the pharmaceutical sector (both state and non-state) that will increase the access of the poor to affordable, quality life-saving drugs

iii. Innovating new tools, technologies and knowledge management systems to improve the quality and availability of pharmaceutical data and strengthening accountability for access to medicines, particularly to communities.

This support is provided through the International MeTA Secretariat (IMS), comprising the World Health Organisation (WHO) and Health Action International (HAI).

This is the programme’s final Annual Review before the Project Completion Review. Expected results Impact - MeTA will contribute to an increase in access to safe, effective and affordable essential medicines, particularly for the poor. It is anticipated that MeTA will increase access to medicines – measured by the average availability of selected essential medicines in public

and private facilities at the baseline or lower real prices (consistent with MDG 8, target 17) [i]

- by 10%, and result in cost-savings of 5% - 10% in public sector expenditure. This is equivalent to £18m - £36m, based on estimated total per annum public expenditure across the seven pilot MeTA countries. Outcome - MeTA will contribute to the development of effective access to medicines policies that are informed and monitored by robust and timely information and evidence on the price, availability, quality and/or promotion of medicines. The outputs of MeTA have been revised since the last Annual Review to be:

Functioning multi-stakeholder groups exist and have national government support;

Capacity built in countries to collect and analyse data, using innovative methods as required;

Transparency of the pharmaceutical sector strengthened which leads to greater accountability;

Civil Society Organisation capacity to support improvements in transparency and accountability of the pharmaceutical sector strengthened;

Policy makers in MeTA countries involved in multi-stakeholder policy dialogue to develop new or review access to medicines policies;

Engagement with MeTA increases and

External evaluation of MeTA outcomes against baselines established during the pilot phase.

Evidence from the pilot and programme monitoring shows that when government, private sector and civil society organisations work together, there is more and better information available to inform policies on medicines. Better medicines policies and practices lead to greater access to medicines. Continued UK commitment to the principles of transparency and accountability in the pharmaceutical sector will be used to leverage commitments by other donors.

5

B: PERFORMANCE AND CONCLUSIONS (1-2 pages) Annual outcome assessment The project has surpassed its target outcomes to date by collecting information on numerous sources, publishing, disseminating widely and by making a wide range of recommendations for both policy and practice. Changes to medicines’ policies and practices have led to improved market conditions and improved availability of better quality and more affordable medicines which has in turn increased confidence in public sector services and improved overall health service utilisation and health outcomes. In some cases, policy and practice recommendations have been taken up already. Capacity has been built in countries and citizens empowered through transparency and accountability efforts which have established baselines for more affordable and more accessible quality medicines. The extension that was granted by DFID until the end of December 2015 will allow countries to finalize some of the ongoing activities which were delayed at the start of the project due to the lag between Phase 1 and 2. In addition to the logframe outcomes, we expect the evaluation to provide new knowledge on promoting transparency within the pharmaceutical sector and beyond, including identification of the combination of factors influencing the achievement of the outcome. Lessons so far, show that it takes time for such a model to work, as the system and culture needs to change, with parallel increases in trust amongst stakeholders, e.g. campaigns on stock-outs have changed the approach of governments, with civil society changing their methodology from activism to advocacy, opening up lines of communication with governments. Progress against outcome indicator targets is noted below:

Outcome: Medicines procurement, pricing and other policies or practices are changed on the basis of a Multi-stakeholder review of robust evidence

INDICATOR

TARGET

RESULTS REPORTED

1: Each MeTA Council demonstrably uses robust pharmaceutical sector data to monitor and review access to medicines.

Comprehensive

sources of key

medicines issues data

reviewed by MeTA

council and findings of

review minuted.

Every country has collected at least one

comprehensive source of data on key

medicines issues and discussed this in

one or several council meetings. The

discussions have led to agreement on

interventions as evidenced in the

workplans and recommendations for

policy.

2: Each MeTA council demonstrates

commitment to principles of

transparency through collection,

publication and dissemination of robust

information on key pharmaceutical

issues.

Data for outcome

indicator 1 target

verified independently

and published in

sources available to

key stakeholders.

All data is verified by WHO and

published in sources appropriate to the

particular context.

3: Policy makers in MeTA countries use

multi-stakeholder policy dialogue

recommendations to develop new or

review access to medicines policies or

practices.

Review from outcome

1 used to identify new

policy requirements

and/or amendments or

practices.

See output 5 on how MeTA-generated

data/information is being used to inform

policy. It is also being used to improve

medicine practice and empower citizens.

4: MeTA countries have new or revised

policies or practice on access to

medicines that are demonstrably

informed by robust pharmaceutical

sector information and policy research

evidence.

All countries able to

demonstrate policy

and practice changes

relating to core

essential medicines.

All countries in the programme have

demonstrated that MeTA is involved in

relevant policy dialogue related to

essential medicines.

6

Overall outcome score and description The table above demonstrates how the outputs contribute to the outcomes. For each MeTA country the type of data collected and analysed, the means of disseminating and holding policy dialogue is summarized and the recommendation or corresponding policy or practice change is indicated. One example from each country is presented here: ● MeTA Jordan carried out a desk review on the availability of national pharmaceutical

information and its disclosure. The review, analysis and discussion led to recommendations for a disclosure policy which was approved and enacted, resulting in a long list of information being published on the Regulatory Authority website.

● MeTA Kyrgyzstan carried out a desk review of all of the information available on the pharmaceutical sector and following analysis and discussion, made recommendations for a new National Medicines Policy which was then widely disseminated and discussed amongst stakeholders and finally approved by government.

● Following pressure on the government to reduce the VAT on medicine, MeTA Ghana carried out a review and made recommendations for a list of medicines to be exempt from VAT. This was accepted by government and MeTA will undertake another study to monitor the effect on price and availability.

● MeTA Peru reviewed and discussed the implications of the high price of Atazanavir for public health and made recommendations for compulsory licensing.

● MeTA Philippines carried out a mapping of benefits programmes and the consequent dissemination and discussion of results led to recommendations for further monitoring at the community level to establish accountability for government funds.

● MeTA Uganda collected data on medicines quality which led to further multi-stakeholder discussions and a series of recommendations such as the development of an appropriate reporting, monitoring and communication system of poor quality medical products.

● MeTA Zambia was involved in the review and recommendation of Accredited Drug Dispensing Outlets which was approved by the government and is in the implementation phase.

● More specifically, related to key medicines issues, MeTA Philippines made progress on raising awareness of counterfeit medicines as well as engaging stakeholders and regulatory authorities. MeTA is now studying barriers to reporting sub-standard and counterfeit medicines in parallel to changing the law around sub-standards. Likewise, MeTA Uganda is working to ensure that the national regulatory authority acts quickly to sub-standard/counterfeit medicines issues.

Key lessons ● The MeTA process is not always linear. In other words, the collection, analysis and

dissemination of information sometimes lead to the need or the request for more information. Dissemination and dialogue often occur simultaneously. Policy or practice recommendation is not an end in itself and MeTA has often been involved in developing implementation plans and sometimes taking the lead on implementation in order to initiate change.

● Improving transparency does not always involve data collection. Sometimes the data is already collected by the government or by other sources and there is additional value in making this information available. It is important to look for existing information before collecting new data. Also the absence of data is information in itself that can be used for advocacy.

● Strategies to address the challenges include developing capacity for formulating policy, bringing in external consultants to contribute, cultivating a shared interest, creating awareness about the public good, having a good strategic plan and strong leadership in the multi-stakeholder platform.

7

Key actions The plans going forward until the end of 2015 include finalizing the ongoing projects in countries, measuring the impact indicators, collecting, analysing and sharing the lessons learnt and supporting countries with sustainability plans. Analysing the lessons learnt will contribute to additional key lessons as we go forward to the end of the project. These lessons should inform future programmes and research agendas that seek to address transparency, accountability and multi-stakeholder approaches and will guide countries and organizations that support countries. Has the logframe been updated since the last review? Yes, a number of amendments are proposed in accordance with the recommendations from the last annual review which stated: ‘IMS to review indicators, milestones and targets to reflect rational use of medicines, other improvements in efficiencies and improvements in practice in addition to policy (by Q3 2014)’. These adjustments principally serve to capture the importance of influencing beyond policy frameworks (to include guidelines, practices, etc.) and beyond procurement and pricing issues alone.

8

C: DETAILED OUTPUT SCORING (1 page per output) Output Title Output 1: Functioning multi-stakeholder groups exist and have

national government support Output number per LF

1 Output Score A

Risk: Medium Impact weighting (%): 20%

Risk revised since last AR?

No

Impact weighting % revised since last AR? No

Indicator(s) Milestones Progress 1.1: Multi-stakeholder groups (MeTA Councils) exist (government, private sector and civil society are represented) produce an annual workplan approved by the IMS.

All countries (7) submit a workplan, containing agreed objectives, timelines and those responsible.

All countries have workplans for year 4 or updated workplans that have been reviewed and approved by the IMS.

1.2: Number of MeTA Councils supported from country level sources (domestic or donor; in cash or in kind. e.g. Office space, goods, services) with great value placed on support leading to integration within existing systems and/or sustainability.

Substantiate country level support as donations or in-kind or demonstrate a tangible increase in previous year support.

All countries can show local support is occurring, but to varying levels (see Table 1). Country support (by Ministries of Health) is strongly observable in three countries. 50% decrease in donations compared to 2014, which had significantly high-levels of donations.

Key Points ● All seven countries have begun delivery against workplans. ● Each country has functioning multi-stakeholder groups. Workplan and performance

management overall has improved. ● All countries have been working on sustainability plans to secure MeTA in-country after

the DFID’s funding ends. ● After four years, MeTA has become embedded in the medicines policy landscape in all

seven countries, actively contributing to policy dialogue. ● Donations in-kind and commitment to MeTA support from various sources has

contributed to operationalisation in lieu of budget. An example of this has been free office space in Zambia, Jordan and Peru likely to be sustainable beyond the end of the project.

● The MeTA Secretariat in each country has reported programmatic donations, and donations in-kind based on market values. Total contributions for 2015 are reported as £97,014 representing an overall decrease of more than 50 % over 2014 (see table below).

9

Table 1: Donations-in-Kind

2015 2014 2013 2014/2015

GBP GBP GBP +/-

Ghana 7,383 6,254 1,690 + 18%

Jordan* 41,340 67,450 38,412 - 39%

Kyrgyzstan 9,632 9,893 9,787 - 3%

Peru 6,788 9,937 8,115 - 32%

Philippines 4,755 4,076 2,770 + 17%

Uganda** 26,658 98,413 76,041 - 73%

Zambia 15,458 13,388 5,763 + 15%

Total 97,014 209,411 142,578

*2014 Canadian Award **2014 funds WBI Overall, MeTA secretariats have been careful and accurate in the reporting of donations-in-kind, so IMS is confident that the figures presented are a true reflection of support. That said, Table 1 (above) appears to show a significant deterioration in support, but needs further explanation. Jordan (-39%) has in previous years received a supplementary award in the sum of USD 50,000 (£31,000; World Bank Institute (WBI)) which was not awarded this year and was beyond the control of the secretariat. Despite this, MeTA Jordan has done extremely well to maintain in-kind support at an increased level to 2013 levels. Peru’s reduction of 32% is due to reduction in operating costs where meetings, supported by third parties have been shorter and provided fewer meals and reduced travel reimbursement. Uganda had a reduction of 73% because the World Bank Institute donation of USD120,000 went directly to Uganda National Health Consumer’s Organisation (UNHCO) to follow up on satisfaction with the previous years’ interventions, so is not directly attributable to MeTA funding. Key Lesson ● MeTA is most likely to be sustainable beyond Year 4 in countries whose governments

have worked to integrate MeTA’s work into their own processes and begun to adapt policies in line with MeTA recommendations (Kyrgyzstan, Jordan, and Peru). MeTA Zambia and MeTA Uganda are now part of a project funded by the Dutch Ministry of Foreign Affairs for 2016. In both Zambia and Uganda, where the secretariat is housed by civil society, MeTA is recognised as the key medicines policy dialogue space. As such, the secretariat and civil society have become financially sustainable and with sufficient budget to convene multi-stakeholder meetings (council meetings and annual forums).

Summary of responses to issues raised in previous annual reviews (where relevant) ● Country sustainability plans were shared with the IMS in January 2015, with all being

feasible with the exception of Ghana.

● Countries recognize the value of MeTA and most are keen to continue to apply the core principles of MeTA in one way or another.

● Progress has been made against all the relevant recommendations from the Annual Review 2014, resulting in improved programme and performance management, sustained support from in-country sources (especially government) across all countries

10

and realistic workplans and sustainability plans to ensure MeTA councils continue to influence medicines policies.

Recommendations 1. Continue to support countries as requested by the countries and as needed, especially in

Ghana. (IMS – ongoing) 2. Ensure balanced stakeholder collaboration with continued inclusion of civil society

organisations in all medicine’ policy and practice discussions (HAI - ongoing).

11

Output Title Output 2: Capacity built in countries to collect and analyse data, using innovative methods as required

Output number per LF

2 Output Score A+

Risk: Low Impact weighting (%): 20%


Y (decreased)

Impact weighting % revised since last AR? N

Indicator(s) Milestones Progress 2.1 Surveys/or data collation (including monitoring) conducted with local team participation and analysis leads to intervention or policy recommendation relevant to outcome 1

Each relevant country makes at least one intervention or policy recommendations based on analysis of data.

All relevant countries1 (six in total) have surpassed the milestone

2.2 New data collection tool, survey, indicators and/or methodology conducted with local team participation and analysis leads to intervention or policy recommendation relevant to outcome 1

Each relevant country produces analytical report using new tools or surveys that is deemed satisfactory by WHO

All 5 relevant countries have produced reports based on novel approaches, tools or methodologies and surpassed the milestone by making policy or practice recommendations based on the data.

Key Points Country capacity has been built through the WHO feedback process, the provision of

technical workshops, participation in courses, and participation in training-of-the-trainer exercises. The quality of this training is evident in the positive changes to medicines, policies and practices.

Ghana, Jordan, Kyrgyzstan and Uganda have all made significant policy or practice recommendations based on MeTA research, analysis and policy dialogue that have already been taken up by the government. Other countries have made recommendations but it is yet to be seen if they will be taken up.

The broadest recommendations were the revisions in National Medicines Policy by Jordan, Kyrgyzstan and Uganda. MeTA Ghana has also contributed to the national medicines policy revision process in their country.

The recommendations in the six countries cover a vast area of the pharmaceutical sector such as VAT exemption for medicines, National Essential Medicines list, disclosure policy, legal framework, medicines promotion, monitoring of indicators and procurement issues.

The use of new tools or methodologies new to the country has taken place in all relevant countries

1 MeTA Zambia is not considered as being ‘relevant’ to 2.1 or 2.2 as it has pursued a different

approach, working more on public and CSO engagement than on data generation. MeTA Zambia has experienced difficulty in finding capacity to carry out data collection despite the development of several study protocols. Options to bring in international expertise to conduct national studies are under consideration. Through the different approach, MeTA Zambia has still been able to make progress on other areas of the project including dissemination of information, policy dialogue and policy recommendation. MeTA Jordan was deemed not relevant to 2.2 after they determined that new technology for monitoring of medicines was inappropriate for the country.

12

Summary of responses to issues raised in previous annual reviews (where relevant) Case study overviews have been prepared for Jordan, Kyrgyzstan, and the Philippines

(in layout phase). Other country overviews are due in July and August. One challenge to completing the overviews was that countries are still implementing at the time of developing the overviews.

IMS has reviewed the logframe to reflect rational use of medicines and improvements in medicines’ policies and practice.

The online platform has been useful for sharing documents with WHO staff, particularly unpublished documents. To mitigate challenges of confidentiality, reports have been shared directly with interested parties upon agreement with the originator. Face-to-face meetings have proven most useful for lesson sharing such as at the MeTA Global Meeting in December in Geneva.

IMS discussed evaluation of the value of new data collection technologies with DFID in January 2015. Some challenges were foreseen with comparing vastly different methodologies such as the price observatory in Peru and traditional price data collection in Uganda. The objective of each data collection type and the evaluation was not pursued due to this.

MeTA will include how opening up data has affected behaviour change and where good practice can be identified in relation to data influencing relationships in their August evaluation.

Recommendations 1. Continue work on a second and final version of the country case studies as the

programme nears completion in order to have a more in-depth and complete story (WHO by December 2015).

2. Continue to share practices and strategies that lead to MeTA outcomes (see output 6) (IMS - ongoing).

3. Incorporate recommendations from the evaluation into the remainder of phase 2 and any further phases (non-DFID funded) (DFID and WHO by October 2015).

13

Output Title Output 3: Transparency of the pharmaceutical sector strengthened which leads to greater accountability


3 Output Score A



N Impact weighting % revised since last AR? N

Key Points

MeTA countries are disseminating information using various formats as appropriate to the country contexts and for the target audience. Jordan and Peru have used government websites to publish information with the purpose of improving transparency. Both countries have successfully established policies and/or legislation requiring disclosure of information.

Kyrgyzstan has held numerous round table discussions and specific stakeholder discussions to engage, inform and get support.

The Philippines has held roundtable discussions and fora to inform, engage and involve stakeholders in dialogue and recommendations. The Philippines and Zambia have used social media such as Twitter, Facebook and YouTube to reach wider audiences.

Uganda has used hard copy distribution and soft copy distribution of information and inclusion in newsletters for dissemination. Ghana and Kyrgyzstan have up to date and useful websites.

In addition to other methods, Zambia has considered radio and TV to reach wider audiences.

Most countries have had media reports on issues related to MeTA work. Examples of MeTA related media reports include: exposure of price asymmetries between public and private institutions in Peru (as a consequence of data transparency through the observatory), coverage of MeTA fora (in almost all countries), coverage of the national medicine policy issues in Kyrgyzstan. While media coverage should lead to improved awareness of the issues and wider ability to hold the government accountable, the impact has not been systematically measured through the MeTA project.

Countries have reported that knowledge and ability to hold governments to account has improved due to increased transparency. The measurement of improvements in knowledge has mostly been outside the scope of the project. There would be some possibilities for future work in this area in countries that have assessed baseline knowledge prior to an intervention. Ghana is the only country to have done this through the study which was conducted to assess knowledge in the community. The purpose of the study was to design a capacity building programme. A follow up study could be conducted following the intervention to assess improvements although this may not be possible in the time-frame remaining. There are anecdotal reports of improvements in ability to hold governments to account such as the case of exposure to price asymmetries in the case of Peru as noted above.

Indicator(s) Milestones Progress 3.1 Findings (from data collection or reports from output 2, reports from the pilot, and other information) disseminated using appropriate messages and methods to other stakeholders.

Key information, analytical reports and findings disseminated to stakeholders and the public as relevant and informs advocacy.

All countries have made progress with disseminating information using a wide range of methods such as press briefings, newsletters, web sites, social media, round table discussions, radio, workshops and briefings.

14

Summary of responses to issues raised in previous annual reviews (where relevant) ● MeTA’s dissemination of information to stakeholders has had varying levels of success.

In Uganda, this is still relatively nascent; however, the stock-out campaign is clearly linked to greater commitment from government. In many of these countries, USAID has been working to strengthen the monitoring system to increase access to data and it would be useful to understand how MeTA links with other such initiatives.

Recommendations 1. Consider using innovative approaches to measure response to different dissemination

methods, to inform future phases and methodologies (WHO - ongoing). 2. Measure changes in knowledge in future interventions, using learning from Ghana

(WHO - by September 2015).

15

Output Title Output 4: Civil Society Organization capacity to support improvements in transparency and accountability of the pharmaceutical sector strengthened


4 Output Score A



N


Key Points ● All MeTA country civil society has grown in capacity and inclusion. This has improved

technical expertise on access to medicines. ● All seven countries achieved their milestones. ● Specific milestones are established by country. These include: representatives of CSOs

are trained to observe the public drug procurement; Information and Education Campaign (IEC) targeting communities and the grassroots; CSO implement joint advocacy to strengthen pharmaceutical sector; capacity building for CSOs improved and CSO representation on decision making structure increased at all levels.

● The assessment of how the messages, used in an antibiotic resistance campaign with special focus on rural populations, influence people found that target groups showed an understanding of rational medicines use and AMR threats.

● Capacity-building activities on undertaking community monitoring, social accountability and community empowerment were scaled up beyond 3 MeTA funded districts.

● Again, Kyrgyzstan and the Philippines exceeded expectations. This has been facilitated by effective stakeholder involvement in Kyrgyzstan and a dynamic chair (ex-Minister of Health) in the Philippines.

● MeTA Ghana was delayed due to inter alia immense and almost overwhelming mistrust of civil society but they are now completing the deliverable.

Summary of responses to issues raised in previous annual reviews (where relevant)

MeTA Philippines completed the survey tools, questionnaires, briefs and forms for the pilot survey phase of Medicines Watch and Philhealth Watch. Meta CSOs (CHAT) organized training workshops for survey teams on the use of the monitoring tools for Medicines Watch and Philhealth Watch in April 2015. The pilot survey is scheduled to take place over a 6-week period to end in mid-August 2015. - MeTA organized two (2) sessions under the CHAT Workshop Series:

1. Trans-Pacific Partnership (TPP) Agreement, intellectual property rights (IPRs), access to medicines and public health (11 September 2014). 2. Theory of Change and community monitoring (17 October 2014).

The CHAT Theory of Change Workshop on 16-17 March 2015 was conducted by HAI, during which a Theory of Change model was developed for the CSO alliance. A planning

Indicator(s) Milestones Progress 4.1 Specific indicators are

established by country. May include: understanding of issues, ability to collect/analyse data, and/or dissemination of information

Specific to each country. Measures include: engagement of civil society in the MeTA council, engagement in policy dialogue and implementation of MeTA objectives

Civil society capacity in all countries has advanced during the last 12 months (see progress against country specific milestones below logframe)

Medicines Watch assesses medicines availability. Philhealth Watch monitors the government provision of the minimum medicines package (universal health package). Both programmes provide a watchdog on the operation of the Governments.

16

meeting was organised to discuss the action plans to ensure CHAT sustainability on 19 March 2015.

At present, given the funding situation and the uncertainty of the future of the programme, it was decided in consultation with key stakeholders that the added value of a global CSO meeting/workshop was unclear. IMS are still considering an end of project CSO meeting, but is in dialogue with implementing partners about the content, which may concern sustainability of the civil society voice in on-going multi-stakeholder dialogues.

Six of the seven countries (Zambia, Uganda, Ghana, Kyrgyzstan, Philippines, Jordan) now have formalised civil society coalitions, with an elected coordinator.

The Knowledge, Attitudes, Practice and Behaviours (KAPB) tool has been finalised, piloted and rolled out in Ghana and has fed into civil society capacity building in that country. However, the tool is still very general so with the aid of the evaluation results on civil society engagement, it is hoped a refined and universal tool can still be developed.

Recommendations 1. Ensure united CSO organisations like ‘Coalition of NGO’s in Health’ (Ghana) have an

independent sustainability plan similar to CHAT (Philippines). (HAI - by September 2015). 2. Utilise Knowledge, Attitudes, Practice and Behaviours (KAPB) tool in other countries to

support strengthening of CSOs. (HAI - by August 2015).

17

Output Title Output 5: Policy makers in MeTA countries engage in multi-stakeholder dialogue to develop new or review access to medicines policies


5 Output Score A+



N


Indicator(s) Milestones Progress Output 5: Policy makers in MeTA countries engage in multi-stakeholder dialogue to develop new or review access to medicines policies

Evidence-based policy recommendations discussed with relevant policy makers and stakeholders

All countries have met the Year 4 milestone and most have had numerous multi-stakeholder discussions on various topics.

Key Points

The MeTA platform has proven to be a good platform for multi-stakeholder dialogue as evidenced by the range and depth of discussion that has taken place with policy makers. MeTA Councils, involving policy makers, are meeting every one to four months and some have set up multi-stakeholder subcommittees to work on specific issues. Some countries have taken the multi-stakeholder dialogue beyond the Council meetings by holding larger fora or targeted round-table type discussions.

In a number of cases, the policy dialogue has led to recommendations that have already been acted upon. Some of the broadest results are the revised national medicines policies in Jordan, Kyrgyzstan (both approved) and more recently Uganda. Other policy recommendations that have been made and approved include the medicines VAT exemption list in Ghana, the disclosure policy and drugs and therapeutics committee (DTCs) policy in Jordan and the Accredited Drug Dispensing Outlets (ADDOs) in Zambia. The disclosure policy in Jordan has already resulted in the publishing of a wide range of information on the Jordan Food and Drug Administration website. A number of other policy recommendations have been made such as recommendations on price, transparency and DTC for the national medicines policy in Ghana, an essential medicines list in Jordan, compulsory licensing for a high priced HIV medicine in Peru, and FDA fee restructuring in the Philippines.

Legislation that has been influenced by MeTA recommendations in countries includes the cheaper medicines act in the Philippines, law creating Health Shops in Zambia, a draft law on pharmaceutical legislation in Kyrgyzstan and the law on reporting of medicine prices in Peru.

A number of practice recommendations have been made and already acted upon, including recommendations to improve rational use of medicines (Ghana, Philippines, Uganda and Kyrgyzstan); to conduct community monitoring (Philippines); capacity building of hospital and therapeutic committees in Uganda).

The MeTA Ghana Council representative is participating in the review of the National Medicines Policy (NMP). With the support of the IMS, this has directly resulted in the addition of the proposed section on transparency/governance into the NMP.


To address the pace at which MeTA Zambia collects and analyses data IMS considered external support for data collection. However, Zambia had a different approach as noted in Output 2 and focusses on public and CSO activities to address this.

18

In MeTA Ghana a number of activities have been carried out this year including a DTC baseline assessment, medicines VAT exemption review, capacity building for a transparency assessment and input into the national medicines policy, bringing Ghana up to speed with project milestones.

Recommendations for 2015 1. Use the evaluation findings to define lessons in supporting policy making processes

(DFID - August 2015).

19

Output Title Output 6: Engagement with MeTA increases


6 Output Score A



N

Impact weighting % revised since last AR?

N

Indicator(s) Milestones Progress 6.1 Global awareness and support for MeTA increases.

IMS meetings held with international donors, partners and stakeholders where support needs are discussed.

Activities have been carried out by WHO and HAI including raising of awareness, funding proposals and discussions on support needs.

6.2 Number of collaborations with global or national stakeholders* with meaningful input to MeTA outcomes and MeTA principles. *(e.g. academic institutions, NGOs, Bilateral and International organisations)

Collaborations maintained

Activities have been carried out by both WHO and HAI and at the country level to improve collaborations.

Key Points ● MeTA Zambia and MeTA Uganda will be involved in a new project (Dutch Ministry of

Foreign Affairs) in 2016, along with Kenya, that was previously not a MeTA country. The focus of the intervention will be sexual and reproductive health medicines and devices (commodities), and targets capacity building of civil society and advocacy. However, the core principles of MeTA remain with civil society driven MeTA secretariats and multi-stakeholder discussion that builds on existing capacities. Partners in the new programme are Amref Flying Doctors, PATH, African Centre for Global Health and Social Transformation (Achest) and Wemos. This model will be expanded in 2017 to include Tanzania, Ethiopia, South Sudan and Malawi.

● HAI has further developed an application with King’s College London to the Economic and Social Research Council, for support to Kyrgyzstan Republic and the Philippines.

● WHO Regional Advisor participated in a meeting in Mongolia to discuss a proposal for developing a multi-stakeholder platform modelled on MeTA model.

● MeTA Ghana, Zambia, Jordan, the Philippines and Kyrgyzstan have presented their work to WHO and/or the EU. MeTA will deliver the key note speech at regional meetings on Access to Medicines.

● MeTA Jordan has applied for the Open Government Awards 2015. The theme this year is Improving Public Services through Open Government. Twenty-one applications were considered in Jordan and MeTA reached the shortlist of three candidates but did not win the nomination. The selected candidate will be put forward for nomination.

● University of Toronto will enter into official collaborations with WHO to support efforts in improving governance including key MeTA principles of transparency and accountability.

● WHO has begun the process of entering into formal relations with Transparency International to collaborate on the development of tools and guidance and support at the country level. WHO has been working with Management Sciences for Health (MSH) and other partners to revise the Transparency Assessment tool.

20


Evaluation to provide information on the levels of awareness concerning MeTA and possible implications of awareness-raising efforts. (August 2015)

Case study brochures have been produced by IMS who is helping countries to create tailored factsheets and brochures which will increase interest at national and international levels. This has been done with country inputs and will collectively produce a global picture of the MeTA project, focussing on success stories. It is hoped that the brochure will enable in-country funding opportunities.

Recommendations 1. Utilise evaluation findings for think-pieces on lessons learned and way forward for Phase

3, resource mobilisation and engaging with partners (WHO - by December 2015).

21

Output Title Output 7: External evaluation of MeTA outcomes against baselines established during the pilot phase (External evaluation to test a theoretical model as applied to MeTA)


7 Output Score B



N

Impact weighting % revised since last AR? Y/N

Indicator(s) Milestones Progress 7.1: External evaluation carried out by DFID to inform future programmes.

Evaluation carried out; insights absorbed among relevant actors and results made public

Expected August 2015

Key Points An external evaluation managed by DFID is still ongoing. Phase II findings are being documented by consultants. Findings so far include:

Multi-stakeholder approach is effective and takes time, but is better than short-term consultancies. The approach builds trust, ownership, shifts perceptions and ensures credibility. Its success is dependent on a few committed focal people, who are available to provide support over the long-term and provide consistency in spearheading change.

Political will is important, e.g. in Kyrgyzstan the deputy minister for health is providing backing, resulting in a redraft of the medicines policy.

Decision making has been influenced by effective advocacy by CSOs who in some countries are having productive dialogue with government and are no longer considered antagonistic.

Summary of responses to issues raised in previous annual reviews (where relevant) Recommendations for the evaluation have been included under the preceding outputs and can be summarised as follows: Enable more cross-country learning with respect to participation in an international

programme such as MeTA to enable some countries to learn how others co-operate more effectively with the IMS without sacrificing their autonomy (through the global meeting, 2014 and through the evaluation process itself).

IMS (and evaluation team, if possible) to identify how MeTA’s approach to opening up data has affected behaviour change and where good practice can be identified in terms of influencing relationships.

Evaluation to take a more systematic look at how MeTA’s dissemination of information to stakeholders leads to improved knowledge of those stakeholders to voice concerns and raise questions and thereby increase accountability (by end of evaluation, August 2015).

Evaluation to provide information on the levels of awareness about MeTA and how this has influenced behaviour.

Recommendations 1. DFID to ensure evaluation is on-track to ensure timely utilisation of findings to feed into

project close-out and future phases funded by other partners (DFID by August 2015).

22

D: VALUE FOR MONEY & FINANCIAL PERFORMANCE (1 page) Key cost drivers and performance The budget has been well managed to cover the core running costs of the International MeTA Secretariat. The key costs are in support of all seven MeTA countries and directed towards staffing (salaries) and travel (to attend key meetings and country visits as per specific need) and all primarily to support in-country technical work. Additional funding is being provided for capacity building and for some areas of national workplan activity related to capacity building.

Where possible, efforts to drive economy and efficiency in procurement are made for all costs (salaries and travel) and all cost drivers (demand for technical support, capacity building and data collection) by the Secretariat: ● Staffing costs have been incurred by HAI. HAI benchmarks its staff salaries against the

CAO Welzijn, (the Dutch Government Labour Code Agreement) which ensures comparable salaries with other NGOs and ensures they are able to attract the right skills at market rates. No pay awards have been made for the past four years under CAO Welzijn.

● WHO also incurs staff costs, both at HQ and in each of the seven MeTA countries. MeTA contributes to salaries (cost-sharing) of existing positions and at country level recruits national experts for cost efficiency purposes. Salaries are in line with the UN scale, like any other UN agency and within this scale, positions and grades correspond to well established competencies and tasks to be carried out. Within the MeTA project no overtime can be charged.

● Travel costs are incurred by WHO and HAI according to each organisation’s travel policy. WHO works with a travel agency that has special rates with airlines for UN travel and chooses the lowest cost fare for each flight. For WHO business class travel may only be used for travel exceeding nine hours. HAI travel economy for all flights. In all cases flights are secured well in advance of travel date in an effort to reduce cost and HAI also purchases discounted ‘NGO’ seats, available from some carriers, when on travel duty for a registered NGO.

● Efforts are also made to ensure civil society and/or MeTA Councils benefit, where possible, from shared office space, meeting rooms and equipment as well as WHO (in-country) logistics and administrative support. This is highlighted elsewhere in the annual review.

● Data collection costs are reduced by first undertaking a review of information needs and making use of available data when appropriate and by using cost-effective methodologies when feasible. Where data collection is contracted out, a competitive tendering process is used.

VfM performance compared to the original VfM proposition in the business case

A number of impact and outcome indicators will contribute to the value for money analysis at project completion. Additional measures to track VfM at the Annual review include:

Economy: Working with WHO and governments in-country means that in-kind contributions can be obtained in lieu of financial support. An example of this is provision of free office space in Zambia, Jordan and Peru. Moreover the MeTA Secretariat reported donations and donations in-kind based on market values. Total contributions are reported as GBP 97,014 for 2015 representing an overall decrease of more than 50% over 2014. While three countries (Ghana, Philippines and Zambia) managed to gain in donations Jordan, Peru and Uganda reported a significant decrease in contributions. This can however be attributed to supplementary once-off donations from World Bank Institute the previous year in Jordan and Peru and improved programme efficiencies (such as improved meeting management and

23

decreases in costs of sponsored travel and meals) in Uganda rather than a decrease in fundraising.

Efficiency: WHO run the Good Governance for Medicines (GGM) programme and others which seek to improve transparency in the pharmaceutical sector in addition to improving aspects of governance such as participation, ethics and management. MeTA is able to benefit from processes already embedded within WHO to increase its own efficiency. WHO (Essential Medicines and Health Products Department) provides technical assistance to national MeTA Councils through in-country staff, regional and HQ staff and the WHO network of international experts (including Collaborating Centres) allowing the programme to benefit from high-level technical capacity which is less expensive than using external consultants. In addition to efficient delivery of technical support, one of the most notable changes in addressing cost-efficiency of the programme, since the pilot, has been the more widespread use of local consultants rather than more costly international consultants. In particular local consultants have been used for data collection, analysis and dissemination of results to influence national policy (Output 2). While the main objective of this was to improve country capacity for collecting and analysing data this also led to cost savings. The role of WHO in providing technical advice on protocols and reports has been beneficial in improving the quality of the outputs (e.g. the National Medicines Policy drafts prepared in Jordan, Kyrgyzstan and Uganda were reviewed by the WHO Country Office, Regional Office and/or Headquarters. It should be noted that the use of country expertise is not always possible due to a small pool of consultants with much demand and limited availability. In one case in Zambia commencement of one study was delayed due to the lack of time available by the qualified consultant. In this case it was necessary to seek the support of a more costly international consultant.

Effectiveness: MeTA has been effective in establishing a ‘neutral’ multi-stakeholder platform for dialogue and policy review. This has stimulated relevant and beneficial medicines policy change in countries. In addition to this, MeTA interventions have enabled a number of recommendations on price, transparency, essential medicines and other specific country priorities. This is detailed under output 5. On the whole countries are demonstrating sustained local support with integration by governments in Kyrgyzstan, Jordan and Peru and new funding from the Dutch Ministry of Foreign Affairs in Zambia and Uganda. This makes MeTA financially and operationally sustainable in the majority of countries albeit DFID funding is to be discontinued.

As noted in previous Annual Reviews, MeTA also has a number of important potential indirect benefits such as reduced corruption, improved market conditions for reputable businesses and better quality health service provision. The independent evaluation currently underway will examine these wider benefits.

Additional, and potentially more significant, savings are anticipated in private expenditure. The next period (until end 2015) will incorporate efforts to better quantify private expenditure and potential efficiency savings.

Assessment of whether the programme continues to represent value for money

Overall, the programme is delivering what is expected, seeks to maintain low costs and is achieving policy or practice change and/or higher levels of civil society mobilisation across the seven countries.

24

Quality of financial management

HAI and WHO jointly comprise the International MeTA Secretariat (IMS) and are jointly responsible for providing support to the implementing partners. WHO procures technical assistance on behalf of DFID under the agreed Memorandum of Understanding (MoU). HAI monitors WHO spend in all countries through an IT system which is regularly reviewed to ensure countries remain on track, both financially and technically.

HAI is audited annually and the audit report is submitted to DFID as well as available online (http://haiweb.org/). Some countries have been asked to submit audited accounts where spend has been more rapid or higher than anticipated. This mechanism has maintained spend accurately. An audit will be carried out at the end of the project as per the MoU.

The approved project budget of MeTA Phase 2 is £6 million and to date:

£2,162,200 has been disbursed to HAI (2011 – 2014) with £550,000 in 2014;

£3,548,448 to WHO (2011 – 2015) with £915,312 in 2014 and £344,856 in 2015;

£140,136 has been disbursed for evaluations with £84,678 in 2014 and £55,458 in 2015.

All financial reporting has been timely and accurate.

Date of last narrative financial report 28/04/2015 Date of last audited annual statement 5/08/2014

E: RISK (½ page)

Overall risk rating: Medium Overview of programme risk The original business case stated that “The impact of MeTA is potentially significant, as the programme aims to improve both the efficiency of pharmaceutical markets and sector, and governance and accountability for access to medicines outcomes. However, the approach is high risk and slower to deliver returns than typical logistics reform and distribution systems support.” This remains true; however the risk pertaining to each output is reduced to low or medium due to the capacity of the IMS to manage the risks that are within its control, giving the project an overall risk rating of medium. Key programme risks and most current risk mitigations are as follows: 1. MeTA support to countries does not gain traction with broader health and pharmaceutical

reforms at country level delaying changes in pharmaceutical sector policies: low to medium risk (high impact). To date this risk has been fully mitigated in all MeTA countries. Concerns from last year regarding Zambia have been alleviated with members of the MeTA council driving the process that will introduce innovative health shops (pharmacies) in the community. This is in full collaboration with the MoH, and part of sector wide health systems strengthening (HSS). Progress in addressing risk to reform is partially dependent on the local political environment and commitment to pharmaceutical policy reform, which has been challenging due to changes in political leadership.

2. Country multi-stakeholder groups are not effective; constituencies do not share information and MeTA is unable to identify areas and approaches for action: low risk (medium impact). Multi-stakeholder approach is working across all countries. Challenges remain in general for CSOs to have a strong or equal voice. Risks have been mitigated by the formation of CSO coalitions. Outputs around private sector engagement are

http://haiweb.org/

25

monitored and the risk remains overall low as the private sector plays an active role in all MeTA countries.

3. MeTA councils are unable to raise resources at country level and remain reliant on international funding: high risk (high impact). Currently this is more a high risk to sustain the MeTA model for the future. To mitigate against this risk, IMS has worked closely with local councils in MeTA countries this year to find sustainable solutions. For countries which have secured high levels of government commitment to MeTA, institutionalization may be an alternative to external funding such as is being explored in Jordan and Peru. Funding options (through alternative funders) have been found for Uganda and Zambia and other options are being explored for the other countries. In most countries the value of the MeTA platform has been recognized and most likely multi-stakeholder platforms will continue in some form.

4. Information generated at country level is not regularly updated and is insufficient to monitor the impact of policy changes or pharmaceutical sector performance: low risk (high impact). Methodologies and tools which are new to the country have been or are in process of being introduced in six out of seven countries. The 7th country, Zambia, has developed a protocol for monitoring pricing in the private sector, but has not yet implemented it. Opportunities for new phone technologies were explored in several countries but the difficulties of integrating these into existing systems outweighed the benefits offered.

5. WHO and HAI delivery of international secretariat functions is ineffective e.g. poor joint working, insufficient capacity, corporate viability (HAI) and/or weak working relationships with countries: low risk (high impact). To date this risk has been mitigated well, dropping from a medium risk rating to a low risk rating due to operational improvements in the last two years.

6. Poor performance by MeTA countries (e.g. focus on process rather than results): low risk (high impact). The risk has been mitigated, overall. Work plans are fully established and clearly linked to desired results (and work/funding is not being approved by IMS unless this is the case). The linkage/theory of change is less clear in Zambia, where the model has focused on CSO communications/public awareness to respond to civil society’s need to understand their rights and the problems rather than a data-specific need; but steps have been taken to address this and to commence evidence generation.

7. Changes in pharmaceutical or health sector (e.g. reduction in health financing etc.) negate potential benefits of MeTA: low to medium risk (high impact). Most countries are well-engaged with policy processes, have generated new evidence and are already able to show results in terms of policy change. Ghana had performed poorly in the past years due to operational capacity issues which slowed delivery of outputs, but more recently evidence has been generated and disseminated and policy and practice discussions held. Zambia has been led to focus on mobilising civil society as a means to affect policy.

8. Weak management results in poor value for money in programme implementation: low risk (low impact). The risk remains low and all management reporting is on time. IMS monitors progress and reports regularly to DFID.

Outstanding actions from risk assessment A high risk to this programme is the continuity and sustainability of the MeTA model. IMS has worked with MeTA councils in countries to find sustainable solutions. For countries which have secured high levels of government commitment to MeTA, institutionalisation has proven to be an alternative to external funding (as in Jordan and Peru). Other solutions include alternative funding options such as in Uganda and Zambia. In most countries the valuable impact of the MeTA platform on medicines policies and practices has been recognised and it is highly likely that multi-stakeholder platforms will continue in some form.

It should also be noted that there is risk that CSOs will encounter challenges to retain a strong or equal voice in the medicines policy dialogues. Of all the stakeholders, civil society

26

funding is the most dependent on external donors which is not always sustainable. As a result, the risk exists that MeTA will have strengthened bilateral engagement between the private and public sectors, but will have failed to include the mitigation of civil society engagement. This has been addressed by the IMS through formation of CSO coalitions and synergies with other programmes which are incorporating MeTA principles.

F: COMMERCIAL CONSIDERATIONS (½ page) Delivery against planned timeframe Despite a late and staggered start across countries, the revised timeframe has allowed for countries to ‘catch-up’. The extension granted by DFID until the end of the December 2015 will allow countries to finalise some of the activities which were delayed at the start of the project due to the lag between Phase 1 and Phase 2. All planned outputs and outcomes, as indicated in the programmatic logframe, are therefore expected to be achieved as targeted. Performance of partnership (s) The partnership between DFID and the IMS is maintained through regular discussions both formally and informally. These interactions ensure that project management issues are addressed timeously and appropriately. The partnership between IMS and in-country councils is particularly strong with regular calls and even country visits when more difficult issues arise. This is supported by capacity-building and training events managed by IMS and have been organised through discussion on need and value. The strength of this relationship is evidenced by the positive achievements and progress in countries. Particulars will be assessed by considering beneficiary feedback which will be briefly referred to in the evaluation and requested at the imminent project completion.

Asset monitoring and control There have been no asset purchases.

27

G: CONDITIONALITY (½ page)

Update on partnership principles (if relevant): N/A.

H: MONITORING & EVALUATION (½ page) Evidence and evaluation The DFID Annual Review process provides annual monitoring of activities and expenditure. No new evidence that challenges the project design or rationale has emerged since the last review. The logframe assumptions remain valid. An independent evaluation is currently in progress and the results will be available in August 2015. Together with results from this annual review, the independent evaluation will increase the wider evidence base on the benefits of greater participation and transparency in medicines policy making. Monitoring process throughout the review process In consultation with DFID, IMS introduced six month reports as a requirement for each MeTA country secretariat. The quality of the reports varied with some countries providing sufficient information to ascertain progress. Follow up through correspondence and phone calls ensured closer attention, especially to any issues arising. This allowed for early detection of obstacles and subsequent consultation to establish practical solutions. An example of this was the most recent report from Zambia which indicated slow progress on one planned study due to lack of capacity which was immediately addressed. Where reports, correspondence and calls signalled specific problems that required face-to-face attention a country visit was scheduled. One such case was in Ghana where the DFID Adviser joined the team.

Based on feedback from countries (from reports and personal interaction) IMS provided DFID with a comprehensive annual report and together with further discussion supporting evidence was critically assessed by Lisha Lala (Health Adviser, DFID Tanzania) with additional technical support from DFID advisers from the Health Services Team (UK). An internal peer review process of the Annual review was undertaken.

Documents

Annual Review - Summary Sheet - WHO · 1 Annual Review - Summary Sheet This Summary Sheet captures the headlines on programme performance, agreed actions and learning over the course