Verification in Results-Based Financing for Health: Summary of Findings and Recommendations from a Cross-Case Analysis
Petra Vergeer, Anna Heard, Erik Josephson, Lisa Fleisher
Results-based financing (RBF) has become increasingly popular as governments with limited resources respond to pressures to improve health service outcomes and the effectiveness and efficiency of health service delivery (Witter 2012, Canavan et al. 2008). At the core of most countries responses is an effort to increase accountability (Brinkerhoff 2004). RBF increases accountability through a system of rewards and punishments (Jack 2003, England 2000, Mills and Broomberg 1998) and results-driven approaches. Verification, defined as the first order substantiation of results paid for in RBF whether coverage rates or quantities of patients seen, quality of services provided or patient satisfaction is critical in assuring that these RBF rewards and sanctions are justly allotted. Verifying results can include ensuring the consistency of routinely collected data, directly observing the conditions of service delivery and care, and conducting patient surveys either at the facility or contacting patients later at home. Ensuring a country has the relevant systems in place to implement and conduct robust verification is of interest to donors and governments which are sensitive to the potential for over-payments based on inflated service reporting.
Despite the increasing popularity of RBF, there is little evidence or documentation of different verification strategies and how strategies relate to the verification results (Naimoli and Vergeer 2010). Documentation of implementation processes including those pertaining to verification of outputs/results is lacking in World Bank-financed RBF projects in the health sector (Brenzel et al. 2009). In demand-side schemes, such as conditional cash transfers (CCTs), verification processes are less documented than other aspects of CCT design and implementation (Fiszbein and Schady 2009). One study has explored the trade-offs between different verification strategies used in RBF programs in sub-Saharan Africa (Ergo and Paina 2012), but there has been little consideration for the influence of the context on verification strategies and limited learning of the effect of verification characteristics on data accuracy and costs.
The overall objective of this cross-case analysis is to expand knowledge about verification processes and practices to address the design and implementation needs of RBF projects. This study adds to available knowledge by comparing the characteristics of verification strategies as well as available data on costs (using level of effort as a proxy), savings, and verification results to date in six countries: Afghanistan, Argentina, Burundi, Panama, Rwanda, and the UK.1 The six case study countries have considerable variability in context and experience with RBF. As a result, each RBF program encompasses a variety of institutional arrangements, foci, levels of implementation, and results purchased. Implementing RBF at different levels of the health system will create different principal-agent problems and the design and implementation of verification strategies are therefore highly heterogeneous across the case countries.
These case studies were purposively selected to explore a number of factors, including: how a variety of results are verified; how the verification strategy is being implemented at different levels in the health
1 Individual case studies, published as six separate HNP Discussion Papers were written on each of these countries (see Annex for references, and download links)
system; and the implications of having different types of actors (that is, third-party versus internal verifiers) involved in the verification process. In this cross-case analysis, the discussion of similarities and differences in verification methods across the six cases as well as the analysis of findings is guided by a conceptual framework developed for this study.
The development of the conceptual framework was informed by a review of the literature on the theory of verification and related topics. The principal-agent problem helps to explain why and how verification is used to increase information, accountability, and motivation. The literature also suggests that the incentives for accountability that are included in the verification strategy, how verification results are used, and the structure of verification, are important to ensure that agents are held accountable throughout the verification process.
The conceptual framework uses a spectrum for each area of focus: context and RBF characteristics, verification characteristics and the use of verification results.
It is important to note that this framework is not intended to be normative: this paper is not suggesting that verification should be structured one way or another based on the context. Rather, the framework is descriptive with respect to how RBF characteristics, and more relevant to this paper, verification
characteristics, and the use of verification results differ. This then allows for consideration of potential implications on data accuracy, cost and sustainability. As we will argue in this paper, these considerations and decisions are critical throughout the implementation of RBF and its verification strategies and the conceptual framework may be helpful for this.
Key findings from across the six case studies are:
Findings Related to Context and RBF Characteristics Finding 1. Implementation of verification is highly dependent on context and RBF characteristics
Finding 2. Verification strategies have changed over time along the spectrum of the conceptual framework
Finding 3. Use of e-data and civil registration systems reduces error rates and cost of verification
Findings Related to Verification Results and their Use Finding 4. Error rates decline after the first few years of verification
Finding 5. Verification and counter-verification databases and reporting present disaggregated data at the level of the contracted party except in Burundi
Finding 6. Verification has spillover effects
Finding 7. Verification identifies both under- and over-reporting
Findings Related to Verification Characteristics
Sampling Strategy Finding 8. Sampling strategy has consequences for cost
Finding 9. Patient tracing confirms patient existence and receipt of service; however it is time consuming
Finding 10. Indicators with high patient volume and complexity have high error rates
Allowable Error Margin Finding 11. Errors not met with sanctions will persist
Advance Warning Finding 12. All countries except Afghanistan give warning for verification visits
Ex Ante vs. Ex Post Verification Finding 13. Ex ante verification is used in all cases
Institutional Setup Finding 14. Varied levels of separation of functions are observed in the countries, influencing (counter-) verification design
Nine recommendations emerge from the cross-case findings:
Recommendation 1. Verification strategies should be dynamic, not static, and use a risk-based approach
Verification strategy should not be static but change based on needs. As context changes, RBF programs evolve and face challenges during implementation, the associated verification strategy (and its characteristics) also will need to change. For example, it became evident that the rationale for RBF influences what is done with the verification results (finding 1): countries which started out more focused on health system strengthening and support, like Burundi and Rwanda, used the verification results mostly for error correction and learning, while countries that were more focused on financial accountability, like Argentina, Afghanistan and the UK, used it mostly for sanctioning or cost-recovery purposes. Over time though, countries made a lateral shift along the spectrum: with Burundi intensifying its efforts to sanction while Argentina moved towards using verification in a more supportive manner. The magnitude and direction of the shift will be determined by a countrys culture and context. It is therefore important to assess periodically whether verification strategies are still aligned with contextual needs.
Verification was found to be an evolving process (finding 2). Changes were made to verification strategies in response to specific challenges that arose, such as making payment for verification visits performance-based due to delays experienced in counter-verification in Rwanda, including a stronger focus on improving quality in Burundi, and an enhanced focus on the number of services provided in Afghanistan.
The country experiences with RBF are also thought to influence verification characteristics: for example, the actor providing the funds seems to influence whether verification and/or counter-verification are carried out by internal actors or by a third party (finding 1): In the UK, verification was put in place following the desire by the government to have much more transparency and a general shift in the political climate, demanding greater financial accountability. In Burundi, the government preferred internal verification but the actors providing external f