Technical Brief

The Africa Indoor Residual Spraying (AIRS) project, funded by PMI, has developed a strong monitoring and evaluation (M&E) system. This system ensures high quality data is produced to guide and strengthen the operations of PMI’s multifaceted IRS campaigns. AIRS collects data from millions of people across 11 countries in Africa. Project data provides evidence of the impact in terms of beneficiaries protected and local people trained. Spray operators and team leaders record this information on paper by hand in the field. But data errors, such as missing figures and arithmetic errors, limit the quality of the data; consequently, supervisors spend considerable time correcting these mistakes.

As a result, AIRS has implemented new M&E tools to help country staff identify and correct common errors that occur during field data collection and data entry. AIRS also conducts an internal post-spray audit to verify the quality of campaign data reported. These M&E improvements generate high-quality data and present PMI with an accurate reflection of IRS campaign results, which are used for program decision-making and planning.

In order to reduce errors and improve the data collection and entry process, AIRS conducted a mobile data collection pilot in Angola in 2013. In 2014, AIRS will pilot mobile data collection in Zambia, using the lessons learned from the Angola pilot to ensure a smooth transition to this new method of data collection.

Strong Systems and Innovative Pilots Improve AIRS Monitoring and Evaluation (M&E)

AIRS is protecting millions of people from malaria by supporting IRS and entomological monitoring in Africa.

www.africairs.net • info@africairs.com

New Data Collection & Entry FormsAIRS developed three tools which help M&E staff spot common data collection errors, verify the data after it has been collected, and clean entered data.

1. The M&E Error Eliminator form was created by the Nigeria M&E specialist to systematically verify that each data point was recorded accurately. Before leaving the field every day, team leaders systematically check data collection forms with the M&E Error Eliminator. If they see that data are written incorrectly, they notify the spray operator and he or she then returns to the household to get the correct information.

2. The M&E Data Collection Verification Form enables M&E supervisor to spot check data recorded by spray operators after they spray homes. During the supervisor’s visit, s/he interviews residents and records information provided by the household on the Data Collection Verification Form and compares it to data collected by spray operators. The Data Collection Verification Form helps the M&E team ensure accuracy and allows the team to periodically audit sprayed villages during the campaign. With this tool, AIRS can correct mistakes with spray operator data in real-time, reducing the likelihood of repeated errors.

3. The Database Cleaner is used by data clerks to identify and correct data that was incorrectly keyed into the database. The database is equipped with automated logic checks that prevent data clerks from entering illogical data. Once data are entered, data clerks use the Database Cleaner to identify data inconsistency and correct these through comparison with the paper form. The Database Cleaner serves as one of the last steps in a series of quality checks before data are ready for analysis.

Post-Spray Data Quality Assurance AuditAfter the IRS campaign is completed, AIRS conducts an internal data audit using the Post-Spray Data Quality Assurance Audit (PSDQA) protocol. This “self-check” is an innovative and transparent data quality assurance measure that is an integral part of the project’s M&E system. After the spray campaign ends, AIRS partners with local government supervisors to survey a representative sample of IRS targeted areas in order to validate the spray coverage. Additionally, the audit allows the field teams to identify strengths and weaknesses in their operations and M&E systems and implement the appropriate changes to improve performance during the next spray round.

Angola Mobile Data Collection PilotIn order to streamline the data collection process and have access to campaign results immediately, AIRS Angola piloted a program that used mobile technology. Instead of collecting data on paper, spray operators in the pilot commune recorded household data on smartphones while team leaders performed concurrent data verification with Android tablets. Smartphones were programmed with data locks that prevented illogical information that is common in paper-based data collection, from being entered; this led to high-quality campaign data. Mobile data collection also provided spray supervisors and the operations team with real-time information on spray progress. The immediacy of the data enhanced its use in guiding spray operations and facilitated the rapid resolution of data issues. By building upon the piloted mobile M&E system, AIRS has the opportunity

to expand the technology to include staff performance monitoring and spatial mapping of spray coverage with epidemiological and entomological trends.