1

Executive  Summary  Implementation  Report  

Author: Jacob Skaggs, Project Manager

PIs: Mark Buntaine, Sarah Bush, Ryan Jablonski, Daniel Nielson, Paula Pickering

Repairing  Information  Underload:  The  Effects  on  Vote  Choice  of  Information  on  Politician  Performance  and  Public  Goods  in  Uganda  

April  2016  

This study is a randomized control trial in Uganda to learn how information about districts’ management of budgets, sub-counties’ supply of public services, and the provision of the information privately or collectively affects candidate support, political participation, and vote choice. The study focused on the February and March 2016 elections for local sub-county and district chairs and councilors.

The implementation team in Uganda comprised one project manager from the U.S., two Ugandan supervisors, and between 18-22 team leaders and 58-80 research assistants depending on the research activity. Research activities were carried out between the end of November 2015 and the middle of March 2016.

Below are important details on the results, challenges and lessons we learned for each of these activities, listed in chronological order.

1) Securing approvals from the Uganda National Council for Science and Technology (UNCST), the Uganda Office of the President, and Resident District Commissioners (RDCs).

UNCST approved our research and provided us an official research approval letter on October 12, 2016. UNCST’s Executive Secretary informed us that with UNCST approval we had explicit permission to start our research while our proposal was being reviewed by the Uganda Office of the President. Our study was finally approved by the Office of the President on February 8, 2016, after majority of our research activities were already completed. Approvals to carry out research in the districts were granted by most of RDCs over a 4-day period. Other RDCs were out of the office and could not be reached. For the districts where RDC approvals were not secured, the implementation team continued seeking approvals from these RDCs during the next subject recruitment phase of the project.

2) Recruiting subjects and conducting public service audits.

During the subject recruitment phase of the project, RDCs from Kyegegwa, Namutumba and Moyo refused to grant us permission to conduct research in their district until the Uganda Office of the President approved our research study. Namutumba District was replaced with Kamuli District since enough time was left in the recruitment phase to carry out research activities in the district.

From November 29-December 19, we recruited 31,310 subjects from 27 districts in Uganda. Since our field activities took place during the rainy season, several teams faced challenges reaching villages due to poor road conditions. With challenging road conditions and issues securing approvals from RDCs, the implementation team recruited subjects in 762 out of the 870 villages selected for the study.

2

The enumeration team faced the challenges one would expect recruiting subjects around the time of elections. Many community members initially mistook our field teams for political party representatives, as several presidential candidates were campaigning in communities around Uganda at the same time. In some communities the enumeration teams faced disgruntled crowds due to election-related issues. Some people expected gifts or money in exchange for phone numbers, which is a common vote-buying tactic. To alleviate these issues, team leaders spent extra time explaining the study to village chairs and garnering their full support for the recruitment effort before started recruitment activities.

At the same time as the recruitment effort, the enumeration team also conducted public service audits on water access points, sub-county roads and sub-county health centers. For water access audits, the parameters we measured were walking time to and from the access point, the number of people waiting in line, and the fee per jerry can. For sub-county roads, we took both road roughness measurements using both an android-based application and a visual inspection of the road. At sub-county health centers we measured cleanliness, drug availability and the wait time of patients.

The audits of sub-county roads were the most challenging to implement. Due to the combination technical issues with RoadLab, a newly develop smartphone application we used to measure road roughness, and challenging road conditions, team leaders were only able to take audit measurements on approximately 60 percent of the sub-county roads included in the study.

Primary education data were provided by our research partner, Twaweza, from its annual Uwezo education study. English literacy scores, numeracy scores, and pupil absenteeism were parameters used to create index scores for primary education. District-level budget management data were gathered from reports from the Uganda Office of the Auditor General. Scores for budget management were calculated using the amount of funds unaccounted for in districts’ budgets.

3) Administering a pre-treatment baseline survey by call center.

The implementation team used call centers to administer the baseline and two endline surveys. Compared to SMS exchanges and in-person interviews, call centers are the least expensive method of gathering information from subjects in Uganda. In comparison, a 7-minute call with subjects costs the same as a single 160-character text message response from a subject.

Our 25-question baseline survey was carried out from January 11-16. Out of the 31,310 subjects we recruited, 17,329 subjects were successfully surveyed in the baseline call center. 218 subjects – 1.3 percent of the subjects we successfully contacted – refused to participate in the survey.

Rather than conducting a large call center at a single location, each of the 19 team leaders worked with small groups of 3-8 research assistants to carry out call centers at locations close to where they live. This helped reduce costs by avoiding accommodation and transportation expenses, and helped the implementation team avoid unwanted attention during the election period. Lastly, this strategy allowed team leaders to employ their own management strategies and provide better oversight over their teams.

Given the long 25-question baseline survey, we found it important to provide subjects with an incentive to encourage participation. Subjects were sent 1,000 Ugandan Shillings (UGX) of airtime for each survey they completed in our study (one baseline and two endline surveys). Services for sending airtime in bulk to subjects across all networks are still being developed in Uganda and we had issues sending airtime in a timely manner to subjects. Cellular networks also offer an airtime loan service that is widely used in Uganda. Our airtime was often used to pay off these airtime loans on subjects’ accounts without their

3

notification. This gave many subjects the impression that we did not send airtime. To mitigate this issue, we sent extra airtime and a message that accompanied the airtime sent.

4. Developing and sending treatment and placebo messages for public service audits and budget management.

There were several tasks involved in preparing treatment messages on water access, sub-county roads, health center 3s, primary school education and district-level budget management. First we calculated scores for public services using audit data collected in the field, primary education data provided by Twaweza, and budget management data gathered from reports from the Uganda Office of the Auditor General. Next we created template messages with placeholders where village-specific scores could be populated. These template messages were then translated by team leaders into the 11 major languages used in the 27 districts included our study. We then used Microsoft Excel to combine the message templates with public service and budget management scores to create messages that were unique to subjects’ villages. Lastly, these messages were sent to subjects using a bulk text message service offered by the Kampala-based company, SMSOne. Each day we sent two to four text messages to subjects over one-week periods leading up to both the district (LC5) and sub-county (LC3) elections. Public service messages were sent out twice, before both the LC3 and LC5 elections. Budget messages were only sent once before the LC5 elections. Overall, approximately 500,000 text messages were sent to subjects in this study. Outgoing text messages sent in bulk are relatively inexpensive with each message costs about 30 Ugandan Shilling - about $0.009.

5. Carrying out the post-treatment endline surveys on the district elections for chairs and councilors.

The LC5 endline surveys were administered by call center over the 5 days immediately following the LC5 elections on February 24th. The implementation team administered the 12-question survey to collect information on who subjects voted for in the LC5 elections and their opinions of LC5 leaders. In the first endline call center we successfully completed over 12,972 surveys out of the 17,329 subjects that were contacted in the baseline call center. Out of the subjects we successfully contacted, 115 subjects – 0.8 percent of subjects – refused to participate in the survey. 3381 subjects – 26.1 percent of subjects we successfully contacted – indicated that they did not see the messages we sent to them. Many of these subjects reported that they had issues distinguishing our messages from election-related messages sent from political parties and other organizations during the same period. Approximately 20 percent of the surveys were completed on the day following the election before the official LC5 election results were announced.

6. Implementing the post-treatment endline survey on sub-county elections for chairs and councilors.

The LC3 endline surveys were administered by call center over the 4 days immediately following the LC3 elections. The Kobocollect survey contained 11 questions about who subjects voted for in the LC3 elections and their opinions of LC3 leaders. In this call center we successfully contacted 12,873 subjects out of the 17,329 subjects contacted in the baseline survey. 748 subjects – 5.8 percent of the subjects we successfully contacted – refused to participate in the endline survey. In Kampala, where support for opposition parties is higher than most other parts of the country, 9.3 percent of subjects refused to participate in the survey. In districts outside of Kampala, 5.3 percent of subjects refused to participate in the survey. 2201 subjects – 17.1 percent of subjects we successfully contacted – indicated that they did not see the messages we sent to them. In both endline surveys, staff contacted the same subjects that they

4

spoke with during the baseline call center. This helped build trust between the subjects and call center staff, which was important in collecting sensitive election-related information.

 

5

Implementation  Report  Repairing  Information  Underload:  The  Effects  on  Vote  Choice  of  Information  on  Politician  Performance  and  Public  Goods  in  Uganda.  

April  2016  

1)  Overview   This report documents the implementation activities of this research study. The report includes the study’s implementation procedures and any deviations from these procedures, management strategies, challenges and valuable lessons we learned throughout the implementation process. This report is meant to complement our Pre-Analysis Plan (PAP), which was registered on November 19, 2015 and contains information on this study’s research design.

Before implementations activates started, our project received approvals from all Co-Principal Investigators’ Institutional Review Boards (Temple & LSE through IAAs), Mildmay Research Ethics Committee, and the Ugandan National Council on Science and Technology (UNCST). For approvals within Uganda, we submitted a research application to the UNCST on October 2, 2015. UNCST approved our research on October 12, 2016. UNCST’s Executive Secretary, Hellen Opolot, informed us that with UNCST approval we had explicit permission to start our research while our research proposal was being reviewed by the Uganda Office of the President. Our study was approval by the Uganda Office of the President on February 8, 2016. Many of the research activities of this study were implemented while our research proposal was pending approval from the Office of the President.

The implementation of this research experiment involved a number of research activities that were carried out between the end of November 2015 and the middle of March 2016.The first activity was to secure approvals from Resident District Commissioners (RDCs) to conduct research in the 28 originally selected for the study. Next, the implementation team spent 15 working days in the field recruiting subjects and conducting public service audits on water access points, sub-county roads and sub-county health centers (health center 3s). Following subject recruitment and public service audits, the implementation team administered a pre-treatment baseline survey by call center with subjects. Treatment and placebo messages were then sent to subjects’ cell phones who could be contacted at baseline using a bulk text messaging service. Finally, call centers were used to administer the post-treatment endline surveys immediately following the elections for local district (LC5) and sub-county (LC3) chairs and councilors. The call center following the LC5 election took 5 days to complete, while the call center following the LC3 election took 4 days to complete.

Below is a timeline of the implementation activities for this study:

Implementation timeline and key dates between November 2015 and April 2016:

• October 10: Received official research approval from the Uganda National Council for Science and Technology

• November 24-27: Secured research approvals from Resident District Commissioners • November 29-December 19: Recruited subjects and conducted public service audits • January 11-16: Conducted a baseline call center to survey subjects on upcoming district and sub-

county elections

6

• January 15-February 15: Developed and translated treatment messages • February 18: Uganda Presidential Election • February 19-24: Delivered public service audit and budget audit treatment messages by text

message to subjects • February 24: Uganda Local Council 5 (LC5) Elections • February 25-29: Conducted an endline call center to survey subjects on the LC5 election and

delivered reinforcement messages for public service audits only • March 2: Uganda Local Council Elections for Division Chairs and Councilors - Kampala Only • March 3-6: Administered an endline call center to survey subjects on the local council elections

for division chairs and councilors in Kampala only. • March 3-8: Delivered public service audit treatment messages by text message to subjects • March 9: Uganda Local Council 3 (LC3) Elections • March 10-13: Conducted an endline call center to survey subjects on the LC3 elections

This report provides details on each of these research activities. Activity protocols, survey instruments, staff contracts and other procedural documents are referenced in the text and documented in the appendices.

Structure and Roles of Implementation Team

The implementation team comprised one in-country project manager, two Ugandan supervisors, and between 18-22 team leaders and 58-80 research assistants depending on the research activity. Team leaders and research assistants were hired based on the survey and language skills that were needed to work in the 28 districts originally selected for the study. Approximately 25 percent of the implementation team were based outside of Kampala, many of them living in the districts where they conducted research.

To maintain institutional memory, the implementation team attempted to use the same individuals as team leaders and research assistants throughout the project. More than 80 percent of the staff involved in the subject recruitment phase of the project were a part of the implementation team for all phases of the project. Staff were replaced in subsequent activities if they failed at meet expectations outlined in the protocols and contracts or had other commitments. Another benefit of maintaining the same team members is that staff were able to build trusted relationships with subjects. To the greatest extent possible, staff interfaced with same subjects they had previously contacted throughout all phases of the study. This strategy of building trust between subjects and implementation staff was particularly important in gathering information from subjects regarding the local government elections given the sensitive nature of elections in Uganda.

2)  Securing  Research  Approvals  from  Resident  District  Commissioners  Without final approval from the Office of the President, even with formal UNCST approval to begin research while the review was pending, several RDCs refused to allow research activities in their districts Summary:

From November 24-27, 11 team leaders traveled by bus out to the 28 districts originally selected for the study to secure research approvals from Resident District Commissioners (RDC). Approvals to carry out research in the districts were granted by most of RDCs during this 4-day period. Other RDCs were out of the office and could not be reached. For the districts where RDC approvals were not secured, the

7

implementation team continued seeking approvals from these RDCs during the next subject recruitment phase of the project.

During the subject recruitment phase of the project, RDCs from Kyegegwa, Namutumba and Moyo refused to grant us permission to conduct research in their district until the Uganda Office of the President approved our research study. Namutumba District was replaced with Kamuli District since enough time was left in the recruitment phase to carry out research activities in the district.

We found that the majority of RDCs preferred to meet with research teams in person and read through hard copies of official introduction and approval letters from UNCST approving our research study.

Challenges and deviation from original protocol:

During the subject recruitment phase of the project, the implementation team first started working in districts where they had previously received approvals from RDCs. As the teams moved onto districts without RDC approvals, three teams encountered issues securing approvals from RDCs. With our research proposal still under review in the Office of the President, RDCs for Namutumba, Moyo and Kyegegwa Districts refused to grant us permission to conduct research in their districts until the Office of the President approved our study. The RDC of Namutumba District notified us of this decision early enough that we were able to select a replacement district. Mayuge and Kamuli Districts were possible replacements based on the language abilities of the implementation team. Kamuli District was chosen by a coin toss (see Table 1 for the list of villages in Kamuli added to the study). Thirty villages in Kamuli District were selected based on villages that were audited in Twaweza’s 2011/2012 Uwezo education study since Kamuli District was not included in the 2013/2014 Uwezo study.

In both Kyegegwa and Moyo Districts, it took nearly a week for the RDCs to inform us of their decisions not to approve our request to conduct research. However, with UNCST approval to start research activities, the implementation team in Kyegegwa started recruiting subjects in villages while awaiting response from the RDC of Kyegegwa. We recruited subjects from a number of villages in Kyegegwa District before the RDC informed us of his decision to not approve our research study until the Office of the President approved our study. Our research project was approved by the Uganda Office of the President on February 8, 2016. The approval letter was sent to the RDCs of Kyegegwa District, allowing us to involve subjects that were previously recruited in district. Kyegegwa and Moyo Districts were not replaced with other districts since there was too little time left to conduct recruitment activities after the RDCs informed us of their decisions.

3)  Recruiting  Subjects Communities often mistook our research efforts for political campaign activities and teams frequently dealt with disgruntled crowds

Summary:

Figure 1. Subjects gather and wait to be read the recruitment script in Lyantonde District.

8

With approval from UNCST to start research activities, the implementation team recruited subjects and conducted public service audits over 15 working days between November 29th and December 19th. Field activities were carried out in 27 districts (see Table 2 listing districts where we conducted subject recruitment and public service audits).

Early on in the subject recruitment effort, several villages in Nakasongola District were added to replace villages that were inaccessible or had very low recruitment numbers. These villages were randomly sampled from the full list of villages in the 2011 census (see Table 3 listing the villages in Nakasongola District that were added). After visiting the resampled villages in Nakasongola, the implementation team quickly realized that resampled villages caused logistical issues that made it challenging to stay on schedule and visit the villages originally selected for the study. The three resampled villages in Nakasongola were used in the study but no other villages were resampled in the study.

A greater number of subjects were recruited in communities in or near Kampala and near major towns and population centers. Lower numbers were recruited in rural villages, particularly in the northern regions of Uganda where village populations are small and few people own cell phones. Since we used SMS and call centers to communicate with subjects, the subject pool could only include adults who owned phones.

Since these field activities took place during the rainy season, several teams faced challenges reaching villages due to poor road conditions. With challenging road conditions and issues securing RDC approvals, the implementation team was able to recruit subjects in 762 out of 870 originally selected villages for a total of 31,310 subjects.

Team structure and responsibilities

For the subject recruitment and public service audit activities, the implementation team comprised 2 supervisors, 22 team leaders and 58 research assistants. The team was split into 11 groups and each group containing two team leaders and 4-6 research assistants. Each group was responsible for carrying out subject recruitment and public service audits in 2-3 districts (see Table 4 for a list of district groups, and the number of villages, research assistants and team leaders assigned to each district group).

When first entering a new village, team leaders were responsible for garnering the support of the village chairs and making sure the research assistants had all the tools needed to recruit subjects (see Appendix A for the mobilization protocol). Research assistants were responsible for recruiting subjects using the guidance provided in the subject recruitment protocol (see Appendix B for the subject recruitment protocol). Research assistants also used a door-to-door recruitment script and consent script to inform subjects about the study, encourage them to participate, and secure their voluntary informed consent (see Appendix C for the door-to-door recruitment script, and Appendix D for the consent script). Team leaders translated both of these script documents into the local languages used in their districts.

Figure 2. A subject is surveyed by a research assistant in a classroom in Lyantonde District.

9

Other details on the responsibilities of team leaders and research assistants were included in the staff contracts (see Appendix E and Appendix F for subject recruitment contracts for team leaders and research assistants, respectively).

Team Management strategies:

To become familiar with the recruitment protocol, research assistants first piloted the survey procedures and instrument in Kampala during a two-day training. Following the pilot effort, we held an in-depth discussion of the issues research assistants had following the protocol and using the survey instrument.

To monitor the progress of recruitment activities in the field and troubleshoot issues that came up, the in-country project manager and two supervisors were in close contact with team leaders throughout the recruitment process. We took a very team leader-centered approach to come up with solutions for issues encountered in the field. Issues experienced by multiple groups were often brought up to the entire group of team leaders over email for discussion. Examples of issues discussed as a group included 1) how to calm down and gain the cooperation of disgruntled crowds that affiliated our research with campaign activities, 2) how to win the support of unsupportive LC1s, and 3) how to plan the complicated logistics of carrying out all recruitment and public service audit activities within their tight schedules. Team leaders that offered valuable advice or insight were recognized for their contributions.

Research tools and technology:

Several electronic devices were used by team leaders and research assistants to recruit subjects and record public service audit. Nexus tablets (2013 version) were used to capture subjects’ signatures for the voluntary informed consent, administer the recruitment survey, and take measurements and record data for public service audits. Tablets were provided to all team leaders and roughly half of the research assistants. We used Kobocollect, a survey application for smartphones and tablets, to record many of the recruitment and public service audit surveys. RoadLab, an Android-based app that measures road roughness, was used for conducting audits on sub-county roads. Each pair of team leaders was provided with a mobile internet router used to upload survey and public service data every evening (see Appendix D for Router, Data and Tablet Management Protocol).

Transportation:

Each pair of team leaders, assigned to cover two to three districts, was provided with two vehicles that were rented from a Kampala-based car rental company. Private vehicles and drivers gave the teams the flexibility to move quickly and safely between villages and manage their various audit tasks, which would not be possible using public transportation. Most of the vehicles we rented were mid-sized sedans such as Toyota Mark IIs. Multi-purpose vehicles like Toyota Ipsums and Hiace Super Custom Wagons were rented for the districts where we expected rougher road conditions.

Challenges and deviations from original protocol:

The enumeration team faced the challenges one would expect recruiting subjects around the time of elections. Many community members initially mistook our field teams for political party representatives, as several presidential candidates were campaigning in communities around Uganda at the same time. In some communities the enumeration teams faced disgruntled crowds due to election-related issues. Some people expected gifts or money

Figure 2. The rainy season made transportation difficult in parts of Kaabong District.

10

in exchange for phone numbers, which is a common vote-buying tactic. After consulting with each other, team leaders decided that the best way to mitigate these challenges was to spend a sufficient amount of time explaining the study to the LC1s and garnering their full support to before recruiting subjects. Team leaders were also advised to immediately leave the village if they felt threatened in any way.

Because our recruitment effort happened during the rainy season, many teams experienced challenges and delays navigating muddy roads that are poorly maintained. Some villages were inaccessible. Teams, particularly in the northern and western regions, found that the Toyota Ipsums and Toyota Super Custom Wagons, though equipped with 4-wheel drive, were not suitable for navigating roads during the rainy season. Time was wasted pushing vehicles and moving around road obstacles as a result. We learned the importance of taking a more careful assessment of road conditions before deciding which vehicles to hire.

In the mobilization protocol (Appendix A), team leaders were asked to visit village local council chairs (LC1s) several days in advance to help mobilize citizens and make subject recruitment process easier and faster. Given the time required to conduct public service audits and provide oversight over research assistants, team leaders often did not have the time to carry out mobilization activities until the actual day of recruitment. Some team leaders had success contacting LC1s in advance over the phone. Overall, mobilizing LC1s in advance was not a good use of time or transportation resources given the large number of tasks assigned to the team leaders.

4)  Conducting  Public  Service  Audits Team leaders worked on a tight schedule to complete public service audits and manage their teams Summary:

Team leaders conducted audits of public water access points, sub-county roads and sub-county health centers simultaneously with subject recruitment activities. In the field, team leaders identified challenges with each of the public service audits. For example, many sub-county roads were inaccessible or impassable and could not be audited using the RoadLab app. A number of sub-counties were missing health-center 3s and instead had health-center 2s or 4s, or lacked a health center altogether. We also found that a number of water access points were privately owned or located outside of the villages we audited. To make things more challenging, our Kobocollect surveys offered few extra data entry fields to document the circumstances we encountered in the field.

Team leaders were encouraged to help develop solutions to the issues that came up in the public service audits. Issues faced by several team leaders were brought up to the entire group for discussion. Team leaders came up with many of the solutions themselves.

Below is an overview of each of the public service audits and the issues we faced conducting these audits:

Sub-­‐county  Road  Audits  Many sub-county roads were inaccessible and impassable by car  

Team leaders were instructed to audit one sub-county road in each village included in the study. Each road audit had two components: a roughness measurement and a visual inspection (see Appendix I for the road audit sampling protocol, and Appendix J for the road audit enumeration procedure). The roughness measurements were recorded using RoadLab, an Android-based smartphone/tablet application (app) that evaluates road conditions by measuring and counting major bumps in a road. One of the limitations of

11

RoadLab was that the app only takes roughness measurements at speeds greater than 20 km/hour. Since many of the roads we audited were difficult to travel on at speeds greater than 20 km/hour, we also conducted a visual inspection of the roads (see Appendix K for the road audit survey instrument for the visual inspection).

Challenges and deviations from original protocol:

RoadLab is a relatively new Android app used to measure and report road roughness measurements. With limited options to measure road quality, we worked with RoadLab to add features to the app so that it met the needs of our research. These added features included entry fields for the village and district where the measurements were taken, and a sign-in feature allowing audit data to be associated with a team leader. Unfortunately team leaders had limited time to pilot the technology on roads in Uganda; due to time restrictions, piloting was limited to a short test-run around Makerere University in Kampala. With only a brief opportunity to pilot RoadLab, team leaders had challenges operating RoadLab over the first few days in the field.

During the first two days of road audits, RoadLab also had technical issues with its server. The district and village information was not recorded in the server when team leaders uploaded the data. Also over the first few days of conducting road audits, team leaders had problems signing into RoadLab. These two factors created problems matching the RoadLab audit data to the districts and villages where the measurements had been taken. Since it was partly their mistake, RoadLab matched the audit data with the locations using GPS information also collected by the app. RoadLab provided the following instructions on how they matched road audit data to the locations where they were taken:

Geo and regular filtering was used to view data in a specific region on one given day that had either the region, village or both parameters missing. Next, GPS coordinates from the water access audit were viewed from a chosen date and region. If we found a match between two sets of GPS coordinates, we compared the timestamp of RoadLab data with the "time of day" parameter (morning/afternoon/evening) on the water access audit. If these temporal parameters matched, we would then assign missing parameters to one road segment using geo filtering.

Additionally, team leaders found that some sub-county roads were inaccessible or impassable using the cars that were rented for the study. Additional instructions were provided for conducting visual inspections of sub-county roads that were inaccessible or impassable by car (see Appendix K for the update to the road audit protocol).

Due to the combination of RoadLab interface issues and challenging road conditions, team leaders were only able to take RoadLab measurements on approximately 60 percent of the sub-county roads included in the study. For the road audits missing RoadLab data, we calculated the quality of these sub-county roads using only the visual inspection (see Appendix X on calculating index scores for sub-county roads).

Water  Access  Audits  A number of water access points were privately owned or located outside of the village

Team leaders were instructed to audit a water access point in each of the villages they visited. The audit included three parameters: 1) the round-trip walking time between the home of the 3rd person recruited and the nearest water access point, 2) the number of people waiting in line at the water access point, and 3) the cost per jerrycan if there was a fee (see Appendix L for the water access sampling protocol, and Appendix M for the water access enumeration procedure). The water access audit was recorded using a Kobocollect survey loaded on the Nexus tablets (see Appendix N for the water access survey instrument).

12

Challenges and deviations from original protocol:

Team Leaders discovered that some of the water access points were constructed by Non-Governmental Organizations (NGO) or built and maintained by individuals. Below are additional instructions to provide team leaders with guidance on this issue:

If the protected water access point is a privately owned, we will not audit it. However, we will audit water access points installed by NGOs since they often work in partnership with the local government to plan and install them. For water access points installed by NGOs, follow the same protocol as before. In KoboCollect, answer the Type of Protected Water Source with “other”. Then on the next data entry page, enter in the type of water source and add “NGO”. For example, if the NGO-installed water source was a borehole, you would type in “borehole NGO”. Fill in the rest of the survey as you normally would.

Several of the water access points selected for the audit were located in adjacent villages. Since the local government might allocate water access resources based on the availability of water access points outside of the village, team leaders were directed to audit the closest public or NGO-managed water access point to the selected household, even if it was located outside of the village.

Health  Center  Audits  A number of sub-counties had health center 2s or 4s, or lacked a health center altogether

Team leaders were directed to audit a single sub-county health center (health center 3) in each of the sub-counties included in the study. The audit explored the following three parameters: 1) the availability of drugs in the health center dispensary, 2) the cleanliness of the facilities, and 3) the wait time of patients (see Appendix O for the health center audit sampling protocol, and Appendix P for the health center enumeration procedure). Team leaders recorded these audits using a Kobocollect survey loaded on the Nexus tablets (see Appendix Q for the health center survey instrument).

Challenges and deviations from original protocol:

In the field, team leaders discovered that many sub-counties lacked a health center 3 but rather had a parish health center (health center 2) or in few cases a county health center (health center 4). Both health center 2s and 4s are managed by the local government. Other sub-counties were served by nearby national hospitals managed by the central government. Other sub-counties in very rural areas or locations with small populations or migrant communities had no health facilities at all. Occasionally team leaders found that there were no staff available at the dispensary. The following guidance was provided to team leaders to address these issues:

If the sub-county where you are working does not have a Health Centre 3, but instead has a Health Centre 2 or 4, conduct an audit on whatever Health Centre is most often used by residents of the village. Conduct the audit in the same way that you would conduct the audit at a Health Centre 3. When entering information into KoboCollect, indicate the type of Health Centre (Heath Centre 2 or 4) in your answer to the question, Name of the Health Center 3. For example, if the name of the hospital is Mulago and it is a Health Centre 4, write Mulago HC4.

Also, if you are unable to find anyone to help you at the dispensary, indicate that there was no help at the dispensary in your answer to the question: Name of Health Centre 3. For example, if there was nobody to help you at the dispensary at the Matatizo HC4, you could enter Matatizo

13

HC4 no help. Continue with the survey the same as before, indicating that that no drugs were available.

5)  Administering  the  Baseline  Call  Center  Call center staff worked in small groups in various locations across Uganda, which reduced costs and helped avoid unwanted attention during the election period

Summary:

The implementation team used call centers to conduct the baseline and two endline surveys. Compared to SMS exchanges and in-person interviews, call centers are the least expensive method of gathering information from subjects in Uganda. Using call centers to administer surveys, we were able to re-contact over 60% of subjects from the initial recruitment. Comparing costs, an 7-minute call with subjects – using an MTN or Airtel voice bundle – can be made for the same cost as an exchange of two 160-character text message. Incoming text messages sent by subjects cost 220 UGX compared to outgoing bulk text messages that cost 30 UGX.

Call center staff used several electronic devices to conduct surveys with subjects over the phone. A simple phone or a smartphone, and sometimes a pair of earbuds, were used to call subjects. MTN or Airtel voice bundles were loaded onto the sim cards used by call center staff to make inexpensive calls. Staff used Kobocollect surveys, loaded onto a tablet or smartphone, to record responses from subjects. Mobile internet routers were provided to each of the team leaders to upload Kobocollect survey data from each of the smartphones or tablets each evening.

The length of the surveys were quite long – 25 questions for the baseline survey – and we found it was important to provide subjects with an incentive to encourage participation. For each survey that was successfully completed, subjects were sent 1,000 Ugandan Shillings (UGX) of airtime. Subjects still received airtime if they skipped or refused to answer questions in the survey.

Services for sending airtime in bulk to subjects across networks are still being developed in Uganda. We worked with two different companies to send out airtime to subjects and experienced issues using both services. We faced challenges with both companies sending airtime in a timely manner to subjects, and accessing reliable records to confirm that airtime had been sent. As a result, many subjects complained when they did not receive airtime when they were promised.

Another challenge we encountered is that the major cellular networks, like MTN and Airtel, offer a service that loans airtime to network users on request. This airtime loan service is widely used among cell phone users in Uganda. When we sent airtime to a subject who owed credit to a cellular network, our airtime was used to repay their credit without the subjects’ knowledge; the networks do not send text messages notifying subjects that their loan balance has been reduced. This resulted in many subjects getting the impression that they did not receive airtime. To alleviate these airtime issues, we asked subjects if they received airtime in both the LC5 and LC3 endline surveys. Subjects that did indicated that they didn’t receive airtime, we sent an extra 1,000 UGX using a bulk airtime service. Also, for the final endline LC3 call center we sent a text message to accompany the airtime that we sent.

Team Management and Strategies:

For all three call centers, the implementation team comprised 2 supervisors, 19 team leaders, 81 research assistants. Rather than conducting a large call center at a single location, each of the 19 team leaders worked with small groups of 3-8 research assistants to carry out call centers at locations close to where

14

they live. Each team leader was provided funding to rent a small venue where their team worked under team leader’s supervision. This strategy helped reduce costs by avoiding accommodation and transportation expenses, which would be incurred if all call center staff worked from a single location in Kampala. Working in small groups around the country – rather than in a large group at a single location – helped the implementation team avoid unwanted attention during the election period. Lastly, this strategy allowed team leaders to employ their own management strategies and provide better oversight over their teams.

To motivate team leaders to effectively manage their teams, the in-country project manager and supervisors held private meetings with 3-5 team leaders at a time to discuss team management strategies and responsibilities before each call center. At these meetings, the call center protocol, survey instrument, and contracts were discussed in line-by-line detail. Feedback was collected from team leaders on how to improve the protocol and survey instrument before they were deployed. These meetings proved to be an excellent way to empower team leaders, exchange ideas on how to improve the call center, and make sure that each team leader understood what was expected of them and their team of research assistants.

During the call centers, the in-country project manager and supervisors held daily calls with team leaders to discuss the progress of the work, gather feedback on challenges that the teams were facing, and brainstorm solutions to any problems. Issues affecting multiple teams were brought up to all of the team leaders by email or phone call. Many of the solutions were developed by the team leaders themselves and they were given credit for their input.

To help set performance goals for the teams, the project manager sent an email to team leaders each morning listing the average number of successful surveys completed per staff member across all the teams. Teams with the highest average number of surveys were recognized. Teams leaders with low survey counts were contacted to find out what challenges they were facing and what other tools they might need to successfully complete their work.

General challenges faced conducting the baseline and endline call centers:

• Poor network issues in some districts made it challenging for teams to connect with subjects, gather detailed information over poor connections, and stay connected during long phone surveys.

• Approximately 35 percent of the subjects we contacted from Kampala – where support for opposition parties is very strong – were suspicion that our research group was affiliated with the ruling party or involved in activities meant to undermine the election. As a result, a higher percent of subjects in Kampala refused to participate in our survey compared to subjects in districts outside of Kampala. To mitigate this issue, call center staff to spent extra time explaining our study to convince subjects to take the survey.

• Over all the call centers, approximately 35 percent of staff first recorded the information collected during the surveys on paper before transferring the information into Kobocollect. Most staff members who used this recording method immediately transferred the information from their notebook to Kobocollect following the call. This resulted in shorter data entry times (see Table 5 listing self-reported data on the percentage of call center surveys that were recorded in a notebook before being transferred to Kobocollect).

Baseline  Call  Center  (pre-­‐treatment)  Out of 31,310 subjects recruited, 17,329 were successfully contacted for the 25-question baseline survey

15

After recruiting subjects and conducting public services audits, the implementation team held a call center to administer a baseline survey. The main purpose of the baseline survey was to gather information on subjects’ opinions of their local government leaders and the delivery of public service, and voting preferences for the upcoming elections for district (LC5) and sub-county (LC3) chairs and councilors.

The call center staff attempted to call each of the 31,310 subjects that were recruited for the study. The baseline call center protocol and staff contracts outline the responsibilities and expectations of call center staff (see Appendix S for the baseline call center protocol, Appendix T for the team leader contract, and Appendix U for the research assistant contract). The baseline survey included 25 questions (see Appendix V for the baseline survey instrument). 17,329 subjects were successfully surveyed in the baseline call center. 218 subjects – 1.3 percent of the subjects we successfully contacted – refused to participate in the survey.

Challenges and deviations from original protocol:

Several questions relating to priors about the quality of health care services were accidentally left out of the survey. A separate call center was held shortly after the baseline call center to survey subject health-related questions (see Appendix AB for baseline survey instrument for health questions only). Since our original plan was to only send subjects messages on the public service most important to them, we only re-surveyed subjects who indicated that their most important public service was health care.

6)  Developing,  Translating  and  Distributing  Treatment  Messages,  and  Sending  Airtime  Incentive  We created over 100 message templates, translated them into 11 languages, and sent over 500,000 SMS messages to our subjects

Summary:

There were several tasks involved in preparing treatment messages on water access, sub-county roads, health center 3s, primary school education and district-level budget management. First we calculated scores for public services using audit data collected in the field, primary education data provided by Twaweza, and budget management data gathered from reports from the Uganda Office of the Auditor General. Next we created template messages with placeholders where scores could be populated. These template messages were then translated by team leaders. We then used Microsoft Excel to combine the message templates with public service and budget management scores to create messages that were unique to subjects’ villages. Lastly, these messages were sent to subjects using a bulk text message service.

Developing  index  scores  for  public  services  and  budget  management  Several parameters were used to calculate index scores for each public service

Scores for each of the public services (sub-county roads, water access, sub-county health center and primary education) and budget management were calculated using data collected from public service audits, Twaweza’s 2013/2014 Uwezo education study, and 2012 and 2014 annual reports for the Uganda Auditor General of Local Authorities (see Appendix W on calculating scores for sub-county roads centers, Appendix X on calculating scores for primary education, Appendix Y on calculating scores for water access, and Appendix Z on calculating scores for health centers).

16

To calculate scores for primary school education, Twaweza provided us with data on from its 2013/2014 Uwezo study. The Uwezo initiative monitors basic literacy and numeracy levels of children aged 5-16 years in districts across Uganda using a household-based survey. Twaweza advised us to use data on pupil attendance, numeracy and English literacy to calculate scores on the quality of primary school education. Education data for Kamuli District was sourced from the 2011/2012 Uwezo study since Kamuli District was not included in the 2013/2014 Uwezo study. Primary education scores were calculated by combining English literacy scores, numeracy scores, and pupil absenteeism scores. In addition to providing subjects with a message about the overall quality of primary education in their village compared to the average quality of primary education in villages in their district, we also provided information about how primary schools in the subject’s village compared to district averages on each of the components of the primary education score: English literacy, numeracy, and absenteeism.

To calculate scores for district-level budget management, we gathered data on the amount of funds unaccounted for and information on common budget irregularities from the 2013/2014 annual reports from the Uganda Auditor General for Local Authorities. Using these audits, we calculated the total sum in shillings of “funds not accounted for”, “procurement issues”, and “payroll anomalies”. This total sum was divided by the LC5’s budget in order to create a normalized measure of financial irregularities which we could compare across districts. In addition to providing comparative information based upon this index in the SMS messages, we also provided two messages per district that provided an examples of common budget irregularity in that district in 2014, such as unaccounted expenditures and contracting problems. Developing  treatment  and  placebo  messages  With a number public service audits missing, some subjects were sent messages on their second-most important public service

Next we created messages templates for the four public services and district-level budget management (see Appendix AA for message templates). All subjects randomly selected for treatment were sent messages on a single public service, and messages on the quality of budget management of their LC5 leader.

For each public service, we developed several types of messages to inform subjects about the quality of that public service delivered to the villages compared to other villages in the district.

• Information on whether the LC3 or LC5 (or both) is responsible for the delivery of the public service

• A message communicating an overall index score for the public service • Several examples relating the parameters that were audited and used to calculate the overall index

score for that public service

For messages on the LC5s budget management, we created several messages to communicate the LC5s record of budget management compared to other districts’ leaders. These messages included the following information:

• An index score for the LC5’s record of budget management compared to other districts’ leaders • The amount of funds unaccounted for in 2014 with reference to the districts’ total official budget • Two examples of budget management activities such as unaccounted expenditures or contracting

problems to add context to the district’s overall budget management score

We also created several placebo messages for each public service. Subjects in the placebo condition received public service messages from a randomly selected list of messages centered on health and

17

livelihood issues relating to the public services being audited. Placebo messages did not contain any information that might communicate the quality of public services or budget management, but did talk about the topic which allows us to isolate the treatment effect of information.

The original plan was to send messages to subjects on the public service that was most important to them. One challenge was that we were missing a large number of audits for sub-county roads, and a smaller number of audits for other public services. Since subjects selected for treatment were only sent messages on a single public services, we decided that subjects would receive messages on their second-most important public service if data was missing on their first choice. If the audit for a subject’s second-most important public service was missing, we sent messages on a randomly selected public service that was available from the subject’s village.

Translating  messages  Translation work is easier when staff work in pairs

After we completed a template for the treatment and placebo messages, 11 team leaders were hired to translate these messages into the 11 languages used in our study. The team leaders were provided extra funding for a colleague to proofread their work. Most team leaders later reported that it was easier to work side-by-side with another translator. Since there are multiple ways of translating messages, team leaders found it helpful to be able to discuss each message in pairs to identify the most correct and concise translation. To ensure that the translations were accurate and clear, many team leaders had their translations reviewed and edited by 2-3 other team leaders or research assistants before producing the final version. The translation process might have been streamlined if translators worked in pairs from the beginning of the activity.

Sending  bulk  SMS  treatment  messages  Outgoing text messages are very inexpensive in Uganda

Approximately 500,000 text messages were sent to subjects for this study. Messages were sent using a bulk SMS delivery service developed by SMSOne, a communications technology company based in Kampala.

Each day we sent two to four text messages to subjects over one-week periods leading up to both the LC5 and LC3 elections. Public service messages were sent out twice, before both the LC3 and LC5 elections. Budget messages were only sent once before the LC5 elections. Messages were typically sent out at 10am and 2pm. In one circumstance, a third pair of messages were sent out at 5pm in a single day. Messages were delivered instantly once they were sent using the bulk SMS interface.

Outgoing bulk messages are very inexpensive in Uganda. Each message costs about 30 Ugandan Shilling - about $0.009.

7)  Conducting  an  Endline  LC5  Call  Center  (post-­‐treatment)  Out of the 17,329 subjects contacted for the baseline survey, 12,972 subjects were successfully contacted for the endline LC5 survey

The LC5 endline surveys were administered by call center over the 5 days immediately following the LC5 elections on February 24th. The implementation team administered the 12-question survey to collect information on who subjects voted for in the LC5 elections and their opinions of LC5 leaders (see Appendix AC for the endline LC5 survey instrument). The responsibilities and expectations of the staff

18

for the endline LC5 call center are included in the call center protocol and staff contracts (see Appendix AD for the endline LC5 call center protocol, Appendix AE for the team leader contract agreement, and Appendix AF for the research assistant contract agreement). To greatest extent possible, staff contacted the same subjects that they spoke with during the baseline call center.

With subjects selected for the treatment, we also read the text message stating the overall index score for their most important public service. We delivered this reinforcement messages to strengthen the treatment effect. In the first endline call center we successfully completed over 12,972 phone surveys from the pool of the approximately 17,329 subjects. Out of the subjects we successfully contacted, 115 subjects – 0.8 percent of subjects – refused to participate in the survey. 3381 subjects – 26.1 percent of subjects we successfully contacted – indicated that they did not see the messages we sent to them. Many of these subjects reported that they had issues distinguishing our messages from election-related messages sent from political parties and other organizations during the same period.

LC5 election results were announced by local radio and in town centers 24 hours after polling stations closed on Election Day. Since the call center took more than 24 hours to implement fully, we randomized order of when subjects were contacted for the survey. We did this to account for bias that might have been created by subjects learning the outcome of the election before they were surveyed. Approximately 20 percent of the surveys were completed on the day following the election before the official LC5 election results were announced.

Challenges and deviations from original protocol:

• Question 5: Call center staff used the option "refused to answer" to indicate "none of the above", since this option was left out of the survey. There was not a single case of a subject actually refusing to answer question 5.

• Questions 2 and 3 in the Kobocollect survey incorrectly listed DP (Democratic Party) as DC. Enumerators were instructed to use DC as a substitute for DP for the entire call center.

• Question 4 about the water basin: Many people were confused if we were asking about the ballot box or basin. Teams clarified by asking the color of the basin where subjects ticked their ballots.

• Question 6 about subjects receiving gifts and money: Many people refused to answer this question because they were afraid of harm to them or their family if they indicated that they received gifts or money from political parties.

• The LC5 chairperson election in Kumi was postponed until a date after all local government elections. For Kumi, question 2 in the survey instrument was dropped and question 5 changed from “chairperson” to “councilor”.

• There was no election for Kisoro District’s chairperson since the position was uncontested. Staff for Kisoro District dropped question 2 and changed question 5 from “chairperson” to “councilor”.

• In Murora sub-county in Kisoro the districts councilor and chairperson were unopposed in election. Staff were directed to indicate that subjects did not vote in the election and continue the survey from questions 7.

• Telecom network issues in Pallisa District created challenges for the call center staff to connect with subjects over the phone.

• A number of subjects indicated that they were confused about who was sending them messages. Subjects received messages from political candidates and the Uganda Electoral Commission over the same period of time that we sent them messages.

• In general Uganda experienced relatively low voter turnout for the LC5 election as a result of presidential election issues.

19

8)  Endline  LC3  Call  Center  (post-­‐treatment)  Out of the 17,329 subject surveyed in the baseline survey, 12,873 subjects were successfully contacted for the endline survey for LC3 elections

In the week leading up to the LC3 elections and our final endline call center, we resent all the public service messages to subjects. This second round of messages was sent to subjects in an attempt to strengthen our treatment effect.

The LC3 endline surveys were administered by call center over the 4 days immediately following the LC3 elections. The Kobocollect survey contained 11 questions about who subjects voted for in the LC3 elections and their opinions of LC3 leaders (see Appendix AG for the endline LC3 survey instrument). In this call center we successfully contacted 12,873 subjects out of the 17,329 subjects contacted in the baseline survey. 748 subjects – 5.8 percent of the subjects we successfully contacted – refused to participate in the baseline survey. In Kampala, where support for opposition parties is higher than most other parts of the country, 9.3 percent of subjects refused to participate in the survey. In districts outside of Kampala, 5.3 percent of subjects refused to participate in the survey. 2201 subjects – 17.1 percent of subjects we successfully contacted – indicated that they did not see the messages we sent to them.

In Kampala only, elections for LC3 division chairs and councilors were held on March 2, 2016. The call center to survey Kampala subjects on the LC3 elections was held on from March 3-6. We were able to carry out surveys with 1594 subjects out of the approximately 1800 subjects from Kampala (see Appendix AH the endline LC3 survey for Kampala subjects only).

Like in the LC5 elections, the LC3 election results were announced 24 hours after polling stations closed on Election Day. Approximately 35% of the surveys were taken on the day following the election before the official LC3 election results were announced.

To greatest extent possible, staff contacted the same subjects that were audited during the baseline and first endline call centers. Team management strategies and staff responsibilities for the endline LC3 call center were similar to those of previous call centers (see Appendix AI for the endline LC3 call center protocol, and Appendix AJ and AK for the team leader and research assistant contract agreements, respectively).

In addition to call center staff first recording survey data in a notebook and then transferring the data over to Kobocollect, there were no major challenges or deviations from the protocol. Many call center staff reported that they were verbally abused by subjects that still believe we were affiliated with a political party or involved in an activity designed to disrupt the elections.

 

20

Tables   Table 1. Thirty resampled villages in Kamuli Districts added to replace villages in Namutumba District

Sub-county Village Balawoli Bugaga Balawoli Bususwa Bugaya Bubeko Bugaya Bukoza- buwande Bugaya Bulegeya Bugulumbya Bufumba Bugulumbya Buswaigira Bulopa Bukaliku Butansi Buleebi Buyende Kigaya Buyende Nabukalu Kagulu Butegere Kagulu Masaba Kagulu Nakauna Kidera Buyanja a Kidera Kyankoole 'b' Kisozi Bukwalu Kisozi Nabukedi Kitayunjwa Bugaga bugobe Kitayunjwa Bulase Kitayunjwa Buloyi Mbulamuti Buwoloma Nabwigulu Buluuta Nabwigulu Bunangwe Namasagali Kabaganda Namwendwa Bulondo Namwendwa Bwagusa Nawanyago Nawanyago tr. Block c Nkondo Kirimira Wankole Lutebe b

21

Table 2. List of 27 districts where subjects were recruited and public service audits were implemented

Agago Kasese Amolatar Kiruhura Arua Kisoro Buikwe Kumi Bulambuli Kyegegwa Bushenyi Lyantonde Butambala Mityana Buyende Mpigi Gulu Nakapiripirit Hoima Nakasongola Iganga Pallisa Kaabong Sironko Kampala Zombo Kamuli

Table 3. Villages added in Nakasongola District to replace villages in Nakasongola that were inaccessible or had low recruitment numbers

Village   Sub-­‐county  Kasambya   Kalungi  Katuugo   Kalungi  Kabaazi   Kalongo  

Table 4. Originally assigned districts for subject recruitment and public service audits

Districts Group Assignment # of villages # of research assistants

# of team leaders

Lyantonde, Mpigi, Butambala Central Group 1 90 6 2 Mityana, Nakasongola Central Group 2 60 4 2 Kampala, Buikwe Central Group 3 90 6 2 Kasese, Kyegegwa, Hoima Western Group 1 90 6 2 Kiruhura, Bushenyi, Kisoro Western Group 2 90 6 2 Pallisa, Kumi Eastern Group 1 60 4 2 Bulambuli, Sironko Eastern Group 2 60 4 2 Buyende, Iganga, Namutumba Eastern Group 3 90 6 2 Kabong, Nakapiripiti Karamoja Group 60 4 2 Gulu, Agago, Amolatar Northern Group 1 90 6 2 Moyo, Arua, Zombo Northern Group 2 90 6 2 Total 870 58 22

22

Table 5: Self-reported data from call center staff who recorded survey data in a notebook before transferring data to Kobocollect

Survey  username   District  group   %  of  time  entering  data  into  notebook  Monica  A   Amolatar,  Gulu,  Agago   50  Laila  Lize   Amolatar,  Gulu,  Agago   5  Faith  A   Amolatar,  Gulu,  Agago   5  John  Baptist   Arua,  Zombo   60  Richard   Arua,  Zombo   60  Foska  Inzikuru   Arua,  Zombo   60  Alesi  Christine   Arua,  Zombo   55  Grace   Buyende,  Iganga,  Kamuli   35  James  Mushebe   Buyende,  Iganga,  Kamuli   30  Nakato  Miriam     Buyende,  Iganga,  Kamuli   25  Simon  Peter   Buyende,  Iganga,  Kamuli   10  I  Obalim   Gulu,  Agago,  Amolatar   60  A  Oola   Gulu,  Agago,  Amolatar   30  V  Laaka   Gulu,  Agago,  Amolatar   25  Mercy  A   Gulu,  Agago,  Amolatar   20  James  Mushebe   Kaabong,  Nakapiripirit   2  Earnest  Dedeng   Kaabong,  Nakapiripirit   2  Patra   Kampala,  Buikwe   90  Christine   Kampala,  Buikwe   50  Keffa   Kampala,  Buikwe   40  Leonard     Kampala,  Buikwe   10  Henry  N   Kampala,  Buikwe   5  Muganzi  Hamis   Kampala,  Buikwe   5  Billy   Kampala,  Buikwe   2  Heather  Namulonndo   Kampala,  Buikwe   1  Augustine   Kampala,  Buikwe   1  Ibra  K   Kamuli,  Buyende,  Iganga   2  Simon   Kasese,  Kyegegwa,  Hoima   80  Sylvia   Kasese,  Kyegegwa,  Hoima   70  Janet   Kasese,  Kyegegwa,  Hoima   50  Sylvia   Kasese,  Kyegegwa,  Hoima   50  Wycliff   Kasese,  Kyegegwa,  Hoima   30  Janet   Kasese,  Kyegegwa,  Hoima   30  Viany   Kasese,  Kyegegwa,  Hoima   15  Joshua  N   Kisoro,  Kiruhura,  Bushenyi   12  Benard   Kisoro,  Kiruhura,  Bushenyi   1  Juliet  Nabankema   Lyantonde,  Butambala,  Mpigi   50  Farouk  Masasi   Lyantonde,  Butambala,  Mpigi   50  Matanda  Musa   Lyantonde,  Butambala,  Mpigi   50  

23

Luzze  Ibrahim     Lyantonde,  Butambala,  Mpigi   50  Timothy  Gaira   Lyantonde,  Butambala,  Mpigi   50  Sylvia  Nabisusunku   Lyantonde,  Butambala,  Mpigi   50  Vivian   Lyantonde,  Butambala,  Mpigi   50  Ivan  Ngono     Lyantonde,  Butambala,  Mpigi   40  Victoria  Kigongo     Lyantonde,  Butambala,  Mpigi   40  Rebecca   Lyantonde,  Butambala,  Mpigi   40  Jack  M   Nakasongola,  Mityana   50  Dan  K   Nakasongola,  Mityana   50  Patience  K   Nakasongola,  Mityana   50  Salima  M   Nakasongola,  Mityana   50  Ronald  T   Nakasongola,  Mityana   5  Immaculate  A   Nakasongola,  Mityana   2  Phillip  T   Nakasongola,  Mityana   1  Roger  K   Nakasongola,  Mityana   1  Claudia   Pallisa,  Kumi   50  E  Emulu   Pallisa,  Kumi   30  Betty  Itadali   Pallisa,  Kumi   10  J  Akurut   Pallisa,  Kumi   2  Shania   Sironko,  Bulambuli     5  Carol   Sironko,  Bulambuli     2  Ronald   Sironko,  Bulambuli     1  

 

24

Appendices  

Appendix  A Mobilization Protocol (for team leaders)

A day or two before recruiting subject in a village, meet with the LC1 (or other local official if the LC1 is not available) if possible. Use the following steps to prepare the LC1 for recruitment activities:

1. Explain the research project. 2. Secure a commitment from the LC1 to help inform people about the information service we are

providing, and organize citizens to speed up the recruitment of subjects. 3. Mention that you will offer 10,000 UGX after the LC1 helps research associates recruit subjects 4. Identify a central location where Research Assistants will be able to quickly recruit adult citizens

living in the village. 5. Kindly request that the LC1 provides a table and chairs for the research assistants to use. 6. Ask the LC1 to offer suggestions on the best way to quickly recruit adult residents. 7. Exchange phone numbers. 8. Leave any informational materials (recruitment protocol and consent script) with the LC1 to help

him/her to inform citizens about our information service and upcoming recruitment effort. 9. Inform the LC1 when the research assistants will likely arrive to start the recruitment of subjects. 10. Contact the LC1 a few hours ahead of time (or the night before) to remind him/her about the

recruitment effort so he/she can prepare for the arrival of the research assistants.

 

25

Appendix  B  Recruitment Protocol (for research assistants)

Arrival and Setup

1. Once you arrive at the village, meet with the LC1 and introduce yourselves. 2. Work with the LC1 and your team leader to set up the location where you will recruit subjects. 3. Start recruiting subjects. 4. Subjects must be adult residents of that village that are a primary user of a cell phone.

Recruitment

1. Gather people into groups if possible. 2. Inform them about the information service we are providing using the door-to-door recruitment

script. 3. Make sure each subject is an adult resident of the village and has a cell phone 4. Read the entire consent word-for-word to people interested in signing up for the information

service. 1. Answer any questions. 2. For the people that agree to the terms of the consent, gather their information using the

KoboConnect app.

Gathering Information

1. Using the KoboConnect app, secure the consent signature of the subject. 2. Ask and record responses for the demographic questions in Kobocollect. 3. Thank the subject for their participation.

   

26

Appendix  C  Door-to-Door Recruitment Script

Hello, my name is ___________. I am working on an outreach and research project being sponsored by the Ugandan nongovernmental organization Twaweza. The project is an information campaign that seeks to help local residents understand how public services like primary schools, roads, water access are provided in the local area.

The campaign also informs people about the responsibilities of local governments in delivering these services and the ways that local councils manage their budgets. The campaign will send this information by text message to some of the participants. This project also has a research component, which will investigate whether this kind of information makes people better informed and more confident about their decisions in the upcoming local elections in March 2016. The project is not associated with any political party and we are not campaigning for any political party. We only seek to provide useful and factual information that may help you make your own choices. Do you have a moment so that we can tell you about the opportunity to participate in this information campaign and research project?

 

27

Appendix  D Subject Consent Statement

You are being asked to participate in a research study that investigates whether factual information about public services helps you make decisions in local elections. This research study is carried out by Twaweza, an organization that works to improve development outcomes by providing citizens with information they need to participate effectively in society.

If you agree to be part of the study, you will provide your mobile phone number. The study has three phases. First, you will be asked to take a phone survey about the upcoming local elections sometime in January 2016. It will take roughly 15 minutes of your time. In the 6-8 weeks after you complete the survey, you will receive a series of approximately 15 text messages on your mobile phone, some of which may contain information about how public services like road quality, water quality, and education compare to other areas in Uganda. The particular messages that you will receive will be decided by a computer and based only on random chance. Immediately after the local elections in March 2016, you will be asked to take another survey by phone call about your participation in the local elections.

All of the answers you give as part of the surveys will be confidential and will not be shared with anyone other than members of our research team. You don't have to be in the study, and there will be no consequences if you choose not to participate, provide your mobile number, or answer the surveys. We will not tell anybody whether you participate or not.

If you participate, it will not cost you anything. Since each phone survey will take approximately 15 minutes of your time, we will provide you with 1000 shillings airtime for completing each of the first and the second phone surveys. If we ask you any question that you don't want to answer, you can skip it. Or you can stop the phone survey by telling the person who calls you that you want to stop. You will still receive 1000 shillings airtime if you skip questions in either of the phone surveys. You can also stop receiving informational messages at any time by sending the message “STOP” to our shortcode.

There are some risks that we want you to be aware of before your decide about whether to participate in this study. There is a small risk that your answers might be intercepted and read by others, but all of the information will be encrypted so that it will be as secure as the SMS you send and any mobile money transactions you undertake normally. We know that elections are contentious in Uganda and that instances of voter intimidation have occurred previously, although only rarely. While we do not expect that participation in this study will pose any additional risks of this kind, we would like you to immediately alert us if you ever have any troubles by sending the message “HELP” to our shortcode or calling our project manager.

If you choose participate, you may receive information that is new to you and that you might find informative as the election approaches. It is hoped that your participation will help us learn about the role of factual information in local elections.

If you have any questions before, during, or after your participation, you may contact the local project manager based in Uganda, Jacob Skaggs at 0780291311, with any questions or concerns. If you have questions regarding your rights as a research participant, please contact the chairperson (Dr. Nabiryo Christini) of the Mildmay Uganda Research Ethics Committee that oversees this kind of research about any concerns at 0392 174 236.

I can answer any of your questions now. Do you have any?

28

Appendix  E Recruitment Survey Instrument

I have read, understood, and received a copy of the above consent and desire of my own free will to participate in this study. [Signature entry field]

1) What is your mobile number?

2) What is your first given name?

3) In which sub-county (LCIII) do you live?

4) In which village (LCI) do you live?

5) What is your gender?

• Male • Female

6) What is your age?

7) How much schooling have you completed?

• No schooling • Some primary school • Completed primary school • Some secondary school • Completed secondary school • Some university

• Completed university (bachelor's degree)

• Some post-graduate school • Completed master's degree • Completed doctor's degree • Refuse to answer

In which language would you prefer to read text messages?

• Alur • Ateso • Japhadhola • Kakwa • Karamojong • Kiswahili • Kumam • Langi • Luganda • Lugbara • Lugisu • Lugwere • Lukhonjo • Lululi • Lumasaba • Lunyole • Lusamia • Lusoga

• Lwamba • Madi • Nubian • Rufumbira • Rukiga • Runyankole • Runyarwanda • Runyolo • Rutooro • Sabiny • English • Other • Refuse

29

Appendix  F  Team Leader Contract for Subject Recruitment and Public Service Audits

By and between Jacob Skaggs (Investigator) referred to as “Investigator,” and _____________________ (Team Leader) referred to as “the Enumerator” on this the 23rd day of November 2015. The Investigator engages the Enumerator as a contractor under the following terms and conditions:

1. Start Date: November 23, 2015

2. DUTIES: The core responsibility of the Enumerator is to conduct subject recruitment in his/her assigned districts. See the research protocol and supporting documents, which detail the specific responsibilities that the Enumerator agrees to take on and complete.

3. COMPENSATION: In consideration of the foregoing, the Investigator shall pay the following sums. From November 30th through December 19th – 15 working days and 3 travel days in total – the Investigator shall pay the Enumerator 80,000 UGX per working day to conduct research activities outlined in the research protocol. On travel days, the Enumerator shall be paid 40,000 UGX. The Enumerator will also receive an additional 14,000 UGX combined for lunch and dinner, and 30,000 UGX for accommodation for each of the working and travel days. Enumerators will receive an additional 10,000 UGX on working days for airtime. On Sundays, the Enumerator will receive 30,000 UGX for accommodation and 14,000 UGX for meals. A full day of research activities for the Enumerator will be defined as supervising his/her assigned Research Assistants (RAs) and overseeing the successful completion of their recruitment responsibilities in compliance with the recruitment protocol. Further, the Enumerator will be responsible for completing all the public service audits required for each enumeration zone in compliance with the audit protocols. The Investigator reserves the right to withhold or reduce pay if the Enumerator’s execution of the work activities does not follow research procedures outlined in the protocol. The enumerator will be receive all per diem expenses – accommodation, meals and airtime - at the beginning of the contract period. The Enumerator will receive his/her full wages at the conclusion of the contract period.

4. DURATION OF CONTRACT: This recruitment and auditing effort is expected to last 15 working days and 3 travel days from November 30th to December 19th. Additional days may be required to complete the assigned work, and these will be dealt with on a case-by-case basis.

5. TERMINATION: This agreement may be terminated upon breach of the agreement by the Enumerator. This agreement may also be terminated if the Enumerator loses, breaks, or otherwise renders inoperative the Samsung Tab4 or Nexus 7 tablet provided by the Investigator to conduct the survey.

6. MISCELLANEOUS: (1) This agreement is undertaken by the Investigator in his private capacities as a researcher and does not implicate Brigham Young University (BYU) or University of California – Santa Barbara, nor is the Enumerator to be considered an employee of any of these institutions. (2) The Enumerator is responsible for his own insurance and taxes. (3) The Enumerator agrees not to disclose any identifiable information about subjects in any form other than the requested data.

In witness whereof, both parties have executed this agreement at Kampala, Uganda, on 23 November 2015.

Investigator Enumerator

Signature:_________ Date __________ Signature:___________ Date__________

30

Appendix  G  Research Assistant Contract for Subject Recruitment

By and between Jacob Skaggs (Investigator) referred to as “Investigator,” and _____________________ (Research Assistant) referred to as “the Enumerator” on this the 23rd day of November 2015. The Investigator engages the Enumerator as a contractor under the following terms and conditions:

1. Start Date: November 23, 2015

2. DUTIES: The core responsibility of the Enumerator is to conduct subject recruitment in his/her assigned districts. See the research protocol and supporting documents, which detail the specific responsibilities that the Enumerator agrees to take on and complete.

3. COMPENSATION: In consideration of the foregoing, the Investigator shall pay the following sums. From November 30th through December 19th – 15 working days and 3 travel days in total – the Investigator shall pay the Enumerator 70,000 UGX per working day to conduct research activities outlined in the research protocol. On travel days, the Enumerator shall be paid 35,000 UGX. The Enumerator will also receive an additional 14,000 UGX combined for lunch and dinner, and 30,000 UGX for accommodation for each of the working and travel days. Enumerators will receive an additional 5,000 UGX on working days for airtime. On Sundays, Enumerators will receive 30,000 UGX for accommodation and 14,000 UGX for meals. Enumerators will work in pairs. A full day of research activities for a pair of Enumerators will be defined as recruiting 50 subjects per enumeration area in compliance with the recruitment protocol, and covering two enumeration areas per working day. The Investigator reserves the right to withhold or reduce pay if the Enumerator’s execution of the work activities does not follow research procedures outlined in the protocol. The enumerator will receive all per diem expenses – accommodation, meals and airtime - at the beginning of the contract period. The Enumerator will receive his/her full wages at the conclusion of the contract period.

4. DURATION OF CONTRACT: This recruitment and auditing effort is expected to last 15 working days and 3 travel days from November 30th to December 19th. If additional time is required to complete the required tasks, additional days may be added on a case-by-case basis.

5. TERMINATION: This agreement may be terminated upon breach of the agreement by the Enumerator. This agreement may also be terminated if the Enumerator loses, breaks, or otherwise renders inoperative the Samsung Tab4 or Nexus 7 tablet provided by the Investigator to conduct the survey.

6. MISCELLANEOUS: (1) This agreement is undertaken by the Investigator in his private capacities as a researcher and does not implicate Brigham Young University (BYU) or University of California – Santa Barbara, nor is the Enumerator to be considered an employee of any of these institutions. (2) The Enumerator is responsible for his own insurance and taxes. (3) The Enumerator agrees not to disclose any identifiable information about subjects in any form other than the requested data.

In witness whereof, both parties have executed this agreement at Kampala, Uganda, on 23 November 2015.

Investigator Enumerator

Signature:___________________ Signature:_____________________

Date:_______________________ Date:_______________________

31

Appendix  H Router, Data and Tablet Management Protocol

Internet Router:

Each internet router has 5 gigabytes of data loaded onto it. This is enough data to synchronize all the KoboToolbox and RoadLab surveys during your three weeks in the field. However, please do not use the data for entertainment purposes as these activities use lots of data. The router ID (SSID) and password (Wifi key) are located on the inside cover of the router. The router battery only lasts about 3-4 hours, so keep it turned off when you are not synchronizing data. To turn it on and off, hold the power button on the top of the router for 3 seconds.

Uploading data using KoboToolbox:

Synchronize the KoboToolbox data each day on each of the tablets. To sync, switch on the WiFi on the tablet and turn on the mobile internet router. Up to 10 tablets can be synced at the same time using the internet router. Open the KoboToolbox app and select send finalized forms on the home screen. There should be a list data entries that are unseleceted. Select toggle all in the lower left hand corner and then select send selected in the lower right corner. The sync should take 1-3 minutes per tablet.

RoadLab:

Synchronize the RoadLab data each day on each of the devices that took RoadLab audits. To do this, switch on the WiFi on the tablet and turn on the internet router. Open the RoadLab app. In the opening screen, you should see a message indicating that the data is automatically synchronizing. Once the sync is finished, another message will indicate that data syncing was successful. This sync should only take several seconds. If you don’t see these messages appear, or the app indicates an error syncing the data, navigate to the settings tab and press synchronize now.

Charging the tablets:

Charge tablets to full capacity every evening. Battery life on the Nexus tablets are 10+ hours, while battery life on the Samsung tablets slightly less. If the power cuts in your hotel, unplug the power strips and tablets until the power returns.

For the Karamojong team, power banks can charge two tablets at once. To start charging, plug in devices and press the green button on the side of the powerbank to start charging.

 

32

Appendix  I Road Audit Sampling Protocol  A. Goals

• Provide a reliable sample of local road quality that is representative at the district level • Provide a sample of road quality that is representative within target enumeration areas • Provide reliable comparisons between similar road types between enumeration areas within a

district • Provide comparable measurements between enumerators, vehicles and tablets

B. Instrumentation

RoadLab in a smartphone app used to measure the surface conditions of roads. The app uses the accelerometer, gyroscope and GPS of a tablet or smartphone to measure and evaluate road roughness using the International Roughness Index (IRI). The app can reliably record road conditions when the vehicle is moving at 20 km/hr or greater. For comparable results using multiple devices, the same model of tablet - Nexus 7 (2013 edition) - were used.

C. Measurement

We will conduct two measurements:

Roughness measurement:

Roughness will be measured using RoadLab which is a tablet based device measuring the level of variation in the evenness of a road. The device will provide a 1-14 measurement where higher values indicate a more rough road.

Visual Inspection:

Enumerators will conduct a visual inspection, noting the condition of features such as road side drainage, shoulders, rail crossing, side walks, culverts and bridges. Using a protocol manual, enumerators will rate such features on a 0-10 scale, which higher values indicate more problematic conditions.

Using these two indexes, we created a condition score.

Condition score = 100 – (Average Value for Distress *5 + Average Value for Roughness * 10)

This condition score will be the primary means for comparing different enumeration areas. One challenge we face is that road types differ between enumeration areas. This is both a measurement challenge and an inferential challenge. At a measurement level, the interval between two condition scores for a paved road is going to be orthogonal to the interval between two condition scores for a dirt road. At an inferential level, it’s not entirely clear whether it is meaningful to compare scores for a dirt road and a paved road as this is likely to correlate very strongly with whether a village is located in a rural or urban area; and thus may not represent a meaningful distinction for voters.

To address these challenges, we propose dividing roads into three categories: (1) asphalt concrete surfaced roads, (2) bituminous surface treated roads, (3) gravel roads, and (4) dirt roads. We will calculate condition scores separately for each of these road types. This will allow us to compare the condition of similar road types across enumeration areas. Or alternatively, this will allow us to weight the index based upon the amount of variation within each road type.

33

Appendix  J Road Audit Enumeration Procedure

Within each enumeration area (village) drive to the location of the largest trading center. You many need to consult with the LC1 or another credible source to identify the largest trading center. If there are multiple trading centers, select the largest one. If more than one trading center exists that are of similar size in the enumeration area, randomly select a trading center using a coin toss.

Identify a local government road (LC3 road) that begins at the trading center and continues towards the largest population center. Confirm with a local government official or other credible source that the road you plan to audit is a local government (LC3) road. The road you audit should not be a national road. Conduct the road audit - including the Roughness Measurement and Visual Inspection - over a distance of 3 km along this road in the most direct route from the trading center to the next population center. Once you have completed auditing 3 km of road, repeat the road audit on the opposite side of the road on the way back to the trading center.

Roughness Measurement:

To take measurements using the RoadLab application, turn on the RoadLab app. In settings, select the district name and village name where you are conducting the audit. Remember to change the district and village name before you take Roughness Measurements in a new village. Next, place the tablet on the floor of the vehicle between your feet. The tablet should be facing up and as close to horizontal as possible. Just before beginning the drive from the trading center towards the identified population center, manually start tracking by pressing the start tracking button in the Roughness Evaluation/Bump Detection tab. You can confirm that the app is tracking by checking if the Roughness Evaluation/Bump Detection tab lists your current speed. Do not pick up or tilt the tablet while Roughness Measurements are being taken. This will cause the app to stop taking measurements. Take Roughness Measurements during the entire 6-km round trip (from the trading center out 3 km, and the 3 km back to the trading center). Once you return to the trading center, press the stop recording button to conclude the Roughness Measurement.

Visual Inspection:

Conduct a Visual Inspection of the road from inside the vehicle while the Roughness Measurements are being taken. For the Visual Inspection, record on a piece of paper the number of major obstacles on the road that force the car to slow down or swerve. However, only record the number of major obstacles you encounter when the automobile is moving at speeds less than 25 km/hr. It may help if you ask the driver to inform you when the vehicle is moving at speeds below 25 km/hr. Record major obstacles for the entire 6-km round trip. These obstacles might include any features -- such as potholes, washed out shoulders (a road that has been narrowed), stagnant water, uncleared vegetation – that cause the vehicle to slow down or swerve.

Also, when you reach the 3 km turning point, record an estimate of the width of the entire road in meters.

Once you return to the trading center after completing the entire Roughness Measurement, start the KoboToolbox app, open the VoteChoice_Roads survey, and answer all the question in the survey.

Instrument Calibration Procedure for RoadLab (you only need to do the following steps once)

1. Open the RoadLab app

34

2. First you will be signing into the RoadLab application. In the settings tab, select employee sign in. For email, type in your surname. For password, type in pwd123. For example, I am Jacob Skaggs. My username is skaggs, and my password is pwd123.

3. In the settings tab, select Suspension Type 4. Select Medium for a 4-door subcompact car such as a Toyota Corolla, or Hard for a mid-sized 4-

wheel drive vehicle such as a Toyota Prado. 5. Lastly, in the settings tab, make sure Automatic detection is turned off

 

35

Appendix  K Update on Road Audit Enumeration Procedure

If the road is not passable by car, team leaders conducted a visual inspection by foot and enter information into the KoboCollect road audit survey. To conduct the visual inspection by foot, walk down the selected sub-county road for 5 minutes, then return to the trading center. During the walk, take note of the following:

1) approximate width of the path at the trading center and turn-around point,

2) any new construction or road/path improvements (if any)

3) the general quality of the road

Once you finish the walk, open up the KoboCollect and answer all the questions that apply. For the question, What type of road is the one being audited?, enter “other”, and describe the road on the next data entry page. For example, you might describe the road as a footpath, or extremely muddy road, or steep muddy road. Mark 0 for the question regarding major obstacles. Answer the rest of the questions based on your visual inspection during your walk. On the last data entry page, indicate that you had issues using RoadLab, and briefly explain why the road was impassable by car.

 

36

Appendix  L Road Audit Survey Instrument

Koboconnect Visual Inspection Survey

Team Leader’s first name: ____________________

Name of village (as listed on the village assignment sheet):______________________

Name of the trading center where the road audit begins: _____________________

Name of the nearest population center where the sub-county road leads: _____________________

1.1. What type of road is the one being audited?

a. tarmac

b. marram

c. dirt

d. Could not determine. Explain ___________________________________

1.2. How many major obstacles did you encounter on the road over the entire 6-km round trip drive at speeds less than 25 km/hr? These include any features – such as potholes, washed out shoulders, standing water, uncleared vegetation – that forced you to slow down or swerve.

a. _____________ obstacles

d. Could not determine. Explain ___________________________________

1.3. On a scale of 1 to 10 (1 being the best and 10 being the worst), how well would you rate the maintenance of this road relative to other roads you have seen in this district.

_____________

b. Could not determine. Explain ___________________________________

1.4. What is the approximate width of the road in meters at the starting point (trading center)?

a. _______________ meters

d. Could not determine. Explain ___________________________________

1.5. What is the approximate width of the road in meters at the turn-around point (3 km from trading center)?

a. _______________ meters

d. Could not determine. Explain ___________________________________

1.6. Did you notice any ongoing construction or improvement work on this stretch of road?

a. Yes, explain ________________

b. No

37

d. Could not determine. Explain ___________________________________

1.7. Did you notice any recently completed construction or improvement work on this stretch of road?

a. Yes, explain ________________

b. No

d. Could not determine. Explain ___________________________________

1.8. Did you have any challenges operating or calibrating the RoadLab application?

No

Yes, please explain the problem in detail __________________________________________________________________________________________________________________________________________________________________________

   

38

Appendix  M Water Access Audit - Sampling Protocol

A. Goals

• Provide a reliable measure of access to a protected public water point from a point of significance to the local community (a primary school, the largest market, or village) at the village level. This paper discusses several options.

• Provide reliable comparisons between access to protected public water points across villages within a district

• Provide comparable measurements between enumerators • Water access from the home of the 3rd subject recruited

B. Instrumentation

A smart phone/tablet with GPS and a timer/clock.

C. Measurement

We could conduct two to three measurements. One of the measurements would be either time or distance. The second would be cost.

1. Time

At a common time of day, the number of minutes needed to walk to the protected public water point, reach the protected public water point (if there is a line, to reach the front of it), and return to the starting point (market) will be recorded using the clock/stopwatch on the smartphone/tablet. During training, enumerators would use their normal cadence to walk a set distance (for example, 50 meters) and record the time needed to complete it.

2. Distance (an alternative to time).

Distance would be measured by using the GPS to locate the market (Option 2) or the LCI office (Option 3) and then the closest protected public water point (shallow well, borehole, tap water, spring, dam..). A produce seller at the market (Option 2) or the village elder/LCI counselor or his/her staff could be asked about directions to the nearest protected public water point. The number of meters from the market/ to the protected public water point and back would be calculated using the GPS coordinates.

3. Cost

Enumerators will record if the water is free, or if there is a fee, the amount of the fee/quantity.

The SCORE/total measure for options 2 & 3 would be an index that combines of the above measures (time/distance & cost).

39

Appendix  N Water Access Audit Enumeration Procedure

Within each village, ask the third person that is recruited to take you to their neighborhood. Ask the subject for directions to the nearest protected public water access point that is currently functioning. A protected water access point can include a borehole, well (deep hole with a bucket, or pipe delivering water from a spring), tap, or spring box. Start the timer on your phone when you leave the subject’s neighborhood and walk at a normal walking speed to the water access point. Make sure to walk all the way to the water access point. While at the water access point, quickly inquire about the fee amount if the water is not free. Quickly count the number of people standing in line (if any), return to the starting point at the subject’s neighborhood, and stop the timer. Make sure you record the time required to complete the entire walk from the neighborhood to the water access point and back to the neighborhood. Open up the KoboToolbox app and answer all the questions in the VoteChoice_WaterAccess survey.

 

40

Appendix  O Water Access Survey Instrument

Team Leader first name: ____________________

Village name (as listed on the village assignment sheet): ______________

Type of protected water source: a) borehole, b) well, c) tap, d) springbox, e) other (explain) _______________

Total time to walk to the water access point and return to the subject’s neighborhood: _______ minutes

Number of people waiting in line at the water access point at the time of arrival: ___________

Cost (if any) for the price/20 litre jerry can: __________________

GPS of subject’s neighborhood

 

41

Appendix  P Health Center Audit - Sampling Protocol

A. Goals

• Provide a reliable sample of local health care quality that is representative at the district level • Provide a sample of local health care quality that is representative within target enumeration areas • Provide reliable comparisons between similar local health care institutions between enumeration

areas within a district • Provide comparable measurements between enumerators

B. Instrumentation

This information will be collected via in-person audits using tablets.

C. Measurement

We will conduct three measurements:

• Drug Supply: Seek out the Health Unit Management Committee (HUMC) at the dispensary. Examine stock cards for the following drugs: erythromycin, chloroquine, septrine, quinine, and mebendazole. Write down the total number of drugs that are currently missing (0-5). If accurate stock cards are not available, count all drugs as missing (5).

• Waiting Time: Over a one-hour period, survey all individuals who are leaving the facility about when they arrived and how long their examination was. The waiting time is the difference between the time the user left the facility and the time the user arrived at the facility, subtracting the examination time.

• Clinic Cleanliness: On a three-point scale in which 1 = clean, 2 = neither particularly clean nor dirty, 3 = dirty, enumerators rate the cleanliness of the health clinic’s main reception and waiting area for the following: (1) floors, (2) walls, (3) furniture, (4) smell.

Using these three indexes, we will create a condition score.

Condition score = 100 – (Average Value for Waiting Time + Drug Supply + Clinic Cleanliness * 10)

42

Appendix  Q Health Center Audit - Enumeration Procedure

Conduct the following three measurements and record the results in KoboCollect using the VoteChoice_Healthcare survey:

• Drug Supply: Seek out the dispensary within the Health Centre 3. Ask a dispensary staff member to show you what drugs are available in the following drug categories: antimalarial, antibiotic (e.g., erythromycin, septrine), deworming, anti-retroviral, pre-natal vitamins. For each category list whether or not the drug is available or unavailable. Record results in Koboconnect.

• Clinic Cleanliness: On a three-point scale in which 1 = clean, 2 = neither particularly clean nor dirty, 3 = dirty, rate the cleanliness of the health clinic’s main reception and waiting area, and nearest washroom (for your gender) for the following: (1) floors, (2) walls, (3) furniture (facilities in the washrooms), (4) smell. Record your results in Koboconnect while you are auditing cleanliness.

• Waiting Time: Over a 30-minute period – only on Mondays through Fridays and after 10 am – survey all individuals who are leaving the Health Centre asking them how long they waited for their examination. Record results in Koboconnect while you are auditing waiting time.

 

43

Appendix  R Water Access Survey Instrument

Team Leader’s first name: ____________________

Name of the District: _______________________

Name of the Sub-county: ____________________

Name of the Health Centre 3: _____________________

GPS of the dispensary: _____________________

Considering the following drug categories – antimalarial, antibiotic, deworming, anti-retroviral, pre-natal vitamins – which of these drugs were available or missing from the dispensary?

Antimalarial _____ Available _____ Unavailable

Antibiotic (erythromycin, septrine) _____ Available _____ Unavailable

Deworming _____ Available _____ Unavailable

Anti-retroviral _____ Available _____ Unavailable

Pre-natal vitamins _____ Available _____ Unavailable

6) How clean were the floors of the main reception and waiting area? 1 = clean, 2 = neither particularly clean nor dirty, 3 = dirty.

1 - Clean

2 - Neither particularly clean nor dirty

3 - Dirty

Unable to determine. Reason: _______________________

7) How clean were the walls of the main reception and waiting area? 1 = clean, 2 = neither particularly clean nor dirty, 3 = dirty.

1 - Clean

2 - Neither particularly clean nor dirty

3 - Dirty

Unable to determine. Reason: _______________________

8) How clean were the furniture of the main reception and waiting area? 1 = clean, 2 = neither particularly clean nor dirty, 3 = dirty.

1 - Clean

2 - Neither particularly clean nor dirty

3 - Dirty

44

Unable to determine. Reason: _______________________

9) What was the smell of the main reception and waiting area? 1 = good, 2 = neither particularly good nor bad, 3 = bad.

1 - Good

2 - Neither particularly good nor bad

3 - Bad

Unable to determine. Reason: _______________________

10) How clean were the floors of the washroom? 1 = clean, 2 = neither particularly clean nor dirty, 3 = dirty.

1 - Clean

2 - Neither particularly clean nor dirty

3 - Dirty

Unable to determine. Reason: _______________________

11) How clean were the walls of the washroom? 1 = clean, 2 = neither particularly clean nor dirty, 3 = dirty.

1 - Clean

2 - Neither particularly clean nor dirty

3 - Dirty

Unable to determine. Reason: _______________________

12) How clean were the facilities of the washroom? 1 = clean, 2 = neither particularly clean nor dirty, 3 = dirty.

1 - Clean

2 - Neither particularly clean nor dirty

3 - Dirty

Unable to determine. Reason: _______________________

13) What was the smell of the washrooms? 1 = good, 2 = neither particularly good nor bad, 3 = bad.

1 - Good

2 - Neither particularly good nor bad

3 - Bad

Unable to determine. Reason: _______________________

14) What was the wait time of the patient #1 in minutes?

45

a) ___________________

15) What was the wait time of the patient #2 in minutes?

a) ___________________

16) What was the wait time of the patient #3 in minutes?

a) ___________________

17) What was the wait time of the patient #4 in minutes?

a) ___________________

18) What was the wait time of the patient #5 in minutes?

a) ___________________

19) What was the wait time of the patient #6 in minutes?

a) __________________

20) What was the wait time of the patient #7 in minutes?

a) ___________________

21) What was the wait time of the patient #8 in minutes?

a) ___________________

22) What was the wait time of the patient #9 in minutes?

a) ___________________

23) What was the wait time of the patient #10 in minutes?

a) ___________________

   

46

Appendix  S Protocol for Baseline Call Center

Overview of call center:

Each call center member will be assigned a list of subjects to contact and administer a baseline survey. You will use a hard copy of this list to record your progress and write any notes to help you if you should need to re-contact them. You will be provided airtime on your personal phone to purchase voice bundles, which you will use to make the calls. If you would like to purchase an extra sim card to conduct the calls, you will be reimbursed for it. Initially, your team leader will send you 30,000 UGX of airtime to purchase an MTN or Airtel voice bundle for 1000 minutes. When your minutes run out, your team leader will send another 30,000 UGX to purchase an additional voice bundle, etc. Below are voice bundle codes:

Airtel: *149*10*2# - 30K for 1000 minutes

MTN: *160*23# - 30K for 1000 minutes

You will be using a separate smartphone or tablet to record answers from subjects. Your team leader will help you install Kobocollect, the app that will be used to record responses. To navigate to the survey, open the Kobocollect app, select Fill Blank Form, and select VoteChoice_CallCenter. Swipe to the right or left to move forward or backward between questions.

You are also provided a translated baseline survey that you will read from during the call. Read the survey questions exactly as they are written on the survey. Avoid making comments or reading the questions in a way that might influence the way subjects respond.

You will use your own phone to make calls, another tablet or smartphone to record answers, and the contact sheet to keep track of the calls you make.

Making calls:

In the first entry field, enter your first name. Next, use the contact list to enter answers into Kobocollect for the next two question: ID number and phone number. Enter this information exactly as it appears on the contact list. Next call the subject. Use the introduction script to introduce yourself. If the subject declines to participate in the survey, mark in KoboCollect that the individual declined. If the subject is not available or you are asked to call at a different time, delete the entry information you made and reenter information for the next subject that you will call. On the contact list, put a tick mark in the attempt box and document any notes that will help you re-contact that person at a later time. (Recording the date and time makes it easy to relocate a partially completed survey). You will attempt to contact each subject three times, perhaps trying at different times of the day or different days. If for any reason you will never be able to conduct the survey with a specific subject, select Other reason the subject can’t participate and document the reason. If the subject agrees, start conducting the baseline survey.

Each question on your hard copy of the baseline survey corresponds to an entry field in Kobocollect. As you ask questions, mark the answers in Kobocollect. Read the entire questions and offer all possible answers except don’t know and refused to answer. Subjects do have the option to refuse (you mention this in the introduction), but we don’t want to encourage them to refuse.

Questions 20a through 20f will only be asked depending on how the subject answers question 18. For example, in question 18 if the subject indicates that primary schools are the most important public

47

services, you will only ask questions 20a and 20b. Kobocollect will only display the appropriate question depending on how the subject answered question 18.

If you are unable to complete the survey for any reason, press the back button on the tablet and select Save Changes. Once you can reconnect with the subject, you can find the partially completed form in the Edit Saved Forms folder on the main menu screen.

Make sure to thank the subject for their time and let them know they will be reimbursed 1000 UGX the following morning.

Technology (for Team Leaders):

• Downloading the Kobocollect app and uploading the survey • Download and install the Kobocollect app from the Google Play store • Open the app and press the setting button (three dots in upper right corner) • Select General Settings • Under Server Settings, press URL • Enter the following URL: https://kc.kobotoolbox.org/mbuntaine/ • Press the back button to navigate back to the main menu • Select Get Blank Form • Select only VoteChoice_CallCenter, and press Get Selected at the bottom of the screen • Now when you press Fill Blank Form, the VoteChoice_CallCenter should appear. • Call me if you have issues: 0780291311

Synchronizing data Kobocollect:

• Synchronize data the end of each day. This information will be used each day to reimburse subjects for 1000 UGX.

• To synchronize data, navigate to the main menu and select Send Finalized Form. Next, select Toggle All, and submit finalized forms. The tablet or smartphone must be connected to the internet to sync.

Router management:

• 3 GBs of data have been loaded onto each router, which should be more than enough to upload information from Kobocollect.

• The username and password of the router is written on the inside cover of the router.

 

48

Appendix  T Baseline Call Center - Team Leader Contract

By and between Jacob Skaggs (Investigator) referred to as “Investigator,” and _____________________ (Team Leader) referred to as “the Enumerator” on this the 8th day of January 2016. The Investigator engages the Enumerator as a contractor under the following terms and conditions:

1. Start Date: January 8, 2016

2. DUTIES: The core responsibility of the Enumerator is to manage a call center team and contact subjects in his/her assigned districts and administer a baseline survey. See the call center protocol and supporting documents, which detail the specific responsibilities that the Enumerator agrees to take on and complete.

3. COMPENSATION: In consideration of the foregoing, the Investigator shall pay the following sums. From January 11-16 – 6 working days in total – the Investigator shall pay the Enumerator 80,000 UGX per working day to manage a call center outlined in the call center protocol. The Enumerator will also receive an additional 24,000 UGX per diem for meals, airtime and transport for each full day of research activities. A full day of research activities for the Enumerator will be defined as supervising his/her assigned Research Assistants (RAs) and overseeing the successful completion of their call center responsibilities in compliance with the call center protocol. The enumerator will receive an advance of 100,000 UGX to cover the costs of hiring a venue for making calls. Further, the enumerator will receive an advance for his/her team to purchase voice bundles for making calls. Receipts and any balance will be collected for the advances at the conclusion of the call center. Further, the Enumerator will be responsible for the successful completion of all the call center objectives in compliance with the call center protocols. The Investigator reserves the right to withhold or reduce pay if the Enumerator’s execution of the work activities does not follow the procedures outlined in the call center protocol. The enumerator will receive all per diem expenses at the beginning of the contract period. The Enumerator will receive his/her full wages at the conclusion of the contract period.

4. DURATION OF CONTRACT: This recruitment and auditing effort is expected to last 6 working days from January 11-16. Additional days may be required to complete the assigned work, and these will be dealt with on a case-by-case basis.

5. TERMINATION: This agreement may be terminated upon breach of the agreement by the Enumerator. This agreement may also be terminated if the Enumerator loses, breaks, or otherwise renders inoperative the Samsung Tab4 or Nexus 7 tablet provided by the Investigator to conduct the survey.

6. MISCELLANEOUS: (1) This agreement is undertaken by the Investigator in his private capacities as a researcher and does not implicate University of California Santa Barbara, London School of Economics, College of William & Mary, Brigham Young University or Temple University, nor is the Enumerator to be considered an employee of any of these institutions. (2) The Enumerator is responsible for his own insurance and taxes. (3) The Enumerator agrees not to disclose any identifiable information about subjects in any form other than the requested data.

In witness whereof, both parties have executed this agreement at Kampala, Uganda, on 07 January 2016.

Investigator Enumerator

Signature:________ Date__________ Signature:_________ Date___________

49

Appendix  U Baseline Call Center - Research Assistant Contract

Research Assistant/Contractor Agreement

By and between Jacob Skaggs (Investigator) referred to as “Investigator,” and _____________________ (Research Assistant) referred to as “the Enumerator” on this the 8th day of January 2016. The Investigator engages the Enumerator as a contractor under the following terms and conditions:

1. Start Date: January 8, 2016

2. DUTIES: The core responsibility of the Enumerator is to contact subjects in his/her assigned districts and administer a baseline survey. See the call center protocol and supporting documents, which detail the specific responsibilities that the Enumerator agrees to take on and complete.

3. COMPENSATION: In consideration of the foregoing, the Investigator shall pay the following sums. From January 11-16 – 6 working days in total – the Investigator shall pay the Enumerator 70,000 UGX per working day to administer a baseline survey outlined in the call center protocol. The Enumerator will also receive an additional 19,000 UGX per diem for meals and transport for each full day of research activities. A full day of research activities for the Enumerator will be defined as contacting subjects to administer a baseline survey and meeting the calling goals provided the Team Leader in compliance with the call center protocol. The Investigator reserves the right to withhold or reduce pay if the Enumerator’s execution of the work activities does not follow the procedures outlined in the call center protocol. The enumerator will receive all per diem expenses at the beginning of the contract period. The Enumerator will receive his/her full wages at the conclusion of the contract period.

4. DURATION OF CONTRACT: This recruitment and auditing effort is expected to last 6 working days from January 11-16. Additional days may be required to complete the assigned work, and these will be dealt with on a case-by-case basis.

5. TERMINATION: This agreement may be terminated upon breach of the agreement by the Enumerator. This agreement may also be terminated if the Enumerator loses, breaks, or otherwise renders inoperative the Samsung Tab4 or Nexus 7 tablet provided by the Investigator to conduct the survey.

6. MISCELLANEOUS: (1) This agreement is undertaken by the Investigator in his private capacities as a researcher and does not implicate University of California Santa Barbara, London School of Economics, College of William & Mary, Brigham Young University or Temple University, nor is the Enumerator to be considered an employee of any of these institutions. (2) The Enumerator is responsible for his own insurance and taxes. (3) The Enumerator agrees not to disclose any identifiable information about subjects in any form other than the requested data.

In witness whereof, both parties have executed this agreement at Kampala, Uganda, on 07 January 2016.

Investigator Enumerator

Signature:___________________ Signature:_____________________

Date:_______________________ Date:_________________________

 

50

Appendix  V  Baseline Call Center Survey Instrument

Introduction

Hello, is this [respondent first name]? (1) yes, (2) no, (3) decline to answer

This is [name of RA] working with Twaweza and their partners. A colleague of mine recently visited you in your village and asked you to sign up to receive information about public services in your village. I would like to ask you a few questions before we start sending information. This survey will take about 10 minutes. You will receive 1000 shillings in airtime if you finish the survey, thought you can skip any questions that you do not want to answer. Your answers are completely confidential and we will not tell anybody about them. We are not affiliated with any political party and only seek to understand how people use information in elections. Is okay that I ask you a few questions? (1) yes, (2) no

[If yes, continue survey. If no, thank the person for their time and wish them well.]

[Important instructions for research assistants: RAs should not read the “refuse” and “don’t know” options when reading the script -- please only read response options that are bolded]

Introduction

(Q1) In general, how do you rate your living conditions compared to other Ugandans? Would you say they are (1) much worse, (2) worse, (3) the same, (4) better, or (5) much better? (8) don’t know, (9) refused to answer

(Q2) Would you say that you share the same tribe as your current LC5 councilor? (1) yes, (2) no, (8) don’t know, (9) refused to answer

Pre-treatment outcomes (intentions)

(Q3) Do you intend to vote in the upcoming LC5 election? (1) yes, (2) no, (3) not sure, (9) refused to answer

(Q4) Do you intend to vote in the upcoming LC3 election? (1) yes, (2) no, (3) not sure, (9) refused to answer

(Q5) If the LC5 election were held today, would you vote to reelect the current councilor or to elect a councilor of the same party? (1) yes, (2) no, (3) not sure, (9) refused to answer

(Q6) If the LC5 election were held today, would you vote to reelect the current chairperson or to elect a chairperson of the same party? (1) yes, (2) no, (3) not sure, (9) refused to answer

(Q7) If the LC3 elections were held today, would you vote to reelect the current councilor or to elect a councilor of the same party? (1) yes, (2) no, (3) not sure, (9) refused to answer

(Q8) If the LC3 election were held today, would you vote to reelect the current chairperson or to elect a chairperson of the same party? (1) yes, (2) no, (3) not sure, (9) refused to answer

Controls and Moderators:

(Q9) Generally speaking, what party do you identify with most? (1) National Resistance Movement, (2) Forum for Democratic Change, (3) Democratic Party, (4) Uganda People’s Congress, (5) other [please state], (6) not aligned, (9) refused to answer

51

[Only ask the next question if the response to Q9 is “other”]

(Q9a) What other party do you identify with most?

(Q10) On a scale of one to seven, where seven means you identify very much with [political party indicated in Q9], and one means you identify very little with [response to Q9], how much do you identify with [political party indicated in Q9]? (8) not applicable, (9) refused to answer

(Q11) Did you vote for the current councilor in the 2011 LC5 elections? (1) yes, (2) no, (3) did not participate in the LC5 councilor election, (8) don’t know, (9) refused to answer

(Q12) Did you vote for the current chairperson in the 2011 LC5 elections? (1) yes, (2) no, (3) did not participate in the LC5 chairperson election, (8) don’t know, (9) refused to answer

(Q13) Did you vote for the current councilor in the 2011 LC3 elections? (1) yes, (2) no, (3) did not participate in the LC3 councilor election, (8) don’t know, (9) refused to answer

(Q14) Did you vote for the current chairperson in the 2011 LC3 elections? (1) yes, (2) no, (3) did not participate in the LC3 chairperson election, (8) don’t know, (9) refused to answer

(Q15) How likely is it that the current LC3 or LC5 councilor, or someone from their party will offer something like food or a gift in return for your vote in the upcoming election? (1) very unlikely, (2) somewhat unlikely, (3) somewhat likely, (4) very likely, (8) don’t know, (9) refused to answer

Salience & Source:

(Q16) When you decide how to vote for your LC5 chairperson and councilor elections, how important is it that the district manages its budget well, for example by accounting for expenditures and preventing fraud? (1) not important, (2) not very important, (3) somewhat important, (4) very important, (8) don’t know, (9) refused to answer

(Q17) When you decide how to vote for your LC5 chairperson and councilor, how important is the quality of public services like primary schools, water access, and health clinics? (1) not important, (2) not very important, (3) somewhat important, (4) very important, (8) don’t know, (9) refused to answer

(Q18) I am now going to give you a list of public services that your LC5 and LC3 councils are involved in providing. When you decide how to vote for these officials, which of these public services are most important to you? (1) primary schools, (2) water access, (3) local roads, (4) local health services, (8) don’t know, (9) refused to answer

(Q19) Which of these services matter to you the second most? (1) primary schools, (2) water access, (3) local roads, (4) local health services, (8) don’t know, (9) refused to answer

Priors on Treatment Information:

[Only ask the following two questions if primary schools are most important issue]

(Q20a) If you compare your LC5 performance in managing the quality of the government primary schools in your villages to other villages in your district how do you think it will compare? (1) much better, (2) better, (3) a little worse, (4) much worse, (8) don’t know, (9) refused to answer

(Q20b) How certain are you about your response to this question? (1) very certain, (2) certain, (3) not certain, (4) very uncertain, (8) don’t know, (9) refused to answer

52

[Only ask the following two questions if roads are the most important issue]

(Q20c) How do you think the quality of the community roads provided by your LC3 in your village compare to the quality of community roads provided to most other villages in your district? (1) much better, (2) better, (3) a little worse, (4) much worse, (8) don’t know, (9) refused to answer

(Q20d) How certain are you about your response to this question? (1) very certain, (2) certain, (3) not certain, (4) very uncertain, (8) don’t know, (9) refused to answer

[Only ask the following two questions if water is the most important issue]

(Q20e) How do you think the water access provided by your LC5 and LC3 in your village compares to the average water access provided to other villages in your district? (1) much better, (2) better, (3) a little worse, (4) much worse, (8) don’t know, (9) refused to answer

(Q20f) How certain are you about your response to this question? (1) very certain, (2) certain, (3) not certain, (4) very uncertain, (8) don’t know, (9) refused to answer

(Q21) Now I am going to give you a list of places where people commonly receive information about the public services provided by LC5 and LC3 officials. Please tell us how much trust you have in the information you receive from each of these sources.

(Q21a) Local politicians: (1) trust a lot, (2) trust a little, (3) do not trust at all, (8) don’t know, (9) refused to answer

(Q21b) Twaweza: (1) trust a lot, (2) trust a little, (3) do not trust at all, (8) don’t know, (9) refused to answer

(Q21c) Non-governmental Organizations, often called NGOs: (1) trust a lot, (2) trust a little, (3) do not trust at all, (8) don’t know, (9) refused to answer

(Q21d) The Uganda Auditor General: (1) trust a lot, (2) trust a little, (3) do not trust at all, (8) don’t know, (9) refused to answer

Priors on Budget Treatment Information:

(Q22) If you compare your LC5’s record of managing its budget expenditures and contracting to other districts in Uganda how do you think it will compare? (1) much better, (2) better, (3) a little worse, (4) much worse, (8) don’t know, (9) refused to answer

(Q23) How certain are you about your response to this question? (1) very certain, (2) certain, (3) not certain, (4) very uncertain, (8) don’t know, (9) refused to answer

Election-level features:

(Q24) How likely do you think it is that powerful people can find out how you vote, even though there is supposed to be a secret ballot in Uganda? (1) Not at all likely, (2) Not very likely (3) Somewhat likely, (4) Very likely, (8) don’t know, (9) refused to answer

(Q25) How likely do you think it is that the counting of votes in this election will be fair? (1) Not at all likely, (2) Not very likely (3) Somewhat likely, (4) Very likely, (8) don’t know, (9) refused to answer

 

53

Appendix  W

Protocol for Calculating Road Index Scores

First, standardize the variables as follows:

• RM = Road maintenance score (1-10 with 1 best) • RW = Average road width (start + turnaround point/2)*-1 • RC = ongoing or completed construction (0 to 1) *-1 • RO = major obstacles (count) • RL = RoadLab 1-14 measure (1 is best)

For each village i and district j calculate the within district standardized z score for each variable:

𝑧_𝑣𝑎𝑟 = !!!(𝑋! − 𝑋!).

This calculation standardizes each variable such that each z_variable equals number of standard deviations above or below the mean in each district.

Calculate the condition score as follows:

Condition score = z_RL*0.5 + z_RM*0.25 + z_RC*0.05 + z_RO*0.15 + z_RW*0.05

Since we’ve standardized each variable, the condition score equals how much worse (in weighted standard deviations) the village is compared to the average for other villages in the district.

The weighting is such that 50% of the condition score is determined by road roughness, 30% by maintenance and construction and 20% by obstacles and width.

In cases where we are missing data, reweight the variables equally such that they add up to one. For instance, most commonly we will be missing data on RoadLab. In this case the condition score equals z_RM*0.5 + z_RC*0.1 + z_RO*0.3 + z_RW*0.1.

 

   

54

Appendix  X Protocol for Calculating Average Education Score

Using Twaweza’s 2014 household-level Uwezo survey database (provided by Twaweza).

1. First, narrow database to include students only grades 3-7, to be consistent with format of the public information provided by Uwezo. The average English literacy and numeracy score at the Primary 2 level is for Primary 3-7 pupils of a village, out of 10.

2. Using 2014 Uwezo household-level data on • L = English Literacy score (variable english: 1=nothing, 2=letter, 3=word, 4=paragraph, and 5 =

story. Recode English Literacy: 1-4 =0, not full competence; 5=1, full competence); • N = Numeracy score (variable math: 1=Nothing, 2=Counting, 3=Numbers, 4=Addition,

5=Subtraction, 6=Multiplication; 7=Divide. Recode Numeracy: 1-6=0, not full competence; 7=1, full competence).

• A = Absenteeism score (variable attendYesterday , 1=subject attended school on day before household survey was conducted; 0= did not attend. Generate absenteeism by reversing coding for attendYesterday: 0=attend; 1=absent.)

3. For each village i and district j calculate the within district standardized z score for each variable:

𝑧_𝑣𝑎𝑟 = !!!(𝑋! − 𝑋!).

This calculation standardizes each variable such that each z_variable equals number of standard deviations above or below the mean in each district.

4. Calculate the condition score as follows:

Condition score (Overall = z_L*0.4 + z_N*0.4 - z_A*0.2)

Since we’ve standardized each variable, the condition score equals how better or worse (in weighted standard deviations) the village is compared to the average for other villages in the district.

5. To compute the quartiles, the egenmore package at: http://fmwww.bc.edu/RePEc/bocode/e/ was installed; A village's OVERALL EDUCATION Treatment (based on OverallEduc_quartile), where much worse=1, a little worse=2, better=3, much better=4)

   

55

Appendix  Y Protocol for Calculating Water Access Index Scores

1. We Winsorized the data first at the 1st and 99th percentiles. (This required installing the winsor2 Stata package from (http://fmwww.bc.edu/RePEc/bocode/w/)

2. We computed z scores for each of the components (walk time, number of people waiting at water point, and fees per jerry can) by district.

3. By district, using the means and standard deviations, we calculated quartile rankings for each of the components – walk time, people waiting, and fees – and the index variable by combining the three component scores.

4. So 1=much worse, 2=a little worse, 3=better, 4=much better.

5. 20 of the 729 villages had no values for any of the three components. If subjects in these villages indicated water access as their most important public service, these subjects were sent messages on their second-most important public services, indicated in the baseline survey. If data from the second-most important public service was missing, subjects were sent messages on a randomly selected public service.  

   

56

Appendix  Z Protocol for Calculating Average Health Center Score

1. First, standardize the variables as follows:

• D = Drug score (0-5 with 5 best)*-1 • C = Cleanliness score (0-16 with 16 best)*-1 • W = Wait time (0 to 336 with 0 best)

2. For each village i and district j calculate the within district standardized z score for each variable:

𝑧_𝑣𝑎𝑟 = !!!(𝑋! − 𝑋!).

This calculation standardizes each variable such that each z_variable equals number of standard deviations above or below the mean in each district.

3. Calculate the condition score as follows:

Condition score = z_D*0.4 + z_C*0.4 + z_W*0.2

Since we’ve standardized each variable, the condition score equals how much worse (in weighted standard deviations) the village is compared to the average for other villages in the district.

The weighting is such that 40% of the condition score is determined by drug availability, 40% by cleanliness and 20% by wait time.

4. In cases where we are missing data, reweight the variables equally such that they add up to one. For instance, most commonly we will be missing data on wait time. In this case the condition score equals z_D*0.5 + z_C*0.5.

5. We propose the following rule to translate individual outcomes into overall assessment:

• If the condition score falls between the 1st and 39th percentiles: better than average • If the condition score falls between the 40th and 60th percentiles: about the same • If the condition score falls between the 61st and 100th percentiles: worse than average

 

 

 

57

Appendix  AA  Treatment and Placebo Messages Template

Language translation: enter the language

Treatment Messages - Budget

General Opening Message #1: Dear [Name], welcome to the Twaweza Information Service. We enjoyed meeting you and talking on the phone.

Day 1.2 Over the next few days we will send you [and many of your neighbors] messages about [public services and] your LC5’s budget.

Day 1.3: The Auditor General conducts yearly audits to record instances where LC5s could not satisfactorily explain how its money has been spent.

Day 1.4: Unexplained spending is often an indicator of mismanagement, fraud or poor quality services.

Day 1.5. Your LC5 did [much better] than most other LC5s in the recent audit.

Day 2.1. In your LC5, the auditor found issues with [21.3 million] UGX from its budget of [7.5] billion UGX. This is [much better] than in other districts.

Day 2.2. This means that [3] out of 1000 UGX in your LC5 budget had issues. In most LC5s [5] out of 1000 UGX had issues. Your LC5 did [much better] than average.

Day 3.1. One reason your LC5 did [much better] than average is that [payments on contracts were explained very well].

Day 3.1. One reason your LC5 did [better] than average is that [it only had a small amount of unexplained spending].

Day 3.1. One reason your LC5 did [much worse] than average is that [four unqualified firms were awarded contracts].

Day 3.1. One reason your LC5 did [much worse] than average is that [111 million UGX in spending could not be explained well enough].

Day 3.1. One reason your LC5 did [much worse] than average is that [14.5 million UGX in spending could not be explained well enough].

Day 3.1. One reason your LC5 did [much worse] than average is that [147 million UGX in administrative spending could not be explained well enough].

Day 3.1. One reason your LC5 did [much worse] than average is that [payments of 98 million UGX were made without proper documentation].

Day 3.1. One reason your LC5 did [much worse] than average is that [payments of 130 million UGX to councilors and government staff could not be explained].

Day 3.1. One reason your LC5 did [a little worse] than average is that [14 employees were overpaid by 65 million UGX].

58

Day 3.1. One reason your LC5 did [a little worse] than average is that [contracts worth 133 million UGX were not properly advertised].

Day 3.1. One reason your LC5 did [a little worse] than average is that [contracts for 100 million UGX were not opened to competitive bidding].

Day 3.1. One reason your LC5 did [a little worse] than average is that [20 million UGX paid out for various activities lacked supporting documents].

Day 3.1. One reason your LC5 did [a little worse] than average is that [158 million UGX was paid to road construction teams without satisfactory work records].

Day 3.1. One reason your LC5 did [a little worse] than average is [that 50 million UGX paid for various activities could not be explained well enough].

Day 3.1. One reason your LC5 did [a little worse] than average is that [53 million UGX in administrative spending could not be explained well enough].

Day 3.1. One reason your LC5 did [a little worse] than average is that [payments of 65 million UGX were not documented well enough].

Day 3.1. One reason your LC5 did [a little worse] than average is that [payments of 75 million UGX were not documented well enough].

Day 3.2. Another reason your LC5 did [better] than average is that [it could explain most of its expenses for payments made to staff].

Day 3.2. Another reason your LC5 did [better] than average is that [all payments made to staff were explained very well].

Day 3.2. Another reason your LC5 did [much worse] than average is that [119 district staff were overpaid].

Day 3.2. Another reason your LC5 did [much worse] than average is that [it could not be determined if funds were used for their intended purpose].

Day 3.2. Another reason your LC5 did [much worse] than average is that [road maintenance contracts for 14 million UGX were not properly approved].

Day 3.2. Another reason your LC5 did [much worse] than average is that [it could not be determined if funds were used properly].

Day 3.2. Another reason your LC5 did [much worse] than average is that [a bid for borehole construction included expenses that could not be explained].

Day 3.2. Another reason your LC5 did [much worse] than average is that [contracts totaling 199 million UGX were awarded without competitive bidding].

Day 3.2. Another reason your LC5 did [worse] than average is that [a construction firm was overpaid by 4.5 million UGX].

Day 3.2. Another reason your LC5 did [worse] than average is that [some contract bids were not administered properly].

59

Day 3.2. Another reason your LC5 did [worse] than average is that [payments to sub-county governments were not properly administered].

Day 3.2. Another reason your LC5 did [worse] than average is that [payments of 24 million UGX remained unexplained at the time of audit].

Day 3.2. Another reason your LC5 did [worse] than average is [that expenditure of 57 million UGX for fuel was not accounted for].

Day 3.2. Another reason your LC5 did [worse] than average is that [excess payments of 3.5 million UGX were made to councilors and chairpersons].

Day 3.2. Another reason your LC5 did [worse] than average is that [22 million UGX paid for various activities could not be explained well enough].

Day 3.2. Another reason your LC5 did [worse] than average is that [salary payments of 2 million UGX were made to people not on the staff list].

Day 3.2. Another reason your LC5 did [worse] than average is that [2 million UGX in administrative spending could not be explained well enough].

Day 4.1. We have provided you [and many of your neighbors] with information about how well your LC5 could explain the way its money has been spent.

Day 4.2. According to the Auditor General your LC5 is [much better] at explaining its payments than most other districts.

Placebo Messages - Budget

Day 2.1. Research suggests that households which open a savings account are better able to save for school books and fees.

Day 3.1. When youth learn to save and manage their money well, they often have higher savings and income later on in life.

Treatment Messages - Roads

General Opening Message #2: Over the next few days we will send you [and many of your neighbors] information about public services.

Day 1.3. You mentioned that you are interested in road quality. Your sub-county roads are managed by your LC3.

Day 1.4. Based on our audits, sub-county roads in your village are [better] than other roads in your district.

Day 2.1. One reason your roads are [better] is that the [number of major obstacles] are [better] compared to others in your district.

Day 2.2. Most other roads in your district have [#] major obstacles per mile. Your sub-county road has [#] major obstacles per mile.

60

Day 3.1. Another reason your roads are [better] is that the [maintenance of the road] is [better] compared to others in your district.

Day 3.2. Most other roads in your district have a score of at least [8/10] in road maintenance. Your sub-county road has a score of [5/10].

Day 4.1 Although your roads are [better] than average, your [roughness of the road] is [a little worse].

Day 4.2. Road roughness in your road has a score of [7/10]. Most other roads in your district have a score of at least [8/10].

Day 5.1: We have provided you [and many of your neighbors] with information about the quality or roads managed by your LC3.

Day 5.2: According to our measurements, these roads are [better] than other roads in your district.

General Closing Message: We hope you found our information valuable. We will be calling you again soon with additional questions. Thank you.

Placebo Messages - Roads

Day 2.1. Careless or reckless driving contributes to 82 percent of road accidents in Uganda.

Day 3.1. Excessive speeding more than doubles the risk of being involved in a fatal accident in Uganda.

Treatment Messages - Education

Day 1.3. You mentioned that you are interested in primary education. Primary education in your village is the responsibility of your LC5.

Day 1.4. Based on Twaweza’s Uwezo Study, your village’s primary school is [a little worse] than other primary schools in your district.

Day 2.1. One reason your primary school is [a little worse] is that [3] out of 10 pupils from P3 to P7 can read and understand a P2 level story in English.

Day 2.2. Most other primary schools in your district have [4] out of 10 pupils who can understand a P2 level story in English.

Day 3.1. Although your village’s primary school is [a little worse], the [number of pupils absent] is [better] compared to others in your district.

Day 3.2. Most other primary schools in your district have [4] out of 10 pupils absent. Your primary school has [2] out of 10 pupils absent.

Day 4.1. Another reason your primary school is [a little worse] is that pupil’s [math skills] are [a little worse] compared to others in your district.

Day 4.2. Other schools have [4] out of 10 pupils from P3 to P7 who have P2 level math skills. Your primary village’s school has [3] out of 10 pupils.

61

Day 5.1. We have provided you with information about the quality of primary schools managed by your LC5.

Day 5.2. According to Twaweza’s Uzewo study, your primary school is [a little worse] than other primary schools in your district.

Placebo Messages - Education

Day 1.3. Good quality primary education is important for development in Uganda.

Day 2.1. Studies in Uganda show that when parents read in English to their children on a regular basis, their students receive higher marks.

Day 2.2. Studies in East Africa show that when parents encourage their children to read in English, their students receive higher marks.

Day 3.1. Studies in Uganda show that strong math skills are important for economic development.

Day 3.2. Studies in East Africa show that people with strong math skills better manage their family budgets.

Day 4.1. Studies in Uganda show that children whose parents strongly encourage them to attend school, attend school more often.

Treatment Messages - Health Clinics

Day 1.3: You mentioned that you are interested in health care. Your Health Centre III is maintained by both your LC3 and LC5.

Day 1.4: Based on our audits, the nearest Health Centre to your village is [much worse] than other health centres in your district.

Day 2.1: One reason your health centre is [much worse] is that drug availability is [much worse] compared to others in your district.

Day 2.2: Most other health centres in your district have [4] recommended drugs available. Your centre has [2].

Day 3.1: Another reason your health centre is [much worse] is that wait time is [much worse] compared to others in your district.

Day 3.2: Most other health centres in your district have wait times of [35] minutes. Your clinic has a wait time of [55] minutes.

Day 4.1: Although your health centre is [much worse], cleanliness is [a little better] compared to others in your district.

Day 4.2: Most other health centres in your district have cleanliness scores of [5] out of 16. Your centre has [10] out of 16.

62

Day 5.1: We have provided you with information about the quality of health centres maintained by your LC3 and LC5.

Day 5.2: According to our measurements, your health centre is [much worse] compared to other health centres in your district.

Placebo Messages - Health Clinics

Day 1.3: You mentioned that you are interested in health care. Your Health Centre III is maintained by both your LC3 and LC5.

Day 1.4: Studies in Uganda show that when people wash their hands in clean water, they are less likely to contract diseases.

Day 2.1: Research suggests that attending regular medical check-ups is important for good health.

Day 2.2: Regular check-ups save money and lives in the long-run by testing blood pressure, cholesterol, and for diseases such as HIV.

Day 3.1: Boiling your water for at least three minutes will kill most harmful bacteria from drinking water.

Day 3.2: Boiling your water for at least three minutes will also kill all harmful parasites and viruses from your drinking water.

Day 4.1: To avoid the spread of HIV/AIDS, it is important to practice responsible sexual behavior.

Day 4.2: To avoid the spread of malaria, it is important to use long lasting nets that have been treated with insecticide.

Day 5.1: Research suggests that engaging in regular physical activity helps prevent heart disease.

Day 5.2: It suggests that engaging in regular physical activity also helps prevent many forms of cancer.

Treatment Messages - Water Access

Day 1.3. You mentioned that you are interested in water access. Protect water access is managed by both your LC3 and LC5.

Day 1.4. Based on our audits, water access in your village is [much better] than other water access in your district.

Day 2.1. One reason your water access is [much better] is that the [walk time to the water source] is [much better] compared to others in your district.

Day 2.2. Most other villages have a walk time of [10] minutes or more to the water source. Your walk time to the water source is [5] minutes.

Day 3.1 Your water access is also [much better] because the [number of people waiting for water] is [much better] compared to others in your district.

63

Day 3.2. Most other villages in your district have a [5] people waiting in line at the water source. Your water source had [3] people waiting in line.

Day 4.1. Another reason your water access is [much better] is that the [fee per jerrycan] is [much better] compared to others in your district.

Day 4.1 Although your water access is [much better], the [fee per jerrycan] is [a little worse] compared to others in your district.

Day 4.2. Most other water access in your district have a fee of [100] UGX per jerrycan. Your fee per jerrycan is [200] UGX.

Day 5.1: We have provided you [and many of your neighbors] information about the quality of water access managed by your LC3 council.

Day 5.2: According to our measurements, your water access is [much better] than other water access in your district.

Placebo Messages - Water Access

Day 1.3. You mentioned that you are interested in water access. Protect water access is managed by both your LC3 and LC5.

Day 2.1. Boiling your water for at least three minutes will kill most harmful bacteria from drinking water.

Day 2.3. Boiling your water for at least three minutes will also kill most harmful parasites and viruses from your drinking water.

Day 3.1. Diarrhea diseases like dysentery, cholera, typhoid are infections that are one of major causes of death for children in Uganda.

Day 3.2. Washing hands with soap after using the toilet or before meals can reduce the risk children getting diarrhea diseases.

 

64

Appendix  AB  Baseline Call Center Survey Instrument for Health Center Only

Subject's ID number

Subject's phone number

Q1 - Quality of nearest health centre compared to others in district

• much better • better • a little worse • much worse • don't know • refused to answer

Q2 - Certainty of quality of healthcare

• very certain • certain • not certain • very uncertain • don't know • refused to answer

Did the subject receive 1000 UGX airtime? (Only ask this question if the subject raises the issue. If you answer "no", we will send them 1000 UGX.)

• Yes • No

65

Appendix  AC Endline LC5 Call Center Protocol  

Conducting an endline survey relating to the LC5 election, and delivering messages to reinforce SMS messages that we sent to subject

Start Date: February 25, 2016

Overview:

In this call center we will be contacting the 17,000 subjects that we were able to contact during the baseline call center. We will be asking 12 survey questions related to how they voted in the LC5 election, and their views of LC5 leaders in response to the text messages we sent them. We will also deliver several messages that reinforce the text messages that were sent to subjects. These reinforcement messages relate to the upcoming LC3 elections on March 8, 2016.

You will attempt to contact each subject three times to conduct the survey and to deliver reinforcement messages. For each subject you will either successfully conducting the audit or attempting to contact the subject three separate times. If you fail to connect with a subject, you should make the next attempt to contact the subject on a different day or at least several hours later.

Here are two very important points regarding this call center:

1. Be very careful that you and your team members are entering the subject’s ID number and phone number correctly into Kobocollect. If this information is not entered correctly, we will not be able to use the survey that you conduct.

2. Make sure you and your team members are conducting the survey with the specific individual listed on the contact sheet. Conducting an interview with someone besides the subject listed on the contact sheet will cause all of our findings to be inaccurate.

Administering and recording the baseline survey questions:

1. Before calling a subject, enter your name, and the subject’s ID and phone number into Kobocollect. Enter this information exactly as it appears on the contact list.

2. Make sure you are speaking with the specific individual listed on the contact sheet. If you can’t connect with the individual listed on the contact sheet, mark the call as an attempt on the contact sheet and re-attempt the call on another day or at least a few hours later.

3. Introduce yourself to the subject – remind them that you are working for the Twaweza Information Service, that you recruited them in their village, and that you conducted a phone survey with them several weeks ago.

4. Let subjects know that if they complete the entire survey, we will send them 1,000 UGX airtime on Monday, February 29th.

5. Record in Kobocollect the subject’s answers to the endline survey questions 6. Read the options for answers – this will helps reduce the call time 7. Do not read out “don’t know” or “refused to answer”

Delivering messages to reinforce SMS messages:

• For this task we are reading messages to the subjects to reinforce several text messages that we sent to them during the previous week. The goal of this task is to remind subjects about the

66

information we sent to them by text message. All the messages relate to the quality of public services in their village.

• Following the survey, read the messages that were sent to the subjects. • Try to stay as neutral as possible when delivering these messages. Putting extra emphasis on

words or phrases could create bias in our survey.

Kobocollect and syncing:

• Synchronize data at the end of each day. Each morning I will post the total number of calls that each district group makes to make sure all teams are on track.

• To synchronize data, navigate to the main menu and select Send Finalized Form. Next, select Toggle All, and submit finalized forms. The tablet or smartphone must be connected to the internet to sync.

Internet router and data management:

1. Each internet router is loaded with 5 gigabytes. 2. This internet bundle will be used for both the endline survey in late February and endline survey

in early March. Team leaders will be assigned the same router for both call centers. 3. Routers should only be used to upload Kobocollect surveys.

Tip: Only turn on the internet router in the evenings when you are syncing data.

Voice bundles:

1. Each call center staff will use approximately 1,400 minutes to contact all their subjects. 2. For MTN numbers, load one 1,900-minute voice bundle at a cost of 50,000 UGX 3. For Airtel numbers, load two 1,000-minute voice bundles at a cost of 60,000 UGX (30,000 UGX

per bundle). 4. These voice bundles are more than enough to cover calls for this call center. Funding for

additional voice bundles will not be provided so staff should use their minutes wisely if they are using their personal lines.

Loading Kobocollect onto a tablet or smartphone:

Here are instructions on how to download the Kobocollect app onto a smartphone or tablet:

1. Download and install the Kobocollect app from the Google Play store 2. Open the app and press the setting button (three dots in upper right corner) 3. Select General Settings 4. Under Server Settings, press URL 5. Enter the following URL: https://kc.kobotoolbox.org/mbuntaine/ 6. Press the back button to navigate back to the main menu 7. Select Get Blank Form 8. Select only VoteChoice_EndlineCallCenter1, and press Get Selected at the bottom of the screen 9. Now when you press Fill Blank Form, the VoteChoice_EndlineCallCenter1 should appear.

   

67

Appendix  AD Endline LC5 Call Center - Survey Instrument

(Note to Enumerator: do not read out loud or prompt subjects with the “don’t know” or “refused” options; select it only if they will not answer based on the other choices provided)

Introduction script: Hello, [name]. My name is [name of RA] and I am working with Twaweza and their partners. In December we visited you in your village and recruited you for the project. About a month ago we also contacted you by phone and asked you questions about the upcoming local elections. More recently, we have been sending you messages about public services in your area. I would like to ask you a few questions about the LC5 election. You will receive 1000 shillings in airtime if you finish the survey, thought you can skip any questions that you do not want to answer. Your answers are completely confidential and we will not tell anybody about them. We are not affiliated with any political party and only seek to understand how people use information in elections. Is okay that I ask you a few questions?

Outcome measures:

1. Did you vote in the February 24, 2016 LC5 elections? (1) yes, (2) no, (3) refused

(Note: if the answer to Question 1 is “no,” skip to Question 7.)

2. If you voted for LC5 chairperson, what party does the candidate that you voted for represent?

(Note to Enumerator: if the subject does not know the party by name, ask if she or he recalls the party symbol, then fill in with the corresponding party; also include “did not vote”, if the subject does not know the party or refuses to answer, enter don’t know or refuse)

3. If you voted for LC5 councilor, what party does the candidate that you voted for represent?

(Note to Enumerator: if the subject does not know the party by name, ask if she or he recalls the party symbol, then fill in with the corresponding party; also include “did not vote” if the subject does not know the party or refuses to answer, enter don’t know or refuse)

4. If you did vote, what color was the water basin in your polling station for the LC5 election?

5. Briefly describe what was most important to you when deciding which candidate to support for chairperson in the LC5 election? (A) Ethnicity (B) Provides benefits to you (C) Provides benefits to district (D) Makes good policies (E) Provides quality public services (F) Has integrity (G) Is endorsed by family or community members (H) don’t know (I) Refused to answer

(Note to Enumerator: Do not read options A-G, simply check all that apply that the subject mentions. Don’t let the subjects talk too long on this topic though.)

6. Have any representatives of LC5 officials provided gifts or money to you or your family in the few months before the election? (1) yes, (2) no, (3) don’t know, (4) refused.

Manipulation checks:

7. Compared to other districts in Uganda, how was your LC5’s record of managing its budget expenditures and contracting? (1) much better, (2) better, (3) a little worse, (4) much worse, (5) don’t know, (6) refused

68

8. How does the quality of the public service that you most care about in your village, whether it be health services, water access, roads, and primary schools, compare to other villages in your district? (1) much better, (2) better, (3) a little worse, (4) much worse, (5) don’t know, (6) refused

Mediators:

9. In your opinion, does your LC5 councilor make much more, a little more, a little less or much less effort to get things done compared to other councilors in your district? (1) much more effort, (2) a little more effort, (3) a little less effort, (4) much less effort, (5) don’t know, (6) refuse.

10. How surprised would you be to hear from a credible source about corruption involving your LC5 chairperson in office before the election? Would you be: (1) very surprised, (2) somewhat surprised, (3) not too surprised, (4) not surprised at all, (5) don’t know, (6) refuse

Message Manipulation Check & Reinforcement:

11. Over the last several days, we have sent you several SMS messages about public services. Did you see these messages? (1) yes (2) no (3) don’t know (4) refuse

This last question is only for subjects that are assigned a reinforcement message. On the contact sheet there is a column including the public service that is most important to the subject, and the score for the public service. If these columns are empty for a subject, skip question 12. If these columns have values, pug in these values to the messages below:

For subjects assigned to reinforcement condition:

12. Our SMS messages to you indicated that [public service] in your village is [much better / better / a little worse / much worse] than most villages in your district. We will send you these messages again in the coming days. Will you read them and think about them when you vote in the LC3 elections? (1) yes (2) no (3) don’t know (4) refuse

 

69

Appendix  AE Endline LC5 Call Center - Team Leader Contract  

By and between Jacob Skaggs (Investigator) referred to as “Investigator,” and _______________ (Team Leader) referred to as “Enumerator” on this the 23th day of February 2016. The Investigator engages the Enumerator as a contractor under the following terms and conditions:

1. Start Date: February 25, 2016

2. DUTIES: The core responsibility of the Enumerator is to manage a team of research assistants to contact subjects in his/her assigned districts and administer an endline survey and deliver reinforcement messages. See the call center protocol and supporting documents, which detail the specific responsibilities that the Enumerator agrees to take on and complete.

3. COMPENSATION: In consideration of the foregoing, the Investigator shall pay the following sums. The Investigator shall pay the Enumerator 320,000 UGX to complete the task of managing a team of research assistants to administer a baseline survey and delivering reinforcement messages, as outlined in the call center protocol. The Enumerator will also receive an additional 52,000 UGX to cover meals and transport expenses. The aforementioned payment amount will be provided to the Enumerator at the conclusion of the call center once all tasks outlined in the call center protocol have been successfully completed. The Investigator reserves the right to withhold or reduce pay if the Enumerator’s execution of the work activities does not follow the procedures outlined in the call center protocol.

4. DURATION OF CONTRACT: This endline call center is expected to last between 3 and 5 working days starting February 25th. Additional days may be required to successfully complete the assigned work outlined in the call center protocol.

5. TERMINATION: This agreement may be terminated upon breach of the agreement by the Enumerator. This agreement may also be terminated if the Enumerator loses, breaks, or otherwise renders inoperative the Samsung Tab4 or Nexus 7 tablet provided by the Investigator to conduct the survey.

6. MISCELLANEOUS: (1) This agreement is undertaken by the Investigator in his private capacities as a researcher and does not implicate University of California Santa Barbara, London School of Economics, College of William & Mary, Brigham Young University or Temple University, nor is the Enumerator to be considered an employee of any of these institutions. (2) The Enumerator is responsible for his own insurance and taxes. (3) The Enumerator agrees not to disclose any identifiable information about subjects in any form other than the requested data.

In witness whereof, both parties have executed this agreement on the dates below:

Investigator Enumerator

Signature:___________________ Signature:_____________________

Date:_______________________ Date:_________________________

70

Appendix  AF Endline LC5 Call Center - Research Assistant Contract

Investigator/Enumerator Agreement

By and between Jacob Skaggs (Investigator) referred to as “Investigator,” and _______________ (Research Assistant) referred to as “Enumerator” on this the 23th day of February 2016. The Investigator engages the Enumerator as a contractor under the following terms and conditions:

1. Start Date: February 25, 2016

2. DUTIES: The core responsibility of the Enumerator is to contact subjects in his/her assigned districts and administer an endline survey and deliver reinforcement messages. See the call center protocol and supporting documents, which detail the specific responsibilities that the Enumerator agrees to take on and complete.

3. COMPENSATION: In consideration of the foregoing, the Investigator shall pay the following sums. The Investigator shall pay the Enumerator 280,000 UGX to complete the task of administering a baseline survey and delivering reinforcement messages, as outlined in the call center protocol. The Enumerator will also receive an additional 52,000 UGX to cover meals and transport expenses. The aforementioned payment amount will be provided to the Enumerator at the conclusion of the call center once all tasks outline in the call center protocol have been successfully completed. The Investigator reserves the right to withhold or reduce pay if the Enumerator’s execution of the work activities does not follow the procedures outlined in the call center protocol.

4. DURATION OF CONTRACT: This endline call center is expected to last between 3 and 5 working days starting February 25th. Additional days may be required to successfully complete the assigned work outlined in the call center protocol.

5. TERMINATION: This agreement may be terminated upon breach of the agreement by the Enumerator. This agreement may also be terminated if the Enumerator loses, breaks, or otherwise renders inoperative the Samsung Tab4 or Nexus 7 tablet provided by the Investigator to conduct the survey.

6. MISCELLANEOUS: (1) This agreement is undertaken by the Investigator in his private capacities as a researcher and does not implicate University of California Santa Barbara, London School of Economics, College of William & Mary, Brigham Young University or Temple University, nor is the Enumerator to be considered an employee of any of these institutions. (2) The Enumerator is responsible for his own insurance and taxes. (3) The Enumerator agrees not to disclose any identifiable information about subjects in any form other than the requested data.

In witness whereof, both parties have executed this agreement on the dates below:

Investigator Enumerator

Signature:___________________ Signature:_____________________

Date:_______________________ Date:_________________________

 

71

Appendix  AG Endline LC3 Call Center – Survey Instrument  Introduction script: Good morning/afternoon/evening, may I please speak with [subject name]? My name is [name of RA] and I am working with Twaweza Information Service. We have been sending you SMS messages about public services in your area. A few weeks ago we contacted you by phone and asked you questions about the LC5 elections. I would now like to ask you a few questions about the LC3 election. You will receive 1000 shillings in airtime if you complete the survey, thought you can skip any questions that you do not want to answer. Your answers are completely confidential and we will not tell anybody about them. We are not affiliated with any political party and only seek to understand how people use information in elections. Is okay that I ask you a few questions? [Do not read out loud or prompt subjects with the “don’t know” or “refused” options; select it only if they will not answer based on the other choices provided.]

1) Do you agree to participate in this survey? (1) yes, (2) no

[If the subject answers “no”, tick the “no” answer, swipe to the end of the survey and save the finalized form.]

2) Did you vote in the March 9, 2016 LC3 elections? (1) yes, (2) no, (3) refused

[If the subject answers “no”, Kobocollect will skip to question 8.]

3) If you voted for LC3 chairperson, what party does the candidate that you voted for represent? (Note to Enumerator: if the subject does not know the party by name, ask if she or he recalls the party symbol, then fill in with the corresponding party; also include “did not vote”, if the subject does not know the party or refuses to answer, enter don’t know or refuse)

4) If you voted for LC3 councilor, what party does the candidate that you voted for represent? (Note to Enumerator: if the subject does not know the party by name, ask if she or he recalls the party symbol, then fill in with the corresponding party; also include “did not vote” if the subject does not know the party or refuses to answer, enter don’t know or refuse)

5) Do you recall the color of the basin where you ticked the ballot for the LC3 election?

6) Briefly describe what was most important to you when deciding which candidate to support for chairperson in the LC3 election? (Note to Enumerator: Do not read options A-G, simply check all that apply that the subject mentions) (A) Ethnicity (B) Provides benefits to you (C) Provides benefits to district (D) Makes good policies (E) Provides quality public services (F) Has integrity (G) Is endorsed by family or community members (H) political party; (I) none of the above; (K) refused

7) Have any representatives of LC3 officials provided gifts or money to you or your family in the few months before the election? (1) yes, (2) no, (3) don’t know, (4) refused

8) How does the quality of the public service that you most care about in your village, whether it be health services, water access, roads, and primary schools, compare to other villages in your district? (1) much better, (2) better, (3) a little worse, (4) much worse, (5) don’t know, (6) refused

72

9) In your opinion, does your LC3 councilor make much more, a little more, a little less or much less effort to get things done compared to other councilors in your district? (1) much more effort, (2) a little more effort, (3) a little less effort, (4) much less effort, (5) don’t know, (6) refuse

10) How surprised would you be to hear from a credible source about corruption involving your LC3 chairperson in office before the election? Would you be: (1) very surprised, (2) somewhat surprised, (3) not too surprised, (4) not surprised at all, (5) don’t know, (6) refuse

11) Over the last several days, we have sent you several SMS messages about public services. Did you see these messages? (1) yes (2) no (3) don’t know (4) refuse

   

73

Appendix  AH  Endline LC3 Call Center - Survey Instrument for Kampala Subjects

Introduction script: Hello, [name]. My name is [name of RA] and I am working with Twaweza and their partners. In February we sent you messages about public services in your area. Two weeks ago we contacted you by phone and asked you questions about the LC5 election. Today I would like to ask you a few questions about the LC3 election. You will receive 1000 shillings in airtime if you finish the survey, thought you can skip any questions that you do not want to answer. Your answers are completely confidential and we will not tell anybody about them. We are not affiliated with any political party and only seek to understand how people use information in elections. Is okay that I ask you a few questions?

[Do not read out loud or prompt subjects with the “don’t know” or “refused” options.]

Outcome measures:

1) Do you agree to participate in this survey? (1) yes, (2) no

[If the subject answers “no”, Kobocollect will skip to the end of the survey.]

2) Did you vote in the March 2, 2016 LC3 elections? (1) yes, (2) no, (3) refused

[If the subject answers “no”, Kobocollect will skip to question 8.]

3) If you voted for LC3 chairperson, what party does the candidate that you voted for represent? (Note to Enumerator: if the subject does not know the party by name, ask if she or he recalls the party symbol, then fill in with the corresponding party; also include “did not vote”, if the subject does not know the party or refuses to answer, enter don’t know or refuse)

4) If you voted for LC3 councilor, what party does the candidate that you voted for represent? (Note to Enumerator: if the subject does not know the party by name, ask if she or he recalls the party symbol, then fill in with the corresponding party; also include “did not vote” if the subject does not know the party or refuses to answer, enter don’t know or refuse)

5) Do you recall the color of the basin where you ticked the ballot for the LC3 election?

6) Briefly describe what was most important to you when deciding which candidate to support for chairperson in the LC3 election? (Note to Enumerator: Do not read options A-G, simply check all that apply that the subject mentions) (A) Ethnicity (B) Provides benefits to you (C) Provides benefits to district (D) Makes good policies (E) Provides quality public services (F) Has integrity (G) Is endorsed by family or community members (H) political party; (I) none of the above; (K) refused

7) Have any representatives of LC3 representatives provided gifts or money to you or your family in the few months before the election? (1) yes, (2) no, (3) don’t know, (4) refused

Manipulation check:

8) How does the quality of the public service that you most care about in your village, whether it be health services, water access, roads, and primary schools, compare to other villages in your district? (1) much better, (2) better, (3) a little worse, (4) much worse, (5) don’t know, (6) refused

Mediators:

74

9) In your opinion, does your LC3 councilor make much more, a little more, a little less or much less effort to get things done compared to other councilors in your district? (1) much more effort, (2) a little more effort, (3) a little less effort, (4) much less effort, (5) don’t know, (6) refuse

10) How surprised would you be to hear from a credible source about corruption involving your LC3 chairperson in office before the election? Would you be: (1) very surprised, (2) somewhat surprised, (3) not too surprised, (4) not surprised at all, (5) don’t know, (6) refuse

   

75

Appendix  AI Endline LC3 Call Center Protocol  

Conducting an endline survey relating to the LC3 election, and delivering messages to reinforce SMS messages that we sent to subject

Start Date: March 10, 2016

Overview:

In this call center we will be contacting the 17,000 subjects that we were able to contact during the baseline call center. We will be asking 11 survey questions related to how they voted in the LC3 election, and their views of LC3 leaders in response to the text messages we sent them.

You will attempt to contact each subject three times to conduct the survey and to deliver reinforcement messages. For each subject you will either successfully conducting the audit or attempting to contact the subject three separate times. If you fail to connect with a subject, you should make the next attempt to contact the subject on a different day or at least several hours later.

Here are two very important points regarding this call center:

1. Be very careful that you and your team members are entering the subject’s ID number and phone number correctly into Kobocollect. If this information is not entered correctly, we will not be able to use the survey that you conduct.

2. Make sure you and your team members are conducting the survey with the specific individual listed on the contact sheet. Conducting an interview with someone besides the subject listed on the contact sheet will cause all of our findings to be inaccurate.

Administering and recording the baseline survey questions:

1. Before calling a subject, enter your name, and the subject’s ID and phone number into Kobocollect. Enter this information exactly as it appears on the contact list.

2. Make sure you are speaking with the specific individual listed on the contact sheet. If you can’t connect with the individual listed on the contact sheet, mark the call as an attempt on the contact sheet and re-attempt the call on another day or at least a few hours later.

3. Introduce yourself to the subject – remind them that you are working for the Twaweza Information Service, that you recruited them in their village, and that you conducted a phone survey with them several weeks ago.

4. Let subjects know that if they complete the entire survey, we will send them 1,000 UGX airtime by Wednesday, March 15th.

5. Record in Kobocollect the subject’s answers to the endline survey questions 6. Read the options for answers – this will helps reduce the call time 7. Do not read out “don’t know” or “refused to answer”

Kobocollect and syncing:

• Synchronize data at the end of each day. Each morning I will post the total number of calls that each district group makes to make sure all teams are on track.

76

• To synchronize data, navigate to the main menu and select Send Finalized Form. Next, select Toggle All, and submit finalized forms. The tablet or smartphone must be connected to the internet to sync.

Internet router and data management:

• Each internet router is loaded with 5 gigabytes. • This internet bundle will be used for both the endline survey in late February and endline survey

in early March. Team leaders will be assigned the same router for both call centers. • Routers should only be used to upload Kobocollect surveys.

Tip: Only turn on the internet router in the evenings when you are syncing data.

Voice bundles:

• Each call center staff will use approximately 1,400 minutes to contact all their subjects. • For MTN numbers, load one 1,900-minute voice bundle at a cost of 50,000 UGX • For Airtel numbers, load two 1,000-minute voice bundles at a cost of 60,000 UGX (30,000 UGX

per bundle). • These voice bundles are more than enough to cover calls for this call center. Funding for

additional voice bundles will not be provided so staff should use their minutes wisely if they are using their personal lines.

Loading Kobocollect onto a tablet or smartphone:

Here are instructions on how to download the Kobocollect app onto a smartphone or tablet:

1. Download and install the Kobocollect app from the Google Play store 2. Open the app and press the setting button (three dots in upper right corner) 3. Select General Settings 4. Under Server Settings, press URL 5. Enter the following URL: https://kc.kobotoolbox.org/mbuntaine/ 6. Press the back button to navigate back to the main menu 7. Select Get Blank Form 8. Select only VoteChoice_EndlineCallCenter1, and press Get Selected at the bottom of the screen 9. Now when you press Fill Blank Form, the VoteChoice_EndlineCallCenter1 should appear.

   

77

Appendix  AJ Endline LC3 Call Center - Team Leader Contract  By and between Jacob Skaggs (Investigator) referred to as “Investigator,” and _______________ (Team Leader) referred to as “Enumerator” on this the 1st day of March 2016. The Investigator engages the Enumerator as a contractor under the following terms and conditions:

1. Start Date: March 3, 2016

2. DUTIES: The core responsibility of the Enumerator is to manage a team of research assistants to contact subjects in his/her assigned districts and administer an endline survey and deliver reinforcement messages. See the call center protocol and supporting documents, which detail the specific responsibilities that the Enumerator agrees to take on and complete.

3. COMPENSATION: In consideration of the foregoing, the Investigator shall pay the following sums. The Investigator shall pay the Enumerator 240,000 UGX to complete the task of managing a team of research assistants to administer a baseline survey and delivering reinforcement messages, as outlined in the call center protocol. The Enumerator will also receive an additional 54,000 UGX to cover meals and transport expenses. The aforementioned payment amount will be provided to the Enumerator at the conclusion of the call center once all tasks outlined in the call center protocol have been successfully completed. The Investigator reserves the right to withhold or reduce pay if the Enumerator’s execution of the work activities does not follow the procedures outlined in the call center protocol.

4. DURATION OF CONTRACT: This endline call center is expected to last between 3 and 4 working days starting either on March 3rd or March 10th depending on of the team’s districts have municipalities. Additional days may be required to successfully complete the assigned work outlined in the call center protocol.

5. TERMINATION: This agreement may be terminated upon breach of the agreement by the Enumerator. This agreement may also be terminated if the Enumerator loses, breaks, or otherwise renders inoperative the Samsung Tab4 or Nexus 7 tablet provided by the Investigator to conduct the survey.

6. MISCELLANEOUS: (1) This agreement is undertaken by the Investigator in his private capacities as a researcher and does not implicate University of California Santa Barbara, London School of Economics, College of William & Mary, Brigham Young University or Temple University, nor is the Enumerator to be considered an employee of any of these institutions. (2) The Enumerator is responsible for his own insurance and taxes. (3) The Enumerator agrees not to disclose any identifiable information about subjects in any form other than the requested data.

In witness whereof, both parties have executed this agreement on the dates below:

Investigator Enumerator

Signature:___________________ Signature:_____________________

Date:_______________________ Date:_________________________

   

78

Appendix  AK Endline LC3 Call Center - Research Assistant Contract  

By and between Jacob Skaggs (Investigator) referred to as “Investigator,” and _______________ (Research Assistant) referred to as “Enumerator” on this the 1st day of March 2016. The Investigator engages the Enumerator as a contractor under the following terms and conditions:

1. Start Date: March 3, 2016 for Kampala District or March 10, 2016 for all other districts.

2. DUTIES: The core responsibility of the Enumerator is to contact subjects in his/her assigned districts and administer an endline survey and deliver reinforcement messages. See the call center protocol and supporting documents, which detail the specific responsibilities that the Enumerator agrees to take on and complete.

3. COMPENSATION: In consideration of the foregoing, the Investigator shall pay the following sums. The Investigator shall pay the Enumerator 210,000 UGX to complete the task of administering a baseline survey and delivering reinforcement messages, as outlined in the call center protocol. The Enumerator will also receive an additional 39,000 UGX to cover meals and transport expenses. The aforementioned payment amount will be provided to the Enumerator at the conclusion of the call center once all tasks outline in the call center protocol have been successfully completed. The Investigator reserves the right to withhold or reduce pay if the Enumerator’s execution of the work activities does not follow the procedures outlined in the call center protocol.

4. DURATION OF CONTRACT: This endline call center is expected to last between 3 and 4 working days starting as early as March 3rd depending on if the districts have municipalities within them. Teams with districts that do not have municipalities will start the call centre on March 10th. Additional days may be required to successfully complete the assigned work outlined in the call center protocol.

5. TERMINATION: This agreement may be terminated upon breach of the agreement by the Enumerator. This agreement may also be terminated if the Enumerator loses, breaks, or otherwise renders inoperative the Samsung Tab4 or Nexus 7 tablet provided by the Investigator to conduct the survey.

6. MISCELLANEOUS: (1) This agreement is undertaken by the Investigator in his private capacities as a researcher and does not implicate University of California Santa Barbara, London School of Economics, College of William & Mary, Brigham Young University or Temple University, nor is the Enumerator to be considered an employee of any of these institutions. (2) The Enumerator is responsible for his own insurance and taxes. (3) The Enumerator agrees not to disclose any identifiable information about subjects in any form other than the requested data.

In witness whereof, both parties have executed this agreement on the dates below:

Investigator Enumerator

Signature:___________________ Signature:_____________________

Date:_______________________ Date:_____________________