Building Resilience Through Asset Creation and Enhancement … · BRACE II Mid-Term Evaluation Compendium 1 1 Introduction to the ompendium (Detailed Analysis of Individual Outcomes

DFID Contract Reference: PO 7532

Mid-Term Evaluation Compendium Department for International Development For the Period: February 2017 to October 2018 Date of Final Report: June 2019 Prepared by: IMC Worldwide In association with:

Building Resilience Through Asset Creation and Enhancement (BRACE) II – Monitoring and Evaluation Services

BRACE II Mid-Term Evaluation Addendum

i

BRACE II Mid Term Evaluation Compendium: Detailed Analysis of Individual Outcomes at the Mid -Term

Department for International Development For the Period: February 2017 – Oct 2018 Date: January 2019 Prepared by: IMC Worldwide

IMC Worldwide Ltd 64-68 London Road Redhill RH1 1LG Tel: +44 (0)1737 231 400 Fax: +44 (0)1737 771 107 www.imcworldwide.com

Forcier Consulting Monitoring, Evaluation & Learning Unit Bilpam Road Juba www.forcierconsulting.com

http://www.imcworldwide.com/

http://www.forcierconsulting.com/

BRACE II Mid-Term Evaluation Compendium

ii

Contents

1 Introduction to the Compendium (Detailed Analysis of Individual Outcomes at the Mid-Term) ............ 1

1.1 Building Resilience through Asset Creation and Enhancement (BRACE II) .................................. 1

1.2 The Mid-Term Evaluation ........................................................................................................ 1

1.3 The Compendium: A complement to the MTE .......................................................................... 2

2 Findings from the MTE Field Work ................................................................................................... 4

2.1 Intermediate Outcomes .......................................................................................................... 4

2.2 Final Outcomes .................................................................................................................... 36

2.3 Impact ................................................................................................................................ 44

References ......................................................................................................................................... 46

Annex 1: Historic Yield and Farm Sizes ................................................................................................. 46

Annex 2: Historic Wealth Groupings and Sources of Income .................................................................. 49

Figure 1: Application of agricultural practices .......................................................................................... 6 Figure 2: Application of techniques by partner and gender ...................................................................... 7 Figure 3: Application of techniques by gender of HH household ............................................................... 8 Figure 4: Social network status, by Component .................................................................................... 10 Figure 5: Types of social support, by Component .................................................................................. 10 Figure 6: Social relationship to network ................................................................................................ 11 Figure 7: Conflict Resolution Mechanisms ............................................................................................. 12 Figure 8: Change in income proxies ..................................................................................................... 15 Figure 9: Increase in cash by beneficiary year ...................................................................................... 16 Figure 10: Expenditure types by Component ........................................................................................ 16 Figure 11: Expenditure types by beneficiary year .................................................................................. 17 Figure 12: Consumption patterns in staple consumables ....................................................................... 17 Figure 13: Consumption patterns in occasional purchases ..................................................................... 17 Figure 14: Consumption patterns in durable goods ............................................................................... 18 Figure 15: Determinants of annual expenditures by type ....................................................................... 19 Figure 16: Changes in cereal production (kg) ....................................................................................... 23 Figure 17: Change in total production (kg and value) ............................................................................ 24 Figure 18: Changes in total production value by sex ............................................................................. 24 Figure 19: Total harvest value by crop (current harvest cycle) ............................................................... 25 Figure 20: Total production value by farmer type.................................................................................. 25 Figure 21: Change in land cultivated .................................................................................................... 26 Figure 22: Change in productivity (yield/feddan kg SE) ......................................................................... 27 Figure 23: Total value and change in production by partner .................................................................. 27 Figure 24: Determinants of harvest value ............................................................................................. 28 Figure 25: Types of crops farmed, by Component ................................................................................. 32 Figure 26: Changes in types of land, by Component ............................................................................. 33 Figure 27: Change in vegetable gardening and irrigation, by Component ............................................... 33 Figure 28: Types of vegetables cultivated by component ....................................................................... 34 Figure 29: Change in income sources by component ............................................................................. 34 Figure 30: Change in income sources, by gender (Component I) ........................................................... 35 Figure 31: FCS by Component ............................................................................................................. 39 Figure 32: HDDS by Component .......................................................................................................... 39 Figure 33: FCS in detail & by Component ............................................................................................. 40 Figure 34: FCS by partner ................................................................................................................... 40 Figure 35: LCSI .................................................................................................................................. 42 Figure 36: Project effect on likelihood of community conflict by sex of head of household ....................... 43

Table 1: Changes in food and income sources by partner ...................................................................... 35


iii

Abbreviations

ACF Action Against Hunger

BPMC Boma Project Management Committee

BRACE Building Resilience through Asset Creation and Enhancement

CFSAM Crop and Food Security Assessment Mission

CFSM Crop and Food Supply Mission

CSI Coping Strategies Index

CWW Concern Worldwide

DFID Department for International Development

FAO Food and Agriculture Organisation

FCS Food Consumption Score

FE Final Evaluation

FHHH Female-Headed Household

FO Final Outcome

Forcier Forcier Consulting

GAM Global Acute Malnutrition

HDDS Dietary Diversity Score

HH Household

ICF International Climate Fund

IMC IMC Worldwide

IO Intermediate Outcome

JAM Joint Aid Management

KPI Key Performance Indicator

LCSI Livelihoods CSI

MHHH Male-Headed Household

MT Metric Tonne

MTE Mid-term Evaluation

NBEG Northern Bahr el Ghazal

NRC Norwegian Refugee Council

PDM Post Distribution Monitoring

RIMA Resilience Index Measurement and Analysis

SAADO Smile Again Africa Development Organization

SAM Severe Acute Malnutrition

SHHS Sudan Household Health Survey


iv

SPEDP Support for Peace & Education Development Programme

SS South Sudan

SSP South Sudanese Pound

TPM Third Party Monitoring

WFP World Food Programme


1

1 Introduction to the Compendium (Detailed Analysis of Individual Outcomes at the Mid-Term)

1.1 Building Resilience through Asset Creation and Enhancement (BRACE II)

BRACE is a five year programme (2015-2020), with two phases of 2.5 years each. BRACE II offers a resilience-building approach, intended to reduce dependence on food aid and relief; improve community relations and reduce vulnerability climate variation. Food insecure families work to create productive assets at the household level (block farms for crops or vegetable gardens) or community assets (e.g. access roads, water ponds, anti-flooding dykes) to bring about a sustained change in the amount of food produced or income generated. Other activities to improve local governance, social cohesion or access to markets are intended to deepen and protect those changes. In exchange for their work, beneficiaries are provided labour opportunities during the lean season (i.e. before harvest when previous food stocks run low) to help them to meet their short-term food needs.

The programme has three components. Component I is managed by the World Food Programme (WFP) and the Food and Agriculture Organisation (FAO). Component II is managed by World Vision (WV). The third is a monitoring, evaluation and learning component delivered by IMC Worldwide (IMC) & Forcier Consulting (Forcier). Component I began with the start of the programme and Component II was launched with the second phase of the programme, in 2018. BRACE II builds on lessons learned in a successful pilot project phase - BRACE I1.

1.2 The Mid-Term Evaluation The mid-term evaluation (MTE) is a theory-based assessment of progress towards programme outcomes which will also gauge the appropriateness of the theory of change. The Department for International Development (DFID) has asked that the MTE should help to answer four questions:

1. To what degree is the programme progressing against its logframe outcomes?

2. Are there differences in performance between Components I and II of BRACE II, and if so, why?

3. What are emerging pathways to resilience and how do we measure actual change and progress along those pathways (and for whom)? What are emerging, actual final outcomes or impacts?

4. Number of people whose resilience has been improved as a result of project support (International Climate Fund (ICF) Key Performance Indicator (KPI) 4)?

Answering the four MTE questions necessitated a unique MTE approach. The MTE adopted a bottom-up process that examines firstly what was done by implementers, secondly, what change (or not) did those activities make and thirdly, how changes fit within the lives of beneficiaries. The MTE seeks to provide unambiguous evidence of change attributable to the programme, to serve as the evidence base for the four

1 2012 to 31 July 2015.


2

MTE questions. The MTE used a mixed methods approach to verify change which occurred and could be attributable to the programme, and to provide critical analysis to interpret those findings.

The field work and data collection for the MTE was carried out in October and November 2018, and the draft MTE report submitted in January 2019.

1.3 The Compendium: A complement to the MTE The Compendium: “Detailed Analysis of Individual Outcomes at the Mid-Term” documents the results of the quantitative data collection exercise and preliminary analysis of each outcome. It provides a wealth of insight into the programme mechanisms of change. It is the basis of the responses to the four evaluation questions for the MTE. The Compendium includes analysis, individually, of performance against each Intermediate Outcome, Final Outcomes and Impact.

This document provides rich detail on not only the performance against indicators, but also insight into which activities resonate with which segments of the community, to what degree and why. These insights are of value for DFID to see how component elements of the programme performed individually. The Compendium can equally be of value to programme designers, implementers or evaluators of not only of resilience programmes but agriculture, social protection, labour/public works, or conflict reduction programmes to learn from the experience of BRACE II.

Intermediate Outcome (IO): Increased capacity to absorb, anticipate and adapt to shocks and stresses (including climate variability and extremes)

IO1: Number of people with improved resilience (ICF indicator)

IO2: Proportion of targeted households using good nutrition practices promoted in BRACE trainings

IO3: Proportion of targeted households using improved and climate-sensitive agricultural practices

IO4: Proportion of households giving or receiving support from other households (Social cohesion)

IO5: Proportion of targeted communities where there is evidence of functional structures for conflict resolution

IO6: Increase in incomes at household level

IO7: Increase in cereal production at household level, disaggregated by increase in production and cultivated area [Metric Tonnes (MT)]

IO8: Increase in number of food / incomes sources at households level

Final Outcome (FO): Improved food security and better community relationships

FO1: Prevalence of poor and borderline food consumption, disaggregated by sex of household head (Food Consumption Group)

FO2: Diet diversity score of targeted households, disaggregated by sex of households head (HDDS)

FO3: Proportion of targeted households implementing crisis and emergency strategies, disaggregated by sex of household head (Livelihoods CSI)

FO4: Reduced vulnerability to climate risks and shocks

FO5: Reduced vulnerability to communal conflict

Impact: Improved food security and better community relationships within the targeted communities & Partners' community feedback mechanisms established

I1: Reduced prevalence of moderately and severely food insecure households (in counties where BRACE II is implemented)


3

I2: Resilience Index Measurement and Analysis (RIMA II) (in counties where BRACE II is implemented)

I3: Crop and Food Security Assessment Mission (CFSAM)


4

2 Findings from the MTE Field Work

2.1 Intermediate Outcomes

IO1: Number of people with improved resilience (ICF indicator)

IO1: Number of people with improved resilience (ICF indicator) is Evaluation Question 4: Number of people whose resilience has been improved as a result of project support ICF-KPI 4. The data required to assess IO1, is drawn from other intermediate outcomes (Particularly IO3, IO4 IO6, IO7). The analytical process, including consideration of resilience dimensions, and organising results under the 3A’s (as suggested by current KPI4 Guidance2) is summarised in Evaluation Question 4.

Results for year 3 indicate that 43% of beneficiaries could be categorised as “Climate Resilient” and 57% are “Shock Proofed”. Consistent with advice, both could be reported against KPI4 (although “Climate Resilient” meets a higher resilience threshold).

IO2: Proportion of targeted households using good nutrition practices promoted in BRACE trainings (Only Component II)

Indicators

Indicators of good nutrition practise are an increased demand for specific nutrition sensitive commodities (i.e. soap from local suppliers) or increased use of soap in the home. Complimentary evidence of a change in diet may also be used to indicate good nutrition practise. These indicators relate to health promotion activities which are in Component II only. Component I does include this activity package.

Results at the mid-term

At the mid-term (October 2018) there was no data provided by WV. Health messaging activities had only begun a few months before. WV felt that no meaningful change could be expected and the indicators were not included in their household survey. Information against this indicator will be collected beyond the MTE, as a part of World Vision’s quarterly monitoring exercises. Progress against this outcome will be reported in the Final Evaluation (FE).

IO3: Proportion of targeted households using improved and climate-sensitive agricultural practices

Indicators

Assessing the changes in proportion of beneficiaries using improved and climate-sensitive agricultural practices among beneficiary households was difficult. No comprehensive list exists of skills that beneficiaries should have been trained in or when. As a proxy of the use of improved or climate sensitive

2 See (Climate Change Compass, 2018)


5

practises, the MTE asked respondents if they apply a wide set of agricultural skills, identified in previous Third Party Monitoring (TPM) missions and if so, on which parts of their land.

Summary

Evaluation question

Findings

Progress against indicator

Due to the lack of baseline and absence of defined skills to be improved, the analysis provides a snapshot of the current situation. Of the skills surveyed, the number of agricultural techniques applied by beneficiaries are 3.3 for Component I and 2.4 for Component II. Conversely, the percentage of beneficiaries applying no skills at all are 4% for Component I and 5% for Component II.

From the analysis, the majority of applied skills are on the home field rather than the households (HHs)’ far fields or the project supported block farms. Only the practice of ridge and row planting was more common in block farms than in other land types.

- Beneficiary year

The total number of skills is very dependent on years participating in the programme (Component I). 2.2 skills are applied for year 1; 3.6 for year 2, and 3.4 for year 3. A difference is observed between years 1 and 2 or 3, but not between years 2 and 3.

- Gender Female-headed households were systemically less likely than men to apply all techniques (and on average applied 0.4 fewer techniques). Differences are similar when looking at single-headed households generally, suggesting other issues, such as labour, may be more explanatory than gender.

- Partner Although some significant differences exist between partners in the number of skills applied by their beneficiaries, it is impossible to attribute it to partner efforts due to the lack of baseline and lack of information on training approaches.

Difference between Components I and II

Clear differences exist between the two Components with Component I beneficiaries reporting higher levels of techniques applied. However, at this stage of the project it is impossible to make attributions to the project’s approach or implementation.


Figure 1: Application of agricultural practices, summarises the results from the survey. The results provide the percentage of the sample that apply the practice and in which fields (only counting those who currently cultivate those land types). No skill has more than 2/3 coverage with weeding and topping at 66% and 64% spread respectively. Weeding is clearly considered by the majority as an important technique, as evidenced by its almost universal application across all farm types (in only block farms is it monetarily incentivised). However, given that farmers are paid to weed on their block farms fields, it would have been expected to be more prevalent. Skills that require either large monetary or labour investment, such as manuring, ridge planting, tree planting, and animal ploughing, are least reported.

The samples for Components 1 and 2 share some significant commonalities. Generally, the Component I sample has a substantially higher spread of practices with the average household applying 3.3 vis-à-vis 2.4 for Component II. It is extremely rare for households to not be using any skills (3.9% for Component I and 5.2% for Component II). A major difference between the components may be manuring, which is reaching almost 40% for Component I, where it’s only 15% for Component II. Manuring is significant both because of its contribution to yields and land longevity, and because it requires investment. The growth within Component I may be an indication of growing wealth, especially households having more cattle (cattle ownership significantly increases the likelihood of reporting use of manure).


6

Both components also show a strong convergence regarding the return period for different techniques and their use in different field. There is less effort and investment into the block and far farm fields compared to home farms, as evidenced by lower application practices such as manuring, crop rotation, intercropping, and tree planting. These techniques deliver value over the longer-term. Both fields are a distance from the household. These fields may play a different role in a households overall strategy to maximise production and minimise risk, and thereby justify different levels of investment and effort.

The exceptions are row planting and ridge planting. In both Components 1 & 2, they are more commonly practised on block farm fields. Possible explanations may be that 1) farmers have a competitive spirit at the block farms (as reported anecdotally by researchers); and 2) skills may not necessarily have full buy-in of farmers as they are not applying them on their household fields where outcomes are more critical than on far or block farm fields.

It is also clear that Component II individuals report substantially lower application of techniques in their far farms than farmers in Component I. It may be a function of the relative newness of the programme in Component II communities. It may also be a function of the farms in Component II, which report to be much poorer than in Component 1. Applying these techniques has labour implications, which are magnified by distances of block farm fields to households. It may be poorer households, or those with lower access to labour, would be less likely to invest in block farm fields.

Figure 1: Application of agricultural practices


7

When examining Figure 2: Application of techniques by partner and gender, some differences by partner emerge. In Component I, Norwegian Refugee Council (NRC) beneficiaries in particular are applying techniques to a much larger degree than other partners (followed by Concern Worldwide (CWW) and Joint Aid Management (JAM). Action Against Hunger (ACF) and World Vision (WV) beneficiaries on the other hand are much less likely to be using the techniques in question. For Component II, Smile Again Africa Development Organization (SAADO) beneficiaries are particularly unlikely to apply the skills. As a whole, the number of skills applied by each partner are CWW 3.7, ACF 2.3, WV 1.6, JAM 3.4, NRC 4.9, SAADO 1.1, Support for Peace & Education Development Programme (SPEDP) 3.1, and WV 2.6. There is no clear information to explain these differences, nor is there a baseline which would indicate if there has been a change in adoption of techniques.

Female-headed households (FMHH) are less likely to apply all techniques by a consistent degree when compared to male-headed households (MHHH). When the data was analysed by single-headed households, regardless of the sex of the household head, it returns a similar result. It suggests that factors other than gender may explains the difference, such as labour availability. MHHHs apply on average 3 skills and FHHHs 2.6.

Key Observations & Discussion

Trainings on most of the practises examined are either features of local farming systems or have been promoted for at least the last 20 years . It is difficult to attribute a change in the adoption of these techniques to BRACE II activities. An attempt to capture this change would risk being fraught with false positives and negatives.

Secondly, in the programme design, some of these trainings represented in part an approach to reduce risk to climate extremes. As such, it is important to have distinguished firstly, the shock or anticipated impacts of climate change and secondly to distinguish trainings that relate to or mitigate the consequences of that shock. It would clarify which training is climate sensitive and which is an improved technique. That distinction specifies the problem which can be remedied through a change in practise or a problem rooted in climate (versus agronomy for example). In its absence, all practices from this list can be considered useful to influence efficiency of plant growth, soil quality (or losses due to erosion) or ability of plants to manage rainfall or temperature fluctuations, or rather a list of good practise which may or may not be of value to a farmer of farm production system. Farmers appear to use them selectively possibly as a strategy to help their plants manage normal rainfall, temperature variations, or threats of pests and disease and provide farmers with more stable production levels over time. Without specifying a problem to be addressed by the adoption of some or all of those practises they are a list of good practises, selectively used by BRACE II farmers, and without clarity of a change desired which would improve farm performance or manage climate variability. It is not clear if or how one can differentiate between good agronomy and managing climate change hazards and as such, does not demonstrate a “risk” to be managed.

0%

20%

40%

60%

80%

100%

CWW ACF WV C1 JAM NRC SAADO SPEDP WV C2

Application of techniques by partner

Mulching Intercropping Crop rotation Tree planting Manuring

Row planting Weeding Ridge planting Animal plough

Figure 2: Application of techniques by partner and gender


8

Beyond discussions of the theoretical benefits to crop production generally or managing climate change specifically, use of these techniques (whether newly adopted or not) do not explain changes in the value or production on farms reported in IO7 (Increased grain production). The changes in value or quantity of crop for BRACE II beneficiaries was in fact explained by the adoption of a new crop and changes in farm size. Moreover, the primary driver of change in the programme design was increased production through land expansion at block farms.

Programme farmers display a different understanding and use of improved and climate proofing techniques than hypothesized in the BRACE II business case. The business case referred to a food security analysis by WFP, which identified lack of good farming practise as a main cause of chronic, acute food insecurity in South Sudan. While that may be true, Figure 1 shows that BRACE II beneficiaries not only understand these techniques, but use them judiciously within their production systems. Labour and cost requirements mean home fields receive greatest investments. It is the priority field in these farming systems. Far farms or block farms are less critical. Manuring for example, is investment more likely to be made on the home farm, not the block farm. Also, block farms were often cut from virgin fields with little need for manure or tree planting. These techniques are important and valuable. Questions of adoption or not of practices, while useful for some purposes, do not capture an important part of the story, which is that . farmers understand the value of these techniques relative to their farming objectives and systems.

IO4: Proportion of household giving or receiving support from other households (Social cohesion)

Indicators

Proportion of households giving vs receiving support was indicated by the self-reporting respondents as to whether or not they felt they were net givers, receivers or engaged in reciprocal sharing relationships. Determining how levels giving had changed as a result of the programme was difficult. There was no baseline. Self-reporting about such a change would be subjective and likely biased. A prototype of the indicator featured in the Component II baseline, but the duration between the two data points and their time difference in the annual cycle do not lend the two measurements of the indicator for robust comparison.

Instead, by comparing current proportions of people giving or receiving together with changes in ability or desire to give (i.e. changes in income level, or sources of food or income) we could provide indicative feedback on general trends in giving or social cohesion. No strong conclusions can be made regarding whether there has been change in social cohesion, but attempts can be made at testing the main mechanisms.

39%47% 45%

15%

29%

42%

67%

10% 7%

34%41% 38%

14%

24%

38%

62%

6% 4%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Application of techniques by gender of HH head

MHHHs FHHHs

Figure 3: Application of techniques by gender of HH household


9

The indicator uses the household survey as its source, along with observations and qualitative interviews.

Summary

Evaluation question

Findings


No baseline or targets exist for the indicator. Generally the data shows that the majority of beneficiaries engage in reciprocal giving and receiving relationships. There is a small minority in unidirectional, either giving or receiving only, though a significant minority seems to be totally unconnected. The programme, if not making a change, has perhaps enabled a deeper engagement in reciprocal relationships.

Component I:

- Reciprocators: 72% - Givers only: 9% - Receivers only: 2% - Unconnected: 17%

Component II:

- Reciprocators: 74% - Givers only: 10% - Receivers only: 4% - Unconnected: 12%

- Beneficiary year

No clear patterns were visible by beneficiary year on this indicator.

- Gender Male-headed households are slightly more likely to be in reciprocal relationships (74% vs 71%) and less likely to be unconnected than female-headed households (12% vs 17%). However, differences are quite small.

- Partner The general distribution and scale are similar across the partners, and small differences are likely simply a factor of location differences rather than partner implementation. The only exception in scale is NRC in Twic where practically all beneficiaries (91%) are in reciprocal relationships.


There are no significant differences between Components, suggesting that the way that such relationships are structured are relatively stable and not strongly altered by project activities. Scale of interactions may have been affected differently, but this could not be measured in the MTE.


The data on social support networks shows reciprocity – i.e. that the household has both received and given some type of support to/from another household (in the form of either money, in-kind, or labour) – is dominant (see Figure 4: Social network status, by Component). Almost three quarters of the beneficiaries belong in this category, a finding remarkably consistent between Components. It is rare for


10

any of the beneficiary households to report only receiving support from social networks. It is in fact more likely to not be in a social support system than to be in a unidirectional one.

Qualitative interviews report there is high pressure to share programme benefits, and cash from labour in particular. One respondent said: “there is no way that I could not share some of my cash with the community”.

In terms of difference with regard to types of support given and received (see Figure 5: Types of social support, by Component), the largest difference is seen for cash. Beneficiaries report giving more cash than receiving (a difference of 18% and 15% respectively for each Component). In contrast, labour is provided and given in equal parts.

Giving or sharing may be correlated to other programme activities. Regression analysis indicated that expenses correlate positively with the likelihood of giving. Income levels are also positively correlated with the likelihood of receiving support, albeit the effect is slightly smaller. Beneficiaries who reported an increase in the value of the farm’s production, e.g. those who adopted groundnuts, were much more likely to be in giving and receiving relationships than beneficiaries who did not report a change in their farm production. The value of the harvest does not correlate with the likelihood of giving (though does slightly decrease the likelihood of receiving support), reducing the explanatory power of the hypothesis that a change in value of production, if not total production, increases likelihood of sharing. The (expected) value of the yield did, however, increase the likelihood of planned sharing of some of the harvest. Results testing assumptions that increased incomes from agriculture lead to improvements in social cohesion (or rather social security networks) is mixed. A consistent finding, however, is that female headed households are less likely to be in the reciprocal relationships, and less likely to plan sharing harvest when compared to male-headed households.

56.8%

38.4%

58.6%

43.3%

61.5%

48.8%

57.5%49.7%

64.2%64.4%

71.0% 68.4%

81.0%74.2%

84.5%78.2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Aid given Aid received Aid given Aid received

Component I Component II

Types of social support

Money In-kind Labour Any

Figure 5: Types of social support, by Component

71.8% 74.4%

2.4%3.7%9.3%

10.1%

16.6% 11.8%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


Social network status

Reciprocator Receiver only Giver only Unconnected

Figure 4: Social network status, by Component


11

The data finally shows that the majority of social support remains within what could be termed as close social networks – within clearly definable blood relationships (see Figure 6: Social relationship to network). This fits with the general norms of Dinka culture where the closeness of a person’s relationship increases the level to which they are expected share. However, these communities are generally ones with rich blood ties and the prevalence of familial relationships should be expected.


We cannot tell from the findings if there are changes in the amount given or if people are more willing or able to assist others. In Dinka culture, giving and receiving is profoundly important for cultural and practical reasons (Harigan & Chol, 1998). The programme may have helped beneficiaries to engage more robustly within these social norms, not actually increasing “sharing”. The fact that such similar levels are recorded for Components I and II would suggest that if the programme is in fact making a contribution towards sharing, it is likely due to the cash transfer. At the time of the MTE fieldwork, cash had already been received and as the harvest was ongoing, any meaningful effects on sharing were unlikely to have been realised. In fact, through the cash payments, the programme may have influenced sharing as beneficiaries were likely to experience considerable pressure to share their entitlements.

The programme may also have indirect impacts, which in turn enable sharing or rather deeper engagement by beneficiaries in social cohesion arrangements. Respondents indicate that their engagement in the programme led to them to be perceived as credit worthy. The programme allowed them to make expenditures on sugar, tea or serving trays (allowing them to invite neighbours to their home), indicating they had improved their standing and reducing their perception as “net recipients”. A better understanding of those dynamics could allow programme activities to help people to work “in them” or at least to avoid the risk of undermining them.

As reciprocity seems to exist in concentric circles from nuclear family to extended family and non-relatives, and as the majority of mutual giving occurs within the inner-most rings, these changes might be better understood as influencing direct familial relationships, if not wider, communal changes.

IO5: Proportion of targeted communities where there is evidence of functional structures for conflict resolution

Indicators

Indications of functional structures for conflict resolution were committees formed by the programme. The related packages of activities only worked with these local governance structures. Respondents were asked if such committees existed, what they did and their perceptions of changes to local security.

41.4% 38.2% 45.4% 40.7%

23.1% 28.1%29.4% 35.0%

8.1% 8.1%6.7% 7.0%

27.3% 25.6% 18.4% 17.3%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Give Receive Give Receive


Social relation to social support network

Immediate family Extended family Lineage Non-relative

Figure 6: Social relationship to network


12


Most respondents felt that there were functional structures to manage conflict resolution (see Figure 7: Conflict Resolution Mechanisms). These functional structures were traditional local leadership who dealt with conflict issues related to neighbours or family as well as with neighbouring communities. For disputes arising from the project – such as land division, invasion of livestock to vegetable farms or disputes about money paid for labour – they would be referred to the Boma Project Management Committee (BPMC). Overwhelmingly, people reported that security has improved over the period of the programme. We may estimate that given the programme coverage, BPMCs existed in 5 to 10% of the communities within a county.

Figure 7: Conflict Resolution Mechanisms

Key Findings & Discussion

The business case spoke of conflict and conflict sensitive approaches and made reference to the war, conflict in the home, gender related conflict, and inter-communal conflict, all of which are dealt with by traditional local leadership and institutions. There was no reported absence or change in traditional local institutions that manage conflict. In contrast, the issues referred to BPMC’s, the ones created and which received conflict training by the programme, are limited to programme related issues. There were isolated reports of conflict related to the programme – delineation of plots, animals invading vegetable gardens – that were managed by the BPMCs. There is no basis for the programme to have claimed it has changed the proportion of functional structures for conflict resolution, beyond managing conflict which the programme itself may have created.

The programme did however create a local institution, mandated to manage project issues. In that sense, BPMCs did what they were created to do. Moreover, researchers reported that local leaders were often recruited to sit on BPMCs. They say that was done to give the BPMC credibility and ensure good programme implementation. It created continuity between the BPMC and local institutions. Rather than changing the proportion of functional structures for conflict resolution, it seems that rather, the programme situated itself within existing local governance arrangements. Researchers reported that beneficiaries were satisfied with those arrangements. Also, given the programmes’ concern that its activities had the potential to negatively impact on women, creating a programme mechanism, BPMCs, to manage potential conflict was important. The fact that it would have linked potential complainants into local governance structures as well may have been a positive by-product.

IO6: Increase in income at household level

Indicators

Evidence of an increase in the household income through the project period was measured through an expenses module of the household survey, which also provided a snapshot into spending patterns. As no baseline or monitoring data exists for income levels, the MTE does not attempt to provide a precise number for a change in income. Instead, more qualitative proxies were developed that would 1) indicate improvements in income and 2) be feasible for beneficiaries to recall. These included increases in the

87.8% 12.2%

0% 20% 40% 60% 80% 100%

Presence of conflict resolution mechanism

Yes No


13

number of livestock, the ability to access education and afford medicine, being able to afford food during the lean season, and so on. The proxies are limited in the degree in which they indicate change. Also, precise numbers should be treated with caution as the value of money has been extremely variable during the past year.

The indicator draws heavily on the household survey and is supported by qualitative interviews.

Summary

This table provides the summary of the in-depth findings section by evaluation question:

Evaluation question

Findings


As this intermediate income, same as the others, was introduced in the spring of 2018, no baseline exists for it. It has no clear milestones, so the MTE cannot show progress against a milestone. It was also not possible to set up a precise baseline retrospectively due to the extreme variance in the currency over the past three years. As such, the MTE provides a less sensitive measure of change through larger proxies that would 1) be possible for beneficiaries to recall, and 2) be tied to levels of income. A more precise snapshot of the current situation acts as a baseline for further study, and the final evaluation.

The proxies (ability to afford health and education services, having sugar and tea, and being able to afford dinner during lean season) show a net positive development for both Components, though more prominent for Component I (as would be expected given its longer implementation period at time of fieldwork).

For Component I, around 30-40% of households indicate positive trends, whereas around 5% report reductions. For Component II, positive change is reported for between 20-30% of beneficiaries depending on the proxy and negative change is between 5-14%.

Changed wealth can be seen in the form of livestock, where goat purchases are quite common and cattle purchases rarer. Households without any livestock are still significant (20% for Component I and 44% for Component II), particularly those without cattle (49% for Component I and 77% for Component II). This suggests that although significant gains have been made during the project period, a significant portion of the beneficiaries are still extremely poor using this traditional measure.

The estimated total annual income (calculated from three different types of expenses) is as follows:

- Component I: South Sudanese Pound (SSP)163,288 annual (around $710); o Consumables: SSP79,581 (52% of total); o Occasional: SSP66,624 (43% of total); o Durable: SSP8,273 (5% of total);

- Component II: SSP102,111 annual (around $444); o Consumables: SSP62,334 (61% of total); o Occasional: SSP35,420 (35% of total); o Durable: SSP4,357 (4% of total% of total).

What can be observed from this data is that consumables still make up the majority of household expenses, though the difference between this and occasional expenses (service payments, less frequent and non-necessary consumables, and small purchases) are substantially higher in Component I. However, durable expenses, which constitute larger investments in goods and livestock remain low for both Components, which


14

suggests that moving from stabilisation to development is still uncommon. See figures 12, 13, & 14 for commodities included in these three expense categories.

- Beneficiary year

Only Component I was analysed on this parameter.

On the proxy indicators, differences are substantial between years 1 and years 2-3, but not between years 2 and 3. Similar patterns are observable in more precise expenses metrics. However, household accumulation of wealth – i.e. purchase of livestock, shows a linear increase between each year. As the investments in durable goods in the expenses module is similar for years 2 and 3, the most consistent hypothesis would be that a level where modest investments can be made is reached by year 2, and the year 3 difference is simply further time on that level.

In a wider regression controlling for demographic and agricultural factors, the number of years is consistently positive, but does not reach statistical significance for total expenditures, only for expenses on durable goods/assets.

It is significant to note, though the finding should be treated with caution due to the difficulty of establishing accurate SSP estimates, that the portion used for investments and wealth accumulation is well below the annual cash transfer value from the project.

- Gender Differences between the change within MHHHs and FHHHs are not substantial in the proxy indicators, though they do suggest slightly higher starting levels for MHHHs. Total expenses are within SSP4,000 of each other, i.e. not significantly different.

However, MHHHs are more likely to be willing or able to invest in livestock and other durable goods. This is likely partially a factor of higher surpluses produced (IO7), but the effect remains even when controlling for total expenses and value of production, which suggests that MH-HHs livestock investments are also a matter of prioritisation over other expenses.

- Partner No consistent story for change in income emerges by partner. However, it’s worth noting that NRC beneficiaries have among the higher recorded changes suggesting that despite the bad harvest of this year (see IO7), progress may have been made in the previous years and sustained.


Significant differences between the two Components exist on all indicators with Component I beneficiaries have consistently higher score on all measures. However, it is at this stage unfair to make clear conclusions from this on the project design or implementation itself given the fundamentally different stages of the partners’ projects.

One difference between the Components does exist in relation to income, as WFP was able to negotiate a higher rate for their cash transfers, meaning slightly more money went to their beneficiaries vis-à-vis Component II.

Findings at the midterm


15

Changes in income are suggested for half or more of Component 1 beneficiaries and perhaps a quarter of Component 2 beneficiaries (see Figure 8: Change in income proxies). The main driver of that change for Component 1 is a change in cash, largely made available through programme supported labour opportunities (Figure 9: Increase in cash by beneficiary year). Also, Component I reports substantially larger improvements across all change in income proxies, (as may be expected due to their length of enrolment in the programme). After cash, most beneficiaries in both Components report there has been no change in expenditures except for less than 10% reporting a decrease in their expenditures. More Component II respondents reported having less cash than in Component I. Closer inspection showed they are more likely to report improvements on other indicators such as school, medicine, or livestock (as is also seen with those reporting less cash in Component I). It would seem that the higher levels of decreased amounts of cash are actually indicative of investments in services, assets, or productive capacity, rather than of HH wealth or status – and suggesting stabilisation or development. The vital question relates to the mechanism through which the positive change is reported. Can these important changes be sustained?

77%

44%

9%

6%

5%

20%

62%

3%

7%

14%

6%

8%

9%

10%

14%

27%

44%

52%

54%

47%

28%

6%

22%

19%

32%

23%

20%

15%

5%

11%

4%

0% 20% 40% 60% 80% 100% 120%

Amount of cash

Number of cattle

Number of goats

Afford school

Afford medicine

Afford dinner (lean season)

Afford sugar

Change in income proxies (Component 2)

Remain bottom Decrease No change Increase Remain top

49%

20%

5%

1%

1%

2%

23%

6%

5%

7%

7%

7%

7%

5%

22%

29%

51%

42%

53%

51%

72%

23%

46%

30%

44%

34%

34%

8%

7%

5%

6%

0% 20% 40% 60% 80% 100% 120%

Amount of cash

Number of cattle

Number of goats

Afford school

Afford medicine

Afford dinner (lean season)

Afford sugar

Change in income proxies (Component I)

Remain bottom Decrease No change Increase Remain top

Figure 8: Change in income proxies


16

Out of those who report more cash available, 65.5% attribute it to the programme supported labour opportunities (cash transfers) and only 26.9% to increased agricultural production (a total under 10% combined attribution to better market access, and other livelihood or wage labour opportunities). However, there is large variance between the Components, as the Component I respondents report 58.6% for cash transfers and 32.5% agricultural production versus Component II’s 82.7% cash transfers and 12.9%

agricultural production. As in IO2, this finding does not suggests any fundamental differences between the two components, but rather an indication of the different stage of implementation of the two projects. However, the percentage for Component I is high considering that a large number of their beneficiaries are about to end their engagement with the project.

Disaggregating the findings by the number of years that the respondent has been in the project illustrates an upward trend in productive capacity vis-à-vis cash transfers (see Figure 9: Increase in cash by beneficiary year), but does not change the prominence of cash transfers, which remains the absolute majority response. This suggests that cash transfers, and therefore a temporary mechanism, has been more responsible for Household (HH)

development.

Using a logistic regression to predict increases in expenditures for schooling, medicine, availability of food, or sugar/tea based on household production variables produces statistically significant effects. The total production value (from SO7) has a significant effect on the likelihood that the household will also report an increase in livestock. This would make sense if income from employment is not sufficient to make a meaningful contribution to household wealth beyond hiring agricultural labour.

A more nuanced look at household incomes and its predictors was done by investigating the household expenditures by category type: consumable, occasional and durable (see Figure 10: Expenditure types by Component). Ascribing precise dollar terms on these values is not possible due to the currency fluctuation, but they nonetheless tell an interesting story through relative distributions. Firstly, Component I beneficiaries report spend significantly more money (requiring more income) than Component II respondents. Despite that difference, expenditures on consumables is not radically different between the Components. Larger relative differences are noticeable in the occasional and durable categories.

26.8%

1.2%2.4% 1.2%

68.3%

29.0%

1.2%1.2% 0.6%

68.1%

35.4%

1.4%6.0% 5.2%

52.0%

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%80.0%

Reasons for increased cash by beneficiary years (Component I)

Year I Year II Year III

Figure 9: Increase in cash by beneficiary year

79,58162,334

66,624

35,420

8,273

4,357

0

40,000

80,000

120,000

160,000

200,000


Expenditure types by Component (SSP)

Consumables Occasional Durable

Figure 10: Expenditure types by Component


17

When looking for differences in expenditures by category over the three years of the programme, Figure 11: Expenditure types by beneficiary year, more money overall is spent, in years two and three than in year one. If total amount of money spent increases in years two and there, there are no observable differences in how it is spent across the three categories. There is no observable difference in proportions of expenditures by category, year on year.

Further interesting findings emerge when expenditures within each category are examined. Figure 12: Consumption patterns in staple consumables, examines expenditures on consumables in depth. The differences between the Component I and Component II beneficiaries are high particularly for sugar and meat, which are more luxury than necessity expenses, which suggests higher levels of income. The difference in sorghum is explained by different consumption preferences. In Magwi County (Component II) there is a wider spread of expenses in different staples, (rice, maize, beans) than in NBEG.

Figure 13: Consumption patterns in occasional purchases, illustrates expenditures on commodities referred to as “occasional expenses”. Differences in spending on medicine, clothing, and school are by far the most significant categories for both components. This is fully in keeping with qualitative data, which suggests that the main uses of the cash from labour, after food, are school fees, medical services, and clothing (along with some other dignity/social status products). It is also interesting to see the degree to which medicine is the main expense in relation to

51211604

25022

8475

19475

6765

20676

15023

0

5000

10000

15000

20000

25000

30000


Consumption patterns - occasional purchases

Tea Coffee Milk Soft drinks Alcohol

Cigarettes Charcoal Wood Bus Mobile credit

Restaurant Hygiene Repairs Doctor Medicine

Clothing School Fuel

6240 49859077 7531

184349667

17184

577232751124710276

20255

0

50000

Component II Component I

Consumption patterns - staple consumables

Rice Maize Maize flour Wheat Cassava Sugar

Oil Bread Meat Chicken Eggs Fish

Figure 11: Expenditure types by beneficiary year

52.9% 55.4%49.3%

42.7% 39.2%45.1%

4.3% 5.4% 5.6%

0%

25%

50%

75%

100%

Year I Year II Year III

Expenditure distribution by beneficiary year (Component I only)

Consumables Occasional Durable

Total 134,542 SSP Total 160,960 SSP Total 155,988 SSP

Figure 12: Consumption patterns in staple consumables

Figure 13: Consumption patterns in occasional purchases


18

health care, as doctor consultations are barely an expense (they are often free/subsidised). The high investment in health and education reflects a beneficial and responsible use of money by beneficiaries.

Figure 13: Consumption patterns in occasional purchasesError! Reference source not found.Error! Reference source not found., illustrates expenditures on commodities referred to as “occasional expenses”. Differences in spending on medicine, clothing, and school are by far the most significant categories for both components. This is fully in keeping with qualitative data, which suggests that the main uses of the cash from labour, after food, are school fees, medical services, and clothing (along with some other dignity/social status products). It is also interesting to see the degree to which medicine is the main expense in relation to health care, as doctor consultations are barely an expense (they are often free/subsidised). The high investment in health and education reflects a beneficial and responsible use of money by beneficiaries.

Durable goods expenditures, Figure 14, are an extremely small expense category for both Components (approximately 5% of total expenditures). The types of assets commonly reported to be purchased relate to one’s dignity (bed) or savings/future wealth (goats for both Components and cattle also for Component I). Of the few household reporting this category of expenditure, just under half had bought goats, about 40% a bed, and 10% cattle in Component I. For Component II, only around 25% had bought a bed, around 20% goats, and just over 1% cattle. An important gender difference is also captured in the category of durable goods/assets purchases, where male headed households are likely to make substantially larger investments in this category and up to 50% more likely to invest in livestock, controlling for harvest value and total expenditure.

Analysis by partner yields no consistent findings on any partner, likely due to the low sample sizes when disaggregating at this level. However, it is worth noting that beneficiaries of NRC do report above-the-mean changes across the proxy indicators, which suggests that despite poorer performance in agricultural production in the current harvest, good progress has been made over the project period. However, NRC beneficiaries are also predominantly 3rd year participants, and as such more significant change could be expected.

Regression analysis on the effect of dosage shows that the effect more years in the programme is predominantly seen in the durable/asset category. Although time in the programme has a generally consistent positive effect, it does not reach statistical significance on other categories.

1377

1832

2567

237

2162

825

0

500

1000

1500

2000

2500

3000


Consumption patterns - durable goods/assets

Bed Furniture Watch Radio

Fan Iron Phone Stove

Bicycle Motorcycle Wheelbarrow Plough

Cattle Goat

Figure 14: Consumption patterns in durable goods


19

Other interesting points that emerge are that male-headed households are spending higher amounts on investments (levels that are statistically significant), though not on other categories or on overall expenses. Furthermore, people in reciprocal giving relationships have significantly higher levels of spending on the occasional category, suggesting they are supporting people outside their own household on these purchases, which include school fees and medicine. In TPMs, it was common for respondents to report paying school fees in particular for their relatives from their programme supported employment income.


It is important to know whether these changes are due to the injection of cash from programme created labour opportunities, which is temporary, or improvements in productive capacity, which is hoped to be sustained.

Along with IO7 (agricultural production), IO6 is a vital indicator to establish the project’s impact on the pathways to resilience. It is not possible to say with certainty if income for BRACE II households has changed, although the proxies suggest on aggregate a change in income is likely. However, for a large proportion of beneficiaries, income seems to have remained the same. The consistency between increased income from agriculture and results of IO7 would suggest that programme activities may be attributable to some degree to the observed change. As a whole, the reported value of this year’s harvest does not correlate with spending on the consumable or occasional categories, but it does for durable goods and assets. It is likely that income from the labour component of the programme is used for the stabilisation of day-to-day spending.

As a whole, beneficiaries are spending their money predominantly on food and services. Very few make expenditures on assets. We don’t know why they spend proportionately little on assets or other durable changes. It may be that cash is enough to cover more immediate needs but not enough to invest in assets or it may simply be a preference – some gender differences may exist on this axis. It must also be highlighted that occasional expenditures are roughly the same proportion as consumable and some occasional

Figure 15: Determinants of annual expenditures by type


20

expenditures are effectively long-term investments in household capacity and potential (particularly education and health services). It may reflect a parent choosing to invest in his or her children rather than a goat, or investing in farming for example.

Trends in expenditures over beneficiary years also tells an important story for BRACE II. Increases in expenditures from year one to year two are seen, but no changes in expenditures from year two to three. It appears that the main benefits are realised after the second year, similar to what has been observed in IO7.

Changes in income in BRACE II were ultimately intended to be based on improvements to agriculture (primarily staple crops, but also vegetables). BRACE II provided employment for those who would work on public works or block farms. The employment income was intended to stabilise household conditions at particular times of the year and allow them to pursue more substantive and sustainable income opportunities. That Impact pathway may have held true, for approximately 30 to 40% of respondents, possibly those reporting a significant change in income from agriculture, (see IO7). For most beneficiaries, however, it was remuneration from programme created labour opportunities that was responsible for changes in cash and possibly income. Qualitative interviews also suggest that many beneficiaries viewed the block farms as working for the partner and it took active and sustained efforts in communication to convince them otherwise – a process that still remains incomplete.

Expenditures were also partially consistent with the theory of change. While expenditures on food, and different types of food were made, as much or more was spent on health, education, clothing or assets. It appears that those categories of expenditure were equally important as expenditures on food. Also, respondents, or rather parents, invested heavily in their children rather than to strictly advancing their own economic future through investments in production. It may reflect a vision by parents that future well-being and safety, or in other words “resilience”, is vested in their children’s ability to seek opportunity in ways requiring education. If true, it would suggest a considerably different vision of programme impact, far beyond the logframe indicators framed around short-term food security.

IO7: Increase in the value of total grain production at household level, disaggregated by increase in production and cultivated area (MT)

Indicators

Progress for this outcome is measured against:

▪ Changes in cereal crop yields (sorghum, maize, millet)

▪ Changes in the total area cultivated by a family (or in polygamous arrangements, the fields managed by a single wife), which include the home fields, block farm and in some instances, far farms;

▪ With the lack of a baseline on production, the MTE had to establish change within the survey.3 All questions asked in the survey related to the harvest prior to the project start. For beneficiaries enrolled in different years, the harvest from which change is measured is different.

With preparation for the MTE, it became clear that the change made by the project would not be captured by investigating cereal crops alone, but through other crops also supported by the programme, which was borne out by the field research. Groundnuts, not sorghum or other cereal crops, emerged as the most important measurement of change.

3 Some FAO Post-Harvest Assessment data exists, but it is inconsistent and does not provide any starting points.


21

To provide a harmonised metric for evaluating value in a climate of extreme currency volatility and lack of reliable currency data, value calculations are presented as kg SE (kilograms in sorghum equivalency, see the Methodology section of the report for details).

Summary

This table provides the summary of the findings presented in detail in the following section.

Evaluation question

Findings


As the indicator was introduced in the spring of 2018, no baseline exists for it, nor have clear milestones been set out. Therefore, the MTE cannot show progress against a milestone.

The change within the indicator captured in the MTE depends on its framing. If only cereal crops (sorghum, maize, millet) are accounted for, the change is the following:

- Component I: Current production per HH was 424kg. This constitutes an increase of 45kg or 12% from pre-project.

- Component II: Current production per HH was 154kg. This constitutes a reduction of 16kg or 10% from pre-project.

However, if we include the total value of agricultural production (including groundnut, sesame, beans, cowpea), which would better capture the changes made by the project:

- Component I: Current production per HH was 1,405kg SE. This constitutes an increase of 661kg SE or 89% from pre-project.

- Component II: Current production per HH was 476kg SE. This constitutes an increase of 77kg SE or 19% from pre-project.

For both components, the change in production value is driven almost exclusively by groundnut farming, specifically by the 33% of beneficiaries that adopted farming groundnut. Up to 33% of beneficiaries also recorded no change or negative change, though in lower quantities, allowing for the substantial increase on aggregate. The rest had modest increases, but not transformative. The project does not account for the opportunity cost of the additional labour required to cultivate the additional feddan, and small increases are likely to be not enough to be good value for money if this was accounted for.

- Beneficiary year

When controlling for other factors, change in total harvest value was not significantly affected by the year of enrolment.

However, as change is pegged to the year before the project, which varies, this may not the best measure. Doing a regression analysis on the current year’s value including number of years, a significant difference was noted between beneficiaries on their first-year versus second- and third-year beneficiaries, though not between second- and third-year beneficiaries.

The difference is consistent with the increased land, as yield per feddan had no statistically significant differences by number of years in the project.

- Gender The value of change in total harvest did not significantly vary by the gender of the head of household. However, male-headed households recorded a higher yield (a factor of roughly 200kg SE) in the current harvest. This difference is driven by MHHHs farming more land (by roughly half a feddan). No difference is recorded on productivity between MHHHs and FHHHs.


22

- Partner The total harvest value and change for Component I partners were:

- CWW total average production of 1,368kg SE (+511kg SE);

- ACF total average production of 1,270kg SE (+665kg SE);

- WV total average production of 1,217kg SE (+419kg SE)

- JAM total average production of 2,259kg SE (+1,549kg SE)

- NRC total average production of 957kg SE (+248kg SE)

The total harvest value and change for Component II partners were:

- SAADO total average production of 318kg SE (+20kg SE)

- SPEDP total average production of 419kg SE (+86kg SE)

- WV total average production of 656kg SE (+114kg SE)

The standout differences by partner can be seen within Component I with JAM recording changes hugely more significant than other partners, and NRC somewhat more modest gains.

The lack of higher changes for NRC is likely a combination of two factors: 1) lower levels of groundnut adopters; and 2) large numbers of block farms of sorghum being destroyed by floods. Although this may signify 2018 as a particularly bad year for Twic, and as such not a signifier of poor performance at large, flooding is a common and foreseeable occurrence that was not sufficiently considered in either the selection of land or protections built around the land.

JAM’s higher performance is partly explained by the highest number of groundnut adopters (63% of JAM beneficiaries) as well as reported availability of suitable land. However, JAM’s level of presence on the ground is also substantially higher than other partners that tend to rely more on community structures to monitor progress. This is likely to be the main structural explanation for the better results. Mechanisms through which this helped with the change include high levels of understanding and enthusiasm regarding the project reported by researchers, as well as the main differences being driven by the amount of land cultivated. The data suggests that when community structures are relied on, the majority of beneficiaries will only farm the land that is tied to the cash payments.


The differences between Components I and II for this indicator are extreme both on the total amount of harvest and the change in harvest from the previous year, with Component I scoring substantially higher on both counts.

However, we do not think that it is at this stage appropriate to draw conclusions on the differences between components, as the stage at which they are in their implementation is fundamentally different. This would lead to a comparison of apples and oranges. The main reasons why the stage itself may strongly affect the findings are:

1. There are strong indications in qualitative data that suggest that the first year in Component I was equally low in impact. In fact, no block farms were created in the first year. Interviews suggest that it took the first year yields being retained by beneficiaries for the trust that benefits of labour would be reaped, which then facilitated more effort to be expended by the farmers. Even Year 1 beneficiaries for Component I predominantly resided in bomas where the project was active and as such the example and social proof existed;

2. It’s highly more likely that Component II beneficiaries would have downplayed their yields for fear that the harvest from the block farm in particular would be taken away, as the trust and social proof from previous years did not yet exist.


23

There are, however, some questions of design and implementation that likely had an impact on changes:

1. Component II decided to conduct seed distributions directly rather than through an Input Trade Fair like Component I. There were delays in the delivery that contributed to the majority of groundnut seeds to have been spoiled, unusable. Although seeds are generally available at the market, expectation of receiving them may have caused Component II beneficiaries to hold off procuring them independently in time for optimal planting;

2. There is some reason to think that the HHs enrolled in Component II are less well off than their Component I counterparts. They score consistently lower on quality of life indicators, but not in other metrics. This would potentially make them less able to make use of the opportunities provided by the project, and was brought up in the TPM visits where many elderly vulnerable persons were observed cultivating land. However, this may also be explained by respondents overplaying their need in fear of losing benefits otherwise.


This section provides detailed results from the data collected in relation to the indicator on agricultural change. We begin with cereal crops, but continue the analysis with total production value, as this better reflects the change made by the project.

As can be seen from

Figure 16: Changes in cereal , below, have been extremely modest across the project. For Component I, the overall increase is below a single bag of sorghum (44.7kg). Some differences exist between male and female headed households, but the difference is only significant in relative terms – in absolute terms the difference in total cereal production change is 8kg of sorghum. For component II, a minor reduction in cereal crops was recorded with no real gender differences.

Figure 16: Changes in cereal production (kg)

However, when examining changes in the total production value, rather than only staple grains, a considerably different picture emerges. A substantial change in groundnut production is seen in Component I and a similar but much smaller change is seen in Component II. Figure 17 presents those changes in terms of sorghum equivalent and kilograms.

44.7 47.839.7

0

10

20

30

40

50

Total (n=729) Male-headed HH(n=450)

Female-headedHH (n=279)

Change in cereal production (kg) Component I

Change in cereal yield

-16.3 -16.8 -15.8

-20

0

20

40



Change in cereal production (kg) Component II

Change in cereal yield


24

Figure 17: Change in total production (kg and value)

The changes can be disaggregated by sex. Although there are some differences by the gender of the head of household, these differences are minor when looking at the change in production value (see Error! Reference source not found.).

Figure 18: Changes in total production value by sex

-15.2

-1 -0.318

629.2 3.4

-15.2

-1 -0.23.8 16 1.8 2.4

-50

150

350

550

750

Sorghum Maize Millet Sesame Groundnut Beans Cowpea

Change in harvested crops (Component II)

Value Yield

460.1

-1.9 -12.5

622

7.1 0.446

0.1-1.2 -2.6

160

1.4 0.3

-50

150

350

550

750

Sorghum Maize Millet Sesame Groundnut Beans Cowpea

Change in crops pre-project to current (kg yield and kg SE value) (Component I)

Value Yield

661 696604

0

200

400

600

800



Change in total production value (kg SE) (Component I)

Change in total production value

77 9749

0

200

400

600

800



Change in total production value (kg SE) (Component II)

Change in total production value


25

Figure 19: Total harvest value by crop (current harvest cycle)

As can be seen in the three previous graphics, groundnuts are the source of significant change among project beneficiaries and represent the largest constituent of total production value (dominantly so for Component I). These findings prompted further enquiry on the role groundnut played in the change. As such, data on beneficiary farmers was examined for their relationship to groundnut farming before the project and currently.

Four different relationships emerged: 1) constant groundnut farmers; 2) groundnut adopters; 3) groundnut dropouts; and 4) constant non-groundnut farmers. Disaggregating data using these four categories of relationships to groundnuts reflects a range of relationships to mechanisms of agricultural change and suggest a role or influence of programme activities in these production outcomes. It appears that certain beneficiaries – groundnut adopters - had a strong positive relationship with the programme’s activities intended to increase production, for example. Equally, it appears that other beneficiaries did not. The two graphs in Figure 1820 summarise changes in total household production (measured in sorghum equivalent) by the four categories.

The amount of land cultivated increased for three of the four categories, and in levels that reflect programme activities and conditionality. Labour payments were conditioned on beneficiaries preparing, seeding and weeding one feddan in the group, block farms. Fields in block farms were intended to be additional to the home and/or far farms of beneficiaries.

404

19

0.39

129

829

20 0.9

Harvest value by crop (kg SE) Component I

Sorghum Maize Millet

Sesame Groundnut Beans

143.8

13.3

0.39

112.9

197

14.5 6.2

Harvest value by crop (kg SE) Component II

Sorghum Maize Millet

Sesame Groundnut Beans

1435

503

1060

494

1683 1740

499602

0

500

1000

1500

2000

Constant groundnut farmers(n=239/212)

Groundnut adopters(n=355/322)

Groundnut dropouts(n=25/22)

Non-groundnut farmers(n=220/185)

Value of production by farmer type (kg SE) (Component I)

Total harvest value before project Total harvest value at 2018

Figure 20: Total production value by farmer type


26

MTE results show increases in the total amount of land under cultivation for each category of farmer/beneficiary (Error! Reference source not found.). Increases are generally of one feddan per household, which are consistent with the incentives provided by the programme. The small number of groundnut dropouts report no change or report an increase in total land under cultivation beyond one feddan, with change the largest among groundnut adopters.

Farmers were asked how their production had increased. 89% of those who reported increases attributed more land cultivated as a significant reason compared to only 37% who felt it was due to improvements in productivity. In contrast, of those who stated that their production had reduced, 34% said it was as a result of farming less land and 80% said this was due to lower productivity.4

Figure 22: Change in productivity (yield/feddan kg SE) contrasts changes in yield (total land cultivated with total harvest in sorghum equivalent) before the project and at the MTE. Consistently, productivity drops or in other words they received less return for their labour (with the exception of Component II non-groundnut farmers, but it’s worth noting that their increase in land is less than 1 feddan, suggesting that they stopped cultivating some parts of their existing land). Decrease in yields may reflect programme activities, which incentivised an clearing, seeding and weeding new land. Incentives were not given to those who produced more than before. In contrast, however, Groundnut adopters, report an increase in productivity, particularly in Component 1 of almost 100%. Their change must be understood to reflect the importance of adopting groundnuts. Using a metric of sorghum equivalent means that a kg of groundnuts produced is

4 It was possible to attribute change to both increased/decreased land and productivity.

656

260

540

281

697569

249 325

0

500

1000

1500

2000

Constant groundnut farmers(n=275/196)

Groundnut adopters(n=195/165)

Groundnut dropouts(n=44/38)

Non-groundnut farmers(n=298/273)

Value of production by farmer type (kg SE) (Component II)

Total harvest value before project Total harvest value at 2018

Figure 21: Change in land cultivated

32.3 2.8 2.4

4.2 3.93.2 3.4

0

1.5

3

4.5

Change in land cultivated (feddans) by farmer type Component I

Land cultivated before project

Land cultivated currently

3.32.2

3.5

2.4

4.43.3 3.5 3.2

0

1.5

3

4.5

Change in land cultivated (feddan) by farmer type Component II

Land cultivated before project

Land cultivated currently


27

almost four times more valuable than sorghum. The 250kg change in productivity of groundnut adopters represents 60kg if it were sorghum and not groundnuts.

In Figure 23: Total value and change in production by partner, when we look at differences between partners, further trends of interest begin to emerge, as can be seen. For Component II, it is at this point still too early to make larger statements regarding differences, and the absolute size of change is not highly different between partners.

However, for Component I JAM has been highly successful in comparison to other partners, with their beneficiaries growing their production by a striking 1,549kg SE, or tripling their production. On the other hand, NRC beneficiaries have recorded a substantially lower change. Interestingly, the starting point for the beneficiaries of all Component I beneficiaries is remarkably similar, suggesting indeed that the change is driven by differences in programming.

Explaining the substantially better results reached by JAM, the researchers who visited their locations attributed the difference to one external and one project-based source. An external factor was the availability of land highly suitable for groundnut production (sandy soil), which aided their beneficiaries in taking full advantage of the opportunities provided by the new crop. Coupled with the highest level of groundnut adopters (63% of the sample), this provided a substantial boost to crop production as a whole.

Figure 23: Total value and change in production by partner

219

111175

103

172 188

79 106

0

200

400

600

Constantgroundnut

farmers(n=238/168)

Groundnutadopters

(n=139/122)

Groundnutdropouts(n=39/34)

Non-groundnutfarmers

(n=262/255)

Change in productivity (kg SE/feddan) (Component II)

Value per feddan before project Value per feddan currently

505

246

443

252

436497

154 179

0

100

200

300

400

500

600

Constantgroundnut

farmers(n=220/203)

Groundnutadopters

(n=334/302)

Groundnutdropouts(n=24/20)

Non-groundnutfarmers

(n=202/171)

Change in land productivity (kg SE/feddan) (Component I)

Value per feddan before project Value per feddan currently

Figure 22: Change in productivity (yield/feddan kg SE)

857605 798 710 709

298 333 542

511665 419

1549

248

20 86114

0

500

1000

1500

2000

2500

CWW ACF WV JAM NRC SAADO SPEDP WV


Total value and change in production by partner (kg SE)

Value prior Increase during project


28

However, it was also noted that their beneficiaries have a significantly higher buy-in and enthusiasm towards the project activities. The best explanation for this seems to be a project management decision to rely less on community structures and have a higher level of presence by JAM staff. This additional staff investment likely led to quicker and better understanding and trust towards the project activities, as well as better oversight on ensuring that project norms are followed. This is supported by the amount of land cultivated, where JAM was the only partner to have an average increase in feddans under cultivation reach two. It is therefore likely that community managed oversight concentrates more on the land cultivated for the year in question, as this ties directly to the cash transfers.

Conversely, the poorer performance of NRC beneficiaries seems to be mainly created through external conditions, particularly large amounts of flooding that entirely wiped out yields in block farms. This seems to have affected particularly those block farms that cultivated sorghum. It is likely that land type suitable for sorghum is more difficult to locate, and the areas available are more prone to flooding. This ties into project management questions as well, as the partner should take care in the selection of land, and also take appropriate steps to reduce the likelihood of shocks affecting the selected land.

Finally, to answer questions regarding the effect of years of enrolment and gender, a regression analysis was conducted. This was limited to the Aweils in order to control for the combination of poor results in Twic and its beneficiary composition, which is almost entirely made up of 3rd year beneficiaries.

What can be observed is that those farming groundnut, whether they were farming it before the project or not, have a highly larger yield than those who do not, strengthening the evidence of its impact. We can also observe a consistent positive effect of the project years from year 1 (the constant line against which other year variables are plotted). However, the changes are not statistically significant unless coupled with the household being a vegetable beneficiary and reaching a level where this has become a source of income.

As such, the data from the MTE suggests that although benefits seem to accrue beyond the first year of enrolment, a third year does not add value beyond this. It should, however, be noted that the third year beneficiaries would have farmed their HH land during the first year, and as such did not follow the norm planned by the project design.

The gender of the household head does have a significant, though modest in comparison, effect on the value of production. This is explained by the larger amount of land cultivated by male-headed households vis-à-vis female-headed ones. In regards to productivity, there is no gender difference. The implication of

this finding is that the likely gender difference relates to the availability of labour either in the household, through social networks, or hired labour. This mechanism is a consistent finding in the TPM visits, where single-headed households struggle to maintain larger farmland.


IO7 is of fundamental importance to the programme design and consequently to the MTE, thus it was important to further interrogate the production data and try to understand the emerging results – some farmers responding, some not and marked differences between Component 1 and Component 2.

Firstly, looking at baseline figures (see Annex 1: Historic Yield and Farm Sizes) it is interesting to note that BRACE II farmers have average farm sizes and some considerably larger than those reported over the last 10 years.

Figure 24: Determinants of harvest value


29

Given the programme’s emphasis on serving the vulnerable, one might expect sizes to be consistently below average. Secondly, yield per feddan ranges between averages of 300 to 600 kg per feddan for the same 10 year period. Very broadly, reported levels of production pre-programme are consistent with what might be expected from baseline farm sizes and productivity per feddan. The changes reported (with the exception of groundnut adopters) might be expected within normal levels of variation of production. However, considering the increases in land size, changes (increases) are less than what one might expect, which is consistent with reports of decreases in productivity.

Groundnut adopters (in Component 1) report changes beyond baseline norms and beyond what might be expected - increases of 1 to 1.5 feddans in farm size and substantial changes in production value. Those changes can be explained by the adoption of a new, higher value (in sorghum equivalent terms) crop, such as groundnut. They also represent 33% of the beneficiaries (42% for Component I and 24% for Component II). The rest of the beneficiaries did not on aggregate see substantial increases in their production.

Explaining why the same activities should affect one group of individuals differently than another is complex. It is not easy, nor necessarily possible to explain these results. For example, one might expect larger farmers already using more complex production systems i.e. groundnut farmers as groundnut production is not strictly subsistence but at least quasi commercial,5 to use programme incentives to make greater and more commercial investments in their farms. Before the programme, their farms were larger and producing more than averages for the counties. However, that was not the case.

A number of explanatory hypotheses are put forward to try and makes sense of the main findings. They are framed around an appreciation of the purposes and limits to different beneficiary farm production systems and an extra-ordinary motivation to move beyond those systems.

Cropping, generally, in traditional, rural livelihoods and for Dinka farming systems in particular is a complementary not primary source of food and income (see 0). External notions of farmers must be put in context of both Dinka norms as well as subsistence household requirements.

Baseline levels of annual household sorghum consumption in South Sudan are approximately 660kg6. It is an amount which correlates to expected production from 1.5 to 2 feddans. Farming 1.5 to 2 feddans means a family can meet its grain requirements with the minimum level of effort and risk. Subsistence levels of grain production are also influenced by the “wealth” of a household, “In most areas of southern Sudan, own crop production alone rarely makes up even half of total needs…” (Fielding, Sharp, Gullick, & Coutts, 2001, p. 14). Historically, “poor households” exchange or purchase as much sorghum as they produce (Muchomba & Sharp, 2006, p. 40).7 Subsistence production systems have evolved to balance household labour resources with household food needs, and is better understood from that perspective.

Current average farm size in Northern Bahr el Ghazal (NBEG) is 2 feddans, or simply the amount of area required to meet grain needs in a subsistence livelihoods system. As food and income for these people are derived as much or more from non-farm sources, one must consider carefully the incentives and costs of farming beyond these levels. Within their cropping systems, household labour – both in number, health and age – are fundamental determinants of crop production (Fielding, Sharp, Gullick, & Coutts, 2001, p. 20)8. Increasing farm production, for many South Sudanese farmers, is challenging for that reason. Command of labour are features of better-off households who are two to three times more productive and cultivate farms on average two to three times larger than poorer households (Muchomba & Sharp, 2006, p. 40), sizes which some of the groundnut farmers in the sample are reaching. If labour is a crucial success factor for crop production (and livestock keeping), its absence, is a mark of vulnerability. The Dinka term,

5 Groundnuts in the areas in question are traditionally grown commonly in small quantities for household consumption and commercialisation has been limited. However, it has great potential as a cash crop, which beneficiaries are also starting to observe. 6 WFP/FAO Crop and Food Supply Assessment mission continues to use this per capita estimate. 7 Findings in income indicators suggests large amounts of spending on sorghum among beneficiaries. 8 It is useful to note, that in this sense a young family may be “vulnerable” according to local notions of the term. It also i llustrates that vulnerability is not equivalent to poverty or structural economic arrangements, as they young children grow up, this family can move out of vulnerability with greater available labour.


30

“nyop”, describes “those within the society and identified by the society who suffer unduly from the harsh conditions that are experienced by all in society” (Harigan & Chol, 1998, p. 20). It refers to transient set of conditions, making one household less able to manage conditions than their neighbours but not permanently. Given the importance of labour to production in the subsistent systems, illness, age of children, pregnancy, etc. can have important consequences in the short term.

Groundnuts, while used in the home, are also a cash crop. Farmers who commit significant parts of their farm to groundnuts are pursuing a more commercial objective than their subsistence farming colleagues. They approach challenges of labour differently, not simply to subsist, but to produce surplus.

Returning to the MTE findings for IO7, this background and perspective on traditional subsistence farming systems, the role of labour as well as the nature of “farmers” themselves in a rural, South Sudan context can be helpful to understand the differentiated results.

Changes in the total harvest value prior to the project in Component 1 for groundnuts adopters and those who did not adopt groundnuts were virtually indistinguishable from each other (503kg vs 494kg SE), which is also similar to current production levels of groundnut dropouts are (499kg SE). We can also see similar patterns with groundnut farmers averaging size around 4 feddans and those who do not at around 3. This may represent the limits of where these type of production systems will reach on aggregate, and gains above it require increased levels of effort and investment. It may be possible for successful commercial farmers to continue investment and grow their land and production through hired labour, but the roof for subsistence farmers is likely to be more firm.

Similar patterns can be observed with Component II, and the fundamental story between the four groups is consistent, except that no group is on aggregate above county averages (from external data). Transformative change can be seen among those who adopted groundnuts, who more than doubled their production value (from 260kg SE to 569kg SE). Those who stopped cultivating groundnut see substantial negative change (540kg SE to 249kg SE), whereas marginal improvements are observable within groups that maintained their farming systems.

This analysis and consideration of different farming systems and agriculture within traditional South Sudan livelihoods systems does not explain why some people appear to have used programme incentives to make significant changes to their levels and types of production. Furthermore, no demographic variables were significant in predicting whether a household would adopt groundnuts – as such suggesting the main determinants are partner decisions and the availability of suitable land. Appreciating the challenges to increasing not only land but production however, could highlight an important disincentive to make those changes.

It also causes reflection on the distinction between a demand for income or cash versus demand for more food or income from farming. Comments by beneficiaries to researchers often stated that they valued labour opportunities and its money but not necessarily the farming. Given the challenges to increasing production and considering the vagaries of weather, pests, and disease to determine yields, choices to pursue income from sources other than farming is rational.

This choice to pursue labour income rather than income from production is also reflected by the fact that rarely do farm sizes increase beyond one feddan. Each year, the programme oversees and financially incentivises the clearance of a single feddan. It is either unable to monitor or enforce whether or not the previously cleared land being is used again (with the exception of JAM). The data here suggests that at least for some beneficiaries, they do not continue farming all the cleared block farm land. It also raised the question of whether or not they will cultivate the block farms at the end of the programme.

IO8: Increase in number of food / income sources at households

Indicators


31

This indicator seeks to measure:

- Change in the number of different sources of either food or income, relating to resilience through potential diversification of livelihood sources and differing risk profiles;

- As no baseline figures or monitoring data exists on livelihoods, and the project directly supports only agriculture and horticulture, the lists of livelihoods relates to the most common types and is based on beneficiaries’ recall for pre-project information.

The indicator is based on the household survey and supported by qualitative interviews and observations.

Summary

This table provides the summary of the findings presented in detail in the following section.

Evaluation question

Findings


No clear operationalisation exists for this indicator on what food and income sources are. The analysis followed four different measures:

1. Number of land plots and size: a. Component I: Increase from 1.4 to 2.3 plots;

b. Component II: Increase from 1.4 to 2.2 plots;

2. Number of crop types cultivated: a. Component I: Increase from 2 to 2.2 crops;

b. Component II: Increase from 1.9 to 2 plots;

3. Vegetable gardening: a. Component I: Increase from 26% to 46%; b. Component II: Increase from 32% to 40%;

4. Number of income sources: a. Component I: No change, remain at 2.2; b. Component II: Increase from 1.7 to 1.8.

Generally, the analysis showed increase in specialisation and transplanting lower value activities with higher value ones, such as away from collecting hay and towards agricultural and horticulture sales. As such, although the number of active sources of income and food did not necessarily change, likely due to limited labour, the current activities are likely to be more lucrative and the ones no longer engaged in are not lost and could still be relied on if needed.

- Beneficiary year

No significant differences found between years of enrolment and any of the measures within the indicator.

- Gender Female-headed households farm slightly fewer crops (2 vs 2.2) and number of income sources (1.9 vs 2.1). This is likely due to lower labour capacity as FHHHs are more likely to be single-adult households. The scale of the difference currently is similar to what it was at the starting point, suggesting again that the differences existed and were not exacerbated by the project – both household types changed in similar ways during the project.

However, when looking at the types of income sources engaged in, changes away from low-value activities such as collecting hay is higher for FHHHs, which suggests higher impacts for FHHHs.


32

- Partner On the main measures for the indicator, no large differences exist between the partners, with the exception of vegetable production, where substantial increases were made by JAM, NRC, and CWW.


Component II locations are consistently lower than Component I on all indicators apart from land size. However, given the fundamentally different stages of the projects, it would not be fair to conclude from this anything in relation to differences in implementation or design.


Enquiry into changing income sources can be started from the livelihoods that were directly supported through the project – agriculture and horticulture. On agriculture, the increase in sources can be measured through new crops and new land.

From the analysis on IO7 on agriculture, it is already established that a proportion of beneficiaries in the sample (42% for Component I and 23.8% in Component II) have already made a significant increase in their food/income source through the adoption of an additional crop. However, on aggregate the data does not support fundamental increases in the types of crops farmed by beneficiaries.

The number of crops farmed by the average household increased from 2.00 to 2.17 for Component I and 1.92 to 2.03 for Component II. This suggests that limitations exist for households on either their willingness or capacity to increase their crop diversity. Large variation can be seen again for the different types of farmers, as adopters grow their crop averages from 1.51 to 2.40 where both farmer types that maintained their farming system reduced their crop diversity (from 2.88 to 2.61 for groundnut farmers and 1.43 to 1.36 for sorghum farmers). This is consistent with the hypothesis that all other things being equal, increasing farming land will come at some cost (diversity in addition to productivity seen in IO2). However, it also shows that those that are farming larger values of total harvest are also able to cultivate more diverse crops, i.e. scale and diversity correlate positively.

Figure 25: Types of crops farmed, by Component

As can be seen in Figure 25: Types of crops farmed, by Component, the lowering diversification does not affect all types of crops equally. Sorghum farming as a core of household cultivating remains unchanged, with reductions seen mainly for other crops such as maize and sesame in Component I. Component II does not show significant aggregate negative changes, but increases in specific crops (particularly groundnut) are also more modest. Overall, the story told by these figures is one of increased specialisation rather than diversification.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Before project Currently Before project Currently


Types of crops farmed

Sorghum Maize Sesame Groundnut Beans Millet Cowpea


33

Figure 26: Changes in types of land, by Component

In relation to the use of plots where food can be produced, the data shows that on aggregate the number of different locations has grown. The block farms have not on average displaced other plots, though very small reductions in the size of far and household farms can be seen in both Components. However, it should be noted that questions remain regarding the sustainability of the block farms, as 38% of 2nd year beneficiaries and 57% of 3rd year beneficiaries (Component I) reported farming only 1 feddan of block farm land.

Moving to horticulture, increases in farming, irrigation sources, and diversity can be seen

in the data. For Component I, around one in five beneficiary households have adopted vegetable gardening over the project period, and just under half of the households now produce vegetables. For Component II, the trend is similar though less pronounced. However, if similar growth was sustained for additional years, it would likely surpass Component I achievements on this measure. Irrigation has also improved, which is positive particularly as it’s done at the same time as many new cultivators began. However, it should be noted that only around 50% of the irrigation sources relate to project activities (ponds – component I only as Component I irrigation sources, if any, would not have been finalised at the time of fieldwork). High levels of irrigation are very positive, as year-round horticulture can make significant contributions to food security during the lean season.

In addition to increasing vegetable production as a whole, respondents also report an increase in the diversity of their vegetable crops. The average farmer is now farming a total of 4.3 (4.5 for Component I and 4.0 for Component II) different vegetable types, an increase of 0.75 (1.2 for Component I and 0.4 for Component II).

1.8 1.7 2 1.9

0.7 0.60.7 0.6

1.5 1.1

0

0.5

1

1.5

2

2.5

3

3.5

4



Changes in types of land (feddans)

Household land Far farm Block farm

26.2%

46.3%

31.9%40.2%

77.9%

89.8%

49.6%

64.8%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%



Vegetable gardening and irrigation

Vegetable gardening Irrigation

Figure 27: Change in vegetable gardening and irrigation, by Component


34

Finally, we can analyse the reported income sources reported by the respondents as a whole. Although we can’t talk about their magnitude in terms of dollars or SSP, there are some noteworthy patterns in the livelihoods. As can be seen in Figure 29: Change in income sources by component, below, there are significant increases in the degree to which beneficiaries report the core activities of agriculture and horticulture being sources of money rather than simply food. For Component I, more than half of the farmers are selling part of their harvest. Although constant groundnut farmers are much more likely to sell portions (around 60% versus 46% for adopters and 34% for non-groundnut farmers), whether the farmer reports planning sales seems to be predicted best by total production value and presence of sesame. This suggests that particularly adopters are not yet entirely clear of the commercial potential for groundnut. Furthermore, almost all Component I vegetable farmers report gaining an income from it, where this is not, at least not yet, the case for Component II.

An additional positive finding visible in both Components is the reduction of households reporting income sources that are 1) economically harmful, or 2) low value, or both. These are particularly collecting wild materials (low value) and felling trees and making charcoal (both). These have reduced for both

0%10%20%30%40%50%60%70%80%90%

100%



Change in vegetable types cultivated

Regela Kudura Sukuma Okra Onion Amaranthus

Karkadeh Tomato Watermelon Eggplant Cabbage Pumpkin

0%

10%

20%

30%

40%

50%

60%

Before project Current Before project Current


Change in sources of income

Agriculture sales Horticulture sales Livestock rearing Agricultural work

Other work Collect wild materials Fell trees / charcoal Business

Hunting Remittance

Figure 28: Types of vegetables cultivated by component

Figure 29: Change in income sources by component


35

Components, though more dramatically for Component I. This is likely contributed to by both the reduction of abject poverty through cash transfers and the further time investment into agriculture and horticulture due to incentivisation.

As a whole, the total number of household income sources have not changed, but they have been replaced with (by and large) better ones – from collecting wild fruits or tree felling / charcoal towards agriculture and horticulture. These are better both in the value they can generate and their environmental effects. Furthermore, it is unlikely that the beneficiaries lose these sources of income by not engaging in them, but rather can resort to them in the future if the need arises. The only potentially questionable development is that the number of businesses have reduced in number. It is difficult to judge the opportunity cost that this may create. However, from qualitative data we know most of the businesses beneficiaries run are low value, such as tea shops, buying basic goods at bulk and selling in small portions, roasting groundnuts, etc.

Some interesting gender differences can be seen when looking at the livelihood profiles of male and female-headed households (Figure 30: Change in income sources, by gender (Component I). Firstly, all changes noted in the aggregate are more radical for FHHHs. Their agricultural and horticultural change is substantially larger, as is the reduction in wild materials collection, along with businesses. Secondly, FHHHs remain substantially lower in the higher value categories of agricultural sales and livestock rearing.

When looking at all these different types of indicators by partner, a roughly similar story can be seen for each partner on the majority of the measures (Table 1: Changes in food and income sources by partner). Most of the differences observed in the current situation reflect the starting point. However, notable divergences from the norm can be seen in the high levels of change in vegetable cultivation for JAM and NRC, and to a smaller degree CWW, where the change is over 20% of the entire beneficiaries and more than 50% of the total sample engages in it. SPEDP beneficiaries were also highly more likely to engage in horticulture prior to the project, which is sensible as they live in a fundamentally different area of the country (Magwi).

Table 1: Changes in food and income sources by partner Component I Component II


# plots prior 1.6 1.4 1.5 1.2 1.3 1.4 1.5 1.2

# plots current 2.5 2.3 2.2 2.2 2.2 2.3 2.4 2

# crops prior 2 1.8 1.7 1.9 2.5 1.2 2.3 2.1

# crops current 2.4 1.8 1.5 2.3 2.6 1.2 2.1 2.7

0%

10%

20%

30%

40%

50%

60%

Before project Current Before project Current

MH-HH FH-HH

Change in income sources

Agriculture sales Horticulture sales Livestock rearing Agricultural work

Other work Collect wild materials Fell trees / charcoal Business

Hunting Remittance

Figure 30: Change in income sources, by gender (Component I)


36

# income sources prior

2.1 2.2 2 2.2 2.5 1.3 2 1.5

# income sources current

2.4 2.1 1.9 2.2 2.6 1.3 2.1 1.8

# vegetable gardeners prior

35% 26% 17% 20% 29% 22% 56% 10%

# vegetable gardeners current

56% 37% 26% 52% 53% 33% 61% 20%

Key Findings & Discussion

While there is likely to be inter-annual variation in sources of food and income (i.e. hardship may require a person to make charcoal one year and not another), the order of the changes (the number of people reporting a different source of income), might suggest changes beyond the norm. Also, changes in sources of food and income closely align with programme supported activities. Researchers also reported that in some communities, respondents commented that due to the project, it was no longer necessary for them to migrate to Darfur for work.

Changes in sources of income also reflect changes observed in other BRACE II activities. The programme intended to not only increase incomes but diversify sources of income and food by creating seasonal labour (public works, and block farms), supporting crop production (subsidised labour, training and seed/tool inputs) and irrigated vegetable gardens (provision of seeds, tools, subsidised labour and water supply). These mechanisms do seem to have made those changes, and to a level commensurate with proportion of people involved with those activities. The exception is labour – which we would have expected to have been on a much larger scale. Contextually, it would appear that there is a demand for income or cash, which at least in part was met by programme activities.

In summary, though the project may not have significantly increased the number of food or income sources, the data suggests that the beneficiaries have been able to become more able to specialise on higher value livelihoods, particularly commercial agriculture and horticulture.

2.2 Final Outcomes It is not possible at the mid-term to indicate change for the final outcomes. Component 1 partner WFP had assessed food consumption score (FCS) and Household Dietary Diversity Score (HDDS) previously in NBEG but those results are not comparable, as they used different sample frames and conducted the surveys at different times of year, when the food situation is considerably different. The MTE assessment of final outcomes does provide a snapshot of current conditions, and will provide a baseline for the assessment of final outcomes in the FE.

FO1: Prevalence of poor and borderline food consumption, disaggregated by sex of household head (FCS)

FO2: Diet diversity score of targeted households, disaggregated by sex of households head (HDDS)

Indicators

The indicators measure the basic food security, caloric intake, and dietary diversity:


37

▪ The FCS module is based on a standard survey used by WFP and its Vulnerability Assessment and Mapping unit;

▪ The HDDS is adapted from the FCS and used by WFP SS in their monitoring. It is not the standard HDDS module, but rather draws on the FCS 7-day recall reconstructed into binaries, rather than the 1-day recall more commonly used. The application of the WFP SS HDDS operationalisation was used in order to ensure figures in monitoring could be compared;

▪ WV conducted a separate HDDS using the standard module, which will feature in their own reporting – there may thus be some differences between the MTE and Component II internal reporting for this indicator;

▪ All Component II data to date was collected in May, and is therefore not comparable with the MTE due to strong seasonal variance in food security. Component I data has been gathered in October for years 2016 and 2017, so is somewhat comparable.

Figures are drawn from the household survey.

Summary

Evaluation question

Findings


For Component I, somewhat comparable data exists from the WFP Post Distribution Monitoring (PDMs) (up to October 2017), but no targets for 2018 have been provided to the MTE team. The WFP data suggests that the prevalence of poor and borderline FCS scores has been reduced from 93% to 60% between 2016 and 2017, with significant gender differences (49% for MH-HHs and 72% for FH-HHs). HDDS has similarly increased from 2.51 to 4.3 during the same time with no significant gender differences at either point. No targets have been provided for 2018 for FCS, but a 15% increase for HDDS was given (from baseline rates).

No comparable data exists and no targets were provided for Component II, though a baseline in May had 82% of the households at poor or borderline FCS levels. As such, the MTE provides solely a snapshot of the current situation.

The FCS and HDDS are as follows:

1. Component I a. FCS: 28.4% poor or borderline;

i. MH-HHs: 25.9% poor or borderline; ii. FH-HHs: 32.3% poor or borderline;

b. HDDS: 6.2 i. MH-HHs: 6.2

ii. FH-HHs: 6.3

2. Component II a. FCS: 63.4% poor or borderline;

i. MH-HHs: 63.8% poor or borderline; ii. FH-HHs: 62.7% poor or borderline;

b. HDDS: 5.1 i. MH-HHs: 5.1

ii. FH-HHs: 5.2

In summary, substantial improvements against these indicators were measured for both Components.


38

- Beneficiary year

For FCS, a significant difference was recorded between 1st year vs 2nd and 3rd year, but not between 2nd and 3rd year beneficiaries. No meaningful difference was recorded for HDDS.

FCS: - Year 1: 41 - Year 2: 45 - Year 3: 46

HDDS: - Year 1: 6.1 - Year 2: 6.3 - Year 3: 6.3

However, it should be noted that most of the 1st year beneficiaries reside in locations where programming has already been taking place and they may as such have benefited more quickly from project activities due to stepping into functional structures.

- Gender For FCS, the mechanism for differences between MHHHs and FHHHs seems to be best explained through the higher yields and higher likelihood of belonging in reciprocal networks. The gender difference only exists for FCS rather than HDDS, suggesting that the difference is driven by the frequency of eating particular foods rather than their inclusion in the general diet.

- Partner Some differences exist between the partners, but the trends are similar across the board. An interesting finding is that JAM beneficiaries have among the lowest FCS scores for Component I, despite their IO7 successes, suggesting they have invested in agriculture which has come at a cost of at least short term food consumption.

For Component II, SAADO beneficiaries are particularly low and the only ones with a mean average within the poor consumption zone, though their HDDS score is not significantly lower.


Due to the different stage of the two Components and differences in data availability, it is impossible to make strong conclusions. The only relevant comparison is that over the first year of programming, Component I beneficiaries had improved on aggregate by 10%-points and Component II by 19%-points (with no gender difference). However, Component I beneficiaries started from a lower point, which may have contributed to slower change.

Findings

Findings of the FCS are summarised in Figure 31: FCS by Component by both Component and gender of HH head. The differences in both FCS and the FCS categories (poor, borderline, acceptable) are substantial between the Components. Component I beneficiaries score positively with almost three quarters of the sampled households reporting acceptable FCS scores and a mean score of 45 well above acceptable limit of 35, and only around 5% of the households with poor FCS scores. The sample for Component II is substantially different, though, with borderline levels of FCS being the majority and poor FCS category including up to a quarter of the households. The mean of 31 is also within the borderline FCS category.


39

A smaller, but important difference, can be observed when disaggregating the data by the gender of the household head. For both Components, male-headed households have a slightly higher FCS score (mean 3 and 2 points higher respectively).

Somewhat similar patterns can be seen when looking at HDDS in Figure 32: HDDS by Component. However, two noteworthy differences exist. Firstly, the percentage of poor and borderline categories is reduced for both Components. Secondly, the difference between the male and female headed households disappears. This suggests that the gender differences can best be explained not by the variety of food groups as a whole, but rather the scale at which they are consumed – i.e. male-headed households eat more in quantity rather than absolute variety.

When we examine the constituent parts of FCS, the differences between the Components become clear and extremely interesting. In Figure 33: FCS in detail & by Component below, it is clear that the large differences in scores are driven almost exclusively by the consumption of meat and dairy, which Component I beneficiaries consume more than twice as often. Because of the high score value of the two food groups (each day consumed is multiplied by four), the differences become extremely large.

Basically no differences exist in staples, legumes, vegetables, fruits or oil. The difference in sweets is large in quantity, but almost irrelevant due to its low factor in the FCS (factor of 0.5). All in all, the consumption pattern differences are consistent with Component I beneficiaries being more well-to-do, whether this is caused by the project or otherwise.

Regression analysis can shed some further light into the potential mechanisms through which the project may have contributed to the FCS. The main statistically significant (p<.5) determinants of FCS scores emerge as being a constant groundnut farmer, value of the harvest, income/expenses, being part of reciprocal social networks, cultivating vegetables commercially or having a business, receiving remittances, and being a year 2-3 beneficiary (though no significant difference was recorded between years 2 and 3, only against 1st year

71.6% 74.1% 67.7%

36.6% 36.2% 37.3%

22.7% 21.4%24.7%

38.7% 41.3% 34.7%

5.7% 4.5% 7.6%24.7% 22.5% 28.0%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Total MH-HH FH-HH Total MH-HH FH-HH


FCS categories

Acceptable Borderline Poor

Mean=45 Mean=46 Mean=43 Mean=31 Mean=32 Mean=30

Figure 31: FCS by Component

80.1% 79.7% 80.8%

48.8% 46.8% 51.7%

16.2% 17.0% 14.9%

34.1% 36.2% 31.1%

3.7% 3.3% 4.2%17.1% 17.0% 17.2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Total MH-HH FH-HH Total MH-HH FH-HH


HDDS categories


Figure 32: HDDS by Component


40

beneficiaries). Importantly, gender differences disappear when controlling for these factors, suggesting that it is better explained by MHHHs having higher yields and a higher likelihood of being in reciprocal

networks rather than by gender itself. A negative gender effect for MHHHs actually exists for dietary diversity, though on aggregate this is negated by the positive effects of higher yields.

Differences exist between partners for FCS (Figure 34: FCS by partner). For Component I, WV and JAM show the lowest FCS groups, though both still with a majority of their households as well as their mean average scores within acceptable scores. Similar trends exist within HDDS, but to a much lower scale, suggesting again that the differences relate to frequencies of consumption – especially of high value meat or dairy products. The finding regarding JAM is particularly interesting in relation to findings from IO7, where JAM beneficiaries reached by far the largest change and final values. This would suggest that they may have

88%77%

60% 57%

71%

26%

45%36%

10%20%

28% 35%

25%

33%

41%

40%

2% 3%13% 8% 4%

41%

14%24%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%



FCS groups by partner


Mean=51 Mean=45 Mean=39 Mean=41 Mean=44 Mean=24 Mean=32 Mean=34

Figure 33: FCS in detail & by Component

0.0

2.0

4.0

6.0

8.0

10.0

12.0

0.0

2.0

4.0

6.0

8.0

10.0

12.0

FCS in detail

Component I Food group eaten Component II Food group eaten

Component II Value Component I Value

Figure 34: FCS by partner


41

made larger investments in their productive capacity rather than food. For Component II, SAADO beneficiaries are particularly low on FCS, both in percentages and mean averages, though do not differ significantly on HDDS.

Discussion

The expenditure component of the household survey indicated that more money had been spent on food during the programme period – regardless of whether or not incomes changed. It may be possible that by affecting the liquidity of households, beneficiaries were able to spend more money on high FCS foods (milk, legumes and meat) than normal.

As a primary objective, BRACE II was largely framed around food security indicators. Implied if not explicit, was that different activity packages should result in the production of more food, more income to buy food or more cash which would be used to buy food. To a certain degree these assumptions were reflected in the results. Even if a small percentage of the total sample reported changes in crops produced or income, more money was spent on food by these individuals than had been spent prior to the programme.

Without a baseline, it is difficult to conclude to what degree BRACE II mechanisms changed FCS, but it may be that current FCS levels are influenced by BRACE II activity packages.

FO3: Proportion of targeted households implementing crisis and emergency strategies, disaggregated by sex of household head (Livelihoods CSI)

Indicators

Questions 24-25 from the household survey, are the basis of the Livelihoods Coping Strategy Index (LCSI).

Summary

Evaluation question Findings


For Component I, the LCSI has been measured by the project monitoring, starting from a baseline of 71%, down to 66.3% in late 2016 and down to 23% in 2017. A target of 50% from baseline was set for 2018.

Component II baseline measured the indicator at 63% and no target was provided for the MTE.

1. Component I: 33% implementing crisis/emergency a. MHHHs: 34% implementing crisis/emergency; b. FHHHs: 31% implementing crisis/emergency;

2. Component II: 47% implementing crisis/emergency a. MHHHs: 43% implementing crisis/emergency; b. FHHHs: 51% implementing crisis/emergency.

- Beneficiary year No consistent pattern between the LCSI and beneficiary years.

- Gender Component I differences by gender minimal. For Component II, FHHHs slightly more likely to report LCSI crisis/emergency.

- Partner For Component I, NRC beneficiaries substantially more likely to report crisis/emergency levels. This is consistent with findings regarding major harvest losses due to flooding in the previous year (IO7).

For Component II, SAADO beneficiaries are substantially more likely to report emergency levels of LCSI, which is consistent with their poorer results across other indicators.


42


At the current stage of the program, reliable conclusions cannot be made.

Findings at the midterm

The findings suggest that aggregate improvements for both Components have been reached during the project period, and those are generally consummate with targets (for Component I). The gender differences that were present in WFP’s previous monitoring have largely disappeared (or reversed for Component II), but it is impossible to say whether this is due to differences in data collection or a genuine difference on the ground.

Figure 35: LCSI

In contrast, actions of approximately 35% of respondents would be classified as Crisis or Emergency. Emergency levels are driven particularly by households reporting that they have engaged in begging for Component I and equally by begging and selling the last female animal for Component II. Crisis levels are driven almost exclusively by reductions in service expenses for Component I and consumption of seeds and reductions in service expenses equally for Component II.

It is worth noting that across the board, N/A options were common (where such coping action is not available). As such, it is difficult to say to what degree the indicator captures the conditions on the ground.

Key Findings and Discussion

Improvements from baselines from both indicators (although for Component II, the different time of data collection makes comparisons difficult) suggests real change. However, it is difficult to know if the coping on non-coping actions are transitory or permanent. Researchers did observe a source of bias, however. Begging is an important determinant of HHs classified as in Crisis or Emergency. The English term, “begging”, translates poorly into Dinka. Actions used to describe begging do not fit well into cultural sharing norms nor understandings of destitution. For example, it is often normal for a child to eat at the home of an extended family member or neighbour. There are however times when a child may be sent to the home of another to eat as the food at home is not sufficient. Researchers said that such actions are considered

48% 47% 49% 49%58%

52%

67%

17%

36% 37% 33%39%

23%

49%

19% 19% 20% 20%

22%22%

16%

17%

18% 19%16%

12%

24%

16%

18% 18% 17% 12%

14%15%

13%

33%

25% 25%

25% 16%35%

20%

15% 16% 14% 19%

6% 11%4%

33%21% 18% 27%

33%

19% 15%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Tota

l

MH

-HH

s

FH-H

Hs

CW

W

AC

F

WV

JAM

NR

C

Tota

l

MH

-HH

s

FH-H

Hs

SAA

DO

SPED

P

WV


LCSI

No coping Stressed Crisis Emergency


43

shameful, and a respondent is unlikely to confide such actions to an unknown person. These challenges may result in an overstatement of households in Crisis or Emergency and understate households forced to exercise extreme coping actions.

Even if there is no means of demonstrating a change in LCSI, changes associated or correlated with positive changes such as having more savings, spending more money on food, or reducing undesirable income activities (cutting firewood/charcoal, collecting forest products) are the types of changes which would lead to more positive responses to Livelihood CSI questions. There may be a relation to the current levels of No Coping or Stressed and BRACE II packages of activities.

FO4: Reduced vulnerability to climate risks and shocks

FO5: Reduced vulnerability to communal conflict

Indicators

Claims by the programme of reduced vulnerability to climate risks and shocks is assessed by an analysis of the intermediate outcomes. The analysis (resilience dimensions analysis) tests for the degree to which individual intermediate outcomes (or demonstrated synergies across intermediate outcomes) are demonstrate BRACED’s five dimensions of resilience. The risk informed element is critical to ascertain reduced vulnerability either to climatic hazards or communal conflict. It considers reduced exposure, specific to those hazards or an increased capacity to manage their consequences.

The results of the analysis are further triangulated with sections of the household survey, enquiring about respondents’ perceptions to changes in security and vulnerability.

Based on this process, the indicator is the score the outcome receives against the resilience dimensions analysis and particularly their performance as risk informed.


Evaluation questions 4: Number of people whose resilience has been improved as a result of project support ICF-KPI 4 in the report provides both the results and discussion on these two final outcomes.

Based on an analysis of the degree to individual intermediate outcomes or demonstrated synergies across intermediate outcomes, it was determined that 43% of respondents could be considered to have made substantive and possible sustained changes (as a result of the programme) in both the short and long term which reduce the impact of climate related shocks. The remaining 67% also made important progress in directions that will also reduce their vulnerability to climatic hazards and their consequences, but those gains will not be sustained.

The activities relating to BPMC, anticipated to directly influence factors to manage community conflict, did not demonstrate the magnitude nor scale of change to influence factors beyond programmatic concerns. Factors governing community conflict remain under the auspices of local governance mechanisms which pre-existed BRACE II. We might say that BPMC has reduced vulnerability to programme beneficiaries to programme created conflict. By influencing “sharing” the programme also sought to increase peace in these communities (i.e. decrease conflict). It was shown that the nature and outcomes of different activities has

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

Less likely No change More likely

Male-headed HH Female-headed HH

Figure 36: Project effect on likelihood of community conflict by sex of head of household


44

likely influenced an increase in the “depth” of participation in social support mechanisms. It could be said to have deepened the inclusion of some of the programme beneficiaries. However, these changes cannot be considered to have modified community (i.e. population level) social cohesion. In fact, there is no evidence social cohesion was lacking at the outset of the programme. We may be able to say, that by virtue of increasing the engagement in social support mechanisms, the programme may claim to have possibly increased their ability to manage the consequences of community conflict. It is difficult to estimate the proportion of respondents who could make that claim. Figure 36: Project effect on likelihood of community conflict by sex of head of household indicates that the majority of respondents feel there they are less likely to be affected by conflict than before the programme. It is a very encouraging response. However, there is no reason not to believe that BRACE II contributed to that outcome. Considering that factors determining conflict reside beyond the scope of influence by BRACE II activities, there may be a positive but limited contribution made by BRACE.

Key Findings and Discussion

The analysis of intermediate outcomes highlighted issues with the BRACE II conception of risk and the nature of resilience to those risks. It also demonstrated what may be perceived as a disconnect with this conceptual goal of the programme and the actual design and implementation of programme activity packages. Pathways to resilience appear to be inter-generational, and vested in the future education and employment or self-employment of beneficiary children. The notions of climate risk are framed around traditional, subsistent means of farming. Resilience to those risks was conceived in increasing their exposure to climate risks and making incremental modifications to farming production systems. While consideration of the means of educating children must be climate risk informed, assumptions that pathways to climate resilience are through farming may be questioned. Furthermore, such a pathway did not resonate with approximately 67% of the programme’s beneficiaries, who made no changes to total farm production, but rather seized the labour opportunities. Programme activity packages showed some changes which may be useful to certain types of farms to negotiate normal seasonal variation but are unlikely to see the farm navigate a real climatic shock (flood or meteorological drought). Similarly, the programme did not demonstrate contingencies they helped develop with respondents, so that in the case of farm failure, they might manage its consequences.

Analysis of intermediate outcomes on conflict and conflict related vulnerability suggests that some programmatic assumptions were not aligned to the reality in BRACE II communities nor could the activities designed be expected to have influenced factors governing peace and to a very limited degree, to manage the consequences of community conflict. While BRACE II assumptions assumed the absence or deficiency of social cohesion leading to community conflict, respondents indicated rather the opposite. Respondents wished to engage further with existing arrangements. Secondly, the magnitude or scale of programme influence, was likely limited to programme beneficiaries. This is a function of the design of BPMC, not a defect in its implementation nor performance. There is no programme activity which showed that it intended to influence the performance of existing and traditional governance structures that influence peace nor inclusion of social support to those affected by conflict.

2.3 Impact

I1: Reduced prevalence of moderately and severely food insecure households (in counties where BRACE II is implemented)

I2: Resilience Index Measurement and analysis (RIMA II) (in counties where BRACE II is implemented)

I3: Crop & Food Supply Mission (CFSM)


45

Rates of changes in prevalence of acute malnutrition are to be reported by WFP in the Annual Needs assessment. FAO conducts both the RIMA and CFSM. The results for 2018 are not yet available.

Summary

We are at the midterm, and examining progress of the programme not assessing its impacts. DFID asked this section to be included allowing for reflection on whether or not the programme is on track to achieve these impacts.

Measurements of impact are all based on national assessments. We must bear in mind that for BRACE II to attribute its outcomes to changes (positive or negative) in these population level assessments:

▪ Its intermediate outcomes must demonstrate a magnitude and scale adequate enough to ensure that changes made within the programme are of an order which could modify results in an entire population. As the CFSM measures primarily sorghum, making only reference to groundnuts, given that sorghum farmers in the programme have reported no change in production, it may be unlikely that a change in CFSM total production or county level yields could be explained by BRACE II.

▪ GAM/SAM and RIMA are multi-factorial conditions and only measured in children under five. Key causal factors include amount of time mothers have to look after their children, and illness which is untreated. As at least half of the under 5 cohort is likely breast feeding, changes in household food availability will not influence those children. For non-breast feeding children, while they have high energy and nutritional requirements, they can be met through relatively small quantities of food. BRACE II intermediate outcomes which might relate to these outcomes could considerably be increased income to pay for medical costs, food diversity – if it is eaten by children and at times of the year, when GAM/SAM rates have their seasonal peak.

While the indicators and metrics of BRACE II’s impact are desirable, programme activity packages, IOs and even FOs do not attribute very directly to those changes both causally or with regard to the scale of those measurements. This does not reflect poorly on the programme, its outcomes or implementation but rather reflects programme design.

BRACE II Mid-Term Evaluation Report

46

References Climate Change Compass. (2018). Number of people whose resilience has been improved as a result of

ICF: KPI 4 Methodology Note. London: DFID.

Dfid. (2018). Building Resilience through Asset Creation and Enhancement Phase Two. South Sudan. London: Dfid.

Fielding, W., Sharp, B., Gullick, C., & Coutts, P. (2001). An Introduction to the Food Economy Research in Southern Sudan 1994-2000. Nairobi: WFP & Save the Children (UK).

Harigan, S., & Chol, C. (1998). The Southern Sudan Vulnerabilty Study. Nairobi: Save the Children Fund (UK).

IMC Worldwide. (2017). BRACE II Inception Report. Redhill: IMC Worldwide.

Muchomba, E., & Sharp, B. (2006). Southern Sudan Livelihood Zone Profiles. Juba: Southern Sudan Centre for Statistics & Evaluation.

Annex 1: Historic Yield and Farm Sizes

Source: FAO/WFP Crop and Food Supply Assessment Mission Reports

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Met

ric

Ton

nes

BRACE II: Historic Yield per Feddan

Aweil East

Aweil Centre

Aweil North

Aweil South

Aweil West

Twic

Magwi


47

Sorghum Yield per Feddan

Year Aweil East Aweil Centre Aweil North

Aweil South Aweil West Twic Magwi

1996 0.105 0.105 0.105 0.105

1997 0.105 0.105 0.105 0.105

1998 0.273 0.273 0.273 0.273

1999 0.286 0.286 0.286 0.286

2000 0.210 0.210 0.185 0.210

2001 0.252 0.252 0.252 0.248

2002 0.134 0.134 0.090 0.210

2003 0.277 0.277 0.290 0.315 0.336

2007 0.294 0.168 0.336 0.336 0.294 0.336 0.378

2008 0.294 0.168 0.336 0.336 0.294 0.462 0.630

2009 0.361 0.403 0.403 0.563 0.361 0.601 0.441

2010 0.420 0.441 0.420 0.462 0.420 0.420 0.441

2011 0.420 0.441 0.420 0.462 0.420 0.420 0.441

2012 0.336 0.336 0.378 0.336 0.420 0.302 0.420

2013 0.378 0.504 0.378 0.420 0.504 0.378 0.420

2014 0.420 0.504 0.462 0.504 0.517 0.504 0.588

2015 0.378 0.504 0.378 0.504 0.504 0.546 0.630

2016 0.336 0.483 0.441 0.462 0.483 0.504 0.588

2017 0.378 0.462 0.441 0.521 0.517 0.504 0.630

Average 2007-17

0.365 0.401 0.399 0.446 0.430 0.452 0.510

Median 2007-17

0.378 0.441 0.403 0.462 0.420 0.462 0.441

Average Low 2007-17

0.292 0.321 0.320 0.357 0.344 0.362 0.408

Average High 2007-17

0.438 0.482 0.479 0.535 0.516 0.543 0.612

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

2002 2003 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

Hec

tare

sBRACE II: Historic Cultivated Hectares per Household

Aweil East

Aweil Centre

Aweil North

Aweil South

Aweil West

Twic

Magwi


48

Average Ha/HH

Year Aweil East Aweil Centre Aweil North Aweil South Aweil West Twic Magwi

2002 0.66 0.66 0.66

2003 0.70 0.30 0.80 0.66 0.70

2007 0.55 0.30 0.55 0.70 0.55 0.80 0.70

2008 0.55 0.30 0.55 0.70 0.55 0.70 0.70

2009 0.55 0.30 0.55 0.70 0.55 0.65 0.80

2010 0.60 0.55 0.60 0.70 0.60 0.75 0.80

2011 0.60 0.55 0.60 0.70 0.60 0.70 0.80

2012 0.80 0.50 0.75 0.80 0.80 1.00 1.20

2013 0.80 0.60 0.75 0.70 0.80 1.00 1.20

2014 0.80 0.60 0.75 0.70 0.80 0.90 1.40

2015 0.70 0.84 0.75 0.84 0.80 0.84 1.20

2016 0.70 0.84 0.84 0.84 0.84 0.84 1.00

2017 0.72 0.84 0.84 0.84 0.84 0.84 1.00

Average 2007-17

0.670 0.565 0.685 0.747 0.703 0.820 0.982

Median 2007-17

0.700 0.550 0.750 0.700 0.800 0.840 1.000

Average High 2007-

17 0.536 0.452 0.548 0.598 0.562 0.656 0.785

Average Low 2007-

17 0.804 0.679 0.821 0.897 0.843 0.984 1.178


49

Annex 2: Historic Wealth Groupings and Sources of Income While South Sudan provides consistent information on approximate farm sizes and yields over time, equally, significant variation in the size of farms and their yields has been noted according to farm size. The traditional agricultural production systems are very sensitive to labour. Labour is required for land preparation, planting, weeding and harvesting. Wealth – in its traditional sense (and to a certain degree in changing poverty definitions, See SHHS 2009) reflect a households control over productive assets – fields and animals. By understanding the nature of the production systems, wealth implies a certain level of labour. In contrast, poverty, or an absence of productive assets (see An Introduction to the Food Economy Research in Southern Sudan 1994-2000) such that a household cannot feed itself or participate/contribute towards communal events, implies a lower level of access to labour.

From this understanding, we wish to better understand hypothetical, historic or baseline levels of production – farm size and yields – by poorer households, which are claimed to be the majority of BRACE I and BRACE II programme clients/beneficiaries.

If the Crop and Food Supply Assessment Mission 2017 estimated that the average farm size for NBEG to be 2 feddans and producing a yield of 450kg/feddan9 (or 9 x 50kg sacks), we might estimate that poor households in the same community have farms of approximately 0.5 feddan, and produce 225kg (or 4.5 x 50kg sacks).

For BRACE II, it will be important to understand the size of fields managed by a household (i.e. woman and her children – in the case of polygamous marriages) prior to the start of the BRACE II programme as well as approximately how many sacks were harvested (and allowing a factor to account for losses or food eaten green)

▪ Estimate how much they produce compare it against baseline % of own production and the CFSAM 2017 assumption of how much dura is eaten by a home of 110kg/pp/an.

▪ 60% of their food is grain – 20% is grown. So, if 110kg = 60%; 35kg = 20% or 35kg x 6 persons = 210kg of their food comes from their farm. Pg 40.

The information is drawn from the 2006 Livelihood Profiles (Muchomba & Sharp, 2006)

2.3.1 Western Flood Plain Livelihood Zone

9 These estimates are gross values and exclude grain that might have been harvested green, losses, or post harvest losses.


50

The average acre or feddan in 2007, was approximately 1.2 feddans; a poor household – or those farming for BRACE II would be significantly less.

2.3.2 Green Belt Zone


51


52

IMC Worldwide Ltd 64-68 London Road Redhill RH1 1LG Tel: +44 (0)1737 231 400 Fax: +44 (0)1737 771 107 www.imcworldwide.co.uk

Documents

Building Resilience Through Asset Creation and Enhancement … · BRACE II Mid-Term Evaluation Compendium 1 1 Introduction to the ompendium (Detailed Analysis of Individual Outcomes