This article was downloaded by: [York University Libraries]On: 20 November 2014, At: 00:16Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number:1072954 Registered office: Mortimer House, 37-41 Mortimer Street,London W1T 3JH, UK

Development SouthernAfricaPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/cdsa20

Using group ratings toassess household foodsecurity: Empiricalevidence from southernAfricaCalogero Carletto , Charles Masangano , GillesBergeron & Saul S MorrisPublished online: 01 Jul 2010.

To cite this article: Calogero Carletto , Charles Masangano , Gilles Bergeron& Saul S Morris (2001) Using group ratings to assess household food security:Empirical evidence from southern Africa, Development Southern Africa, 18:5,541-552

To link to this article: http://dx.doi.org/10.1080/03768350120097423

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of allthe information (the “Content”) contained in the publications on ourplatform. However, Taylor & Francis, our agents, and our licensorsmake no representations or warranties whatsoever as to the accuracy,completeness, or suitability for any purpose of the Content. Anyopinions and views expressed in this publication are the opinions andviews of the authors, and are not the views of or endorsed by Taylor& Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information.Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilitieswhatsoever or howsoever caused arising directly or indirectly inconnection with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private studypurposes. Any substantial or systematic reproduction, redistribution,reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of accessand use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

0:16

20

Nov

embe

r 20

14

Development Southern Africa Vol 18, No 5, December 2001

Using group ratings to assesshousehold food security: empiricalevidence from southern Africa

Calogero Carletto, Charles Masangano, GillesBergeron & Saul S Morris1

This article addresses the reliability and validity of household welfare rankings using the GroupRatings (GR) method. The GR aimed to measure the food security status of 142 households inseven villages in Malawi. Sets of informant groups rated households from their own community.Results show that the reliability of the method was no more than fair to moderate, and wasparticularly low for households falling in the middle category of ‘intermittently food insecure’.Consensus ratings from the GR sessions were then compared with a number of alternativeindicators of food security from a quantitative household survey. GR were associated with themore visible aspects of food security, such as household asset and livestock holdings, butassociations with less visible aspects of food security were weaker. The strength of theseassociations varied from village to village.

1. INTRODUCTION

The proper identi� cation of vulnerable households in a community is central to theeffective implementation of targeted poverty alleviation and food security programmes.Timeliness in generating this information is also critical for economic and logisticalreasons, and rapid appraisal techniques are therefore used frequently by donor agenciesfor this purpose. The same limitations attributed to other qualitative methods – lowexternal validity and lack of statistical precision – are widely acknowledged, but it isgenerally assumed that rapid appraisal techniques do provide useful data quickly andinexpensively. In particular, they are seen to be adept at incorporating local perspec-tives, enabling development workers to capture dimensions that may escape moreformal data collection approaches (Grandin, 1983; Chambers, 1997).

One rapid appraisal technique frequently used by development agencies is that ofGroup Ratings (GR) of household welfare, wealth-ranking being the best-knownvariant. The approach involves having households ranked by informants from the samecommunity according to a speci� ed criterion (wealth, prestige, food security, vulner-ability, etc). The method has been put to many uses, including stratifying householdsfor screening purposes (Scoones, 1992), identifying local criteria of wealth (Mukherjee,1992) and establishing criteria for measuring vulnerability through formal surveys

1 Respectively, Economist, Development Research Group, World Bank, Washington, DC; Head,Department of Rural Development, Bunda College of Agriculture, University of Malawi, Malawi;Food Security Specialist, Food and Nutrition Technical Assistance Project, Academy forEducational Development, Washington DC; and Senior Lecturer, Department of Epidemiologyand Population Health, London School of Hygiene and Tropical Medicine, London. Work reportedin this article has been supported by a grant from the International Fund for AgriculturalDevelopment (IFAD) to the International Food Policy Research Institute (Grant 301-IFPRI). Theauthors wish to thank IFAD for its support for this work, but stress that the views and opinionsexpressed here are their own and should not be attributed to IFAD.

ISSN 0376-835X print/1470-3637 online/01/050541-12 Ó Development Bank of Southern AfricaDOI: 10.1080/0376835012009742 3

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

0:16

20

Nov

embe

r 20

14

542 C Carletto et al

(Seeley et al, 1995). (See also Feldstein & Jiggins, 1994 and Mearns & Shombon,1992, for more details on such approaches.)

The popularity of GR has led various researchers to examine critically the conditionsunder which the method performs best. Methodological questions tend to focus on thereliability and internal validity of the GR method. Reliability concerns the extent towhich a measuring procedure yields the same results on repeated trials, whereasvalidity refers to how well a particular tool captures the referent it is intended tomeasure (Carmines & Zeller, 1983; Nunnally & Bernstein, 1994).

Several pieces of research have shown problems in both areas. Chadwick et al (1995)found that the wealth ranking method was not a precise tool for establishing thesocio-economic pro� le of an area. Social differences, in this case gender and caste,affected ratings (see also (Welbourn, 1992 for similar � ndings). Criteria used forevaluating wealth also tend to differ between groups (Chadwick et al, 1995). Otherauthors (Scoones, 1992; Mearns & Shombon, 1992) report similarly that results fromwealth rankings are susceptible to variations depending on the criteria used. The latterauthors suggest prudently that the method be used as a complement, rather than as analternative, to more conventional survey approaches. Direct empirical tests of thereliability of the GR method have also been undertaken, revealing a low consistencyof responses between sets of raters (Bergeron et al, 1998).

The research reported in this article highlights problems with reliability. It alsoillustrates the dif� culty that GR methods may have in measuring multidimensionalconstructs, such as food security and vulnerability. The research � ndings suggest that,whereas GR may provide reasonably reliable estimates of the readily observabledimensions of food security (as proxied by physical assets), they do not fare well in theassessment of less visible, yet equally important, dimensions of food security asassessed by dietary diversity, food expenditures, calorie intake and our index of copingstrategies.

2. FIELD METHODS

In November 1998, the International Food Policy Research Institute, in collaborationwith the Rural Development Department of Bunda College of Agriculture in Malawi,conducted a number of studies in several villages of the Kandeu Extension PlanningArea in central Malawi. First, two rounds of a detailed household survey wereconducted in the region to assess quantitatively the level of household food security.Secondly, a series of qualitative exercises were conducted in seven villages, selectedrandomly from the communities included in the quantitative study, to generatequalitative classi� cations of households’ food security status using the GR method.

The household surveys covered approximately 700 rural households. The � rst roundwas administered in January–February 1998, and the second round took place inJuly–August of the same year. The timing of the two rounds was based on the need togather information relative to the pre-harvest – coinciding with the peak of the ‘hungryseason’ – and the post-harvest, or ‘plenty season’. Data on a number of food securityindicators were collected in both rounds. These included conventional indicators, suchas food acquisition, and frequencies of consumption, as well as less orthodox measuressuch as information on households’ responses to food shortages. Also, information onhouseholds’ asset ownership was gathered, together with complete expenditure � gures.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

0:16

20

Nov

embe

r 20

14

Using group ratings to assess household food security 543

A more detailed presentation of the survey methodology, the variables and some of theresults can be found in Carletto (1999).

For the group ratings, 142 households were selected from seven communities. In viewof the small size of the communities, the selected households in each community kneweach other quite well. Also, in several villages, they belonged to the same farmers’association.

The procedure used in the GR exercises was to have the selected households that weredivided into small groups classify themselves into three mutually exclusive categories:food insecure, intermittently food insecure and food secure. The facilitators de� nedfood-insecure households as those households that experience food shortages systemat-ically every year. The food-secure group was de� ned to include all those householdsthat had access to suf� cient food for all their members throughout the year, and eachyear. Intermittently food-insecure households were de� ned as those households that areusually food secure but become exposed periodically to the risk of food shortagesfollowing unforeseen shocks or short-term circumstances.

Each GR session began with a brief presentation of the study objectives and a de� nitionof food security by a trained facilitator. Most of the facilitators had previous experiencewith GR techniques and one of the authors, responsible for the � eld coordination andthe facilitator’s training, is a lecturer on participatory methods at a national university.Next, the informants were asked to split into smaller groups of between three and sixinformants. Except in a few cases, the groups divided along gender lines.

The Index Card method was used for the food security rating exercise. The name ofeach household was written on a separate index card and a complete pack of names wasgiven to each informant group. Each group then sorted its pack into three separate pilesbased on the degree of food insecurity of the household, as perceived by the membersof the group. Beans were used to illustrate the categories. Food-secure households wererepresented by a large heap of beans and a card labelled ‘Food Secure’. Intermittentlyfood-insecure households were represented by a medium heap of beans and a cardlabelled ‘Intermittently Food Insecure’ and food-insecure households were representedby very few beans and a card labelled ‘Food Insecure’. The cards with the names werehanded to one member of the group who was asked to read each name aloud. Thisperson and the other group members then decided on which pile to place the card. Ifno one in the group was able to read, the facilitator assisted by reading out the names.All the group sessions generated very active discussions. When the group memberscould not agree on the rating of a household, they were instructed to put the card asideand rediscuss it after all the other households had been ranked. Once all cards had beenplaced, the informants were asked to go though each pile again to review theirclassi� cation. On several occasions, households were reclassi� ed during this revision.

Once all households were classi� ed, each group was asked to explain the criteria theyused for rating the households the way they did, eg the factors causing a household inthe food-insecure pile of cards be classi� ed as such. Once the individual groups had� nished their exercise, all the groups came together in a plenary session during whichthe rankings of the different groups were compared. Intergroup disagreements in theclassi� cation of particular households were discussed further in the plenary session inorder to force a convergence of opinion on a ranking on which all or the majority ofinformants were able to agree.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

0:16

20

Nov

embe

r 20

14

544 C Carletto et al

Table 1: Percentage of households in each food security class, based ontheir average score across informant groups

Number of Food Sporadically Food

Site households insecure (%) food insecure (%) secure (%)

Chibalala 16 63 25 12

Fasi 15 35 47 18

Kadam’manja 13 38 28 31

Mazonda 16 39 47 14

Muwalo 30 53 44 3

Phonya 20 50 35 15

Tchale 32 34 40 26

Average 45 38 17

3. DISTRIBUTION OF FOOD SECURITY STATUS USING GROUP RATINGS

Tables 1 and 2 show the distribution of food security status in each community asdetermined by using GR. In the � rst table the reported ratings are based on the averagescores of the different smaller groups of informants in each community, while in thesecond table the classi� cation is based on the forced convergence of the differentsmaller groups’ rating.

For computational purposes, a value of 1 was associated to a food-secure household,a value of 2 to sporadically food-insecure and 3 to all remaining households. Based onthe individual rankings of the small informant groups, averages were computed andhouseholds were classi� ed as food secure for average scores lower than 1,5; sporadi-cally food insecure for values greater or equal to 1,5 and lower than 2,5; and foodinsecure for average scores greater or equal to 2,5.

Close to 50 per cent of the households were ranked as chronically food insecure,whether the average values of the GR were used (Table 1) or the convergence valuesfrom the plenary sessions were used (Table 2). This � nding is not surprising and isconsistent with previous studies in Malawi (Carletto, 1999; World Bank, 1996). In each

Table 2: Percentage of households in each food security class, based onforced convergence scores

Number of Food Sporadically Food

Site households insecure (%) food-insecure (%) secure (%)

Chibalala 16 69 19 13

Fasi 15 33 53 13

Kadam’manja 13 38 31 31

Mazonda 16 50 50 0

Muwalo 30 57 40 3

Phonya 20 40 45 15

Tchale 32 41 25 34

Average 47 37 16

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

0:16

20

Nov

embe

r 20

14

Using group ratings to assess household food security 545

Table 3: Assessment of inter-rater agreement using Cohen’s Kappa

(1) (2) (3) (4) (5) (6) (7)

Number of Kappa: SFI Kappa:

households Number of Kappa: FI vs vs (FI 1 Kappa: FS vs weighted

Village rated groups (SFI 1 FI) FS) (FI 1 SFI) average

Chibalala 16 4 0,51 0,22 0,35 0,38

Fasi 15 4 0,19 0,15 0,67 0,29

Kadam’manja 13 3 0,40 2 0,20 0,28 0,15

Mazonda 16 4 0,67 0,37 0,10 0,43

Muwalo 30 5 0,41 0,31 0,10 0,35

Phonya 20 5 0,52 0,24 0,63 0,44

Tchale 32 5 0,71 0,32 0,50 0,52

Average 0,49 0,20 0,38 0,37

community the two types of aggregation yielded similar results, and the averagecorrelation coef� cient between the average and the forced convergence scores was high(r 5 0,85), with a minimum value of 0,77 in Tchale and a maximum of 0,95 inKaddam’manja. However, these consolidated values hide large differences between theindividual GR, con� rming the results of an earlier study that found low consistency ofresponses among raters (Bergeron et al, 1998).

4. RELIABILITY OF THE GROUP RATINGS METHOD

To assess the reliability of the GR method, the investigators � rst measured the degreeof inter-rater agreement in each community, using the Kappa coef� cient (Cohen, 1960).This statistic measures the level of inter-rater agreement among the different groups ofrespondents, the assumption being that the stronger the agreement, the more reliable thetechnique in that particular context. The Kappa statistics reported in Table 3, columns4–6, are computed by reducing two adjacent categories (eg FS and SFI, or SFI and FI)into one, and then comparing this to the third category. The overall Kappa (column 7)is calculated as a weighted average of the individual Kappas based on two-ratingoutcome categories. The Kappa statistic can take any value between 2 1 and 1 1. Avalue of 0 indicates that the amount of agreement observed is exactly equal to thatwhich would be expected by chance, while a value of 1 indicates perfect agreement.For intermediate values, Landis & Koch (1977) suggest the following interpretation:

Below 0,00 Poor agreement0,00–0,20 Slight0,21–0,40 Fair0,41–0,60 Moderate0,61–0,80 Substantial0,81–1,00 Almost perfect

The summary results (column 7) suggest only fair to moderate levels of inter-rateragreement in classifying households based on their food security status. When consid-ering each category individually, it appears that raters � nd it easier to agree on whichhouseholds are chronically food insecure than any other class. The intermittently

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

0:16

20

Nov

embe

r 20

14

546 C Carletto et al

food-insecure category is the least well de� ned, exhibiting only slight to fair inter-rateragreement in any village. Only in a few of the communities, such as Tchale andMazonda, is there any substantial agreement among raters, even for the food-insecureclass.

5. ASSESSING THE VALIDITY OF THE CONSENSUS RATINGS AS A MEA-SURE OF FOOD SECURITY

Food security is usually de� ned as a situation where all people at all times have bothphysical and economic access to suf� cient food to meet their dietary needs for aproductive and healthy life (USAID, 1992). The concept encompasses three distinctnotions: availability (existence of adequate food supplies in a geographic area), access(capability of households to acquire food) and utilisation (capability of individuals touse and metabolise the food effectively so they can meet their speci� c dietary needs)(Riely et al, 1995). Although the research reported in this article refers to food securityin a general sense, the � eld exercises concentrated on food access. This is because thelevel of analysis was the household, and access is the dimension that varies at that levelof aggregation.

For comparison between the group informant ratings and the quantitative surveyresults, we used a number of indicators collected in the household survey covering allthe various domains that we posit re� ect the household’s access to food. Theseincluded coping strategies, dietary diversity, food consumption, food expenditure,household income and asset ownership. Each of these indicators is brie� y describedbelow.

· The Coping Strategy Index (CSI) relates to how households respond to the presenceor threat of food shortages. The most knowledgeable person in the household withrespect to food preparation – generally the female spouse – was asked a number ofquestions related to the occurrence and frequency of, � rstly, the substitution ofpreferred foods with less preferred items, secondly the reduction of portion sizeserved at each meal and thirdly, the reduction in the number of meals served in aday. This information was aggregated, either as a simple count of the strategies usedwithin the reference period (CSI Sum), or as weighted sums (CSI Weighted). In thelatter case, the weights re� ected both the frequency with which the strategy was usedover the reference period and the severity of the response, eg skipping a meal wasattributed a larger weight compared with reducing the portion within the meal.

· Dietary diversity is the sum of the number of different foods consumed by thehousehold over a speci� ed time period, and is computed as a simple count of allitems consumed.

· Calorie acquisition is a rough estimate of the number of calories available forconsumption by the household over a speci� ed reference period, equal to one weekin this case. The household member with primary responsibility for the preparationof meals was asked about the use of approximately 60 food items in meal preparationduring the reference week. This served as the basis for computing per capita calorieequivalent, and it is used here as a proxy for calorie intake.

· Total household income is proxied by computing the sum of all food and non-foodexpenditures. Food expenditures are based on monthly expenditure data collected formore than 60 food items in each of the two survey rounds. For purchased items, theexpenditures re� ect the value of the purchase. Median prices were used to calculatethe value of items donated to them or produced by the household and consumed

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

0:16

20

Nov

embe

r 20

14

Using group ratings to assess household food security 547

during the period of reference. To make the � gures of the two surveys comparable,the value of the post-harvest consumption of own-production and food donations isestimated using median prices of the pre-harvest survey.

· Data on non-food expenditures were also collected in each round and added to foodexpenditure to obtain total expenditure aggregates. For durable goods, eg furniture oragricultural utensils, the period of reference is six months prior to the survey. Forsmaller disposable items, eg soap, candles or paraf� n, the period of reference is themonth preceding the survey. Annual values of non-food expenditures were computedby multiplying the reported value of each item by the corresponding factor.

· The value of both assets and livestock was computed, based on the respondents’valuation of all items owned by the household.

· Finally, the total amount of land owned by the household at the time of the surveywas included as an additional measure of the household’s wellbeing. This includedall land that, although not held as private property, was the household’s own undercustomary law.

To assess the validity of the GR, Spearman rank correlation coef� cients were computedbetween the forced convergence scores from the GR exercise on the one hand, and theset of individual indicators from the household survey on the other. This approachrecognises the fact that there is no single criterion measure of food security, but rathera series of closely related constructs. Rank correlation coef� cients were chosen inpreference to Pearson’s correlation coef� cients because the aim of the exercise was toapproximate the households’ rankings on the criterion measures, not to estimate actualvalues. The resulting correlations are shown in Table 4.

Overall, correlations between the ratings based on the GR and the other indicators werelow. The only clear exception was the value of household assets and livestock. Thecomparisons suggest that the assessments made during the rating exercise were morestrongly associated with visible indicators of wealth and food access, such as assets andlivestock, than with less visibly manifested aspects such as dietary diversity, foodcoping strategies, calorie intake or per capita food expenditures.

6. CHARACTERISATION OF THE THREE FOOD SECURITY CLASSES

Participatory methods generate information that is not easily captured in a conventionalquantitative survey. One of the objectives of the qualitative exercise was to comp-lement the quantitative data by shedding light on some of the underlying causes of foodinsecurity for different groups in the communities. Table 5 provides a summary of thereasons that in the informants’ view caused the households to belong to one of the threecategories, as classi� ed during the Index Card exercise.

Food-insecure households were described as families that lack suf� cient income topurchase adequate amounts of fertilisers and improved seeds. These families spent mostof their time working in other people’s � elds in exchange for food during the growingseason, which was also the peak period of food shortage. This resulted in a neglect ofactivities in their own � elds, and hence poor yields. For these households, lowproduction in one year led to food shortage, which also led to low production in thenext. By harvest time they had already eaten part of their crop. This situation is whatMkandawire (1988) referred to as the vicious cycle of food shortage. These householdswere also described as having large family sizes and experiencing frequent illnessesduring the crop-growing season. The informants indicated that illnesses in these

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

0:16

20

Nov

embe

r 20

14

548 C Carletto et al

Tab

le4:

Spe

arm

anco

rrel

atio

nco

ef�c

ient

sbe

twee

nco

nver

gen

cesc

ore,

grou

pra

ting

s,an

din

dica

tors

ofho

useh

old

food

secu

rity

deri

ved

from

the

quan

tita

tive

surv

eys

Cop

ing

Cop

ing

stra

tegy

stra

tegy

Per

capi

taT

otal

Num

ber

inde

xin

dex

Die

tary

Cal

orie

food

per

capi

taL

ives

tock

valu

eA

sset

valu

e

Vill

age

obse

rved

(sum

)(w

eigh

ted)

dive

rsit

yin

take

expe

ndit

ure

expe

ndit

ure

(log

)(l

og)

Lan

d

Chi

bala

la16

0,62

0,58

20,

302

0,06

20,

372

0,31

20,

512

0,50

20,

35

Fas

i15

0,33

0,42

20,

110,

172

0,10

20,

472

0,78

20,

412

0,10

Kad

am’m

anja

130,

290,

372

0,05

20,

302

0,31

20,

252

0,61

20,

662

0,32

Maz

onda

160,

120,

250,

010,

172

0,43

20,

332

0,11

20,

462

0,08

Muw

alo

300,

350,

382

0,40

20,

192

0,38

20,

562

0,49

20,

462

0,25

Pho

nya

200,

330,

382

0,08

20,

162

0,45

20,

412

0,41

20,

382

0,69

Tch

ale

320,

600,

512

0,08

20,

102

0,14

20,

210,

122

0,36

0,16

0,33

0,38

20,

082

0,10

20,

372

0,33

20,

492

0,46

20,

25

Not

e:Fi

gure

sin

bold

are

sign

i�ca

ntat

the

0,10

leve

l.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

0:16

20

Nov

embe

r 20

14

Using group ratings to assess household food security 549

Table 5: Characteristics of food security categories

Food secure (FS) Sporadically food insecure (SFI) Food insecure (FI)

· Have access to adequate fertilisers · Have small landholding sizes · Have no access to fertilisers

· Have sources of income such as · Do not have access to adequate · Do not use improved seed

remittances, income-generating fertilisers · Have large family sizes

activities, employment, retirement · Sickness and deaths affect the · Work in other people’s

packages from previous availability of labour gardens in exchange for

employment · Experience labour shortages food

· Have adequate labour · Do not use improved seeds · Lack income

· Able to hire labour either for cash · Experience shortage of money · Labour shortages due to

payment or in exchange for food · Large families illnesses

· Have livestock · Work in other people’s gardens · Harvest very little, as they

· Have small family size · Poor rainfall distribution start eating the crop before

· Buy supplementary food whenever the harvest

necessary · Loss of soil fertility

households were common from the months of November, coinciding with the begin-ning of the rains, to February. The most common forms of illnesses were diarrhoea,dysentery and malaria. These illnesses greatly affected the availability of labour for� eld activities.

The sporadically food-insecure households were described as households producingjust enough food for their families, with no surplus for sale or for hiring labour to helpin � eld activities. They had inadequate access to fertilisers and generally did not useimproved maize seed varieties. They had large families and yet suffered occasionallabour shortages because, often, they had to work in other people’s � elds in exchangefor food. Their landholding sizes were quite small. The food production base of thesehouseholds was very fragile. Any shock, such as illness or death of one of thehousehold members, or poor rainfall during the growing season, would result in foodshortage.

The food-secure households were described as having access to income through varioussources such as remittances, off-farm employment and other income-generating activi-ties. This income helped these households to access adequate amounts of fertilisers andimproved seeds and, consequently, they obtained higher yields in staple production.Thanks to surplus food production, these households were also able to hire labour for� eld activities in exchange for food during food-shortage months. Apparently thefood-shortage months coincide with the time when labour is most required for � eldactivities. This has helped these households to have their � eld activities done in time.As a result, good production of food in one year led to another good food productionyear in the following year. These households also had a better stock of assets andlivestock, which served as a buffer in the case of unexpected shocks. Generally,households in this category were described as having large landholdings and smallerfamilies.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

0:16

20

Nov

embe

r 20

14

550 C Carletto et al

7. CONCLUSIONS

This article set out to examine the reliability and validity of group ratings when usedto assess the food security status of rural households. The concluding commentsaddress two aspects, the � rst relating to GR as a method; the second relating to foodsecurity as a concept and its ‘measurability’ through the use of rapid appraisaltechniques.

7.1 Reliability of GR

Overall, GR were shown to be only fairly to moderately reliable in identifyingfood-insecure households. Speci� cally, the experience reported here suggests that anysingle group of raters will usually be right some of the time, but not all of the time.The rating may be improved by forcing a convergence among multiple groups ofinformants, but this may be impractical since the typical situation is to have only onegroup of people providing information, not to derive multiple ratings as we did here forexperimental purposes.

In this study, the extent of misclassi� cation varied by community. However, it is clearthat targeting food-insecure households on the basis of GR scores will always includeboth errors of inclusion and errors of exclusion. Some comfort may, however, be takenfrom the fact that in most cases, the misclassi� cation occurs between contiguouscategories – in other words, it rarely happens that a household gets classi� ed by onegroup as most food insecure, and by another group as most food secure.

The consequences of using a relatively imprecise instrument to select bene� ciaries willalso vary according to the speci� cs of the programme. Issues such as to what degreeresources are constrained, how critical issues of social equity are and – if theprogramme pursues secondary aims such as environmental protection – how centralfood-insecure households are to these aims, will all affect how critical it is to identifyall food-insecure households correctly, and only those.

In sum, the GR approach shows some potential, but much remains to be clari� ed at themethodological level before it is possible to claim a truly reliable instrument. A‘positive deviance’ type of approach could be used, focusing on those communities thatshowed the greatest natural convergence, and trying to identify which factors made thispossible. Lessons learned from such an exercise could promote better practices whenusing GR.

7.2 The concept of food security and its assessment

Overall, the comparison between the GR � ndings and the indicators from the quantitat-ive survey reveals poor correlations. The only notable exceptions are the correlationswith asset and livestock holdings. This seems to suggest that group assessments of thehousehold food security status of fellow villagers may ultimately be based on a numberof factors distinguished more by their visibility than by their accuracy in predictingfood security. Community members seem to make little distinction between theconcepts of food security and poverty, perhaps because these two dimensions areinextricably linked in these exceptionally deprived communities.

Group discussions con� rmed that the food security status of these rural households isclosely linked to their ability to generate income and accumulate wealth. A comprehen-sive food security strategy must then focus on investment and income-generating

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

0:16

20

Nov

embe

r 20

14

Using group ratings to assess household food security 551

opportunities, including off-farm and non-agricultural activities. The income generatedfrom such activities can play a major role in helping households to invest in productiveassets like livestock, and to access inputs such as improved seeds and fertilisers whichwould, in turn, increase the household’s ability to acquire food.

In this environment, the main coping strategies as ranked by food-insecure householdswere reducing the amount of food per meal, skipping meals and working on otherpeople’s plots in exchange for food. While the � rst two strategies have a direct negativeimpact on malnutrition, the latter results in a vicious cycle of food shortage. This cycleis very dif� cult for individual households to break, since food shortage in one year hascumulative effects on food shortages in the next year.

In the opinion of the respondents, credit schemes such as the ones that operate currentlyin the region do not provide an adequate solution to the food security problem. Perhapsa better solution lies in programmes that provide both inputs and food in thefood-shortage months. Programmes such as input and food-for-work and loans orgrants for both inputs and food may provide better solutions. While the inputs wouldhelp increase crop yields, the food would help the households inasmuch as they wouldspend more time working on their own plots during the most critical months, insteadof having to work on other people’s farms in exchange for food.

REFERENCES

BERGERON, G, MORRIS, SS & MEDINA BANEGAS, JM, 1998. How reliable aregroup informant ratings? A test of food security ratings in Honduras. World Develop-ment, 26: 1893–902.CARLETTO, C, 1999. Strengthening the impact of market- and trade-oriented devel-opment programs on food security and nutrition. Report prepared for the United StatesAgency for International Development. Washington, DC: International Food PolicyResearch Institute.CARMINES, EG & ZELLER, RA, 1983. Reliability and validity assess-ment.Quantitative Applications in the Social Sciences Series No 17. Newbury Park,CA: Sage.CHADWICK, MT, SEELEY, JA & SHERCHAN, GR, 1995. Preliminary comments ona wealth ranking exercise performed with forest user groups from three geographicalareas of Nepal. Kathmandu, Nepal: Nepal–UK Forestry Research Project.CHAMBERS, R, 1997. Whose reality counts? Putting the � rst last. London: Intermedi-ate Technology.COHEN, J, 1960. A coef� cient of agreement for nominal scales. Educational andPsychological Measurement, 20: 37–46.FELDSTEIN, HS & JIGGINS, J, 1994. Tools for the � eld: methodologies handbook forgender analysis in agriculture. West Hartford, CN: Kumarian.GRANDIN, B, 1983. The importance of wealth in pastoral production: a rapid methodof wealth ranking. In International Livestock Center for Africa (ILCA), Pastoralsystems research. Addis Ababa: ILCA.LANDIS, JR & KOCH, GG, 1977. The measurement of observer agreement forcategorical data. Biometrics, 33: 159–74.MEARNS, R & SHOMBON, G, 1992. Direct and indirect uses of wealth ranking inMongolia. RRA Notes, 15: 29–38, May.MKANDAWIRE, RM, 1988. Smallholder survival strategies, perceptions, and respon-siveness to extension and credit in the Salima Agricultural Development Division. A

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

0:16

20

Nov

embe

r 20

14

552 C Carletto et al

study prepared as part of the European Economic Community-funded mid-termevaluation of Salima Agricultural Development Division, Phase IV. Unpublisheddocument. Lilongwe: Commission of the European Economic Community.MUKHERJEE, N, 1992. Villagers’ perceptions of rural poverty through the mappingmethods of PRA. RRA Notes, 15: 21–6, May.NUNNALLY, J & BERNSTEIN, I, 1994. Psychometric theory, 3rd ed. New York:McGraw Hill.RIELY, F, MOCK, N, COGILL, B, BAILEY, L & KENEFICK, E, 1995. Food securityindicators and framework for use in the monitoring and evaluation of food aidprograms. Impact project. Washington, DC: United States Agency for InternationalDevelopment.SCOONES, I, 1992. Wealth ranking: insights into patterns of rural differentiation insouthern Zimbabwe. London: International Institute for Environment and Development.SEELEY, JA, KAJURA, EB & MULDER, DW, 1995. Methods used to studyhousehold coping strategies in rural South West Uganda. Health Policy and Planning,10(1): 79–88.UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID),1992. Policy Determination No 19. Document PN-AAV-468. Washington, DC:USAID.WELBOURN, A, 1992. A note on the use of disease problem ranking with relation tosocioeconomic well being: an example from Sierra Leone. RRA Notes, 16: 86–7, July.WORLD BANK, 1996. Malawi human resources and poverty: pro� le and prioritiesfor action. Report No 15437-MAI. Washington, DC: World Bank.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

0:16

20

Nov

embe

r 20

14