The ESDinds project

Analysing participation in a cooperative research project on values-based

indicators

Marie Harder, Gemma Burford, et al.Sustainable Development Coordination Unit,

University of Brighton

Co-inception: the need for ESDinds Initial discussions between academic

researchers, an international consultant, and representatives of 10-15 CSOs working in Education for Sustainable Development

CSOs identified values-based indicators for project monitoring and evaluation as a priority: “making the invisible visible”

Meet specifications of EU call for proposals while leaving room for interpretation and co-design by participating CSOs

ESDinds Project ConsortiumFour CSOs:

Earth Charter Initiative, Alliance of Religions and Conservation European Bahá’í Business Forum People’s Theater, Germany

Two university-based research groups: Sustainable Development Coordination Unit, University

of Brighton, UK Charles University Environment Center, Prague

Independent adviser with expertise in the field of SD indicators

Co-design: defining goals & activities Face-to-face meetings of whole consortium

(every 6 months) Consultative decision-making Independent adviser acts as mediator Reflection on how to improve collaboration PBworks wiki and e-mail list Reporting back to EU

Values and indicators First phase: learning about values and

outcomes that CSO partners associated with ‘successful’ projects

Then: “How would these values be lived?” Initial list of indicators suggested by CSOs Iterative development of indicator list Identifying possible measurement

methods Planning field visits to test indicators

Case study: Sierra Leone YABC project “Youth as Agents of Behavioural Change”

International Federation of Red Cross & Red Crescent Societies (IFRC): Principles & Values

Sierra Leone Red Cross Society Promoting a culture of peace and inclusion Agricultural projects and peer education

bringing together vulnerable youth from ‘opposite sides’

ESDinds field visit in February 2010 Consortium’s need: Test the indicators!

(NOT co-inception with respect to IFRC)

Defining research goals and activities IFRC Director seeking effective ways to

evaluate impact and measure change in individuals’ values

Youth leaders (members of YABC network) prioritised 10 indicators for testing

Indicators incorporated into a workshop to test IFRC’s ‘draft toolkit’ of YABC exercises in Sierra Leone

IFRC and SLRCS representatives selected research methods from a ‘menu’ provided by researchers

Methodologies Quantitative:

Spatial survey (Never – Sometimes – Always) Secret ballot survey (as above) Structured observation of participation in

decision-making and discussion Qualitative:

Conventional focus groups Focus groups with theatre performance

CSO staff preferred to watch researchers facilitating the exercises

Interpretation of data Some data fed back to youth immediately

through focus groups (survey findings) Other results discussed in expert group Some results invalid due to:

Conformity bias (`following the crowd’) Uncomfortable environment (hot sun) Lack of competent female translator

BUT Red Cross still learned something important and useful…

Less discrimination in teams than villages!

Sustainability ESDinds exercises complemented IFRC’s

draft toolkit very well National Youth Coordinator confident of

being able to incorporate them into SLRCS programmes without further input from researchers

IFRC has acknowledged values-based indicators as a useful approach for projects elsewhere

Assessing participation at each project stage

1. Consultation (Experts present pre-determined issues)

2. Cooperation (Community offers advice, but decision-making rests with experts)

3. Participation (Equal decision-making by experts and community)

4. Full control (Community controls decision-making, experts advise)

Ref. Naylor et al. (2002) Soc Sci Med 55: 1173-1187

Relationships of participation Participation varied at different stages of

the project – this is an overview! Within consortium: generally 3

Equal decision-making (CSOs/research groups) Between researchers and CSO staff: 3-4

Decision-making mainly led by CSO staff Between CSO staff and youth: (generally) 1-2

Very short, centrally organised workshop Large group Language & literacy barriers

Lessons learned Participatory process was critical in

responding to challenges, e.g. conformity bias in spatial survey

Consider all relationships of participation before concluding that research is truly ‘participatory’!

Youth to become co-evaluators of their own programs: Identifying research needs Defining goals and activities More ‘hands-on’ involvement in M&E activities

Policy implications Who represents ‘the community’?

Roles of local experts and leaders/elites? Beneficiary populations are diverse

Who are the most marginalised people in this project? (age, gender, poverty, etc…)

What are the barriers to their participation - and can they be removed?

Is co-inception a realistic possibility? How can they be involved in meaningful

ways in co-design? More participation = more benefit?

Thanks for listening, and please feel free to join our online community!

www.wevalue.orgwww.esdinds.eu

Conference 16-18 December 2010at the University of Brighton

Making the Invisible Visible:An Emerging Community of Practice in Indicators, Sustainability and Values