Values and outcomes in public spacean evaluation framework for Third Sector Organizations

Eloisa Helena de Souza Cabral São Paulo | Brazi l

Third Sector (TS) is considered asintermediate area (Evers 1995) between the State, the market and the community, a publicspace of social protection and development, endowed with the values (Cabral 2005):

• representation of collective interests; • democratization;• quality;• effectiveness;• visibility;• public culture; • universality;• autonomy;• social control; • and sustainability.

2

Social Management

has a double nature, as a social mode of producing outcomes and reproducing values,as an aims mission accomplishment providing public goods. The values are assumed by, and the facts impact on, the public constituencies of the Third Sector Organizations (TSO).

3

Public constituencies

• founders,• donors,• workers,• volunteers,• and beneficiaries.

have different Expectations, Needs, Capacities, Interests and Representations (ENCIR) about social issues in the public space of TS (Cabral 2007).

4

Economic and Impact evaluation, alone, although well established procedures, deserve some attention because:

• The intangibility of some benefits;• The TSO purpose to disseminate values as well as produce

material benefits;• The possibility to fall in a kind of fact/value dichotomy

(Putnam 2002);• The presence of tensions between ENCIR associated with

different public constituencies; • The difficulties of TSO to prepare control groups; • The limitation of methods adapted from the area of public

policies and the market.

5

Arguments to evaluate impact,economy and values• The importance to gauge mission accomplishment;• The condition of social control as an insufficient management

tool in TSO (Cabral 2005); • The political concern among TS that operates more closely to

its frontiers with the market and with government resources, public funds and public policies in developing countries;

• The accreditation of TS capacity to provide development conditions and public goods; and

• The financial agencies request of both impact and economic evaluations, as a regular procedure.

6

We propose that

The evaluation of TSO social achievements must consider the entanglement of facts and values, measuring the impacts and learning its effectiveness from the communication of public space values. ENCIR are constraints between facts and values, appraised by public constituencies and must be taken as parameters in the evaluation. The ensemble of public space values make up an evaluation framework, or an evaluative space (Sen 1997), to identify the impact.

7

The results of impact and economic evaluations projected onto the framework of a qualitative evaluation of those public values, indicate the public nature’s flourishing, or the mission accomplishment by social management in TS, with the effective reproduction of values tied to outcomes.

Conclusion: public space flourishing

8

Application of this framework and objectives of this survey

1. To establish a hierarchy of values through public constituencies’ perceptions on the public space values associated with benefits (done).2. Verify, by a qualitative survey, the presence of values pursued by the TSO (in progress).3. To establish, analytically, the links between the economic and impact evaluation findings and the PC’s perceptions (to be done).

9

Methodology

Focusing one outcome, we have asked the public constituencies to order values from the most important to the least important one. sample• Beneficiaries 918• Founders 198• Donors 150• Workers 948• Volunteers 366Total n= 2580 in 50 TSOs

10

Findings• Different PC have different hierarchies of values,

associated with the benefits.• Quality is the most important value to all.• Social control is the most important value only to

Beneficiaries (excluding quality). • Social control is the least important value to Donors,

Founders and Volunteers.• Values associated with the benefits (quality,

effectiveness) are, generally, more important than values associated with the public nature (representation, public culture, autonomy, visibility).

• Standard deviations may be interpreted as tensions, dissensions of perceptions.

11

Perception of values (ALL)Order Value m

1 Quality 7,59 2,342 Effectiveness 5,78 2,753 Sustainability 5,48 2,974 Democratization 5,36 2,785 Universality 5,31 2,846 Representation 5,30 2,707 Public culture 5,16 2,618 Social control 5,16 2,979 Visibility 4,96 2,74

10 Autonomy 4,90 3,02

12

Perception of values (DONORS)Order Value m

1 Sustainability 6,80 2,942 Autonomy 6,60 3,123 Quality 6,40 2,684 Universality 5,80 2,635 Visibility 5,40 2,636 Representation 5,40 2,967 Effectiveness 5,28 2,898 Democratization 5,24 2,899 Public culture 4,52 2,52

10 Social control 3,56 2,38

13

Perception of values (BENEFICIARIES)Order Value m

1 Quality 7,56 2,342 Social control 6,24 2,893 Effectiveness 5,82 2,764 Democratization 5,63 3,075 Representation 5,51 2,706 Public culture 5,11 2,587 Universality 5,02 2,678 Visibility 4,97 2,829 Autonomy 4,59 2,9010 Sustainability 4,55 2,72

14

Perception of values (FOUNDERS)Order Value m

1 Sustainability 7,30 2,112 Quality 6,36 2,563 Universality 6,24 2,874 Visibility 5,73 3,025 Effectiveness 5,61 2,626 Autonomy 5,21 3,227 Democratization 5,00 2,868 Representativity 4,88 2,699 Public culture 4,73 3,0110 Social control 3,94 2,51

15

Perception of values (WORKERS)Order Value m

1 Quality 7,93 2,182 Effectiveness 5,97 2,793 Public culture 5,43 2,594 Democratization 5,39 2,555 Universality 5,22 2,926 Sustainability 5,18 3,007 Representativity 5,06 2,568 Social control 4,97 2,969 Autonomy 4,96 3,0910 Visibility 4,87 2,72

16

Perception of values (VOLUNTEERS)Order Value m

1 Quality 7,92 2,182 Sustainability 6,77 2,853 Representativity 5,66 2,954 Effectiveness 5,49 2,685 Universality 5,49 3,006 Public culture 5,05 2,517 Democratization 4,93 2,598 Autonomy 4,61 2,809 Visibility 4,54 2,47

10 Social control 4,54 2,88

17

Perception of values (EXCLUDING BENEFICIARIES)Order Value m

1 Quality 7,60 2,352 Sustainability 5,93 2,993 Effectiveness 5,76 2,754 Universality 5,46 2,915 Democratization 5,23 2,626 Representativity 5,20 2,707 Public culture 5,18 2,638 Autonomy 5,06 3,079 Visibility 4,95 2,70

10 Social control 4,63 2,87

18

References• Cabral, Eloisa Helena de Souza (2005) Une double perspective de controle

sur da gestion sociale. In: Concepts of the Third Sector: The European Debate ISTR-CRIDA, Paris available at www.crida-fr.org, accessed 06/20/2007.

• Cabral, Eloisa Helena de Souza (2007) Terceiro Setor: gestão e controle social, São Paulo, Saraiva.

• Evers, Adalbert. (1995) Part of the welfare mix: the third sector as an intermediate area, New York, Voluntas, 6, (2) 159-182.

• Nussbaum, Martha and Sen, Amartya. (1995) The Quality of Life, Oxford, Clarendon Press.

• Putnam, Hilary. (2002) The Collapse of the Fact/Value Dichotomy,Cambridge, Harvard University Press.

• Sen, Amartya. (1997) Quality of life and economic evaluation, AcademiaSinica Economic Papers, 25, (3).

19

banner_AF.indd 1 29/6/2008 23:40:23