Through Assessment and Organizational LearningSeveral groups have been particularly suc-cessful in integrating assessment and learn-ing into their organizational culture. Assessment

StrengtheningSocial Change

ThroughAssessment andOrganizational

Learning

One Dupont Circle, NW - Suite 700 Washington, DC 20036 [email protected] 202/822-6006

Community Learning Project

The Community Learning Project explores ways of building a stronger infrastructure forlearning in the field of community and social change. It works to expand writing, research, eval-uation, teaching, training and learning opportunities which give grassroots community groupsand their supporters easier access to helpful and provocative lessons from the extraordinaryefforts and experience of their peers.

Community Learning ProjectAndrew Mott, Director

Suite 700One Dupont Circle, NWWashington, DC 20036

202/822-6006www.communitylearningproject.org

Conference participants Chief Mike Retasket, Jagadananda andSarah Okwaare Otto

Ruben Daria Espinosa Alzate (center), interpretors Silvia SanMiguel and Martha Lopez, and other conference participants

StrengtheningSocial Change

ThroughAssessment andOrganizational

Learning

Report on The Gray Rocks Conference -An International ExchangeWith Grant Support from

The Ford Foundation

Conference participant Sheela Patel

By Andrew Mott

In September, 2003, a remarkable cross-section of people came together in Canadafor an international exchange on the differ-

ent approaches they were using to strengthenthe forces for positive social change in differ-ent parts of the world. The Gray Rocks con-ference convened more than three dozen com-munity organizers and activists, evaluatorsand other “learning partners” from elevencountries to focus on how they were buildingstrong systems to help social change organi-zations analyze, explain and strengthen theirwork. Over three and a half days they dis-cussed how, as organizers or outsiders, theywere helping activist groups meet their inter-nal need to keep assessing, reflecting on andstrengthening their work so they couldincrease their impact, while also meeting theirexternal need to help their partners, fundersand others outside their organizations under-stand, evaluate and learn from their work.

The Gray Rocks conference onStrengthening Social Change ThroughAssessment and Organizational Learning wassponsored by four organizations in the UnitedStates, Canada and the United Kingdom.1Involving participants from Asia, Africa andLatin America as well as North America andEurope, it was coordinated by the CommunityLearning Project and supported by a generousgrant from the Ford Foundation’s Governanceand Civil Society Program to the PrattInstitute.

Conference planners had two primarygoals for Gray Rocks. First, the conveningcreated a unique learning opportunity for peo-ple working in different parts of the world tolearn from each other’s experience in settingup evaluation and organizational learningsystems in the context of social change.Second, it enabled participants to beginexploring steps they might take individuallyor collectively to expand support for assess-ment and learning strategies which aredesigned specifically to help organizations

which are undertaking the enormous task ofpressing for fundamental economic, social,and/or political reforms in their societies.

Conference planners brought together peo-ple from very different worlds. They invited amix of activists, organizers, people from sup-port organizations and networks, and evalua-tors and academics – people with very differ-ent roles, training and perspectives.Approximately 60% of the participants werefrom the United States, Canada and Europe,and 40% were from the Global South andinternational “nongovernmental organiza-tions”, or NGOs, which work in developingcountries. The stage thus was set for a richdialogue among people with very differentexperiences and views.

While they came from different worlds andplay a variety of roles in social change,throughout the discussions it was clear thatall participants were united in their commit-ment to fundamental social, economic and/orpolitical reform that helps poor people andothers who face discrimination, marginaliza-tion and exclusion. They used different ter-minology – community organizing, socialchange, rights-based work, advocacy, democ-racy-building, and development – and therewere differences in their approaches whichdeserve deeper exploration, but there wasconsensus on –

the necessity and great challenge of bring-ing about fundamental reform in theirsocieties;

the central importance of strong move-ments and effective organizations in press-ing for reform; and

the necessity of helping those forces devel-op and refine their systems for assessingtheir work so they can continually learnhow to increase their effectiveness andimpact and so funders and others canevaluate and learn from their experience.

5Report from the September, 2003 Gray Rocks Conference

“All participants were united in their commitment to fundamentalsocial, economic and/or political reform that helps poor people and

others who face discrimination, marginalization and exclusion.”

The conference proceeded in stages, fromthe first day which was devoted to establish-ing common ground, through discussion ofhow activist groups, their partners, and eval-uators assess key aspects of social changework, to exploration of how they might worktogether to support the growth of assessmentand learning systems which meet their needs.Both plenaries and small group sessions fea-tured case studies, or “stories”, of how partic-ular organizations or evaluators/learningpartners were working to assess and learnfrom the experience of specific campaigns forfundamental reform. Time was also set asideto enable participants to initiate sessionswhere they could share their experience andlessons or discuss pressing issues with oth-ers.2

The meeting concluded with early explo-ration of how a cross-section of activists andacademics, evaluators, other learning part-ners, and donors might work together toexpand support for –

1. assessment and learning systems whichmeet the groups’ internal and externalneeds and thus help strengthen theorganizations which are leading the strug-gle for social change; and

2. evaluation approaches which help fundersand others evaluate, communicate about,and learn useful lessons from the organi-zations’ experience, and which reinforcerather than overload or undermine thegroups’ own learning systems.

This paper3 pulls together the mainthreads and conclusions from the wide-rang-ing discussions at Gray Rocks, highlightingareas of broad agreement, areas aroundwhich tensions or disagreements wereexpressed, and areas for further exploration.It concludes by summarizing the action prior-ities which participants discussed as the con-ference ended.

Areas of Broad Agreement

There was broad agreement on four keypoints. These common views wereshared across international lines and

among activists, evaluators and donors.

First, it is particularly difficult to assessand evaluate work in the social changearena, and it requires innovative approach-es to evaluation and learning. Socialchange is an area of great complexity, withmany actors, viewpoints, trends and coun-tertrends. It is a nonlinear process withmany ups, downs and surprises. It thereforeis difficult to develop a shared understandingof what has occurred and why. Tracing causeand effect is a major challenge.

Furthermore, by definition “social change”requires overcoming the status quo, makinginnovations and taking risks, often againstgreat odds. It therefore involves trial anderror, and messy, uncertain processes whichare difficult to track and evaluate. Questionsraised during this discussion included –

What constitutes “success”?

How do you evaluate progress fairly whennegative trends and resistance to changeare so great that it may be unreasonableto expect anything more than “holding thefort” or small victories against a landscapeof losses?

How do you allow for inevitable “failures”without unfairly damaging a group orjeopardizing a program officer’s credibilitywithin a foundation?

How can you judge the relative importanceof the contributions different actors makein a campaign involving many organiza-tions?

Second, all the participants, including thefunders, agreed that the paramount goal for

6 Strengthening Social Change Through Assessment and Organizational Learning

“It is particularly difficult to assess and evaluate work in the socialchange arena, and it requires innovative approaches to

evaluation and learning.”

evaluation and learning in the socialchange field should be strengthening theorganizations which are leading efforts tobring about fundamental reforms. Socialchange cannot be achieved without strongsocial movements or social change organiza-tions, and those groups need access to feed-back so they can analyze and improve theirperformance. That assessment should bedesigned to meet their internal and externalassessment needs, helping them reflect andlearn how they can enhance their work, whilealso expanding the outside world’s ability toevaluate and learn from their experience.

Third, there was agreement that few fun-ders currently give priority to meetingtheir grantees’ needs when they requireevaluations. Far more often private and pub-lic funders develop evaluations with little con-sultation with their grantees. Most show littleconcern with whether the assessments willprovide information and analysis which willhelp their grantees learn and develop, or helpthem explain and demonstrate how they arehaving an impact. There are, however, somefunders who are committed to evaluationapproaches which are participatory and sen-sitive to their grantees’ needs, and who wantto help other funders understand the advan-tages of developing better approaches to eval-uating social change.

Fourth, many agreedthat social change organi-zations, evaluators andlearning partners, andfunders committed tofostering organizationallearning and evaluationstrategies which helpstrengthen social changeorganizations can benefitfrom working togetherover the long run. Theyshare a strong interest inproducing credible infor-

mation on the role and value of social changeorganizations and movements, evidence whichwill help good groups survive and grow withbroader support and public backing. Theyare interested in increasing support forassessment and learning approaches whichwill advance the work of the organizations,movements and individuals that lead reformefforts. Several spoke of their commitment tofostering learning by the next generation ofchange agents and the general public. Andeach is limited in its current influence with itscolleagues in the activist, evaluation, andfunding communities. The conference con-cluded with a strong sense of the benefits ofcontinued exploration of how people likethose participating in Gray Rocks can collabo-rate to increase support for assessmentapproaches which serve these purposes, espe-cially the need to increase the sophistication,capacity and power of groups pressing for thefundamental reforms our societies need sodesperately.

Assessments Designed to Meet theOrganizations’ Learning Needs

During the discussion there was consid-erable emphasis on the fact that someparticipant organizations have already

established strong assessment and learningsystems by themselves or with evaluators or


“Some participant organizations have already establishedstrong assessment and learning systems.”

“Our objective is developing the abilities and confidence of thepartners involved and the frontline staff so they can use that learn-ing to improve that effectiveness.

“We want it to be a process which empowers our partners tocommunicate more effectively with us. We're a big organizationand we're called "partners", but there are power issues in that rela-tionship, so it's essential that we help our partners communicatemore effectively about what's important to them rather than usingour language and trying to second-guess what they think is impor-tant.”

-Jenny Chapman

partner organizations. Many communityorganizing groups, for example, have stronginternal disciplines of reporting andreflection4, and many activist organizationsare pioneering various approaches to self-assessment, peer assessment, and collabora-tion with outside evaluators and learningpartners.5 Furthermore, during the confer-ence organizations described how they weresetting up monitoring and assessment sys-tems to create greater transparency andaccountability to the grassroots people theyserve, thus transforming power relationshipsand programs.6 We are “attempting to beaccountable in what we do but mainly down-wards to the poor.”7 One group describedthese systems as helping them function as a“school for the community” with residentsfeeling strong ownership of the knowledge andthe organization.8

Several groups have been particularly suc-cessful in integrating assessment and learn-ing into their organizational culture.Assessment has become an integral part oftheir daily work and strategic planning.Furthermore, they have ensured that thelearning leads to action by assigning someoneresponsibility and authority for guiding the

learning and regularly monitoring implemen-tation of the lessons.

Such organizations want to increase thevisibility and credibility of the assessmentsystems they have found useful so thatfunders will recognize their value andshape their own evaluation requirementsto build on, rather than ignore or under-cut, those valuable systems. Discussionbegan on strategies for accomplishing this,including having groups publish case studiesand other assessment reports on their socialchange work to provide outsiders with evi-dence of how well the groups’ own systemswere serving their need for documentationand analysis. However, there was insufficienttime at Gray Rocks to explore this questionfully.

Because of time constraints conferees alsodid not explore the question of what can bedone to persuade and assist other organiz-ers and activist groups to create strongersystems for assessing, reflecting andreporting on their work. While those invitedto Gray Rocks were selected because theywere committed to assessment and learning,many other activists do not share that com-mitment. They devote little time to self-

assessment or reflection ontheir work, feeling pressedby the many demands ontheir time and/or skepticalthat investing time inassessment and reflectionwill pay off for their organi-zations. While participantsat Gray Rocks also citedthe enormous pressures oftime and resources, it wasclear throughout the dia-logue that they felt theirorganizations’ investmentin assessment and learningwas well-spent.


“While some organizers said they were unconvinced of the value ofoutside help with evaluation, other participants cautioned about

strictly internal assessment processes.”

“We're trying to help communities produce change. It is essen-tial to build the capacities of communities to sharpen and definetheir experiences in an environment where they can visit othergroups, see what they've done, offer their advice, and share experi-ences with each other. This serves several functions. First of all, itcirculates the knowledge among hundreds and hundreds and hun-dreds of people. It produces a huge power base, and people nolonger feel marginalized, they no longer feel isolated. The secondimportant thing is that it produces the tools that produce the knowl-edge which is needed for advocacy.

The same data which are used to mobilize, teach and learn arealso used to monitor and evaluate. Can you take the people'sissues and build on them and help us to sharpen and refine thisprocess?”

-Sheela Patel

While some organizers said they wereunconvinced of the value of outside help withevaluation, other participants cautionedabout strictly internal assessment processes.Participants pointed to the limited credibilitysuch internal evaluations may have with out-siders. Some stressed the danger that inter-nal assessments might not question funda-mental assumptions or bring new, independ-ent, perhaps challenging perspectives andinsights to an organization. In discussinginternal systems, conferees pointed out thatan organization’s internal culture will deter-mine whether internal assessments will besufficiently rigorous and revealing.Participants cited the central importance oftrust and openness to learning as prerequi-sites to candid internal assessments. Theycited several key questions –

Does the organizational culture permitpeople to admit and embrace “failure”,seeing it as an inevitable and importantpart of organizational learning in an arenawhich requires creativity and risk-taking?

Is there safety for people who challengefundamental assumptions, overall strate-gies, any straying from the organization’soriginal mission, or people at the top?

Some warned of “overly romanticizing”internal knowledge and missing the insightsand unusual ideas which may come from oth-ers whose methodologies or knowledge sup-plement those inside an organization.

It was evident throughout the discussionthat a broad variety of organizations and indi-viduals in addition to professional “evalua-tors” serve as “learningpartners” for social changeorganizations. In additionto evaluators whose com-mitment and approachmake them natural part-ners for groups wanting

access to evaluation experience, skills andcredibility, there are several less widely recog-nized types of people who help groups assessand learn from their work. These include var-ious kinds of networks, support organiza-tions, technical assistance and traininggroups, and consultants that incorporateorganizational assessment and learning asintegral parts of their organizational develop-ment assistance to front-line organizations.Some are “critical friends”, people closeenough to the organization to be trusted, anddistant enough to have some degree of inde-pendence and perspective. Others are peerswho through peer review and peer learningcircles often include their assessments andreflections in the context of broader sharing ofexperience and lessons. Time was not devot-ed to exploring how these learning partner-ships, peer exchanges and internal self-assessment processes might be systematicallysupported and developed as important evalu-ation resources for the future, nor was timeallocated to exploring the pros and cons ofthese different relationships.

Assessments to Meet ExternalAssessment and Learning Needs

An increasing number of funders areincreasing their emphasis on evalua-tion and increasing their insistence

on numbers. This pressure is especiallystrong in developing countries where fundingcomes primarily from either government agen-cies or international institutions.9 Theselarge bureaucracies have elaborate systemsfor tracking, reporting and evaluation, and


“A broad variety of organizations and individuals in addition to professional ‘evaluators’ serve as ‘learning partners’ for

social change organizations.”

“We had 20 interviews with 20 different organizations and askedthem what they were working on and then derived eight indicatorsfrom what they told us and then tested them a lot with the groups.”

-Anne Hallett

their systems seldom are tailored to the par-ticular groups and projects they are support-ing. In the United States, where most fund-ing for social change comes from private foun-dations, pressure for evaluation is alsoincreasing. Several factors are converging tocreate this situation, including pressuresfrom foundation boards and leaders for –

evidence of the impact their grants arehaving;

lessons from their grantees’ experience;and/or

a better basis for deciding among compet-ing grant requests during an era of cut-backs and tough choices.

Donors stressed that the strongest philan-thropic and public supporters of socialchange need credible proof of the value ofthese efforts so they can justify fundingthem.10 As a donor said, “The key to sustain-ability is to demonstrate effectiveness…. Ihave a keen interest in impact and what we’relearning… But can we create a candid rela-tionship?” An activist echoed that “theaccountability, legitimacy, credibility of ourwork is essential for us. We are not only sup-posed to BE results-oriented; we also have toAPPEAR results-oriented.”11

Conference participants repeatedly dis-cussed the fact that several characteristics ofsome conventional funder-supported evalua-tions cause problems for social change organ-izations. These problems are likely to increaseas evaluation pressures grow. They include –

the reality that evaluations are often “add-ons”, adding work without addingresources, and imposing approacheswhich grantees see as flawed or burden-some and which bypass rather than rein-force the organizations’ own learning sys-tems;

the “power imbalance between those whohave the money and those who want it”12

which makes it difficult for groups to becandid or to press for approaches to eval-uation which they feel are better than afunder’s approach;

the unrealistically short time-frame whichis often used to gauge progress despite thefact that social change requires long timehorizons;

the contrast between the nonlinear waysocial change happens, with many upsand downs, and many donors’ corporateand bureaucratic frame of reference whichis based on a linear view of how progressis made;13

the competition for funds and recognitionamong groups which undercuts their abili-ty to work together to marshal sufficientcollective power to make progress on com-mon issues;

the difficulty of admitting “failures” –which are inevitable in the challengingand unpredictable world of social change –to funding sources which must make andjustify decisions about whether the grouphas been sufficiently “successful” to meritrenewed funding;

the difficulty of depending upon corporate,government, or conservative funders whoare not committed to social change, adilemma which tempts groups to avoiddiscussing controversial aspects of theirwork and instead isolate activities whichare “safe” – a situation which can distortcommunications between funders andgrantees, divert an organization from itspriorities, and complicate candid compar-isons of an organization’s goals with itsaccomplishments;


“Several characteristics of some conventional funder-supportedevaluations cause problems for social change organizations.”

many funders’ desire to quantify results –an emphasis which gives little weight tosuch vital but hard to quantify issues aschanges in the group’s relative power tobring about long-range change, or increas-es in its leaders’ sophistication, leadershipcapacity, confidence, vision, ambition andwillingness to act as they set goals fororganizing and politicizing the communityor reforming policy and institutions;

the “flavor of the month” approach ofsome funders who change priorities fre-quently;

the increasing number of funders who“have their own agendas, making doneesbecome subcontractors facing rigid andnonsensical reporting relationships”;14

the frequency with which funders ignorethe results of their evaluations as theymake funding decisions, including decid-ing to drop grantees after giving them highmarks on their performance; and

the current tough funding climate inwhich evaluation is often used to cut pro-grams.15

The central strategic question is how toachieve a fair and balanced approach whichmakes evaluation work for both funders andthe groups they support.16

As one activist said to fun-ders who are committed tosocial change and evalua-tion, “What can we do soyou won’t leave the founda-tion and we will survive?”17

There was agreement thatthis would require seekingmajor changes in therelationships and negoti-ations between fundersand grantees regardingevaluation and learning.

Participants stressed the need to recognizethe power imbalance between grantmakersand grantees and how that influencesgrantees’ attitudes toward external evaluation.They spoke of “democratizing the process”and helping nonprofits “negotiate in parity”with funders on the details of an evaluation.They discussed the need to increase thegrantee organizations’ power, enabling themto negotiate as full partners concerning howtheir work is assessed and their internallearning needs met.

There was general agreement thatgrantees can increase their influence on eval-uation by taking the initiative and clarifyingwhat they want to assess and how, and thenusing that plan as the basis for negotiatingwith funders. It is “useful to ask ourselveswhat change we would like to see happen inthe next year, through our work. This is aninteresting way of starting to ‘detect’ the smallchanges that we find meaningful on a dailybasis.”18 One donor spoke of seeking ways toconsult with grantees as his foundation con-siders changing funding priorities. Headvised social change groups to get togeth-er to strategize about how to increase theirinfluence with donors regarding their fund-ing priorities and their approaches toassessment and learning – to seek ways totake the lead in developing a “knowledge


“The central strategic question is how to achieve a fair and balancedapproach which makes evaluation work for both funders and

the groups they support.”

“How can we deal with these external evaluation forces?Internal learning interests are very often at odds with externallearning interests, which are very functional, involve allocation ofresources, are sometimes too short-term, do not consider or coin-cide with the interests of the community.

Evaluation is only useful if it's helpful to organizations. Are theorganization's goals being accomplished? Are institutions keepingin mind the interests of the community? Do community membersparticipate, become involved? Have we opened up spaces for par-ticipation, democracy? Has it become a school for the community,ethics, unity?”

-Ruben Daria Espinosa Alzate

development strategy” which serves every-one’s needs.19

While there are examples of strong rela-tionships between grassroots groups andprofessional evaluators, there clearly aretensions and obstacles which often compli-cate such collaboration. There are consider-able differences in style, skills and ways ofthinking between activists and academics andother evaluators. Pointing out that somegroups have had bad experiences with aca-demics and researchers, one participant said,“To some groups, the evaluator is like a den-tist, you don’t want to see him. Evaluatorsmay be called in when the situation is desper-ate.”20 Some evaluators have had equallyunfortunate experiences with grantees theyhave been asked to assess.

Some organizers and activists assertedtheir view that their internal systems are suf-ficient and questioned the need for externalhelp in meeting their evaluation needs. Onecited “a myth that lack of familiarity with agroup and its work equals objectivity.”21

However, other activists called for more helpfrom evaluators. One asked for professionalevaluators who would stay on site sufficientlylong to set up systems and train people in theorganization to document and assess thegroup’s work, thus helping the organizationbuild its capacity while bringing additionalskills and experience to bear on knotty evalu-ation questions.22

Needless to say some evaluators stronglydisagreed with the view that internal evalua-tions are sufficient. They cited the advan-

tages of bringing their methodological expert-ise, evaluation experience, and “distance” andperspective into the assessment process, butdoing it in ways which respect the organiza-tion and its own learning systems and whichare designed to be helpful.

All agreed that evaluators are in an awk-ward position when they are hired by funders.Funders often impose rigid standards anddata requirements or time limits, leaving littleroom for negotiating for changes which evalu-ators and the groups to be evaluated wouldprefer. As a matter of principle some havedecided not to conduct evaluations for fun-ders when this is the situation.23

Furthermore, funder-driven evaluationsunderstandably raise groups’ concerns aboutevaluators whose findings may threaten theirrefunding. For their part some evaluatorscomplained that groups frequently are unco-operative or, at a minimum, so busy or souncommitted to an evaluation that they areunprepared to provide the level of reporting,access, candor and responsiveness the evalu-ators need to fulfill their responsibilities.

Nevertheless, there appears to be com-mon ground upon which stronger relation-ships could be built between activists andevaluators. Evaluators frequently echoed theviews of organizers and activists as theyspoke of the dilemmas of being caughtbetween funders and grantees who don’tagree on evaluation and learning priorities.As evaluators discussed the principles behindtheir practice, their views responded to manyconcerns raised by organizers and social

change leaders. These prin-ciples include –

Social change organiza-tions should be involved indeveloping, interpretingand communicating theresults of the evaluation


“Sufficient flexible ‘core’ and program funding is critical to supportthe assessment systems groups need.”

“What we can do is surface the differences in our goals from anevaluation, be explicit about them, put them on the table and asequals negotiate some resolution so we can go forward clearly andin agreement.”

-Pat Kelly

and receive adequate support to carry outthose responsibilities;

Evaluation should be designed to be use-ful in improving the work of grantees, thefield, and others;

Evaluation should build the group’s inter-nal capacity for self-evaluation, and/orbuild on existing mechanisms for reflec-tion and self-assessment;

Evaluation should respect and acknowl-edge the context in which the organizationis operating;

All the costs of conducting the evaluationshould be fully funded; and

Candor should not be punished, inside anorganization or by funders.24

In retrospect, it is unfortunate that, unin-tentionally, the allocation of time for presen-tations at Gray Rocks was somewhat imbal-anced. While there were several presenta-tions by organizers and activists, less timewas allocated for evaluators to tell “stories”about their work, including examples of howthey have worked with organizations anddonors that are committed to strongly partici-patory approaches which strengthen front-line organizations and speed up socialchange. It would have been very useful todiscuss more examples of how evaluators andother learning partners have helped groupsstrengthen their internal capacity and sys-tems for self-assessment and/or supplement-ed those approaches with complementaryanalyses and feedback.

While discussion at Gray Rocks surfacedissues concerning the relationship betweensocial change organizations and evaluators,the group did not decide how to explore themfurther. One suggestion was to create a taskgroup of activists, evaluators and funders toexplore the broad question of how they might

work together to expand support for assess-ment and learning which fosters socialchange. An alternative suggestion was forsmall cross-sector groups to focus on suchconcrete issues as –

thinking through alternative ways evalua-tors can help community organizing andsocial change groups develop stronger,more useful learning systems; or

analyzing how activists can help evalua-tors refine their strategies for buildingstrong, mutually helpful relationships withthe groups they evaluate.

Throughout the discussion participantsstressed the importance of sufficient flexible“core” and program funding to support theassessment systems groups need. There arestill few funders which provide core fundingto cover the central costs of running anorganization and developing the basic sys-tems, internal strength, and flexibility it needsto function thoughtfully and effectively.Furthermore, program grants frequently failto cover the full costs of the assessment andplanning which are essential to the organiza-tion’s success, and underfunded evaluationscan be so inadequate that they are harmful.Activists, evaluators and funders agreed onthe crucial nature of adequate funding tocover these costs and ensure the organiza-tional health of groups leading the movementfor social change.

Current Areas of Exploration inAssessing Social Change

Participants in the conference are doingpathfinding work on a series of toughassessment and learning challenges.

These include their analyses of an organiza-tion’s progress in impacting issues of publicpolicy and institutional change and organiza-tional development questions.


“Another area of discussion related to the importance ofevaluation and learning systems as integral parts of

organizational capacity-building.”

There was agreement on the importanceof continuing to refine systems for docu-menting and assessing success in changingpublic policies, the practices of majorinstitutions, or politics – an area which iscentral to social change and difficult to evalu-ate. Several groups have set up quite elabo-rate internal reporting and reflection systemsto track their progress on policy and keepimproving their strategies and tactics.25

Others have informal but nevertheless rigor-ous internal processes for constantly assess-ing how they can increase their chances ofvictory on a particular campaign or longerrange change strategy.26 Several conferenceparticipants had worked with an intermediaryand researchers to develop a common theoryof change which they used in developingwritten case studies documenting theirimpact and how they achieved it.27 Othersare testing ways of involving the people whoare most affected by a policy or institution indefining which issues most concern them,identifying indicators for assessing progress,and then conducting an ongoing assessmentthemselves.28

In this area of power, politics and policythe particularly difficult areas to assessinclude:

causal issues including the relative contri-bution made to policy change by insidersand outsiders, and by organizations whichplay different roles in trying to promotechange;29

the impact of the immediate policy work ineducating and politicizing people, givingthem hope and encouraging them tobecome active in policy arenas;30 and

its impact in building the organization’spower, expertise, alliances, influence andsustainability or in opening up new politi-cal space where it can work to expand itsreach and influence.31

Another area of discussion related tothe importance of evaluation and learningsystems as integral parts of organizationalcapacity-building. During discussion severalorganizers and activists detailed how theyassess their progress in broadening and deep-ening their constituency and being account-able to it, or developing an expanding base ofgrassroots leaders which grows in numbers,knowledge, skills, political consciousness andinfluence.32 Some cited the statistics theykeep to measure their success in, forinstance, reaching new people, seeing themparticipate in meetings and actions, and hav-ing them begin taking leadership roles.

Another challenging area relates toassessing changes in attitudes, includingpeople’s sense of personal empowerment.“It’s a challenge to talk about the level ofempowerment: we are trying to give a sensethat they have power to people who did notthink they had any.”33

Assessment and learning questionsbecome more complex when several organ-izations work on the same or closely relat-

ed issues or projects. Ifthey work together, formal-ly or informally, to achievechanges in public policy, itis extremely difficult foreither insiders or outsideevaluators to determine therelative impact and value ofeach party’s contribution.


“All of the sectors represented at Gray Rocks expressed theirstrong desire to work and collaborate with their peers.”

“Fifty or sixty people in the villages meet every quarter andmonitor what's happening. Did electing councilors from the com-munity make a difference? How did they deliver? Attitudes?Behavior? How did they use our training? Have they sharpenedand improved their skills?”

-Sarah Okwaare Otto

In achieving success, what is the relativeimportance of each group, the coalition itself,those doing the research or communicationswork, various back-up organizations, insidersand bridge people? This is further complicat-ed if a group is part of a network or collabora-tive which has been constructed to foster peerlearning among groups involved in parallelactivities or facing similar challenges.

Several other evaluation and learningchallenges were mentioned during the discus-sions. These included:

How do you foster multi-stakeholderlearning when there are such powerimbalances (between funders and theirgrantees, for example) and groups are con-cerned about their reputations and fund-ing?

How do you sustain organizational learn-ing when there is constant staff and boardturnover?

How do you evaluate progress in rapidlychanging situations?

How do you foster candid reflection inconflict situations when it may be best forpeople to avoid certain important subjectsbecause they are too divisive or volatile?34

How do you use power analysis in evalua-tion and learning?

How do you weigh the value of action “tochange things that never change – pover-ty, wealth redistribution, power relation-ships”?

Desire for Gatherings andExchanges with Peers

All of the sectors represented at GrayRocks – organizers and activists,donors, evaluators and learning part-

ners – expressed their strong desire to havemore opportunities to work and collaboratewith their peers.

In particular, during the conference organ-izers and activists expressed a hunger forcontinuing exchange with others who areengaged in rights-based work in very differentcontexts and cultures. Groups from all cor-ners of the world feel isolated from each oth-ers’ experience and see increased exchangesas having great potential in helping themreflect on their work, raise their vision andgain new inspiration and ideas. They citedsuch exchanges as essential for sharingmethodologies and building enduring relation-ships which would make ongoing collabora-tion possible.

During the Gray Rocks conference thisdesire focused on creating new opportunitiesfor international exchange and site visits onassessment and learning questions. However,it was clear from the dialogue that groupsfrom all corners of the globe also cravedopportunities to broaden this exchange sothey could compare different philosophicalapproaches and practical strategies for bring-ing about change. Throughout the confer-ence, key questions about social change keptsurfacing, and time was insufficient to dis-cuss them satisfactorily. This left manywanting to explore these issues more thor-


“These donors’ goals are to help redirect assessment and learningapproaches so they are more useful and used, and to

increase financial support for groups which are tacklingfundamental reform issues.”

“One challenge of doing peer exchanges is how we challengeeach other to go deeper with our analysis, with our understanding.Sometimes that involves bringing in new information or otherapproaches which help us raise different questions.”

-Valerie Miller

oughly in the future. What do differentgroups mean by “social change”? What arethe different approaches they are using toachieve it? And what lessons can be drawnfrom experience with those differentapproaches?

Some professional “evaluators” from uni-versities, consulting firms, and nonprofit sup-port organizations also mentioned the advan-tages of having closer relationships with col-leagues in their own countries and worldwide. They share a commitment to assess-ment and learning systems which are forma-tive, participatory, and useful to social changeorganizations and their allies. Because thiscommitment is uncommon in the evaluationfield, they are isolated from others who sharetheir values and approaches, and they wouldrelish having more time with their colleagues,exploring how they might learn from eachother and influence their field.

Similarly, donors committed to socialchange expressed their interest in workingwith other funders to discuss issues of organ-izing and social change as well as assessmentand learning and to explore joint strategies forworking together. These donors’ goals are tohelp redirect assessment and learningapproaches so they are more useful and used,and to increase financial support for groupswhich are tackling fundamental reform issuesin different parts of the world. A centralquestion posed by one funder – Can we get acritical mass of funders to collaborate onthis?35

Possible Future Areas ofExploration and/or Collaboration

The meeting concluded with participantsdeveloping a list of four priorities forpossible future action. Three of these

issues were discussed by small groups whichreported back to the full conference in the

concluding session. These report-backs over-lapped on several points. The conferenceended with agreement on the importance offinding ways to enable people who are con-cerned with such issues to move forward andexplore them further.

The first discussion group concentrated onthe importance of devising ways in whichsocial change organizers and activists,evaluators, learning partners and donorswho share the goal of strengthening socialchange through improving assessment andlearning strategies can develop commonand separate strategies to advance thatgoal. This discussion led to agreement on anumber of points:

the need to develop more shared, open,interdependent spaces in which peoplelike those at Gray Rocks can develop rela-tionships and strategies which will fosterthe growth of “just evaluations”;

the need to identify and connect with oth-ers around the world who are discussingthese issues within their sectors – fun-ders, evaluators, social change groups – inorder to enlarge the exploration and helpsharpen, affirm, question and develop themany observations raised and recordedduring the conference regarding assess-ment, learning and evaluation;

the need to expand the base of people whoare debating these questions and furtherdiversify the voices, including reaching outto people who are excluded; “When youbring in excluded people it expands thethinking about how change happens,helping us understand better how we canassess it.”36

the need to clarify our different and mutu-al understandings of what “social change”is, how change happens, and how we see


“The conferees were interested in having new ways of sharing tools,techniques, strategies and lessons concerning assessment and

learning in a social change context.”

it from where we “sit” so that we can thendevelop better ways of assessing change;

the advantages of spending time in eachother’s organizations to help build a histo-ry of mutual trust and greater under-standing of each other’s internal politicsand processes; and

the need to shift the discussions awayfrom differences and focus around con-crete situations, perhaps in particularcommunities and in dialogue with grass-roots groups, dealing with any differencesin those practical contexts.

A second group began discussion byfocusing on the conferees’ interest in hav-ing new ways of sharing tools, techniques,strategies and lessons concerning assess-ment and learning in a social change con-text. Stimulated by the stories and strategieswhich they had heard at Gray Rocks, partici-pants stressed the importance of finding newways to share lessons and practical advicewith each other. They emphasized that learn-ing often works best when it is based on “sto-ries” which allow people to understand thecontext in which others are working, how theyare approaching issues, what obstacles andopportunities they face, how they are learningas they go ahead, and how their approachcan be adapted to other circumstances. Theytalked about the value of having a “virtuallibrary of materials”, including case studiesand “stories”.

However, they quickly broadened the dis-cussion to stress the value of having a virtualnetwork which would pro-vide space for continuingdialogue and learning onwhat it takes to be aneffective social changeorganization and howlearning and assessmentcan sharpen, strengthen

and legitimize such organizations. Such anetwork should facilitate exchange at boththe practical and theoretical levels. It shouldbe much more than just a passive repositoryfor materials. Instead it should be a hub foractively supporting exchange visits andfacilitating dialogue which fosters criticalthinking and learning. It should be a placewhere funders, groups and learning part-ners can go to find innovative assessmentand learning approaches and people.

This virtual network should also havean advocacy dimension. It should provide aforum for developing and pursuing jointstrategies to influence donors, the evaluationcommunity, social change activists and othersin ways which expand the use of assessmentand learning approaches which meet theneeds of social change groups and their allies.It should raise the visibility and credibility ofthe systems which social change groups andtheir supporters are finding most useful.

This small group pointed out that theseideas for sharing would require shared leader-ship through committees as well as dedicatedtime and resources. The group stressed theimportance of avoiding premature decisionsabout how it might be housed and supported:those decisions should be deferred until (1) there has been clarification of the commonvalues, goals, and ways of operating for thisjoint effort, and (2) there has been joint devel-opment of criteria for selecting one or moreappropriate “institutional hub(s)”. The grouprecommended that each participant con-tribute “stories” which would enable others tolearn from some aspect of their experience,


“They would like a space for continuing dialogue and learning onwhat it takes to be an effective social change organization and how

learning and assessment can sharpen, strengthen and legitimize such organizations.”

“Politics of scarcity… leads to competition among groupsrather than groups getting together to expand the pie…. Evaluationstrategies should avoid competition and promote movement andcommon action.”

-Ron Shiffman

and that there be a follow-up workshopamong interested parties to further developthe values, goals and criteria for decidingwhere to house these activities.

Another group concentrated on people’sstrong interest in devising internationalexchange programs which could help dif-ferent actors in the social change processlearn from experience in other countries.This small group stressed several key ingredi-ents to worthwhile exchanges. These includ-ed:

exchanges which are structured so thateveryone learns, the visitors and the visit-ed;

the importance of facilitation, especially incross-cultural exchanges so that peoplecan understand different contexts as theytalk about issues of power, politics andsocial change and how lessons can beapplied in their own contexts; and

the importance of careful preparation foreach exchange, and serious reflectionafterward.

They also discussed the advantages ofalliances between grassroots groups and uni-versities in fostering learning and documenta-tion from the exchanges, and the need fordonors to pool funds to support them.

Although this small group focused oninternational exchanges, this interest mayrelate to the issue of peer learning and peersupport which was stressed repeatedlythroughout the conference. Activists, evalua-tors, and donors frequently emphasized howmuch they learn and reflect on their ownwork and the possibility of changing theirapproaches when they are in dialogue withtheir peers, locally or internationally. As dia-logue continues on these issues, thoseinvolved in the discussions should consider

whether they want to broaden their discus-sion to encompass this broader set of issues.

Fourth, there was agreement on the needto increase the number of people who havethe values, skills and experience needed toevaluate social change efforts and to helpsocial change organizations reflect on, assessand evaluate their work and put what theylearn into action. This shortage exists withinsocial change organizations, in the evaluationprofession, and among learning partners anddonors. There is a need to devise remediesfor this problem and increase the numbers ofpeople who can help as external evaluators orin building strong systems of self-assessmentand peer assessment and learning. However,because discussants on the concluding daygave higher priority to the other small groups,these issues were not explored in depth atGray Rocks.

These four topics rose to the top as partic-ipants made relatively quick decisions ontheir priorities for possible future action.Other possible follow-up ideas surfaced earli-er in the conference and are reflected in thesenotes, and further exploration may well revealadditional thoughts regarding future priori-ties. As the process for continuing explo-ration of these questions unfolds, it thereforeis vital that it be open, inclusionary, respon-sive to different viewpoints, and centeredaround the participants’ commitment to –

advancing fundamental social change incountries throughout the world, and

finding ways to grow practices of assess-ment, critical reflection, evaluation, learn-ing and action which maximize thechances that the social change will be fun-damental and will benefit those who mostneed a stronger voice and greater opportu-nities.


“Participants agreed on the need to increase the number of peoplewho have the values, skills and experience needed to evaluate social

change efforts and to help social change organizations reflect on,assess and evaluate their work and put what they learn into action.”

Moving Ahead

While the dialogue was rich and strongrelationships were built, conferenceparticipants shared a frustration

that, having reached an initial understandingabout the key issues that unified this diversegroup, they had just begun to share experi-ences on many issues and there was littleclarity about how dialogue might continue.There was, however, a strong sense amongmany participants that those who weremost interested in these questions shouldcreate ways of working with others to con-tinue the exploration and seek majorchanges in assessment and learning as itrelates to organizing and social change.

The group agreed on several key principlesconcerning possible next steps.

First, the next stage of exploration shouldstart with far deeper discussion of values andthe extent to which they are shared. Itshould deepen discussion of what constitutessocial change and how it is achieved. Itshould clarify the extent to which there isagreement on these issues, and provide thebasis for exploring what should be the mostessential goals regarding future assessmentand learning in a social change context. Therewas consensus that these issues must bethoroughly discussed before people candecide how interested they are in proceedingfarther with this exploration, with whom, andwhat kind of vehicle or vehicles would bemost appropriate for moving ahead.

Second, as participantsgrappled for terms todescribe their vision of howthey might best structureor support further explo-ration and possible collabo-ration, they spoke of “virtu-al” networks and institu-tional “hubs”. They spoke

also of openness, inclusiveness, and trans-parency, of the necessity of shared values,of an approach which depends on individ-ual and collective initiatives rather thanbeing overly centralized, and which helpsthe separate sectors work on their own andtogether, as appropriate. They also spoke ofthe need to devise some way for loosely man-aging these different activities toward theshared vision and goals.

Third, they stressed the need to buildtrust and understanding over time. Withouttrust, people’s capacity to reflect and learnwill be limited. They see a need for sufficienttrust and safety that people feel comfortablechallenging each other and stimulating criti-cal thinking, and being “critical friends”. Thisvery likely requires investing time in visitingand getting to know each other, working onrelatively small practical projects together,and building relationships which will providea firm foundation for more ambitious agendasin the future.

The conference concluded with agreementthat a summary report on the conferencewould be developed to capture thinking atGray Rocks and furnish the base for fur-ther exploration. There was also agreementthat opportunities should be created forpeople to step forward to volunteer to par-ticipate in thinking through possiblefuture action on the issues which mostconcern them.


“Those who were most interested in these questions should createways of working with others to continue the exploration and seek

major changes in assessment and learning as it relates to organizing and social change.”

“Rather than sit down and look at all the differences we have…we should start identifying specific projects and from those workthrough our differences, surface them, acknowledge them, andmove on to build new understandings about how we can createsocial change.”

-Lance Evoy



Endnotes1 The Pratt Institute for Community and Environmental Development in New York City, the

Participation Group at the Institute of Development Studies at the University of Sussex inBrighton, England, the Institute in Management and Community Development at ConcordiaUniversity in Montreal, and the Center for Community Change in Washington, DC.

2 The conference planning and facilitation team consisted of Andrew Mott of the CommunityLearning Project, John Gaventa and Samuel Musyoki of the Participation Group at theInstitute of Development Studies, Irene Guijt of Learning by Design in the Netherlands,Wesley Woo of the Center for Community Change, Victoria Creed of Learning Partners inKnoxville, Tennessee, and Lisa VeneKlasen of Just Associates in Washington.

3 This report was prepared by Andy Mott with input from several members of the planning andfacilitation team. Irene Guijt’s extensive documentation of the conference was of immensehelp.

4 Participants from ACORN, CLOUT and PICO, DART, and the Northwest Federation ofCommunity Organizations in the US and ACFODE in Uganda spoke of such systems.

5 Representatives of ActionAid Uganda, Community Development Resource Association inSouth Africa, the Virginia Organizing Project in the US, the Association of IndigenousCouncils of Northern Cauca in Colombia, and the Center for Youth and Social Developmentin India cited their approaches.

6 Uganda Land Alliance involving communities in monitoring the work of their paralegals ascited by Sarah Okwaare Otto of ActionAid Uganda.

7 Jenny Chapman, ActionAid UK, discussing their ALPS (Accountability, Learning andPlanning System) process of self-assessment as well as assessment by their grantee grass-roots groups.

8 Ruben Daria Espinosa Alzate, Association of Indigenous Councils of Northern Cauca.

9 Several groups in developing countries and the US mentioned that American governmentagencies are increasing the extent to which they assess groups on the basis of their standson such policy issues as abortion and family planning and whether they are “faith based”.

10 Tom David, Director of Organizational Learning and Evaluation, Marguerite CaseyFoundation.

11 Jagadananda, Member-Secretary of the Center for Youth and Social Development in Orissa,India.

12 Andy Mott, Community Learning Project in Washington.

13 Ron Shiffman, Professor of Planning, Pratt Institute, and former Executive Director of thePratt Institute Center for Community and Environmental Design.

14 Susan Soal, Community Development Resource Center in South Africa.

15 Heather Weiss, Executive Director, Harvard Family Research Center.

16 Susan Soal.

17 Joe Szakos, Executive Director, Virginia Organizing Project in the US.

18 Mireille Landry, Institute in Management and Community Development, ConcordiaUniversity in Montreal.

19 Tom David.

20 Lisa VeneKlasen, Just Associates in Washington.

21 Francois Pierre-Louis, of Queens College and the PICO Network in the U.S.

22 Diana Bustamante, Executive Director, Colonias Development Council in New Mexico.

23 Edna Co, National College of Public Administration, University of the Philippines.

24 Heather Weiss and Tom David.

25 SPARC in India, ACORN, Northwest Federation of Community Organizations, and others.

26 Center for Community Change in the US, Future Ways in Northern Ireland, and others.

27 Evaluation of the reciprocal relationship between community organizing and public schoolreform efforts (Anne Hallett, Cross City Campaign for Urban School Reform, partnered withResearch for Action in the US).

28 Involvement of newly elected officials and villagers in assessing the impact of elections onland issues (Sarah Okwaare Otto, ActionAid Uganda); the involvement of young people inevaluating youth organizing (Barry Checkoway, School of Social Work, University ofMichigan).

29 Evaluation of coalitions and technical assistance groups (Imoyase Group in Los Angeles).

30 Highlander Center.

31 Participation Group, Institute of Development Studies, University of Sussex; Center forCommunity Change.

32 ACFODE in Uganda, Center for Youth and Social Development in India, CLOUT and PICO inthe US, Just Associates in international consulting work.

33 Lance Evoy, Institute in Management and Community Development. Concordia University.

34 Karin Eyben, Future Ways, Northern Ireland.

35 Michael Edwards, Ford Foundation.

36 Lance Evoy.


Planners and Participants in Gray Rocks Conference

Ruben Daria Espinosa Alzate Association of Indigenous Councils of Northern Cauca, ColombiaSimone Arsenault-May Administration, Concordia University, CanadaDiana Bustamante Colonias Development Council, USJenny Chapman ActionAid UKBarry Checkoway University of Michigan, USEdna Co University of the PhilippinesVictoria Creed Learning Partners, USTom David Marguerite Casey Foundation, USMichael Edwards Ford FoundationLance Evoy Institute for Management and Community Development,

Concordia University, CanadaKarin Eyben Future Ways, Northern Ireland, UKJohn Gaventa Participation Group, IDS, U of Sussex, UKCheryl Grills Imoyase Group, USPeter Gubbels World NeighborsIrene Guijt Learning by Design, NetherlandsLee Ann Hall Northwest Federation of Community Organizations, USAnne Hallett Cross City Campaign for Urban School Reform, USJagadananda Center for Youth and Social Development, IndiaPatricia Kelly Kellidge Group, USSteve Kest ACORN, USChannapha Khamvongsa Ford FoundationMireille Landry Institute for Management and Community Development,

Concordia University, CanadaMartha Lopez InterpreterValerie Miller Just Associates, USAndy Mott Community Learning Project, USGrace Mukasa ACFODE, Action for Development, UgandaSammy Musyoki Participation Group, IDS, University of Sussex, UKSonia Ospina New York University, USSarah Okwaare Otto ActionAid UgandaRobert Owens CLOUT, DART, USSheela Patel SPARC, Society for Promotion of Area Resource Centers, IndiaSuzanne Pharr Highlander Center, USFrancois Pierre-Louis PICO, Queens College, Haiti and USChief Mike Retasket Bonaparte Indian Band, CanadaSilvia San Miguel InterpreterRon Shiffman Pratt Institute Center for CED, USSue Soal Community Development Resources Association, South AfricaJoe Szakos Virginia Organizing Project, USUrvashi Vaid Ford FoundationGloria Vela Independent consultant, ColombiaLisa veneKlasen Just Associates, USHeather Weiss Harvard Family Research Center, USBessa Whitmore Carleton University, CanadaWesley Woo Center for Community Change, US

Documents

Through Assessment and Organizational LearningSeveral groups have been particularly suc-cessful in integrating assessment and learn-ing into their organizational culture. Assessment