Creating Change and Keeping it Real

Creating Change and Keeping it Real:How excellent child-serving organizations carry out their goals

Cross-Site Findings for Phase I of Community Based Theories of ChangeA five-year study of the Research and Training Center for Children’s Mental Health

Department of Child and Family Studies Louis de la Parte Florida Mental Health InstituteUniversity of South Florida

December 2002

Co-Principal Investigators:Mario Hernandez, PhD and Sharon Hodges, PhD

Authors:Sharon Hodges

Mario HernandezTeresa Nesman

Lodi Lipien

Acknowledgements

We gratefully acknowledge the assistance and support of the following people:

Administrative Support: Nancy Burrus and Sonya JonesConcept Mapping Support: Svetlana YampolskayaDesktop Publishing: Kathy AnthonyInterview Coding and Analysis: Claire Porter, Svetlana YampolskayaInterview Logistics Coordinator Sharon LardieriInterview Team: Eloise Boterf, Don Dixon, Ruby Joseph, Sharon Lardieri,Debra Mowery, Lynn Pedraza, and Claire Porter.On-Site Logistical Support: Marilyn Daniels, Blended Funding Project; LizCowart, United Way of Central Ohio; Michael Curtis and Lynn Jones, WashingtonCounty Mental Health.

Suggested Citation:

Hodges, S., Hernandez, M., Nesman, T. & Lipien, L. (2002). Creating Change andKeeping it Real: How Excellent Child-Serving Organizations Carry Out Their Goals. Cross-Site Findings for Phase I Community-Based Theories of Change. Tampa: Louis de la ParteFlorida Mental Health Institute, Research and Training Center for Children’sMental Health, University of South Florida.

Table of Contents

Introduction ..........................................................................................................................1Methods................................................................................................................................3

Site Selection ....................................................................................................................3Identified Theories of Change ...........................................................................................4Data Collection and Analysis ............................................................................................7

Document Review.........................................................................................................7Key Informant Interviews..............................................................................................7Concept Mapping ..........................................................................................................8

Cross-Site Findings............................................................................................................. 10Organizational Characteristics......................................................................................... 10Organizational Facilitators .............................................................................................. 15Sustaining the Theory of Change..................................................................................... 17Benefits and Challenges .................................................................................................. 19

Benefits....................................................................................................................... 19Challenges................................................................................................................... 22

Discussion .......................................................................................................................... 25Balancing Regulative and Generative Functions.............................................................. 26Figure 1........................................................................................................................... 28Facilitating the Balance ................................................................................................... 28Figure 2........................................................................................................................... 30

Conclusions ........................................................................................................................ 31Works Cited........................................................................................................................ 32Appendix A: Research Questions........................................................................................ 33Appendix B: Community-Based Theories of Change Descriptive Materials ........................ 34

Cross-Site FindingsPhase I: Community-Based Theories of Change

1

Introduction

The question of how human service organizationstranslate their ideas into action is an important one.For human services in general and children’s servicesin particular, little is known about how service-relatedpolicy is conceptualized and operationalized at thelocal level and how this translates to actual servicedelivery. For example, what structures and processeswithin a human service organization support a sharedcommitment to its mission and goals? How does anorganization consistently communicate its purposeand philosophy to those responsible for carrying outits mission and goals? How does an organizationsustain its focus in the face of complexity and changein its environment? Research suggests that becauseorganizational decisions are made under complexconditions, it is important to study the processes thatcontribute to these decisions (Rogers, 1995).

Community-Based Theories of Change is a national studyfunded by the federal Center for Mental HealthServices and National Institute on Disability andRehabilitation Research that is designed to addressquestions such as those mentioned above. The studyinvestigates:

• How community-based systems and programsconceptualize, operationalize, and implementservice delivery for a particular population ofchildren and families;

• How they evaluate the results of this servicedelivery;

• And how local service policies are integratedand transferred into daily operations.

From August 2000 through January 2002, three sitesparticipated in Phase I of this five-year study. Thepurpose of Community-Based Theories of Change is tobetter understand organizational processes thatsupport local policy implementation. The studyinvestigates how organizations communicate and

Community-BasedTheories of Changeinvestigates how humanservice organizationscarry out their missionand goals, how theytransfer their policyagendas acrossstakeholders, and howthey sustain their servicestrategies over time.


2

sustain their purpose and guiding principles amongthe people responsible for carrying out theorganizational mission. The central question of thisstudy is: How do human service organizationssuccessfully implement their policies at the local level?Phrased differently, the study investigates how humanservice organizations turn their ideas into action.

A theory of change can be understood as theunderlying assumptions that guide a service deliverystrategy and are believed to be critical to producingchange and improvement for children and families (asdiscussed in Hernandez & Hodges, 2001). Community-Based Theories of Change is grounded in the assumptionthat a clearly articulated, widely held theory of changefacilitates local policy implementation. It is furtherassumed that a participating site’s ‘theory of change’reflects the organization’s mission and goals andrepresents the implementation of policy by theorganization.

Phase I of this study sought to identify organizationalstructures and processes that support policyimplementation across three participating sites. Eachsite operated under a well articulated, widely heldtheory of change, although the theories of changewere different at each site.

This report will summarize the cross-site findings ofCommunity-Based Theories of Change and present lessonslearned across the three participating sites. Anoverview of the research questions for this study canbe found in Appendix A. Study descriptive materialsthat were distributed to potential and participatingsites are included in Appendix B.

Community-BasedTheories of Change isgrounded in theassumption that a clearlyarticulated, widely heldtheory of changefacilitates local policyimplementation.


3

Methods

Site Selection

Six sites were initially identified as potentialparticipants in Phase I; three sites met the selectioncriteria and agreed to participate. The participatingsites were: the Blended Funding Project (KingCounty, Washington); United Way of Central Ohio(Ohio); and Washington County Child Mental Health(Vermont).

The goal of the site selection process was to establishthat participating sites anchored their activities in aclearly articulated and widely held theory of change.Site selection began with telephone interviews of keyinformants and a detailed document review. This wasfollowed by visits to each potential site that involved:a) direct observation of decision making aboutadministrative or service delivery issues, and b) keyinformant interviews to determine the site’s theory ofchange from the perspectives of participants. Resultsof the site selection visits were summarized using alogic model format that included a concise statementof the identified theory of change as well as datarelated to the conceptualization, operationalization,and implementation of their theory. In addition, theframework included site observations related to theproject’s history, financial structure, evaluation, andpopulation of focus. Sites were given the opportunityto provide feedback and correction to their theory ofchange logic model.

The site selection process confirmed that each of theparticipating sites demonstrated the ability to clearlyarticulate and communicate local service deliverypolicies to their staff and other stakeholders. Sitesalso demonstrated their ability to report the results ofservice delivery to local stakeholders in a predictableand timely manner. In addition, they demonstrated aclear set of observable managerial processes for

Site selection criteria forPhase I of this studyincluded that participatingsites have thedemonstrated ability toclearly articulate andcommunicate their localservice delivery policies totheir staff and otherstakeholders.


4

eliciting the involvement of service deliveryimplementers from various levels of a site’sorganizational structure.

Identified Theories of Change

Having a theory of change should be distinguishedfrom members of an organization having a sharedpurpose or sharing a deep concern related to an issue,such as reducing out of home placements orincreasing community-based services and supports forchildren with serious emotional disturbance. Havinga theory of change requires an idea or theory for howto affect change related to a shared purpose, goal, orconcern. The idea must focus on how anorganization will bring about change related to apurpose, cause or specific population. The emphasisof change must be specific to a population and mustarticulate intended action and the expected result.

Using a theory-based approach, organizationalmembers are compelled to examine the underlyingbeliefs and assumptions they use to link expectedoutcomes to strategies for achieving those outcomes.This causes them to focus on why they believe certainservices or policies will lead to positive changes in theidentified population.

A brief description of the participating sites and theiridentified theories of change follow:

Blended Funding Project:

The Blended Funding Project of King County,Washington, was created as a new and collaborativeapproach to serving children with emotionaldisturbance and their families. Blended Fundingcombines funds from three participating children’sservice systems (i.e., child welfare, mental health, andspecial education) into a single resource pool. Parent-led child and family teams have access to these funds

Having a theory ofchange requires an ideaor theory for how toaffect change related to ashared purpose, goal, orconcern.


5

to provide mental health, educational, recreational,social and other supports to meet child and familyneeds. The project recognizes that the family isusually the child’s most valuable resource and theproject’s theory of change hinges on families beingsufficiently empowered to take advantage of theopportunities offered through the Blended FundingProject.

The Blended Funding theory of change is “With thesupport of the child and family team and with theability to purchase and create neededsupports…families will become empowered;everyone- the child, family, care manager, communityteam and service systems- will become more hopefuland motivated to change; service systems and familieswill collaborate more effectively on behalf of children;children’s connections to natural communities (family,school, neighborhood) will strengthen; the family willbe better able to care for the child’s needs; children’sneeds will be met across multiple domains; andchildren’s behavior and functional status willimprove” (Williams, Vander Stoep, & Jones, 1998).

United Way of Central Ohio:

The United Way of Central Ohio, an establishedcommunity agency, undertook planful and targetedorganizational change intended to redefine the agencyas a community impact organization rather than afundraising and allocation organization. Throughcommunity needs assessment, seven areas of focuswere identified: education, employment, health,housing, neighborhood development, race relations,and safety. Community impact goals were identifiedfor each of these focus areas, and Vision Councils(staffed largely by volunteers) were created to identifyand fund programs specifically intended to achievethe community impact goals. Vision Councilvolunteers described the Vision Council structure asshifting the focus of United Way away from being a


6

fundraising vehicle to becoming a partner in thecommunity. Fundamentally, this change representeda shift away from the traditional view of the fundedagency as customer to a ‘donor as customer’approach.

The United Way theory of change is “Focusing effortson targeted community initiatives will allow UnitedWay to improve critical community conditions”(United Way Site Selection Summary, 2000).

Washington County Child Mental Health:

Washington County Child Mental Health is part ofthe Vermont System of Care, a service delivery andsystem change model for children with seriousemotional disturbance and their families that wascodified into state law in 1988. Strategies fordeveloping and sustaining the Vermont System ofCare include blended funding, local decision-making,family involvement, and a wraparound model ofservice delivery. In Washington County, acollaboration between Washington County MentalHealth and the Division of Social and RehabilitativeServices provides services to children with seriousemotional disturbance and their families through theSystems in Collaboration-Higher Ground Program.

The Systems in Collaboration-Higher Ground theoryof change is “The development of a comprehensiveand coordinated system of care that providesindividualized and strength-based local services andsupports will reduce the need for out-of-homeplacement and prevent children from entering intoout-of-home care” (Vermont System of Care SiteSelection Summary, 2001).

Data collection includedthree methods: ongoingdocument review, keyinformant interviews, andconcept mapping.


7

Data Collection and Analysis

Data collection for the three participating sitesincluded three methods: ongoing document review,key informant interviews, and concept mapping.

Document Review

Document review was used to collect data on therecorded theory of change and to better understandthe results of having a theory of change. The reviewof documents typically included published literature,promotional materials, evaluation reports, and annualreports as well as forms and other documents thatverified organizational processes. Key investigators,including the co-principal investigators and theproject manager, completed the document review.

Key Informant Interviews

Individual interviews with managers andadministrators, system or program staff, and servicerecipients were used to provide more detail about therecorded theory of change, and all interview data wereused to better understand the results of having atheory of change. The interviews were scheduled inadvance and conducted by trained staff from theDepartment of Child and Family Studies at the Louisde la Parte Florida Mental Health Institute at theUniversity of South Florida. On average, interviewslasted 45 minutes. Interviewers summarized theanswers to each question in writing and these answerswere combined across respondents into a composite.Notebooks were prepared so that all respondentanswers to a single interview question could beviewed together for comparison.

Thirteen people were interviewed as part of theBlended Funding Project, including 5administrator/managers, 4 direct service providers,

Individual interviews withmanagers andadministrators, system orprogram staff, andservice recipients wereused to provide moredetail about the recordedtheory of change . . .


8

and 4 family members. In Ohio, 13 people wereinterviewed including 5 staff of United Way, 4 VisionCouncil volunteers, and 4 providers funded by UnitedWay. For the Systems in Collaboration-HigherGround Program in Washington County, Vermont,17 people were interviewed including 4 managers andadministrators, 6 direct service providers, and 7 familymembers.

A team of five staff read the interview compositesindependently, coded the data according to the eightresearch questions, and completed interview dataanalysis. Following their independent analyses, teammembers met to discuss the data and identifyemergent themes. A matrix was developed thatallowed the analysis team to consider these themes asthey related to a specific research question. Thematrix also prompted the team to identify specificpassages in the interview data that either supported orrefuted each theme under discussion. Special care wastaken to consider each potential theme acrossrespondent type so that differences in perception onthe part of administrator/managers, direct service,and family member respondents were reflected.

Concept Mapping

Concept mapping, using Concept System software(Concept Systems, Inc.) was used to collect andanalyze data related to the structures and processeswithin each of the sites that support how its missionand goals are carried out. Concept mapping is aprocess that allows a group of stakeholders to expresstheir ideas on a certain topic and then look at all ofthese ideas as they relate to one another. The resultof this process is a visual map that illustrates:

• The group’s ideas• How the ideas are related to one another• How the ideas can be organized or clustered

into general concepts

Concept mapping is aprocess that allows agroup of stakeholders toexpress their ideas on acertain topic and thenlook at all of these ideasas they relate to oneanother.


9

• How concepts are rated by the group interms of importance and effectiveness.

The sampling strategy for the concept mappingprocess was to involve individuals who wereresponsible for carrying out the mission and goals ofthe organization. There were 12 participants in theconcept mapping process for Blended Funding, 9participants from United Way of Central Ohio and 13participants from Washington County’s Systems inCollaboration-Higher Ground program. All conceptmapping participants were familiar with the servicesprovided by the participating sites and with the theoryof change used by these sites.

Concept mapping begins with a structuredbrainstorming process in which participants are givena focus statement and guided to generate statementsin response to the prompt. The focus statement wasconsistent for each of the sites:

“Generate a list of things that are done [in yourorganization] so that you and others understand howto carry out its mission and goals.”

Following the brainstorming, participants at each sitewere provided with a complete set of the statementsand asked to sort the statements into piles in a “waythat makes sense” to them. After the individualscreated their piles of statements, they labeled each pileaccording to the type of statements included in thatpile. In addition to sorting the statements, eachparticipant was given a list of all statements and askedto rate their importance on a scale of 1 to 5, 1 beingnot important and 5 being very important. Finally,participants rated the same statements as to theireffectiveness.

The Concept System software makes use ofmultivariate statistical techniques for the analysis ofdata, including multidimensional scaling and cluster

Generate a list of thingsthat are done [in yourorganization] so that youand others understandhow to carry out itsmission and goals.


10

analysis. This analysis was used to create a conceptualmap for each sites based on the perspectives of theparticipants. On these maps, statements perceived tobe similar to one another by the group are positionedclose to each other and statements perceived to bedissimilar are located farther apart. Similar statementsare grouped together in non-overlapping categoriescalled clusters based on their special proximity to oneanother.

Individual concept mapping results have been madeavailable to each participating site.

Cross-Site Findings

Data from document reviews, key informantinterviews, and concept mapping were analyzed forthe purpose of identifying emergent themes commonacross the sites. The themes identified as commonacross the participating sites included:

• Four characteristics that shape the nature ofthese organizations as they carry out theirmissions and goals.

• Two organizational facilitators.• Three elements that sustain the theory of

change consistently and over time.• Benefits and challenges identified with having a

clearly articulated and widely held theory ofchange.

Organizational Characteristics

Four characteristics emerged from the data that werecommon across the sites and seemed to significantlyshape the nature and complexion of theseorganizations as they worked to carry out theirmission and goals. These were: Identity, Integration,Initiative, and Innovation. Each of these

Four organizationalcharacteristics: Identity,Integration, Initiative,and Innovation.


11

characteristics is presented below along with specificexamples from the sites.

Identity: The participating sites shared thecharacteristic of having a strong organizationalidentity. Stakeholders within and outside of theseorganizations had a clear and shared understanding oforganization’s purpose, what the organization intendsto accomplish and why. Interview respondentsindicated the organizations are anchored in vision,mission, and values. Concept mapping data alsoindicate strong anchoring across sites in organizationalvision and mission as well as values and principles.

Organizational identity was sometimes expressed ashaving a clear organizational focus. A fundedprovider at United Way commented, “The UnitedWay is faced with the same issues all of us are in ourdaily lives or organizational lives, and that is, there aretoo many demands for the resources- time, talent andtreasure available. And if you’re really gonna move theneedle and show progress you have to focus. And thatI believe is the major objective of Vision Councils isto be able to focus on specific needs.”

An administrator for Systems in Collaboration-HigherGround in Vermont discussed the importance ofhaving a clear philosophy to fall back on, particularlygiven the challenges of reducing out-of-homeplacement and meeting individual needs, “We canalways go back to the philosophy when [we’re] notclear”. Similarly, another Vermont respondentcommented that having a strong organizationalidentity provides reinforcement for “what we’re allabout.”

A direct service provider with the Blended FundingProject commented on the importance of having aclear organizational vision, even when an organizationstruggles for consistency, “I totally believe in thevision that Blended Funding has laid forward. I know

“ . . . if you’re reallygonna move the needleand show progress, youhave to focus.”

United Way


12

they’ve struggled in figuring out what to do and howto do it. They’re now discovering strengths thatfamilies bring to the table and how important it is tohave that joint partnership.”

Integration: Organizations participating in Phase Iexhibited characteristics of organizational integration.That is, their structures and functions were wellaligned and individual roles within the organizationswere clear and well supported. Structure and functionworked together to support the achievement of theorganizational mission and goals.

The alignment of structure and function ensures thatthe intended mission can be carried out withoutstructural or functional impediments to the process.Such alignment prevents situations in which dedicatedworkers function to fulfill the mission of theorganization while facing structural barriers such aseligibility criteria or funding restrictions that preventservices being delivered as intended. Alignment ofstructure and function also restricts the ability of staffto function independently in individualized efforts tocarry out goals regardless of the agreed-uponorganizational goals and strategies.

A volunteer with United Way indicated that thetheory of change provides direction that allows morefocus on meeting identified community need and lessfocus on allocation of resources. The respondentdescribed the funding decisions made as part of hisVision Council responsibilities as “more integrated”and “broader.” This respondent commented that thefocus now is on identifying and funding “the kind ofprograms that will support [a specific] set ofobjectives” rather than simply allocating a certaindollar amount of funds.

Examples of structural and functional alignment citedby interview participants across sites include itemssuch as the availability of written program guidelines

The alignment ofstructure and functionensures that the intendedmission can be carriedout without impedimentsto the process.


13

for new and existing staff as well as regularly held staffmeetings in which management and service deliveryissues are discussed. In addition, the convenientphysical location of meeting space as well as the co-location or convenient location of collaborating stafffrom partner agencies contributed to alignment.

Role clarity adds to organizational integration ofstructure and function in that staff members clearlyunderstand how their responsibilities relate to thoseof others in the organization and to achieving thegoals of the entire organization. A Vermontrespondent indicated that role clarity supportsconsensus around what organizational roles shouldbe, “I think it’s important for roles to be defined. Sothat everybody knows . . . this is what I’m gonna bedoing, this is what you’re gonna be doing, [and] weboth agree on that, and if anybody steps out of role,it’s by agreement.”

Initiative: Participating sites also demonstratedorganizational initiative, meaning they wereachievement oriented and believed themselves to beaccountable for the results of their strategies andactions both inside and outside of the organization.

Staff, volunteers and funded providers with UnitedWay talked about the organization’s achievementorientation and willingness to take initiative bybringing critical issues before the community. Theorganization’s goal was to take a leadership rolesolving community problems and to be accountableto the community for the results of these efforts. AUnited Way volunteer commented, “The United Wayis good at evaluating their own processes, as well ascritiquing [them] openly… there is an ongoingseeking-out of improvement.”

Achievement in the Blended Funding Project wasdefined through their ability to empower families andachieve the results laid out in their model of change.

“I think it’s important forroles to be defined sothat everybody knows. . .this is what I’m gonna bedoing, this is what you’regonna be doing, [and] weboth agree on that . . .” Washington County, Vermont


14

Family members in the Blended Funding Project citedthe individualized and participatory approach toevaluation as an important measure of the project’sachievements and of its accountability to the familiesbeing served. One family member said, “I neverwould have had a broad picture of what a child lookslike as a typical teenager if I didn’t have the researchinformation. It gave me a broader scope and helpedme know what we were going toward.” In addition, aBlended Funding administrator suggested that theevaluation demonstrates the achievements of theproject when he commented, “Clearly our evaluationsshow that families are becoming empowered, thatsystems are collaborating, that children are gettingbenefits… in terms of functionality, but moreimportantly in terms of successful relationships withthe community that facilitate their growth andhealth.”

Systems in Collaboration-Higher Ground dataindicate that this program defines its achievementthrough its ability to meet challenges. The commentsof a direct service provider in Vermont describe theachievement orientation for that organization, “Ithink the best thing about the Higher GroundProgram is that we really don’t accept failure. If we’regiven a challenge, we are always looking for a way towork with that challenge.”

Innovation: Phase I sites also share theorganizational characteristic of innovation, meaningthe organizations are willing to challenge establishedconvention by taking new and creative approaches toservice delivery. These innovative strategies representcalculated and well-considered risks rather thanimpulsive decisions. And innovative approaches arealways measured against their ability to achieve theorganizational mission while maintaining anorganizational flexibility to allow for adaptation andchange in strategy. Participants at each of the sites

“I never would have hada broad picture of what achild looks like as atypical teenager if I didn’thave the researchinformation.”

Blended Funding Project


15

expressed a certain pride in their organization’s abilityto be innovative.

A funded provider with United Way identified theorganization’s efforts to encourage new providers toparticipate in their funding process as a significantinnovation, “A major change has been in theopenness of the system to encourage new providers.Before we were locked into traditional providers, nowa bigger proportion of money is set aside for newprograms (not necessarily new agencies). There is anintentionality to increase the proportion of funds fornew programs.”

Family members with Blended Funding described theproject’s flexible approach to family team planningmeetings as an important innovation in theircommunity. One family member commented, “[Theteam approach] allows you to have more flexibilityand more services to meet your child’s needs becauseyou’re not limited.” Another family memberdiscussing the Family Team process said, “Frommonth to month we’re constantly trying to come upwith different solutions . . .”

Systems in Collaboration-Higher Ground, which is anestablished interagency collaboration, definedinnovation as their ability to keep their collaborativeefforts fresh and well-functioning. An administratorin Vermont talked about the importance of keepingthe program vital, saying, “We have to educate newpeople into it and there’s this ever-present need . . . tokeep it fresh, keep it working, keep the relationshipsgood, keep the program a living and vital program.”

Organizational Facilitators

The cross-site data suggest that two facilitatorssupport the organizational characteristics of identity,integration, initiative, and innovation. These

Two OrganizationalFacilitators: Leadership,Communication.


16

facilitators are: leadership and communication.These facilitators enhance the impact of theorganizational characteristics by reducing anyobstacles to their accomplishment. Without thesefacilitators, the influence of strong organizationalidentity, integration, initiative, and innovation wouldbe impeded.

Leadership was also identified as crucial to the sites’ability to achieve their mission and goals. Across thesites, leadership provided inspiration, guidance, anddirection. The leadership style observed across thesites can be characterized as both strong andempowering. Participants at each site were clearabout lines of authority and the established decisionhierarchy. However, across the sites, authority fordecision-making was decentralized within clearlydefined roles and responsibilities. Stakeholders at alllevels (including staff, volunteers, and family membersat some sites) were empowered to problem solve andmake decisions within their areas of responsibility. Inthis way, leaders inspired and guided organizationalactivities without maintaining absolute control.

Communication served the purpose of transmittinginformation and ideas within and outside of theparticipating sites. Across the sites, communicationcan be characterized as open, multi-directional, andcontinuous. In addition to formal methods ofcommunication such as regular staff meetings, writtenreports, and training events, the cross-site dataindicate that informal lines of communication are alsostrong. These include easy access to co-workers andother stakeholders by means of impromptu phonecalls, office visits, and lunch gatherings in whichconcerns and issues related to policy implementationare discussed as needed. Ease of communication wasidentified through interview and concept mappingdata as a crucial factor in the sites’ ability to carry outtheir intended mission and goals.

Stakeholders at all levels(including staff,volunteers, and familymembers at some sites)were empowered toproblem solve and makedecisions within theirareas of responsibility.


17

Sustaining the Theory of Change

In addition to characteristics and facilitators, threeelements were identified across the sites that affecttheir ability to sustain their theory of change withconsistency and over time.

These elements can be summarized as follows:

The first element is organizational commitment to thetheory of change through adequate and consistentsupport. Cross-site data indicate that theories ofchange were well supported and frequently reinforcedwithin and outside of the participating organizations.Adequate resources were provided for training and aswere descriptive or promotional materials that clearlystated the theory of change and what it intended toaccomplish. This ranged from family empowermenttraining in the Blended Funding Project to recordedtelephone messages used to describe the purpose andgoals of the United Way Vision Council structurewhen incoming callers are put on hold.. Consistentreinforcement of the theory of change included thetheory as well as values and principles beingprominently hung on meeting room and office wallsand in formal and informal meetings in which thetheory was clearly restated.

Without adequate and consistent organizationalsupport of the theory of change, multiple theoriesmay be implemented simultaneously as stakeholderswithin and outside of the organization adapt andrefine their activities in ways that are not consistentwith the theory.

The second element is that organizations must have away to know if the strategies that are actuallyimplemented are the same strategies that are in thetheory of change. Information about implementation

1st Element:Commit to the theory ofchange.


18

can be gathered informally or through a formalevaluation process. Regardless of the informationsource, however, it is important that an organization isable to:ß Confirm that their strategies are in fact serving

whom they intended to serve;ß Confirm that they are providing the services

and supports they intended to provide.

Without information confirming that implementationstrategies are consistent with the theory of change,any information about the results of the programcannot be attributed to the impact of the programstrategies.

The third element is that the organization must have away to know if their strategies are producing thedesired impact. To accomplish this, evaluation andother information sources can be used to determine ifthe results of service delivery are what were expectedfrom implementation of the theory of change.Information about the impact or results of theirstrategies can help determine if there are unexpectedchallenges to implementation and sustainability.

Without information regarding the results of servicedelivery, organizations cannot determine if theirtheory of change for children and families continuesto make sense under current conditions.

Each of the sites demonstrated a strong reliance oninformation for guiding decisions related toimplementation of strategies and the achievement oforganizational mission and goals. All three sitesdemonstrated a strong reliance on informal sources ofinformation. In addition, both Blended Funding andUnited Way relied on ongoing formal evaluationefforts as an important source of information aboutwhether they were implementing intended strategiesand if these strategies were producing the expectedimpact. Although there is no ongoing evaluation of

3rd Element:Know if theory-basedstrategies are producingtheir desired impact.

2nd Element:Know if implementation isconsistent with thetheory.


19

Systems in Collaboration-Higher Ground, thisorganization has been part of more than a dozenevaluation efforts during the past decade.

Benefits and Challenges

Interview participants were asked to identify bothbenefits and challenges that they associate with usinga theory of change approach to policyimplementation. Across sites, several themes emergedrelated to these benefits. Challenges and limitationspresented by using a theory of change approach werealso identified. These are discussed below.

Benefits

Four benefits to having a theory of change wereidentified across the participating sites.

Focused Effort: Participating sites report that havinga theory of change is linked to being better able totarget or focus services in order to produce desiredoutcomes. For example, respondents in Vermontemphasized the importance of having a philosophythat can focus efforts, guide decisions about servicesand be linked to positive outcomes, “ . . . it’simportant that the folks who are closest to the workcontinually have [the theory of change] to go back toand continue to have that as a set of guidingprinciples, as a set of goals . . .” United Wayparticipants emphasized the benefits of being able tofocus on issues that have been identified by thecommunity as concerns and being able to say no tothings that do not fit the overall goals. And BlendedFunding emphasized the ability to focus on effortsthat will be most beneficial to their target population,children and families, with one respondent saying, “Ithink we are seeing quite a few benefits, and I thinkthe theory of change helped us direct our work tothose issues we felt were going to be most important

Theory of Change Benefits:• Focused effort• Expanded creativity• Improved ability to

demonstrateeffectiveness

• Increased engagement


20

and beneficial to children and their families in termsof improving their life situation. If we hadn’tapproached, or come up with, a theory of change, wemight have done the same kind of scatter-shot workthat we tend to do in the child-serving system whenwe don’t collaborate and come up with a commonvision about what we’re trying to accomplish.”

Expanded Creativity: All sites described the abilityto be creative and flexible in designing and providingservices. Although not directly attributed to having atheory of change, in each example the theory ofchange provided a philosophical and strategic basethat supported creativity and flexibility for the site. Inthe Blended Funding Project, the theory of changesupports the idea of choice and options for families.A family member commented, “The mental healthteam from before gave no options; they weren’t there;they weren’t available. With Blended Funding, therewas a lot more to choose from, more variety andfreedom in what kind of stuff we needed. We gotmore appropriate help.” Another family membercommented, “Families don’t always fit into this nicelittle system we have, so we [Blended Funding] try tobe creative and kinda mold services for familiesinstead of expecting the families to mold to us.”

In Vermont, creativity is linked to development ofservices to meet needs rather than fitting needs intoexisting services. “The fact that each plan isindividualized really looks at a particular child. There’sno formula.” And at United Way, creativity ispromoted through the flexible development ofproposals for funding and the development of goalswithin Vision Councils that tie to the overallcommunity objectives. Central Ohio United Waygives agencies freedom to propose strategies, but theymust be based on a logic model format that linksthem to overall objectives for the community and aproposed budget. A United Way funded providercommented, “. . . There’s quite a bit of flexibility. . .

“Families don’t always fitinto this nice little systemwe have, so we try to becreative . . .”



21

We can name our own criteria, own objectivemeasurements, but have to make sure we are meetingthem the way we stated them and are still targeted onthe original goal…They will say, ‘You tell us howyou’re going to meet this goal, and if it’s reasonableand you’ve got all the pieces together the way you seethem, then we can go for that.’ ”

Improved Ability to Demonstrate Effectiveness:Having articulated a theory of change givesstakeholders a means by which to judge theeffectiveness of the organization’s efforts by linkingidentified needs to services and outcomes. In theBlended Funding Project, outcomes identified acrossall respondents were directly related to stated goals inthe theory of change. These included familiesbecoming empowered, child-serving agenciescollaborating, and children benefiting in terms offunctionality as well as developing successfulrelationships with their community. Vermont linkseffectiveness to their ability to work collaboratively toreach the desired outcome of reduced out-of-homeplacement. “… if kids have to be placed out of theirhomes, that they’re placed in Washington County asopposed to be placed anywhere else in the state. Wehave data that supports that.” United Wayrespondents link effectiveness to having a clearmessage that supports the ultimate goal of increasingdonations and having an impact on the community.A respondent commented, “We do annually what wecall a Caring Report that looks at communityconditions, and generally peruse the data, and surveyseveral hundred residents. That gives us a snapshot orperception of how we’re doing.”

Increased Engagement: Sites report that theinvolvement of stakeholders and the quality of theirinvolvement can be improved by having a theory ofchange because people understand the desiredoutcome of the effort and their role in it. United Wayinterview responses suggest that having a clear theory

Sites report that theinvolvement ofstakeholders and thequality of theirinvolvement can beimproved by having atheory of change.


22

of change has increased commitment to the UnitedWay mission. United Way has seen a growth in itsdonor base as well as increased ability to attractknowledgeable volunteers and agency proposalstargeted to the mission. A United Way staff membercommented, “I think the most important piece of it isthe fact that we are focusing on initiatives that thecommunity has spoken about versus those that we’vedeveloped ourselves . . . In working on communityissues we are working with and in conjunction withthe community.” Respondents with Systems inCollaboration-Higher Ground believe that increasedengagement of stakeholders is a result of havingshared goals, values and philosophy. A direct serviceworker commented, “I think the most helpful thing isthat the team members are all communicating andthat we all understand what the goals are.” And aBlended Funding family member commented, “Thepeople involved are more involved, they know it’s notjust business as usual. There are certain things theyhave to do to be part of it, so they’re just moreinvolved.”

Challenges

Four challenges were identified as associated withhaving a theory of change.

Resistance to Change: Although the theories ofchange varied across participating sites, eachorganization was invested in creating change throughits implementation of policy. Participating sitesreported that using a theory of change approach canproduce resistance to change at both the individualand organizational levels. In the Blended FundingProject, achieving family empowerment wasunderstood to mean that families would be involvedat all levels of the program. A Blended Fundingadministrator discussed their theory of familyempowerment and the resistance to involvingfamilies, “When we formed the Blended Funding


23

Project . . . we wanted families to be represented at alllevels of the organization. Some of our traditionalpartners felt there was some conflict of interest, thatyou couldn’t have families on the [interagency]steering committee.”

United Way respondents discussed the tension andresistance created by their new priority for fundingprograms that are specifically designed to have impactin defined community need areas. The theory ofchange represented a significant shift in the UnitedWay funding process. The new approach emphasizesfunding programs that target their programs to meetidentified community needs and uses the VisionCouncil structure to make these funding decisions. Afunded provider commented, “I believe that the roleand the place of an agency has been minimized,whereas agencies were more powerful, moreimportant in the past, they are subordinate to theVision Council and the United Way brandmarketing.” Commenting on resistance to the moretargeted funding strategy, an administrator said,“There are still people who want [United Way] to beall things to all people . . .I think, slowly, we’reunderstanding that’s not possible.”

Turnover: Staff turnover was identified as achallenge to implementing a theory of change becauseturnover affects how widely the theory is acceptedand shared. A Blended Funding administratorcommented, “We’ve had leadership changes at thecounty level in foster care and mental health, andevery time that has happened we’ve had to go back inand completely re-explain how we’re blendingmoney.” Turnover in the care management staff hasalso affected Blended Funding. A family memberobserved that her new care manager was “doing theopposite of how the Blended Funding Project wasstarted” by making foster placement decisions withoutteam input. Systems in Collaboration-Higher Groundhas worked to reduce similar problems with staff

Theory of ChangeChallenges:

• Resistance to change• Turnover• Balancing the real and

the ideal• Need for capacity

building


24

turnover through an emphasis on teamwork andconstant communication. One staff membercommented that communication with team membersbuilds understanding of the goals and helps peoplework toward similar goals.

Balancing the Real and the Ideal: Participatingsites report that the theory of change must beresponsive to the realities of implementation,including budget, community, and child safety needs.Without this, sustainability is an issue. BlendedFunding respondents reported that, initially, theirtheory of change called for replacing formal supportswith informal supports. They found that familiesbenefited from increases in both formal and informalsupport because they found that children with seriousemotional disturbance have ongoing and complexneeds and their necessity of formal supports is notnecessarily reduced by an increase in informalsupport. Respondents from Systems inCollaboration-Higher Ground found that their theoryhad to be based in the reality of practice in order to beeffective. They found that holding people to a theoryof change should be linked to reality. A Vermontadministrator said, “Frequently we come up withconstructs and they sound all sweetness and light . . .this work is extremely difficult, it’s trying, it can wearpeople down, and if you don’t have a basic set ofvalues to fall back on that remind you why you’rehere, it’s not going to work.” United Wayrespondents indicated that viewing their theory ofchange as a developmental process rather than a staticone has helped make their commitment to communityimpact reality rather than theory. One United Wayfunded provider indicated that the Vision Councilstructure has been criticized as too categorical toaccommodate groups with a more comprehensive,widespread mission, commenting, “We are stillstruggling with this, but I think [we’re moving in] apositive direction. The United Way does try to bereceptive to reality and they’re not afraid to hang onto

“We’ve had leadershipchanges . . . and every timethat has happened we’vehad to go back andcompletely re-explain whywe are blending money.”



25

what’s good and move forward to some areas thatseem more appropriate.”

Need for Capacity Building: Sites also found thatthe capacity of their service systems needed to beimproved in order to better implement their theoriesof change. For example, the ability to provideindividualized care is limited if there is inadequateservice capacity. The best intentions of systemcollaboration cannot be realized if access to services isrestricted by the availability of a broad range ofservices and supports. A Vermont family membertalked about the need to expand the capacity of thesystem to provide services in order to better carry outthe theory of change. Similarly, Blended Fundingrespondents identified the importance of expandingcapacity for services such as treatment foster care inorder to better meet the needs of children andfamilies.

Sometimes it is important to build capacity in areasother than direct services. A Blended Funding serviceprovider noted an increased capacity in accountingprocedures was necessary in order to support theproject, “The biggest challenge we have is meetingaccounting needs . . . we need to develop this processso that money to support families is not hung up forweeks on end.” United Way data suggest the need tobuild the capacity of all stakeholders to implement thetheory rather than just understand it. Acknowledgingthat volunteer staff can understand and even articulatethe theory of change, a United Way administratordiscussed the need for training that would betterprepare them to implement the theory of change intheir work.Discussion

The cross-site findings provide insight into theplanning and management processes of human serviceorganizations that have a clearly articulated and widelyheld theory of change. A useful concept emerging

Sometimes it is importantto build capacity in areasother than direct services.


26

from this analysis is the existence of a balancebetween regulative and generative organizationalprocesses in human service organizations. Drawingfrom Uzzell’s (1990) description of regulative andgenerative organizations, regulative processes can beidentified as those which rely on power for decisionmaking authority, employ standardization of workpractices, filter out information that would providefeedback, and treat actions as final rather thanconditional. In contrast, generative processes can beidentified as those which rely on information fordecision making authority, allow for idiosyncratic orcontextual design, incorporate information that willprovide feedback, and treat actions as experimentaland open to adaptation when necessary.

An example of an organization that we would expectto be highly generative is a street vendor whoseproducts, prices, and location are open to change andadaptation depending upon the relative success ofcurrent strategies. Customers appreciate a generativeapproach in circumstances such as these because theybenefit from a vendor who can adapt quickly to thechanging the needs and expectations of customers.

In contrast, an example of an organization customershope will be highly regulative is a nuclear power plant.In this case, an authoritative structure, standardizationof practice and careful monitoring of criticalindicators ensure public safety as well as a reliablepower source. Customers welcome adherence to theregulatory aspects of planning and management thatare in place in a nuclear power plant.

Balancing Regulative and Generative Functions

Although these examples illustrate the extremes ofgenerative and regulative planning and management,most organizations must develop processes that fallsomewhere between the extremes. These processesmust allow for organizational responsiveness built

A useful concept arisingfrom this analysis is theexistence of a balancebetween regulative andgenerative organizationalprocesses in humanservice organizations.


27

upon a foundation of proactive and strategicimplementation. To achieve this, human serviceorganizations cannot be operated through entirelygenerative or entirely regulative processes.

The organizations that participated in Phase I of thisstudy demonstrated a balance between regulative andgenerative organizational processes that wasconducive to their ability to carry out their missionand goals. The data from these sites suggest thatusing a theory of change supports a balance betweengenerative and regulative functions. For example,reliable service delivery and accountability for resultsrequires an adherence to rules and standardization ofprocedure that can only be characterized as regulativein nature. Regulative processes are important becausethey help ensure consistent and dependable access tocare for children and families spanning a side varietyof community contexts and populations of focus.However, in their extreme, regulative processes typifythe categorical nature of eligibility, funding, andservice delivery in child-serving systems. Regulativeprocesses often restrict the flexibility that is needed sothat services can respond quickly to individual needs.In order to be strength-based and individualized,systems and programs must be able to respond in atimely manner to information that can only beprovided through generative processes. A balancebetween generative and regulative planning andmanagement may be necessary in order for child-serving agencies to provide individualized andresponsive services and supports that respond tocomplex and changing family and community needs.

This balance between generative and regulativeprocesses is illustrated in Figure 1 below.

. . . human serviceorganizations cannot beoperated through entirelyregulative nor entirelygenerative processes.


28

Figure 1

Facilitating the Balance

The relationship between the identified organizationalcharacteristics and facilitators that is suggested by thisresearch is that, in combination, they help humanservice organizations establish a balance betweenregulative and generative planning and management.

Examples of this balance are seen across the sites. InVermont, the collaboration between WashingtonCounty Mental Health and Social and RehabilitativeServices provides an illustration of a generative-regulative balance. On the generative side, thecollaborating agencies are attempting to provideservices that are strength-based, clinically appropriateand individualized. A family member describes thegenerative aspects of service delivery as, “the wholeteam is trying to get him placed in a more appropriateresidential setting than the one he is in. [Systems inCollaboration] is basically a middle man to get [my]son services from other agencies.” This generativeapproach is challenged by a regulative need to assure

Regulative GenerativeBalance

• Reliance on Power

• Employs Standardization

• Treats Actions as Final

• Filters out Feedback

• Reliance on Information

• Allows for Idiosyncratic, Context-sensitive Design

• Treats Actions as Experimental

• Incorporates Feedback

Planning and Management ProcessesFigure 1


29

community safety and budget resources equitablyacross clients. A direct service provider describes theregulative aspects of service delivery, “Each child hasa budget that we have to deal with, and we’re fairlylimited to …how much time and how much moneywe’re able to spend on each client…. But, above that,with the population I work with, they’re restrictedbased on their treatment. They have to make sure thecommunity is safe first, before [the clients are] able togo out and do things. So that’s another guideline.That [the clients are] doing their treatment and they’resafe to enter certain types of activities.”

The balance between generative and regulativeplanning and management is, perhaps, most notablein the Blended Funding Project where a significantstructural change has altered service delivery processesand relationships. Responding to serious costoverruns, the decision was made in 2001 to shift thefunding structure from one that was flexible andwithout limitation (completely generative) to a flexiblebut capitated monthly rate per child (more regulative).Some interview respondents noted that when thiscapitated rate has not been enough to meet the needsof an individual child and family, they have appealedto Blended Funding administration and receivedadditional funding. However, some family memberssuggested that this change restricts the ability of theprogram to respond adequately to individual needsand may signal a shift in understanding of the theoryof change. What was noted by respondents atadministrative, direct service and family levels is thathaving a well-articulated theory of change as theirfoundation has allowed the program to survive in thistime of significant challenge because it providedcommon ground from which everyone could work.

The relationship between regulative and generativeplanning and management is illustrated in Figure 2below.


30

Figure 2

• Identity• Integration• Initiative• Innovation

Facilitating the Balance

RegulativePlanning

andManagement

GenerativePlanning

andManagement

Communication

Leadership

Figure 2


31

Conclusions

Often, the greatest challenge to successful policyimplementation is turning ideas into action. Phase Iof Community-Based Theories of Change was designed asan inductive process to investigate how human serviceorganizations implement policy at the local level.This phase of the study observed that operating froma clearly articulated and widely held theory of changeprovides an anchor that enables organizations tomove from conceptualization to implementation in away that remains true to the organization mission andgoals. In addition, the study identified organizationalstructures and processes that support anorganization’s ability to carry out its mission andgoals. Using a theory-based approach was notregarded as a neutral or benign action by participatingsites. Although participants at each site easilyidentified significant benefits to having a theory ofchange, they also observed that the approach requirescommitment and consistency in order to bemaintained.

Phase II of this study will seek to confirm ordisconfirm the initial findings of this study. Thesecond phase will focus on five child-serving agenciesthat use the same theory of change. Documentreview, concept mapping and individual interviewswill be used to gather information about how theseagencies transfer policy agendas across theirstakeholders, structural and relational factors thataffect how they carry out their mission and achievetheir goals, and how these organizations sustain theirlocal service strategies over time.


32

Works Cited

Concept Systems, Inc. (2002). The Concept System (Version 1.75) [Computer Software].Ithaca, NY: Author.

Hernandez, M. & Hodges, S. (2001). Using theory-based accountability to supportsystems of care. In M. Hernandez & S. Hodges (Eds.), Developing Outcome Strategies inChildren’s Mental Health. Baltimore: Paul A. Brookes Publishing Co.

Rogers, P. & Hough, G. (1995). Improving the effectiveness of evaluations: Makingthe link to organizational theory. Evaluation and Program Planning, 18, 321-332.

United Way Site Selection Summary. (2000). Tampa: Louis de la Parte Florida MentalHealth Institute, Research and Training Center for Children’s Mental Health.University of South Florida.

Uzell, D. (1990). Dissonance of formal and informal planning styles, or can plannersdo bricolage? City and Society 4, 114-130.

Vermont System of Care Site Selection Summary. (2001). Tampa: Louis de la ParteFlorida Mental Health Institute, Research and Training Center for Children’s MentalHealth, University of South Florida.

Williams, M., Vander Stoep, A. & Jones, B. (1998). Families as full research partners:Blood, sweat, and laughter. Presented at the Building on Family Strengths Conference,Research and Training Center on Family Support and Children’s Mental Health,Portland State University, Portland, OR.

Documents

Creating Change and Keeping it Real