1

The problem of problem structuredness in the

Implementation Research literature: a state of the art

*Gustavo Valdivieso1

The body of literature on Implementation Research has been built steadily since the

publication of Pressman and Wildavsky’sImplementation (1973).Successive approaches

have dominated the literature in a rather short time span, and the field itself has dispersed

so much that most of the research on implementation affairs cannot even be found now in

the journals of public administration or political sciences, but on others of thematic fields

such as healthcare and education (Saetren, 2005; Sanfordt, Roll and Moulton, 2013).

There have been several comprehensive reviews of the Implementation literature including,

among the most well-known, those by Sabatier and Mazmanian (1989, orig. Sabatier 1986),

O’Toole (1986) and Hill and Hupe (2007). A very recent review is that by Sanfordt, Roll

and Moulton (2013).

In order to guide us throughthe relevant literature we will follow the paths of the previous

reviews, incorporating additional elements only when needed. We will pay close attention

to the number and level of units of analysis included, and the existence of a multi-actor

approach (O’Toole, 1986). There is another element that we want to look at, however: the

presence of considerations about the “problem” being dealt with, and the level of

structuredness of that problem.

Problem structuredness was a concern for the first generations of policy analysts and seems

now confined to the dominion of post-positivists, constructivists and critical policy

analysts, far away from the interests of implementation research. In our view,

however, the recent interest on a multi-level analysis of implementation which

takes into account not only the program, but also the organizational level and even

the policy field level (Sandfort, Roll and Moulton, 2013) and even more, the levels

of program, policy regime and governance mode (Hill and Hupe, 2007; Howlett,

2011) paves the way for a reconsideration of the role of consensus on problems and

consensus on values –what we are calling problem structuredness- for specific

dependent implementation variables.

1 PhD student, Universidad Externado de Colombia. This article was originally completed in January 2014.

2

Along the following pages, we will first proceed to delimitate our concepts for problem

structurednessand implementation. Then we will proceed with a thorough review of the

implementation research literature. In the third place, some conclusions will be drawn on

the subjects of implementation research and the role of problem structure among them.

A. Problem structuredness and implementation: basic

definitions

Despite all the clarifications about previous research on the subject made by O’Toole

(2000) and Hill and Hupe (2007), 1973 is still the year of the official birth of

implementation research, with the publication of the first edition of Implementation by

Jeffrey Pressman and Aaron Wildavsky (Pressman and Wildavsky, 1973). It was also the

year that Rittel and Weber used the words wicked problems to name a whole category of

problems that were basically unsolvable using analytical techniques (Rittel and Weber,

1973).

The first known use of the concept of structurednessis in Simon, 1973, with the author not

defining it but the different levels of structuredness instead, and defining problem

structuring as methametods that, in a multi-level process of policy analysis, come before

methods of problem solving. Problem structuring “…is a a continuously recurring phase of

policy inquiry in which analysts search among competing problem formulations of different

stakeholders, is no doubt the most important activity performed by policy analysts” .

The essential dimensions of problem structuredness for Dunn (2008) are three: a. Number of

decision makers b. Consensus on utilities (values) and c. number of alternatives. Well-

structured problems are those with less decision makers, more consensus on values and a

smaller number of alternatives.

Robert Hoppe (Hoppe, 2010) bases his analysis on this subjecton Dunn’s, yet identifies only

two relevant dimensions for problem structuredness: agreement on relevant knowledge and

agreement on values. Being more “cognitive” in his approach, it is natural that he understands

the number of decision makers as mainly another factor that may or may not increase the

divergence on values. After all, problems are, both for him and for Dunn, essentially cognitive

constructions.

Policy implementation was defined by O’Toole (2000) as

“whatdevelopsbetweentheestablishmentofanapparentintentiononthepartofgovernmenttodosome

thing, ortostopdoingsomething,andtheultimateimpact intheworldofaction”. The author

distinguishes this definition from the one of Sabatier and Mazmanian, that includes “the

passage of the basic statute”, the decisions of the implementing agencies, the compliance of

target groups and “the actual impacts –both intended and unintended- of those outputs, the

3

perceived impacts of agency decisions, and finally, important revisions (or attempted revisions)

in the basic statute” (Sabatier and Mazmanian, 1989).

O’Toole’s is a bottom-up approach to implementation, according to Hill and Hupe (2007), yet

the more convenient to focus, not on policy learning or policy evolution over rather long

periods of time –like Sabatier began to do after the mid-1980s- but on the rather

straightforward question of what happened after a problem was “found” and a decision was

made.

B. The evolution of implementation research

Three periods can be identified in the evolution of this field of research. The first one, from

1973 to the end of the 1980s, when implementation was viewed as a process and explored

mainly through case studies, without consensus on the most adequate approach to analysis

(top-down vs. bottom-up). The second period starts during the second half of the 1980s

(1986), when the hows of implementation cease to be the key questions and the presentation

of a process starts to vanish in favor of a more political perception of implementation and an

increased interest in causality, and finally a third period from the 1990s to the present, when

a) no paradigmatic framework has been developed, although there is a large consensus on the

relevance of a multi-level and multi-actor analysis of implementation, as opposite to the

earlier dominance of single case studies looking at the fate of specific policies and programs

and b) the implementation research literature has expanded and diversified beyond the

boundaries of policy analysis, while a more “scientific” approach to implementation (the so-

called implementation science) insists on finding its own space.

The widely agreed-upon time zero for implementation research is the study by Pressman and

Wildavsky (Pressman and Wildavsky, 1973), Implementation: How great expectations in

Washington are dashed in Oakland or why it’s amazing that federal programs work at all.

The title in itself tells much about the approach: how the decision made at the top faces

several difficulties on its way to completion –and may never reach it. Two of the major

conclusions of the authors are the complexities of joint action –and therefore the need to keep

the number of coordination points as low as possible- and the need to avoid long causal

chains between the intended action and the intended effect –since it is difficult to forecast

what the concrete effect might be- that was understood by Sabatier and Mazmanian (1989) as

“the need for sound causal theories”. Pressman and Wildvasky’s only unit of analysis was the

program, and therefore there was no attempt at a multi-level analysis of implementation.

Their Oakland case involved multiple actors, and thus one of their main conclusions –the

complexity of low probability of success in joint action- is related to the specific problems of

multi-actor implementation.

4

In recent years, however, a reappraisal of the significance of earlier literature has taken place

(Hill and Hupe, 2007; Sanfordt, Roll and Moulton, 2013), and the role of Phillip Selznick

(Selznik, 1949) has been highlighted. As the latter authors write, Selznick´s classical study on

the Tennessee Valley Authority (TVA),TVA and the Grass Roots, although essentially

sociologic, included observations on how the goals and outcomes of the federal economic

development initiative were substantially influenced by the implementation context where

cooptation by the local leaders occurred. In broader terms, it has been acknowledged that

scholarship informing implementation began long before 1973 in the fields of public

administration, political science, sociology and economics, also including inter alia the works

of Blau (The Dynamics of Democracy, 1955) and Kaufman (The Forest Ranger, 1960),

(Sandfort, Roll and Moulton, 2013; Hill and Hupe, 2007). Hill and Hupe go further back until

the times of the Greeks and the Romans, since “a significant amount of the Greek and Roman

literature is about the implementation problems confronting those who sought to organize

societies and engage in war” (Hill and Hupe, 2007).

For the first period –implementation perceived as a process and its analysis mostly through

case studies- our first list of reference will be that of Sabatier and Mazmanian (1989), to then

complement it with the extensive review by O’Toole (1986).

Sabatier and Mazmanian (1989) include in their list of relevant works for what they call the

“first generation” of implementation analysis, with a defined top-down approach and

“pessimistic conclusions”, the works of Derthick (New towns in town, 1972), Murphy (The

education bureaucracies implement novel policies, 1974) and Bardach (The implementation

game, 1977). We can include here the writings of Paul Berman (1978) and Rein and

Rabinovitz (1978). If we broaden the scope to include the whole period before the shift to

causality and the disaggregation of the until-then unitary (idea of the) implementation

process it is possible to identify two first attempts at frameworks for analysis (Van Metter

and Van Horn, 1975; Sabatier and Mazmanian, 1980) both within the top-down logic. And

also, the beginning of the bottom-up reaction signaled by Lipsky’s reflections on the role of

street-level bureaucrats (Lipsky, 1980) and the publications of Benny Hjern (1981, 1982)

among others. The contributions of Laurence O’Toole (1984,1986, 2000) and

MacolmGoggin(1984,1986) also fit here.

According to Aguilar (1993) Derthick’s “merit was to launch for the first time the rhetorical

question Why a federal program failed? …her research highlighted “the limits of

centralization”, questioning the assumption of the easy and subordinated articulation of the

State and local governments with the federal government. The Feds happened to have a

limited capacity to influence local governments, and a tendency to set ideal goals for

achievement (Derthick, 1972, p. 83, quoted by Aguilar, 1993, p. 37). Commenting on

Derthick’s work, Aguilar wrote that “idealism and federalism seem to go hand in hand”

(Aguilar, 1993, p.38).

5

Bardach’sThe Implementation Game (Bardach, 1977) introduces the notion of a political

dimension to the implementation process –hence the idea of it involving “games”- and also

the advice for policy makers to “fix the game” by following through the implementation of

decisions. According to Hill and Hupe, the book provided ammunition both to the future

top-downers, with its advice for a careful scenario writing, and to the future-bottom-upers,

with the acknowledgement of an inevitable political dimension to implementation.

There are few better examples of the top-down approach, anyway, than Bardach’s picture

of the “assembly line”:

“Whatever else it is, a policy –or program- implementation process is an assembly

process. It is as if the original mandate, whether legislative or bureaucratic or

judicial, that sent the policy or program in motion were a blueprint for a large

machine that was to turn out rehabilitated psychotics or healthier old people or

better educated-children or more effective airplanes or safer streets. This machine

must sometimes be assembled from scratch. It can sometimes be created by

overhauling and reconstituting an older, or preexisting, machine. Putting the

machine together and making it run is, at one level, what we mean by “the

implementation process”. “Implementation problems”, as we said above, are control

problems, but they are specific to the assembly activities that constitute some

“implementation process”. (Bardach, 1977, p.36).

Is there anything more top-down than an assembly line?.

Rein and Rabinovitz (1993, orig. 1978) are among the first to talk about “implementation

politics” and devote their attention to “what happens between inputs and outputs: how

policies are modified as they are translated from administrative guidelines to practice”

(Rein and Rabinovitz, 1993, p. 147). They do not want to test a theory, but to develop a

framework and define implementation as 1) a declaration of government’s preferences 2)

mediated by several actors that 3) generate a process characterized by power relations and

negotiation and identify three kinds of imperatives influencing it: the legal imperative of

complying with the legislative demand, the rational-bureaucratic imperative of doing what

can be rationally defended and the consensual imperative of facilitating agreements

between stakeholders. Tensions between these three imperatives influence differently three

stages of the implementation process: policy elaboration, resource distribution and

supervision (Rein and Rabinovitz, 1993, p. 158). To them, the best way of understanding

implementation politics is to think of it as an attempt to bridge the gap between the three

imperatives (Rein and Rabinovitz, 1993, p. 184). The specific way in which the conflicts

are solved –and with it, the form that the implementation process will take- depends on the

clarity, relevance and consistency of the goals, the type, level and opportunity of the

resources and the complexities of implementation’s administrative process (Rein and

Rabinovitz, 1993, p. 184).

6

Just like Rein and Rabinovitz, in 1978 Berman considered that enough knowledge was

available about implementation, and that time was ripe for a framework. He seems to

borrow from Derthick for the idea of what he calls “macro-implementation” –how to make

federal policy influence local organizations- (Berman, 1993, p. 291) but he also studies the

“micro-implementation” of organizational change in the local level and how “members of

organizations can adapt to demands of change in expected ways” (the problem of micro-

implementation). (Berman, 1993, p.304).

The three imperatives of Rein and Rabinovitz seem to be represented in Berman’s macro-

implementation (the legal imperative) and micro-implementation (the bureaucratic and the

consensual imperative interacting with each other).

Berman is fully aware not only of the inevitable “politics” between national and local

governments in implementation, but also between local governments and a series of actors

in specific settings that can avert implementation if the program does not fit their needs. His

approach is mainly top-down , but this other players cannot be “contained” like Mazmanian

and Sabatier wrote after him, with the improved design of an statute. The program will have

to adapt to them. (Berman, 1993, pp. 314-315). Of course, his is a multi-actor, yet not

multi-level approach.

Van Metter and Van Horn (1975) had also worked on a model, one in which the key feature

is the distinction between the “impact” of the policy and its performance, acknowledging

that a policy can be well executed and still have no substantial impact, be it because it was

not well designed or because unexpected circumstances emerged (Van Meter and Van

Horn, 1993, p. 101). They classify policies in terms of their likely implementation

difficulties, considering two dimensions: a. Magnitude of attempted change and b.

Consensus on goals. And one more thing: scores in both dimensions are interchangeable.

So although normally “implementation will be most successful when only incremental

change is needed and goal consensus is high” it is also true that “high consensus may make

high change possible, as in a war time situation”. (Van Metter and Van Horn, 1993, p. 115-

116).

Van Meter and Van Horn developed a six-variable model in which the outcome is

“performance”, and warn that relationships between variables should not be identified in

only one moment in time –like in a snapshot- but longitudinally instead.

The variables (or more precisely clusters of variables) are:

Policy standards and objectives, which …elaborate on the overall goals of

the policy decision…to provide concrete and more specific standards for

assessing performance

The resources and incentives made available

7

The quality of inter-organizational relationships (including a discussion of

aspects of federalism, the commentary is ours)

The characteristics of implementing agencies

The economic, social and political environment and

The “disposition” or “response” of the implementers, involving three

elements: their cognition (comprehension, understanding) of the policy, the

direction of their response to it (acceptance, neutrality, rejection) and the

intensity of that response

Adapted from Hill and Hupe, 2007

Van Meter and Van Horn have their unit of analysis in the policy/program itself, and do not

consider more than one level of analysis –that of the program precisely- nor differentiate

between uni-actor and multi-actor implementation, although they do consider multi-actor

situations and have “quality of inter-organizational relationships” as one of their six key

variables. Having consensus as one of their two key dimensions and including the

implementers’ “cognition” of the policy links their analysis to our focus on problem

structuredness, yet the consensus they seem to worry about is more around the policy itself

than about the problem.

One could say that Berman and Van Meter-Van Horn agree on the key importance of

required change for implementation, yet a difference is that for Berman it is an unavoidable

barrier and the policy will have to adapt to its implementers, whereas for Van Mater and

Van Horn its influence can be moderated if there is huge agreement on goals. Either way,

for Van Meter and Van Horn, as well as for Berman (and top-downers in general) the focus

is not on “the problem” but on the program, and consensus on goals is relevant just because

it makes the program more or less implementable.

Sabatier and Mazmanian (1989, orig. 1980) also start from a top-down perspective and

identify three kinds of factors affecting implementation processes:

Factors affecting the “tractability” of the problem

“nonstatutory variables affecting implementation” and

The “ability of the statute to structure implementation”

By the time they wrote, several different attempts on models had identified variables

deemed relevant for implementation success. In their own words, Sabatier and Mazmanian

“sought to synthetize the large number of variables” identified “into a shorter list of six

sufficient and generally necessary conditions for the effective implementation of legal

objectives”:

8

1 Clear and consistent objectives –that they admit to have taken form Van Metter and

Van Horn- providing both a standard of evaluation and an important legal resource to

implementing officials

2 Adequate causal theory –admittedly taken from Pressman and Wildavsky- with an

emphasis on the adequacy of the jurisdiction and policy levers of implementing officials to

those causal assumptions

3 Implementation process legally structured to enhance compliance –borrowing again

from Pressman and Wildavsky, they look at the number of veto points, sanctions and

incentives available to overcome resistance, and the intensity of support from

implementing agencies

4 Committed and skillful implementing officials, taking into account, i.a, the work of

Lipsky, acknowledging that this is largely a product of post-statutory political forces

5 Support of interest groups and sovereigns –recognizing the need to maintain their

support throughout the long implementation process (quoting here Downs, Murphy,

Bardach and Sabatier himself)

6 Changes in socio-economic conditions that do not substantially undermine political

support or causal theory (mentioning Hofferbert and Aaron)

In their own words, they were knowledgeable in 1980 of the arguments about the

substantial limitations of programed/hierarchical control but “did not accept the pessimists’

conclusions concerning the inevitability of “adaptive” implementation in which policy

makers are forced largely to acquiesce to the preferences of street-level bureaucrats and

target groups” whose behavior “could be kept within acceptable bounds over time if the six

conditions were met” (Sabatier and Mazmanian, 1989, p. 291).

As a matter of fact, they are clear in the 1989 edition of Implementation and Public Policy

(Sabatier and Mazmanian, 1989) that their framework for implementation analysis

incorporates “those basic, yet usually uncontrollable, factors deemed critical by general

social systems theorists in determining the capacity for, and constraints on, self-conscious

social change” (including available resources, economic capacities, technological know-

how and prescribed (constitutional) political rules. They understand those factors as not

able to predetermine policy “in a literal sense” but establishing “the boundaries of possible

action” and complement them with examinations of policy subsystems and resource

interdependencies within and the importance of interests groups (Sabatier and Mazmanian,

1989, pp. 19-20). This could be an example of a macro-level approach. On the other hand,

they acknowledge that general systems theory is not enough to understand implementation

and incorporate organization theory and the limitations of hierarchical integration within

and among agencies as additional explanations for the difficulties to obtain compliance

9

from street-level bureaucrats (a mezzo-level analysis) and finally include a micro-level of

analysis around “the rational pursuit by individuals of their desires for power, security and

well-being” citing the works of Mancur Olson about the problems of collective action

(Sabatier and Mazmanian, 1989, p. 20).

So we can affirm that Sabatier and Mazmanian’s framework is a first example of a multi-

level analysis of implementation, in which the units of analysis are policy subsystems and

society characteristics (for the macro-level), organizations (for the mezzo-level) and

individuals (not programs) for the micro level.

They still view implementation as a process that “normally runs through a number of

stages” and associate part of the systems restrictions with “non-statutory variables” ,while

“the problem” is associated with a series of variables: technical difficulties, diversity of

target group behavior, target group as a percentage of the population, and extent of

behavioral change required –where they meet again with Van Meter and Van Horn.

Problem “structuredness”, or the concepts of consensus on values or on relevant knowledge

are not discussed.

Besides the three groups of independent variables, Sabatier and Mazmanian identify five

dependent variables. It is worth noticing that these variables are conceived as stages of the

implementation process: 1. policy outputs of implementing agencies, 2. compliance of

target groups, 3. actual impact 4. Perceived impacts (by sorvereigns and constituency

groups) and 5. Changes in statute. (Sabatier and Mazmanian, 1989, p. 40). So the idea of a

process still dominated at this stage their understanding of implementation and, at the same

time, elements not related to a process are mixed with the process sequence and the general

concern is not only with the process, but also with the final results of the policy:

“If one is concerned only with the extent to which actual impacts conform to

program objectives, then only the first three stages are pertinent. In our view,

however, one should also consider the political system’s summary evaluation of a

statute, which involves the latter two stages in addition” (Sabatier and Mazmanian,

1989, p. 35)

The confusion between process and results as foci for implementation research that Goggin

(1986) will discuss later on was already present here.

Sabatier and Mazmanian compare themselves to the conception of implementation in Rein

and Rabinovitz(1993, orig. 1978) andtheir three basic “forces” or competing imperatives

shaping the implementation process: the legal (with an emphasis on statutory intent), the

rational-bureaucratic (with emphasis on organization) and the consensual (with emphasis

on a modus operandi with major interest groups). They include the three forces in their

framework, but are clear about their reasons to focus more on the statute: 1. Because they

believed “the so-called behavioral era has tended to ignore the importance of legal

10

variables” and 2. Because of their conviction that “insofar as possible, policy decisions in a

democracy ought to be made by elected officials rather than civil servants”. (Sabatier and

Mazmanian, 1989, p. 43).

Summarizing, Sabatier and Mazmanian based their 1980 framework upon case studies as a

methodology –with the 1970’s Clean Air Act as their main subject- and they a) used a

multi-level analysis b) had policy subsystems, organizations and individuals as their units

of analysis c) gave importanceto “the problem” but did not discuss its structure and d)

addressed multi-actor implementation problems.

“The bottom-up challenge”

This is the term that Hill and Hupe (2007) use to refer to the bottom-up reaction to the idea

that policy was the most important influence upon those targeted by it, and that

interestingly enough, as they also point out, was not even a “reaction” really, since the first

article by Michael Lipsky (1971) on the procedures of street-level bureaucrats actually

precedes the book by Pressman andWildavsky.

We will briefly discuss here the contributions of Lipsky and of Benny Hjern, with an even

more brief mention of Barrett and Fudge (1981), also relieved by Hill and Hupe. Hjern is

also the main bottom-up reference in Sabatier and Mazmanian’s (1989) review of the

bottom-up literature.

“Most citizens encounter government, if they encounter it all, not through

letters to congressmen or by attendance to school board meeting but though

their teachers and their children’s teachers, or through the policeman on the

corner or in the patrol car. Each encounter of this kind represents an instance

of policy delivery” (Lipsky, 1980, p. 3)

There is no valid opposition to this key Lipsky’s insight. Nor to the truth of the “dilemmas”

he identifies for this street-level-bureaucrats: not having enough resources to

enforce/implement a policy, nor adequately clear goals, nor viable performance measures, but

still have to enforce/implement (Yates, 1982, p. 145). It is nevertheless possible to discuss

whether all last-level implementers of public policy feel the samepressureaspolicemen and

teacher towards discretionary action, or whether all public policies have street-level

bureaucrats implementing them at all (using Lowi’s terms, one could doubt that they are so

important in the case of constitutive or redistributive policies, where there is less everyday

contact with the “target population” and to some extent question as well if all regulatory

policies have an important role for street-level bureaucrats at all. Where there are these

pressures, however, it is normal that “control from the top to combat the alleged failures of

street-level staff” only worsens them, and that, since the implementation of public policy is

really about what these lower staff do, the preoccupation about “how great expectations in

Washington are dashed in Oakland is really beside the point… This means that different

11

approaches are needed to secure the accountability of implementers” (Hill and Hupe, 2007,

p.53).

Regarding our basic questions, Lipsky does not perform a multi-level analysis, he has one

clear unit of analysis in the individual street-level bureaucrat. And he does deal with multi-

actor implementation –though not multi-organization implementation. The problem of

problem “structuredness” is not obvious in his analysis, in spite of the fact that the

consequences of discretion for street-level bureaucrats are obviously going to be different if

they share the decision maker’s problem definition and if they don’t.

Benny Hjern: Implementation structures

Sabatier and Mazmanian (1989) profile Benny Hjern as possessing “an acute awareness of

the methodological weaknesses of the top-down approach, a commitment to the development

of an intersubjectively reliable methodology, and a concern with policy areas –e.g manpower

training- involving a multitude of public and private organizations”. (Sabatier and

Mazmanian, 1989, p. 297).

Hill and Hupe (2007) noticeHjern’s focus precisely on those policies which “depended upon

interactions between several different organizations” and how he perceived a role for

networks in them, therefore –together with his colleagues- developing a methodology that

“constructed empirically the networks within which each field-level decision making actors

carried out their activities without predetermining assumptions about the structures within

which they occurred”. (Hill and Hupe, 2007, p.54).

Of the three “forces” of Rein and Rabinovitz –legal, organizational, consensual- Hjern and

Porter, for instance, (Hjern and Porter, 1981) clearly pick the second and, within it, seek to

introduce a new unit of (administrative) analysis: implementation structures. This quote from

their introduction is very telling:

“We suggest that perceptions of deficits in program implementation are distorted an

exaggerated by analytic frameworks which use organizations and individuals as the

basic unit of analysis. A universal finding in studies of public implementation is that

clusters of public and private actors are involved. To cope with this condition, a

theory is needed which bridges the gap between the atomistic theories in economics

and comprehensive planning and management theories in public administration. We

suggest an administrative theory with “implementation structures” as the core unit of

analysis to bring this gap” (Hjern and Porter, 1981, p. 211).

Observing policy settings with large numbers of actors participating Hjern and Porter

were positively surprised that outcomes were produced at all. Yet there were multiple

problems anyway. Noting that “public administration theories, grounded on the

Lonely Organization Syndrome, were not sufficient if used alone” because “no single,

12

comprehensive organization can command all the resources” but also that

“conventional market analyses were also insufficient” because of significant market

failures they concluded that the need was for “a strategy analyzing purposive action

within a framework where parts of many public and private organizations cooperate

in the implementation of a program” (Hjern and Porter, 1981, p. 214).

“Implementation structures are not organizations” they wrote.

“They are comprised of parts of many organizations; organizations are

comprised of parts of many programs. As analytic constructs, implementation

structures are conceptualized to identify the units of purposive action which

implement programmes. They are “phenomenological administrative units”

and “phenomena which has been described as anomalous or deviant when

analyzed within the context of organizations become explicable within

implementation structures” (Hjern and Porter, 1981, pp. 222-223).

Implementation structures seem very much like advocacy coalitions, just focused on

implementation when advocacy coalitions are mainly usefulto explain policy formation –and

implementation loses importance within their framework.

As Hill and Hupe notice, Hjern is critical of the top-down approach basically for not

contributing to improve policy effectiveness in itself. “The aim of the exercise for

Mazmanian and Sabatier is to help federal and state politicians better to control public

administration. This is not perforce to ensure policy implementation”. (Hjern, 1982, p. 302).

Hjern (1981; 1982) openly discusses Weber and Wilson and the relationship between policy

and administration when exposing the foundations of his theory. “Implementation analysis

started as an attempt to transcend the sharp separation between politics and administration in

public administration research…Specially commissioned type of research has exposed

implementation analysis to the danger of becoming more and more entrapped in the public

administration notion of stable and sequential relations between politics and administration”

(Hjern, 1982, p. 307). And concludes discussing Sabatier and Mazmanian: “A legalistic

approach is necessary but an inter-organizational structuring is indispensable in

implementation analysis” (Hjern, 1982, p. 308).

It seems useful to highlight the priority given by Hjern to policy effectiveness vis-à-vis

“top’s” control over policy in his argument about the need for a bottom-up approach. He does

not mention “the problem” or its structure, yet he seems to think “solving the problem” –an

effective policy- is better served by a non- hierarchical approach to implementation. Needless

to say, Hjern considers several levels of analysis (individual, organization, policy) but focuses

on one –organization- and on one unit of analysis (his organization structures). Like said

before, he did not address “problem structure” but his discussion has implications for that of

the problem structure.

13

Hill and Hupe also review the work of Barrett and Fudge in 1981 (Barrett and Fudge, 1981)

and their critic of the “tendency in the top-down literature to depoliticize the policy-action

relationship”. They question that “if implementation is defined as putting policy into effect

then compromise by the policy makers should be seen as policy failure” and “if

implementation is seen as “getting something done”, then performance rather than

conformance is the main objective and compromise a means of achieving it. (Barrett and

Fudge, 1981, p. 258). Their discussion will come back to us in a few pages.

Richard Elmore (Elmore, 1979; 1985) is catalogued by Hill and Hope among the

“synthesizers” between top-down and bottom-up approaches. He, however, did not develop a

framework.

He noticed that “in many policy areas in the United States implementation actors are forced

to make choices between programs that conflict or interact with each other” (Hill and Hupe,

2007, p. 58) but also that what he calls forward mapping is the typical way of designing

public policy and understanding its implementation (Elmore, 1979, p. 603). A key

assumption of that underlying logic is that policy makers control the processes that affect

implementation.

Backward mapping, in turn, “begins not at the top of the implementation process but at the

last possible stage, the point at which administrative actions intersect private choices”. It

begins not with a statement of intent, but with a statement of the specific behavior at the

lowest level of the implementation process that generates the need for a policy. Only after

that behavior is described does the analysis presume to establish an objective. With a target

established at the lowest level of the system, the analysis “backs up” asking at each of those

levels two questions: a. What is the ability of this unit to affect the behavior that is the

target of the policy and second, b. what resources does the unit need in order to have that

effect? (Elmore, 1979, p.604).

Backward mapping does not assume that policy is the only –or even the major- influence

on the behavior of people targeted by it, writes Elmore, making us aware that forward

mapping does make that assumption. In Elmore’s approach, the definition of success is

conditional, depending on the ability of organization at one level of the implementation

process to influence the behavior of other organizations and, in turn, of private individuals.

(Elmore, 1979, p.604). So you do not choose your target but, in a way, the probability of

reaching them defines the universe of possible targets for you.

Forward mapping assumes that organizations in the implementation process are linked in

essentially hierarchical relationships, with two corolaries: the closer to the top exert a

higher control upon the process, and the ability of complex systems to respond to problems

depends upon the establishment of clear lines of control and responsibility, whereas

backward mapping assumes, first, that the closer one is to the source of the problems the

14

greater is his ability to influence them, and, therefore, that the ability of complex systems to

respond to problems rests on their capacity to allow discretion at the point which is closer

to the source of the problem (it is worth noting here that these insights are in line with those

by Pressman and Wildavsky, heavily associated with the top-down approach, not only on

the need for sound theories, but on the need of shorter chains of causality).

Very little can be done about the problems of the complexity of joint action, if analysts and

policy makers persevere in viewing implementation as a hierarchically ordered set of

authority relationships. (Elmore, 1979, pp.604-605).

Taking the example of Young employment programs, Elmore argued that discussions at

several levels were –at that time- more about how to enhance coordination between existing

federal programs, and less about how to really reduce Young unemployment.

In 1985 (Elmore, 1985) the author revisits his former contribution and provides a more

detailed comparison between forward and backward mapping, associating the later not only

with the identification of a concrete target behavior at the bottom end of the process, but

also with a careful consideration of the alternatives of the target group, as it might

appreciate them. (e,g Elmore, 1985, p. 36). He recalls that “the value of reversible logic is

not that it helps us anticipate implementation problems, but more importantly that it affects

the way we frame and evaluate alternatives (Elmore, 1985, p. 51) . In the conclusions of his

1985 article he recalls his example of looking at the possibilities for reducing energy

consumption in buildings from both forward and backward perspectives. “On the forward

leg, we started with a standard set of implements; we then asked what external conditions

might affect those implements, how implementing agencies would be expected to respond

to the implements, to whom the implements were addressed, with what expected effect. On

the backward leg, we started with a set of decisions that policy would have to affect in

order to influence energy consumption” asking later what outcomes would have to follow,

what external conditions would affect those decisions, how implementing agencies would

have to adapt to the goal they were trying to reach, and what implements could be used to

influence the decisions not only of the target group, but of implementers as well. (Elmore,

1985, p.51).

From process to causation as the focus of the analysis: 1986-

Perhaps the main charge of bottom-uppers against top downers was the lack of explanatory

power for causation in their analysis that, then, in Hjern’s words, were more useful for

advising the top on “how to improve control” than for improving effectiveness, if

effectiveness is performance and not compliance.

The writings of Laurence O’Toole and Malcolm L. Goggin between 1984 and 1990 showed

discontent with the methodologies employed (small numbers of case-studies) and the

15

theoretical developments of implementation research, and began to demand a shift on the

focus away from the idea of sequence and a process, even if not denying it.

Laurence O’Toole (1986) reviewed not only the theoretical developments specifically

regarding multi-actor implementation –in the tradition of Pressman and Wildavsky, yet

perhaps influenced by Hjern and other networkers- but also the practical recommendations

made by the authors and the empirical evidence of their use. Although he provides

remarkable insights into the (scarce) impact of policy authors’ recommendations upon

policy practitioners, we will focus here on his inventory of causes associated with

implementation results.

Yet it is important to clarify first what O’Toole means by Implementation. In his words

(O’Toole, 2000, p.266):

“Some scholars include here both the assembly of policy actors and action, on the one hand,

and the cause-effect relationship between their efforts and ultimate outcomes, on the other

(forinstance,MazmanianandSabatier1989). Others, including myself, have emphasized the

importance of making a conceptual distinction between implementation (action on behalf of the

policy) and ultimate impact on the policy problem (for initial statements, see Montjoy and

O'Toole 1979; O'Toole and Montjoy 1984). Implementation research concerns the development

of systematic knowledge regarding what emerges, or is induced, as actors deal with a policy

problem”.

So O’Toole is closer to Van Meter and Van Horn here.

TABLE I. VARIABLES VIEWED AS IMPORTANT IN THE IMPLEMENTATION

LITERATURE (O’Toole, 1986):

Author(s) and Variables

Ackermann and

Steinmann (1982)

resources, interorganizational structure

Alexander (1985) stimulus, policy, program, implementation; contextual, organizational,

environmental, perceptual variables

Altenstetter and Bjorkman

(1976, 1977)

goal ambiguity

Ball (1976) decisional clarity, consensus, and reiteration; personnel altitudes,

enforcement resources, reputational authority, homogeneity of issue

public, cost of abatement, role of local media

Baum (1976, 1981)

clarity of directives, accuracy with which decisions are communicated,

interests of subordinates, subordinate policy preferences, authority,

insulation, sanctioning, persuasion, branch of government, structure of

interorganizational interdependence, Sabatier and Mazmanian checklist

Berman & McLaughlin (1976);

Berman (1978);

(local) institutional context has major impact, especially organizational

climate and motivations of participants (materials development, staff

16

Author(s) and Variables

McLaughlin (1976) training, planning, frequent meetings); federal policy, technology, and

resources: little influence

Berman (1980) Important variables depend on context (organizational, political, social

and legal). Clarity of policy goals, number of actors participating,

implementers' degree of resistance, ineffectualness, or inefficiency;

degree of control exerted from top.

Bowen (1982) clearances, number of actors, persistence, time

Bowman (1984) tension, contextual factors (e.g., economic context)

Browne and

Wildavsky (1984)

formal policy (clarity of objectives and priorities, validity of theory of

causality, sufficiency of financial resources, sufficiency of power);

learning/adaptation

Browning, Marshall,

and Tabb

(1981, 1984);

Browning and Marshall

(1976)

local context primary (especially orientations of local targets, ideology

of local dominant coalitions), statute secondary; time

Bryner (1981) clarity of statute, level of general political support

Bullock (1980)

clarity of statutory standards or objectives, degree of agency support,

'presence or absence of programmatic bias favoring the attainment of

the agency's responsibilities,' 'altitudes of the agency's sovereigns'

Bunker (1972) 'The degree to which an organization (a) is consensually clear about its

task, (b) is appropriately differentiated into parts related to its pertinent

sub-environments, (c) is integrated by information exchanges and

effective conflict management devices, (d) has clear knowledge about

its performance, and (e) is self-reflective and able to make corrective

adjustments in its own behavior'; issue salience, power resources,

agreement

Chase (1979) operational demands, resources, authority, support: 44 'factors for

consideration"; structure of interdependence

Cleaves (1980) complexity of the change mechanism, degree of change, number of

actors involved as targets, number of goals, clarity of goals, duration

Davies and Mason (1982) the economy, resources, legitimacy

Durant (1984) extent of behavioral change required, complexity of joint action,

sociopolitical environment, 'noncompliance delay effect,' implementers'

dispositions, validity of policy's causal theory, 'adequacy and credibility

of enforcement resources'

Edwards (1980)

communication (transmission, clarity, consistency), resources (staff,

information, authority, facilities), disposition or altitudes of

implementers, bureaucratic structure (standard procedures,

fragmentation), complexity

Elmore (1976, 1977, 1978, structure of power relationships and incentives, discretion, resources

17

Author(s) and Variables

1979-80, 1985)

Goodwin and Moen (1981)

sound technical theory, constituency support, socioeconomic

environment, learning

Grindle (1980, 1981)

content of policy (interests affected, types of benefits, extent of change

envisioned, site of decision making, program implementors, resources)

and context of implementation (power, interests, strategies of actors

involved, institution and regime characteristics, compliance and

responsiveness)

Gross et al. (1971)

implementers' clarity about innovation, needed skills and knowledge,

availability of materials, compatibility of organizational arrangements

with innovation, degree of staff motivation

Gunn (1978) nature of policy, implementation structure, outside interference,

Hogwood and Gunn (1984) control over implementers, resources, valid theory

Hambleton (1983) policy message; multiplicity of agents, perspectives, and ideologies;

resources; politics of planning

Hargrove (1983) type of policy (distributive, regulatory, redistributive)

Hays (1982) local political conflicts, degree of mediation of local agency

Hucke (1978) conditions in agency's implementation field

Ingram and Mann (1980) degree of policy demand, accurate causal theory, incentives

Jones (1980) differences in local polities

Kelman (1984) complexity of program, organizational capabilities

Kirst and Jung (1982) Time

Larson (1980) policy goals, implementation procedures, complexity, changes in

economic environment

Lazin (1980) administrative linkages

Levitt (1980) timescale for introducing legal powers, public expenditure policy,

opportunity cost, policy as a 'sacrificial victim,' outside pressure, policy

instruments, certainty of benefits from action, policy formulation as an

end in itself, stability of interorganizational relationships, elements of

surprise and power after governmental change, images, consultation,

learning, public opinion, trends in interest groups' perceptions, media,

international policy-making

Luft (1976) Incentives

McLanahan (1980) goal specificity, degree of centralization, interest-group participation,

public accountability

Majone and Wildavsky (1978) objectives, resources, theory underlying policy, constraints emerging in

the implementation process

Mandell (1984) multilateral brokerage role

18

Author(s) and Variables

Marvel (1982) number of levels of government involved

Mazmanian& Sabatier

(1981, 1983);

Sabatier &Mazmanian

(1979, 1981, 1983);

Sabatier and

Klosterman (1981)

tractability of the problem (four variables); ability of statute to structure

implementation (seven variables); nonstatutory variables (five); initial

implementation success

Mead (1977) background economic and social forces, funding level, incentive

structure, weakness of provider institutions, administrative weaknesses,

political constraints

Mechling (1978) technical uncertainty, internal conflict

Menzel (1981, 1983) statutory deadlines, 'configuration of intergovernmental and institutional

relationships,' supportive relationships among clientele, organizational

structure and management, rules, enforcement styles, timing

Mitnick and Backoff (1984) elaborate incentive system model, including characteristics of sender,

sender-receiver relation, receiver, organizational setting, and distal

environment

Montjoy and O'Toole (1979);

O'Toole and Montjoy (1984);

O'Toole (1983)

policy specificity, resources, agency goals, routines, world view,

structure of interdependence, technical requirements of the task,

facilitator, perceived risk for implementers

Moore (1978a and 1978b) incentives, number of clearances, specificity of criteria, resources

Mueller (1984) degree of change required, level of community support, fixers

Murphy (1971, 1973,

1974, 1976)

degree of support among implementers, adequacy of staff, strength of

monitoring, law and tradition, constituency pressure

Nakamura and

Smallwood (1980)

specificity of policy, technical limitations, actors, arenas, organizational

structures, bureaucratic norms, resources, motivations, communication

networks, compliance mechanisms

Nixon (1980) clarity and consistency of communication

O'Brien (1980) overlapping and conflicting provisions of law

Pesso (1978) rules, supervision

Porter (1976) task technology, type of interdependence, environment

Pressman and

Wildavsky (1984)

multiplicity of participants, perspectives, decision points, intensity of

preferences, resources

Raelin (1980, 1982) power, authority and mandate distributions; quantity of organizations;

network leadership; level of competition and conflict

Rawson (1981) clarity of organizational goals, support of organizational leaders, degree

of discretion over means and ends, new vs. established organizations

19

Author(s) and Variables

Rein and Rabinovitz (1978) goal saliency, complexity, nature and level of resources, number of

levels, number of agencies, number of participants

Ripley and Franklin (1982) type of policy

Rodgers and

Bullock (1976)

perception of legal standard, degree of agreement with legal standard,

perceived costs and benefits for the implementers, environmental

factors, degree to which law clearly defines who is responsible, whether

the law specifies the type and amount of compliance required, perceived

sanctions, whether beneficiaries are cohesive and able to take strong

actions

Rosenbaum (1980) specificity (four variables) and enforceability of statute (four variables)

Ross (1984) implementation strategy, tractability of policy problem, content of

policy, structure of broader sociopolitical and policy systems, number of

actors, extent of power diffusion, personal and institutional dispositions

of actors, clarity, adequacy of resources, support of leaders, institutional

routines

Sapolsky (1972) environment, resources, skill at bureaucratic politics, ability to manage

complexity

Scharpf (1977, 1978);

Scharpf et al. (1978)

level of conflict, available capacity for conflict resolution, structural or

organizational variables

Scheirer (1981) decision and control processes, resources, relations with environment,

supervisory expectations, routines, technical requirements,

communication flow, work group norms, behavioral skills, incentives,

cognitive supports

Skelcher, Hinings, Leach,

and Ransom (1983)

structure of interorganizational linkage

Smith (1973) various tensions among idealized policy, implementing organization,

target group, environmental factors

Sorg (1983) individual implementer behaviors

Thomas (1979) local propensity to accept a program, blend of policy incentives with

conditions, how the issue develops

F. Thompson (1982) discretion in policy; agency consensus, ethos and leadership; type of

pressure from environment; capacity of oversight actors

J. Thompson (1982) hierarchy, socioeconomic variables

Van Meter and

Van Horn (1975),

Van Horn (1978,

1979a, 1979b),

Van Horn and

Van Meter (1976)

policy standards, resources, enforcement, Communications,

characteristics of implementing agencies, political conditions, economic

and social conditions, dispositions of implementers

Weatherley

and Lipsky (1977)

resources, coping behaviors of street-level bureaucrats

20

Author(s) and Variables

Weiler et al. (1982) availability of baseline data, extent to which individual programs are

affected by required standards of performance, agency attitudes,

resources

Weimer (1983) time, civil service system, bureaucratic environment

Williams (1980, 1982) bargaining and fixing, institutional arrangements, staff competence,

marketlike pressures, information process, resources

Yaflee (1982) statute (prohibitive character)

Taken from O’Toole (1986)

While identifying the common grounds, O’Toole himself warns of the lack of unity:

“Anexaminationof thematerialsummarizedin the tableindicatesthatthereseemtobepossibilities

forbuildingsomeconvergenceinthefield,butthusfarlittleaccumulationhasactuallytakenplace…

thetypicalsituationinimplementationresearch,evenmoresothanfor

othertypesofsocialscience,hasbeenfortheretobeverylittleconsciouseffortto

developandtestsystematicallytheinsightsgeneratedinpreviouswork,andthustoseparatethepromisingfro

m themerelyplausiblebutunproductive” (O’Toole, 1986, p. 189)

He finds two major explanations for the divergence on variables identified as “key” for implementation: First, normative disagreement and second, an insufficient development of the field’s empirical theory.

For normative disagreement O’Toole means a difference between the values deemed important by some regarding implementation –e.g coordination, speed, consistency- and the values important to other authors –e.g diversity, access, widespread participation (O’Toole, 1986, p. 197). As for the state of empirical theory, although he finds a discrete progress “understandable” since researchers must first map “the complexity and diversity of organizational pattern through which implementation occurs” he highlights how promising ideas “are left ignored by others” and makes a particular distinction of the work by Bowen (1982) which basically contradictions one of the main conclusions of Pressman and Wildavsky about the probabilistic reduction of the chances for successful implementation as the number of actors involved in a program goes up. (O’Toole, 1986, p. 198-199).

His claims about promising ideas “left ignored” apply to a work of his own that might be at least as relevant as his 1986 review: his 1984 article –together with Montjoy (O’Toole and Montjoy, 1984) - in which he questions Pressman and Wildavsky’s conclusions on the stable, negative impact of the number of actors upon implementation results –defining implementation as “action” in the concretion of the policy/program- and proposes a typology of interdependencies for each one of which the implications of the number of actors vary.

According to O’Toole and Montjoy there are three types of interdependencies, and Pressman and Wildavsky’s predictions hold constantly for only one of them: a. Pooled interdependency, when multiple agencies operate upon the same object during implementation but none of them must receive inputs from one another in order to do its part of the job –although coordination could improve the result. b. Sequential

21

interdependency, where “a delay or break down at one point will affect the operations of the subsequent units in the chain of implementation” and c. Sequential operating interdependence, where agencies “must adjust mutually to cooperate with each other”. (O’Toole and Montjoy, 1984, pp. 493-494). Based upon the results of a series of cases of the U.S’ General Accounting Office (GAO) they conclude that: a. Where the relationship is pooled, the probability of a relatively quick start of some kind of implementation is considered high, while the chances of coordinated action effectively diminish as the number of involved units goes up. b. When implementation is designed to be sequential coordination becomes imperative and a longer delay to start has to be expected, and those –and the costs of coordination- will climb as the number of units involved increases. c. Finally, in reciprocal interdependence coordination costs will always be high and a direct function of the number of units participating. So delays are not necessarily “a function of the number of decision points, the number of participants at each point, and the intensity of their preferences” like Pressman and Wildavsky argued a decade before. (O’Toole and Montjoy, 1984, p. 499). Indirectly, O’Toole and Montjoy also identify, among the many variables with each implementation is associated, one of the most obvious and omnipresent ones: Time.

O’Toole (1986) also points to the U.S scholars’ “ignorance” of the work of their European peers, and is perhaps the first author to present implementation research as a field of “applied sciences” largely dependent upon developments on the mores basic fields contributing to it –his partial list includes, i.a, policy analysis, organization theory, inter-organization theory, exchange theory, intergovernmental relations, public choice and the theory of the agency- and also affected by the changes and lack of stable basic concepts in those very fields.

This is not the same as having nothing, anyway: “Effortsshouldbeundertakentobuildsystematicallyandcumulatively ontheresearchthathas focusedonpolicycharacteristics,resources,implementationstructure,implementerdisposition,implementer-clientrelationship,andtiming”.

Roughlyhalfof thepublishedstudiesinthefieldidentifypolicycharacteristics(especiallyclarity,specificity,and/orflexibilityof goalsandprocedures,andvalidityof apolicy'scausaltheory)assignificant;approximatelythesamenumberclaimthatresources(financialandother)arecrucial.Otherfrequently­identifiedcategoriesofvariablesinclude:implementing-actorormulti­actor structure,number ofactors, attitudes and perceptions ofimplementingpersonnel,alignmentofclientele,andtiming(includingthepossibilitiesforlearningamongimplementers).

O´Toole’s mood was more optimistic in 2000 (O’Toole, 2000) for several reasons. First, he

notices a series of efforts at synthesis and a quieting of the top-down vs. bottom-up

confrontations. “Sufficient evidence has been accumulated to validate partially both top-

down and bottom-up articles”. This “peace”, however, had not helped overcome the

problem of “too many variables” that Goggin pointed at back in 1986 (Goggin, 1986). “A

multitude of candidate variables continue to float through the research literature” (O’Toole,

2000, p.268)

22

One of the biggest reasons for optimism were the methodological developments

overcoming the problem of “proverbial dependence” by what he had called an overreliance

on case studies, through the development of a) “smaller-

nstudiesofanexceedinglyhighstandard” and b) several large-n studies of implementation, where

making large-n studies compatible with a longitudinal approach was a new challenge. The call

for a “third generation” of “more scientific” implementation research studies “explaining why

that behavior changes across time, places and units of government” was also considered a

promise (O’Toole, 2000, p. 271-272).

Other reasons for optimism were the contributions of several fields in the social sciences to a

better understanding of implementation issues, namely institutional analysis –and in particular

ElinorOstrom’s Institutional Analysis and Development (IAD) approach-, actor-centered

institutionalism, networks and network management and…the study of governance (O’Toole,

2000, pp. 273-282).

Before moving forward to review the first frameworks for implementation research –that of

Sabatier and Mazmanian and the one by Goggin et al. which O’Toole himself suscribes ,

(Goggin et al.,1990) it is fair to step back a little bit and look to Goggin’s early contributions to

the above mentioned problem of “too many variables”.

In 1986 (Goggin, 1986) Malcolm L. Goggin drew attention to the fact that “the complex of

dynamic nature” of implementation problems had lead many researchers to leave aside statistical

and experimental methods in favor of the case study approach. He identified three main sources

of implementation variation addressed in the existing literature: a) the content of the

policy/program being implemented b) the capacity of the organization in charge and c) the

skills/will of those in charge of implementing. The understanding of implementation as a

process whose stages had to be looked at, for one part, and the confusion of the term itself

between those who only include the process and those who add the result, for another, had

resulted in a large number of variables, both independent and dependent, being investigated by

different researchers simultaneously (Goggin, 1986, pp. 329-331).

Goggin recounts how the three strategies suggested by Lijphart (1971) to deal with this

disproportion between cases and variables had been tried: reducing the number of variables,

increasing the number of cases, and introducing an element of control by selecting cases on the

basis of comparability and similarity. He concludes, however, that as long as the number of

cases remains relatively small, this type of comparative analysis “will poses neither the

randomization and multivariate analysis used in statistical correlation studies nor random

selection and assignment and the use of control groups of experiments” (Goggin, 1986, p. 334)

and proceeds to propose two new strategies: combining large and small-n studies, and “building

and testing theory” with small –N studies that can precede the large-N ones, developing theory

23

to be tested in the latter, or can follow them, with N=many studies being used to identify “the

most promising case studies for inclusion in a more limited, but more intensive comparative

study” (Goggin, 1986, p. 339).

For the purpose of this review, of the same importance as Goggin’s methodological

developments are the implementation variables he singled out, especially as a result of his own

review of the efforts to “reduce the number of variables and focus on the critical ones”: First, a

cluster of independent variables associated with the policy itself: Promises (in this case, policies)

are more likely to be kept if they: 1. Fit within the existing beliefs and practices 2. Are perceived

as helping rather than hurting the interests of those involved in implementation 3. Distribute

benefits widely enough to result in a minimal coalition of interests “committed to making good

on policy as intended” 4. Are based upon sound theory 5. There is internal coherence between

their objectives 6. Those objectives are clear and specific 7. There are provisions for rewarding

compliance and punishing non-compliance. Gogging presents this seven as components of just

one variable –the policy- but it seems more appropriate for the sake of precision to divide that

huge variable in these seven components that are easier to study. Then, another key independent

variable to Goggin is 8. The “will and skill” of those in charge of implementing –one might

consider the need to separate will and skill to be obvious. (Goggin, 1986, pp. 331-332)

Apart from this eight independent variables, Goggin identifies two dependent variables: 1

Duration of the implementation process 2. Change in the policy with respect to the designers’

original intent –we could use here the term variation. Goggin identifies two implementation

“styles” from the behavior of those two variables: a more “political” implementation style

identifiable when there are delays and significant variation with respect to the original plan and a

more “administrative” implementation when implementation is prompt and with little changes.

Political implementation is identified with failures, and the administrative style with success.

(Goggin, 1986, p.332).

Goggin (1986) insists in a proposition made by himself while reviewing Sabatier and

Mazmanian’sImplementation and Public Policy in 1984 (Goggin, 1984): to identify two

generations of implementation research since 1973: a) a first generation represented by

Pressman and Wildavsky for which implementation a) is a process and b) will almost inevitably

fail, compared to expectations, and for which the main questions were how the process works

and why a particular experience turned the way that it did and b) a second generation,

represented by Sabatier and Mazmanian’s Implementation and Public Policy (orig. 1980) , but

also by Van Meter and Van Horn (1975) and others, for whom implementation was really

political in nature and its outcomes also varied according to time and space. All of the above

with the aim of proposing a third generation of implementation research that evolves beyond the

who, hows and whys and poses new questions to illuminate “the full range of outcomes that lie

between the extremes of success and failure”, the various causal paths leading to each type of

outcome, and the independent effects of each independent variable, among other things.

(Goggin, 1984, p. 160).

24

In 1986 Paul Sabatier wrote a new review of literature, this one specifically on the top-

down/bottom-up debates and, listing four key bottom-up criticisms (we will discuss the

1989 version of the article, published as a chapter of a re-edition of Implementation and

Public Policy): 1 “they (top-downers) start from the perspective of (central) decision-

makers and thus tend to neglect other actors” 2 top-down models “are difficult to use in

situations where there is no dominant policy (statute) or agency, but rather a multitude of

governmental directives and actors, none of them preeminent”–like Hjern’s cases, add we-

3) they are likely to ignore, or at least underestimate, the strategies used by street-level

bureaucrats and target groups to get around (central policy) and/or to divert it for the own

purposes…A related point is that such models are likely to neglect many of the

counterproductive effects of the policies chosen for analysis” 4) “the distinction between

policy formation and policy implementation is misleading and/or useless” because some

organizations are involved in both, and because target groups frequently ignore central

administrators and deal directly with each other, and because it is difficult to isolate policy

decisions (Sabatier and Mazmanian, 1989, p. 296).

Sabatier acknowledges the first three criticisms as “reasonably persuasive”, but not the

fourth about the distinction formation/implementation. Reviewing the cases written using

the Sabatier/Mazmanian framework, he affirms that in only one of them (among 24) it can

be said that there is no “policy decision” but a series of very incremental steps over time.

He writes that:

“…obliterating the distinction between formation and implementation will have two significant

costs…First, it makes it very difficult to distinguish the relative influence of elected officials and

civil servants…Second, the view of the policy process as a seamless web of flows without decision

points (Majone and Wildvasky, 1978; Barrett and Fudge, 1981) precludes policy evaluation (because

there is no policy to evaluate) and the analysis of policy change (as there is never a defined policy at

t0 that changes into another definesd policy at t1)”. (Sabatier and Mazmanian, 1989, p. 297).

Sabatier also finds strengths in the bottom-up approach (represented by Hjern in the

following lines) since it 1 has developed “an explicit and replicable methodology for

identifying a policy network (“implementation structure”) 2 is “able to assess the relative

importance of a variety of governmental programs vis-à-vis private organizations and

market forces” in order to solver actors’ specific problems 3 since it doesn’t “start with a

focus on the attainment of formal policy objectives” is free to “see all sorts of (unitended)

consequences of governmental and private programs” 4 “is able to deal with a

policy/problem area involving a multitude of public (and private) programs, none of them

preeminent and 5 “is better able to deal with strategic interaction over time”. Yet of course,

there are shortcomings: 1 “the focus on actors’ goals and strategies…may underestimate the

Center’s indirect influence over those goals and strategies through its ability to affect the

institutional structure in which individuals operate 2 “Hjern et al. take present participants

in an implementation structure as given without examining the previous efforts of various

25

individuals to affect participation rates” 3 A failure “to start from an explicit theory of the

factors affecting its subject of interest. Because it relies very heavily on the perceptions and

activities of participants, it is their prisoner…” (Sabatier and Mazmanian, 1989, p.300).

The most important element in the review is the solution to the top-down/bottom-up

dispute: a new framework borrowing from both of the approaches. That is the way the

Advocacy Coalitions Framework (ACF) is introduced (although the original ACF article by

Sabatier and Hank Jenkins-Smith was published a year before, in 1988).

The main strategies are: 1 Start from a policy problem or subsystem –rather than from a

policy decision- and then explore the strategies of different actors in both the public and

private sectors, at several levels, to deal with that problem in a manner consisting with their

objectives. (this is bottom-up). How to identify those interest? “The most useful principle

of aggregation seems to be belief system. This produces a focus on advocacy coalitions, i.e

actors from various public and private organizations who share a set of beliefs and who

seek to realize their common goals over time” (Sabatier and Mazmanian, 1989, p. 304).

Just as others (e.g O’Toole 1984, 1986) has demanded more theoretical contributions from

implementation research and the bottom-uppers had blamed top-downers for focusing more

on “compliance” than on “performance” the ACF is described as “primarily concerned with

theory construction rather than with providing guidelines for practitioners or detailed

portraits of particular situations” (Sabatier and Mazmanian, 1989, p. 305).

“In short, the synthesis adopts the bottom-uppers’ unit of analysis- a whole variety of public and

private actors involved with a policy problem- as well as their concerns with understanding the

perceptions and strategies of all major categories of actors (not simply program proponents). It then

combines this starting point with top-downers’ concerns with the manner in which socio-economic

conditions and legal instruments constrain behavior. It applies this synthesized perspective to the

analysis of policy change over periods of a decade or more. This time-frame is required to deal with

the role of policy-oriented learning –a topic identified as critical in several top-down studies-

although by no means inherent to that approach. Finally, the synthesis adopts the intellectual style (or

methodological perspective) of many top-downers in its willingness to utilize fairly abstract

theoretical constructs and to operate from an admittedly simplified portrait of reality” (Sabatier and

Mazmanian, 1989, pp. 304-305).

Top-down approaches taking a decision/mandate as their starting point are limited to

explain policy change not emanating from an statute, a judicial or administrative decision,

and clearly inadequate for explaining policy change in the long run, something even more

important when Sabatier and Mazmanian had already noted in their 1980 framework that

original decisions/intents are often modified as a result of implementation.

In 1993 (Sabatier and Jenkins-Smith, 1993) Sabatier identified “at least four basic

premises” for the ACF: (1) A time perspective of a decade or more is needed to understand

the process of policy change (2) “policy subsystems” –the interaction of actors from

different institutions who follow and seek to influence governmental decisions in a policy

26

area- is the most useful way to think about policy change over such a long time span (3)

Those subsystems must include an intergovernmental dimension, that is, they must involve

all levels of government (at least for domestic policy) and (4) public policies (or programs)

can be conceptualized in the same manner as belief systems, that is, as sets of value

priorities and causal assumptions about how to realize them. (Sabatier and Jenkins-Smith,

1993, p. 16)

There is a key role in the ACF for policy-oriented learning, as “an important aspect of

policy change and can often alter secondary aspects of a coalition’s belief system” but

changes “in the core aspects of a policy are usually the results of perturbations in

noncognitive factors external to the subsystem, such as macroeconomic conditions or the

rise of a new systemic governing coalition” (Sabatier and Jenkins-Smith, 1993, p.20).

The key aspect about the ACF regarding implementation research, anyway, is not that it

denied the relevance of implementation (because it did not, as Weible, Sabatier and

McQueen, 2009; Weible et al., 2012 show ) but that it shifted interest from the question of

how policy changes are implemented to how policy changes happen.

When reviewing the use of the ACF in research, Weible, Sabatier and McQueen (2009)

found it was being used to explain implementation and it had been notices that “while ACF

portrays policies as translations of beliefs, the framework does not describe how those

belief-based policies are put into practice at the operational level or how implementation

affects subsequent learning and policy change (Smith, 2000). They also found (Weible,

Sabatier and McQueen, 2009, p.136) that several researchers were combining the ACF with

other policy process theories, including the stages heuristics, perhaps –we think- for the

simple reason that the ACF is good at explaining why policies change over time, rather than

how exactly they do and left unanswered the perennial question from public administrators,

policy advocates and practitioners: how do we make it happen?. So, by 2009, Weible and

Sabatier acknowledged that “clearly, issues of agenda setting, formulation, adoption,

implementation and evaluation are important conceptual topics and are part of the policy

process”. They acknowledged, too, that the ACF and the stages heuristic –so connatural

with the original top-down approach of implementation research- could in fact be together,

providing some conditions: 1 beliefs are accepted as a casual driver, and the stages

heuristics identified with a political strategy or a venue (at a particular moment) 2 coalitions

operate across multiple policies (not just one that is usually the focus of analysis of the

stages heuristics) 3 coalitions operate across stages, and they can be simultaneously

fighting the implementation of one policy and trying to reformulate it 4 the stages

heuristics does not address group formation or strategies (Weible, Sabatier and McQueen,

2009, p.136).

More specificity and more science

27

Not everybody abandoned implementation research in the 1990s, however. One example in

the list of Hill and Hupe (2007) is Robert Stoker (1991) who takes on (once again) the

analysis of the implications of American federalism but from a different perspective: the

disability thesis” –that the U.S Government is disabled by design because the whole system

was made so that it can’t control the States and no one can control all the agencies and

relevant players at the same time- ignores that “it may be possible to manipulate the

conditions of the implementation process to encourage co-operative responses to conflicts

of interest” (Stoker, 1991, p. 50 cited in Hill and Hupe, 2007, p.72).

Stoker finds three ways to solve implementation problems: one is authority (simplifying or

circumventing the barriers for compliance), a second one is exchange (the achievement of

cooperation) and the third one is governance, “an activity in which reluctant partners are

induced to collaborate” (Hill and Hupe, 2007, p.72)

Walter Kickert, Erick-Hans Klijn and JoopKoppenjan (1997) dig more in the policy

network approach and assume that “policy is made in complex interaction processes

between a large number of actors which take place within networks of independent actors”

and “because cooperation is central in the policy network approach, explanations for the

success or failure of policy processes are based on the extent of cooperation achieved”.

(Hill and Hupe, 2007, p. 78).

Consensus is not an independent variable, but a dependent one, and “the evaluation of

success or failure of a policy is based on the process used to arrive at a possible common

problem formulation” Two main strategies are devised to achieve consensus: process

management and network constitution.

There are more authors in the review of Hill and Hupe (2007) for this period of the 1990s.

There is the Swedish Bo Rothstein who, concerned about how to build just institutions,

reviews the implementation literature trying to answer the basic question of how different

ways of organizing public administration affect the prospects for carrying out programs

successfully (Rothstein, 1998, p. 7 in Hill and Hupe, 2007, p. 78). After finding no

applicable universal model –including the “have clear goals” advice, and looking at

different taxonomies, he suggests that “the best ways to organize policy implementation

depend on the kind of task the organization must carry out”, in line with a substantial body

of organization theory, including contingency theory (Hill and Hupe, 2007). It seems the

research of Rothstein was only documental.

There are the contributions in the book of Dennis Palumbo and Donald Calista,

Implementation and the policy process (Palumbo and Calista, 1990), from their observation

taken from the British experience that “implementers are present in in every stage of the

policy cycle” (Palumbo and Calista, 1990, p. 15 in Hill and Huppe, 2007, p. 70) to that of

Soren Winter (Winter, 1990) about the impact of policy formulation upon implementation.

28

There continued to be top-down contributions focused on the importance of mandate, e.g

those by Peter J. May (1993) based on three different cases for which statutory coherence

could be substituted by tools (arrangements including facilitating features) and a high

agency commitment to objectives, or about the difficult translation, at least in the U.S, of

implementation lessons into policy learning (May, 1992).

Ingram and Schneider (1990) go back and look at the top-down/bottom up debate as an

argument about the level of discretion implementers must enjoy, and distinguish several

approaches to this: the strong statute approach, with the whole policy designed mandated at

the top (the spirit of Sabatier and Mazmanian in 1980 perhaps), the “Wilsonian

perspective”, where clear goals are set but administrative agencies are given discretion to

organize their implementation, the “grass-roots approach” in which street-level bureaucrats

and even target populations can influence the process and the “support-building” approach,

in which policy content is bargained between top and bottom (Schenieder and Ingram,

1990, pp. 74-82). They conclude that the statute design most reinforce that which is lacking

in its specific context: “Depending upon the context of the policy design, implementers can

be encouraged through the allocation of discretion, to build support for the policy goals, to

add to the information about problems necessary to solve them, or to provide resources and

incentives necessary to motivate targets to engage in actions that fulfill policy goals”

(Schneider and Ingram, 1990, p. 82)

Coming from the field of Public Administration, and distant from the top-down/bottom-up

disputes in the policy process theory, some authors, including Hill and Hupe, include in this

period the contribution of James Q. Wilson (Wilson, 1989) in his Bureaucracy. Wilson not

only draws a clear line between outputs -“the work the agency does”- and outcomes –“how,

if at all, the world changes because of the outputs” or “results” (Wilson, 1989, p. 158). He

also creates a typology of agencies according to how observable their outputs and outcomes

are, which in turn depends on their tasks. Introducing tasks as a unit of analysis is of great

help for finding the theoretical link between implementation and effectiveness, that is,

between outputs (the measurable result of implementation) and outcomes (the measure of

effectiveness which evaluation usually looks at). In the words of Robichau and Lynn (2009,

p. 25) “if we think of tasks as the work agencies and their agents do, then depicting outputs

as necessary to achieving outcomes seems intuitive”.

So tasks are theoretically relevant for implementation research because of the link they help

make clear between implementation and effectiveness/evaluation, but also because they

help take the approach dispute to more concrete terrains: the relevance of a top-down,

bottom up or network approach to implementation analysis –and to implementation itself-

will very likely be a function of the kind of tasks to be performed in a given program.

Perhaps the most commented contribution of the period, however, is that of Richard

Matland (1995) who, like O’Toole, or Sabatier and Mazmanian before O’Toole, is

29

overwhelmed by the large numbers of variables already identified, but instead of trying to

identify “the critical ones” asks himself “the conditions under which this variables are

important and the reasons we should expect them to be important” hat “have been neglected

to a large degree or have been treated superficially (Matland, 1995, p.153).

Matland notices –once again- the lack of consensus about what is successful

implementation, and identifies it at the very base of the top-dow/bottom-up dispute. In his

view, the degrees of ambiguity in and conflict about policy are decisive to determine the

best way to implement it. It is important to note that, here, ambiguity is not intrinsically

negative, but rather a characteristic of policy, another variable to be taken into account, and

ambiguity can even be useful for policy formation and implementation (these are

observations much in the line with bottom-u approaches).

He finds reason in the argument (adopted by Sabatier and Mazmanian, 1980 among others)

that democratic theory backs the idea of a superiority for the objectives set by elected

officials –because they are elected- at least for cases in which policy goals have been

clearly stated. In those cases, the measure of successful implementation is loyalty to the

prescribed goals. “When a policy does not have explicitly stated goals, the choice of a

standard becomes more difficult, and more general social norms and values come into play”

(Matland, 1995, p. 155). Matland, it must be said, is more ambiguous about defining

implementation success in situations where the policy is ambiguous too, and does not tell

us even what parameters should be kept in mind.

Looking again at O’Toole’s complaints about “contradictory recommendations”, Matland

explains them “because the underlying antecedent characteristics of a policy are analyzed

insufficiently. The factors that help implement policy under one set of conditions

exacerbate problems under another” (Matland, 1995, p. 159).

So, trying to reconcile top-down and bottom-up approaches, he builds a typology of

policies around two dimensions: conflict and ambiguity, and identifies a central principle

determining outcomes for each of one of four combinations.The results are the four types:

1 Administrative implementation (where resources are the central principle because goals

and means are clear): When a policy is characterized by a high degree of consensus and the

means for reaching the policy goals are known, the implementation process becomes

dominated by technocratic questions of compliance and follow-up (Matland, 1995, p. 162)

2 Political implementation (with low ambiguity and high conflict making power decide the

balance): “Since some of the actors whose cooperation is required may disagree with the

policy goals, successful implementation depends on either having enough power to force

one’s will on the other participants, or having sufficient resources to be able to bargain an

agreement on means” (Matland, 1995, p. 164).

30

Type 3 corresponds to Experimental implementation (when ambiguity is high and conflict

low, and context becomes very important): Outcomes depend on which actors and with

which resources are participating, the implemented program differs from site to site. Type 4

corresponds tosymbolic implementation when both ambiguity and conflict are high. “The

policy course is determined by the coalition of actors at the local level who control the

available resources” (Matland,1995, p. 168).

Ambiguity-Conflict Matrix: Policy Implementation Processes

CONFLICT

Low High

AM

BIG

UIT

Y

Low

Administrative

Implementation

Resources

Example: Smallpox erradication

Political

Implementation

Power

Example: Busing

Hig

h

Experimental

Implementation

Contextual Conditions

Example: Headstart

Symbolic

Implementation

Coalition Strength

Example: Community action

agencies

Taken from Matland, 1995, p. 160

Matland’s is surely among the most parsimonious proposals for analyzing implementation,

and helps the building of theory when transcends description of problems –for instance,

delays- and looks for their sources in the general importance of conflict and ambiguity in

policy. It does not deal with ambiguity or conflict about “problems”, however, but about

specific programs. It seems to focus on multi-actor implementation, but looking at jus one

level of analysis (that of the program) not paying much attention to organizations, policy

subsystems or …problems.

The millennium

31

Hill and Hupe (2007) finish their author’s review at this point, the mid-late 1990s, and

devote the rest of their book to exploring the interaction between implementation, the

declining influence of the State and governance. A powerful stream of the implementation

literature has taken that direction, and many expressed concern that the “old”

implementation research was basically abandoned since then (e.gdeLeon ,1999; 2006). But

the opposite seems to be true: In spite of not being widely represented in the journals of

public administration and political science, implementation research is very much alive in

field-specific publications and a new wave of “Implementation Science” journals.

Sanfordt, Roll and Moulton (2013) quote Saetren (2005) when commenting in the

surprising good health of implementation research. The number of articles on the subject

has kept increasing since the 1990s in field-specific and core journals. And even in core

political science and public administration journals the number of articles did not decline

after the 1990s. Implementation research “just went out of fashion” he wrote, considering

“symptomatic that the last symposium on policy implementation research was published in

Policy Studies Review in the late 1980s” and that only 16 per cent of all book reviews on

implementation cited in the Social Science Citation Index date after 1990” (Saetren, 2005,

p. 566).

Nilsen (2013) puts the issue in context addressing “the changes that occurred in state-

society relations in many industrialized countries…In the 1990s, there was more reliance in

market-based policy instruments and less governmental intervention. Policy

implementation research shifted emphasis to address the effects of institutional and inter-

organizational relationships, with governance and policy networks emerging as important

research topics” (Nilsen, 2013, p. 3) Those were times of privatization, outsourcing and

new public management. He compares policy implementation research with the younger

literature of Implementation Science, whose origins, interestingly, “can be traced to the

origins of evidence-based medicine and its wider application as evidence-based practice in

the 1990s”. (Nilsen, 2013, p. 4).

In 2007, Hill and Hupe (Hill and Hupe 2007, orig. 2002) highlighted the relations between

changes in the state, increasing citizen participation, consultation-decentralization and a

reduced interest in implementation. But they are clear that they expect that “the mainstream

of studies in which the implementation of a specific public policy is described and analyzed

will continue” (Hill and Hupe, 2007, p.113). And they present their own contributions,

essentially compatible with the shift in attention to governance and networks.

They propose to identify three levels of governance: constitutional, directive and

operational “levels of action”, that can also be designated as institutional setting, policy

setting and micro setting. There are also modes of governance more adequate in one

context than the other, and “two preconditions of fundamental importance for a judgment

on the appropriateness of a specific mode of governance. These preconditions concern the

32

level of steering ambitions and the degree of independence of the government actor

involved, as observed on the directive level of governance, particularly in the policy

setting” (Hill and Hupe, 2007, p.188). So a “networked” approach to implementation is not

necessarily the norm. “In the case of a specific public task like national defense, the level of

steering ambitions will be high, while government has the competence necessary to fulfill

this task “in house”. (Hill and Hupe, 2007, p.188).

There is an action perspective for every governance mode:

“For the governance-by.-authority mode, the congruent action perspective is the enforcement

perspective on managing implementation…Managing policy processes implies the assignment of an

explicit responsibility to fulfill such a task when and where the specific policy is applied…The

perspective on managing implementation congruent with governance by transaction can be called the

performance perspective…In the management of policy processes creating “interfaces” is

important… For governance via persuasion the compatible perspective is the coproduction

perspective on management implementation…Managing of policy processes here means leaving

discretion to other actors and inviting them to participate (Hill and Hupe, 2007, pp. 188-189).

Robichau and Lynn (2009) try to improve the way in which public policy theories and

governance theories complement each other.While working on a multi-level theory of

governance and after reviewing the recent public policy literature –specially the

frameworks in the 2007 edition of Sabatier’s Theories of the Policy Process (Sabatier,

2007) they conclude that a) the policy literature is not very clear when distinguishing

outputs from outcomes or explaining relations of causality between them–with a few

exceptions, including the simple yet tremendously useful distinction by James Q. Wilson

(1989) and b) policy theories “give short shrift to the influence of administrative systems on

the production of outputs.

These systems arguably account for much of the fundamental complexity found on

policy subsystems. Expanding upon implementation in the policy subsystem, based

on what has already been studied in the governance literature (i.e, the role of

management and distinguishing outcomes from outputs) would give both

governance and policy scholars insights into one another’s fields. (Robichau and

Lynn, 2009, p. 32).

Robichau and Lynn were not the last ones in linking governance and implementation. In

2011, Michael Howlett (Howlett, 2011) also took on the subject, although from a different

perspective. His main concern was with implementation tools and the way they contain the

universe of alternatives available for policy designers. Howlett reviews the current situation

in several policy fields to find a) that despite all the recent interest in governance, the most

common form of policy design continues to be the legalistic mode and b) more attention

should be paid to the constraints imposed by implementation tools, since “in a typical

33

policy design situation, not all elements of a policy are at play and the range of choices left

to designers at the micro-level of concrete targeted policy tool calibrations is restricted by

general policy aims and implementation preferences which, in turn, inform meso-level

considerations about alternative policy objectives and policy tools combinations”. (Howlett,

2011, p. 38).

So in short, there is a cascade of causality from governance mode through policy regime

through program level constraining the choices for policy designers. That is Howlett’s

contribution. We can add that if designers choose to ignore those constraints, they will end

up with a program with a tremendously difficult implementation, since it will contradict the

way other actors in the system are used to operate.

Just like other authors felt that a focus on individual programs and the way they evolved

through the stages heuristics told us very little about the causalities of change, Howlett feels

that the focus on contextual variables when studying implementation does not help us to

completely understand the restrictions for design (which, if neglected, will become

restrictions for implementation). He borrows valuable insights from Linder and Peters

(1989) and Peters (2000) in order to build a typology of policies according to involved

resources.

In our view, Howlett’s description of three levels of constraints for policy design

(governance mode, policy regime and program level) is compatible with Hill and Hupe’s

institutional setting, policy setting and micro setting. And just like Hill and Hupe warn that

the choice of “governance level “ will be a function of steering ambitions and autonomy –

so networked governance is not always the preferred option- Howlett warns that

governance mode constrains and policy regime constraint will just make some policy tools

inappropriate. Perhaps preventing an actor from designing a program based on them will

not be feasible, but reaching successful implementation with them will be an almost granted

impossibility.

Howlett is clearly focused on a multi-level analysis of implementation, based upon his

conviction that there are regularities affecting several programs at the same time –so the

unit of analysis to understand the relevant variables in implementation research cannot be

the program alone- yet those regularities are not only those associated to a policy regime,

that we can link to Sabatier’s policy subsystem and its proceeding at a given point in time,

but there is an even higher level of determination: the governance mode.

So another, perhaps more effective way to deal with the problem of “too many variables” in

implementation research is to find the level for which a given variable is theoretically

relevant. Mandate clarity, for instance, may be relevant at the program level. Support from

advocacy coalitions should be relevant at the policy regime level.

34

Propensity for tool use by governance mode

Governance mode NATO

resource

category

Legal Corporatist Market Networtk

Organization Direct

government

Administrative

tribunals

State-owned

entreprises

Privatization

Contracting

out

Special

operating

agencies

Public-private

partnerships

Clientele

agencies

Task forces

Public

hearings

Authority Laws

Direct

regulation

Administrative

procedures

Independent

regulatory

commissions

Delegated

regulation

Advisory

councils

Deregulation

Voluntary and

self-regulation

Stakeholder

conferences

Treasure Excise taxes

Insurance

Subsidies,

grants,

interest group

mobilization

Tax incentives

Procurement,

vouchers

Exhortation

and suasion

Interest group

creation

Informational Product

information

campaigns

General

information

campaigns

Surveys

Statistics and

data collection

Freedom of

information

Taken from: Howlett (2011, p. 129)

Conclusion: Implementation research and the role of problems

35

This review of implementation research has led us to understand the evolution of the debate

from the original question: why did this program not work according to plan?,through the

fundamental confusion between implementation and evaluation, the proposal of several

frameworks until Matland’s ambiguity and conflict, and the new focus on governance.

I believe there are enough reasons to avoid a new interpretation of implementation

exclusively through the eyes of governance and networks, assuming now that the “natural”

way of planning is networked, just like the “natural” way was top-down thirty years ago.

What is relevant then? We know now that it is not only mandate clarity or the top-down

variables. We also know it is not only context variables, and we have now a much clearer

picture of what “context” means. Context is not necessarily about being in France or in

Colombia, but about a governance mode, a policy regime, advocacy coalitions and specific

actors too.

What we have not done is going one step further in causality. If conflict and ambiguity

(Matland, 1995) are the basic two dimensions to judge the possibilities of successful

implementation, for instance, is there a variable affecting both the probability of conflict

and the desirable level of ambiguity? We believe there is, and that variable is problem

structuredness.

It seems easy to understand why problem structuredness has not been taken into account in

implementation research. Although implementation research and problem structuredness

literature sprang and about the same time in 1973, the first was initially rooted on the stages

version of the policy cycle, which was focused on studying programs for which the

“problem” was supposed to be given.

Despite the existence of literature on the “wrong-problem problem” (Raiffa, 1968; Dunn,

1988, Dunn, 2008) it was not a major concern for implementation research. Only recently,

the “governance” approaches to implementation and the constructivist literature in policy

analysis put the problem again in a central stage, The former, because what explains the

interaction between networks and, more concretely, between coalitions, is a “problem” that

different coalitions perceive in different ways. The later, because the very core of

constructivism is the socially-constructed representation of reality –that is, in policy

analysis, of problems. So the wrong-problem problem is no longer a possibility as a result

of bad diagnose. By definition, every description of a problem by one sector risk being

considered “wrong” by another sector –or coalition.

A quick look at the literature in problems shows us that:

Rittel and Weber (1973), in the context of reflections on a general theory of planning, came to

the conclusion that not some, but most social problems are what they call wicked, meaning

essentially that they cannot be solved through analysis. To them, in a context where popular

36

protest “may have become a social movement”, social services professionals were under attack

“precisely when they are becoming good at their jobs” because their work had changed from

dealing with problems that “appeared to be definable, understandable and consensual” to pay

attention to others “that are much more stubborn”. “The seeming consensus, that might once

have allowed distributional problems to be dealt with, is being eroded by the growing

awareness of the nation’s pluralism and the differentiation of values that accompanies

differentiation of publics” (Rittel and Weber, 1973, p. 156). So analysis lacked its first and

more important input: clear, pre-defined goals (call them “revealed preferences” or just goals).

By now we are beginning to realize that one of the most intractable problems is that of defining problems

(of knowing what distinguishes an observed condition from a desired condition) and of locating

problems (finding where in the causal networks the trouble really lies). (Rittel and Weber, 1973, p. 159).

The problems of government planning -.and above all those of social planning- are wicked, as

opposite to “tame” or “benign” problems, as difficult as the latter can be, because they cannot

be solved, but at best continuously re-solved “and they rely upon elusive political judgment for

their resolution” (Rittel and Weber, 1973, p.160).

Wicked problems share 10 characteristics: 1. There is no definitive formulation of a wicked

problem 2. Wicked problems have no stopping rule –you never know when you reached a

“solution” 3. Solutions for them are not true-or-false, but good-or-bad 4. There is no immediate

and no ultimate test of a solution 5. (Because of the social context) There is no opportunity for

trial-and-error, and every solution is a “one-shot opportunity”. 6. There is no enumerable set of

potential solutions 7. Every wicked problem is essentially unique –for all the similarities that

might be with others, there might always be an additional distinguishing property. 8. Every

wicked problem can be considered to be symptom of another problem 9. The problem can be

understood in numerous ways, and the choice of explanation determines the nature of the

resolution 10. The planner has no right to be wrong

Interestingly enough, in their text, Rittel and Weber propose that “the systems-.approach “of

the first generation” is inadequate for dealing with wicked problems. Approaches “of the

second generation” should be based on a model of planning as an argumentative process in the

course of which an image of the problem and of the solution emerges gradually among the

participants, as a product of incessant judgment, subjected to critical argument” (Rittel and

Weber, 1973, 162).

Hebert Simon (Simon, 1973) went farer. He defined well-structured problems as having “some

or all” of the following characteristics:

l.Thereisadefinitecriterionfortestinganyproposedsolution,andamechanizableprocessfor

applyingthecriterion.

2. Thereisatleastoneproblemspaceinwhichcanberepresentedtheinitialproblemstate, the

goalstate,andallotherstatesthatmaybereached,orconsidered,inthecourseof

attemptingasolutionof theproblem.

3. Attainablestatechanges(legalmoves)canberepresentedinaproblem

37

space,astransitionsfromgivenstatestothestatesdirectlyattainablefromthem.Butconsiderable

moves,whetherlegalornot,canalsoberepresentedthatis,alltransitions

fromoneconsiderablestatetoanother.

4. Anyknowledgethattheproblemsolvercanacquireabouttheproblemcanberepresented

inoneormoreproblemspaces.

5.Iftheactualprobleminvolvesactingupontheexternalworld,thenthedefinitionofstatecha

ngesandoftheeffectsuponthestateofapplyingany

Operationreflectwithcompleteaccuracyinoneormoreproblemspacesthelaws(lawsofnature)that

governtheexternalworld. 6.Allof theseconditionsholdin

thestrongsensethatthebasicprocessespostulatedrequireonlypracticableamountsofcomputation,andtheinformationpostulatediseffectivelyavailabletotheprocesses-i.e.,availablewiththe helpofonlypracticableamountsofsearch.

Then the author provides several examples of problem –a chess game, the building of a house-

of how problems that are generally thought of a “well-structured” are not really such. So,

almost every problem can be conceived as being ill-structured.

Simon’s solution for ill-structuredness is not less, but more analysis -.in the original sense-

instead. Yet we must recall that he published his article in a journal of Artificial Intelligence

and his subject for examples were some kind of (individual) intelligence and the a concrete

robot: “As a matter of fact, the entire procedure could conceivably be organized as a system of

productions” and “the evocation of relevant information and sub-goals from long-time memory

can be sequential”. He quotes Reitman:

Justassentence·transformsto'subjectpluspredicate',and'subject'maytransformto“articleplusnounphrase'...so'fugue'maybethoughtofastransforming ‘exposition plusdevelopment plus conclusion', 'exposition' to 'thematic material pluscountermaterial',and'thematicmaterial'to'motiveplusdevelopmentofmotive'." (Simon, 1973, p. 190)

Although coordination problems among the well-structured sup-problems might arise, they can be handled, according to Simon, through “the architect’s” skill to provide organization to the overall process. The key feature in his approach is the availability of “ a very large long-tem memory of potentially evocable information that canbeusedtobringaboutrepeatedmodificationsintheproblemspace” (Simon, 1973, p. 200).

So Simon bypasses the absence of a “definite criterion” for testing solutions, since every decision has been broken into so many specific parts that relatively straightforward choices are available. His ill-structured problems are such because of the absence of “definite” criteria to decide, but those criteria are may not be necessary provided the problems are simplified in multiple smaller parts. For him the problem is of complexity, which makes “relevant knowledge” less obvious, not of values. His robots were individual decision makers with no need for coordination.

William N. Dunn´s perception of ill-structured problems is different from Simon’s, since Dunn is focused on the normative consequences of a multiplicity of decision-makers.

In his introductory book to policy analysis, Dunn (2008) identifies three types of

38

problems: ill-structured problems, well-structured problems and moderately structured

problems. The well-structured problems are those that involve “one or a few decision

makers and a small set of policy alternatives. Utilities (values) reflect goal consensus and

are clearly ranked in order of decision makers' preferences”. According to him “the

prototype of the well-structured problem is the completely computerized decision

problem, where all consequences of all policy alternatives are programmed in advance”.

Moderately structured problems, in turn, are “those involving one or a few decision makers

and a relatively limited number of alternatives. Utilities (values) also reflect consensus on

clearly ranked goals. Nevertheless, the outcomes of alternatives are neither certain

(deterministic) nor calculable within acceptable margins of error (risk)”. Ill-structured

problems are “those that typically involve several decision makers whose utilities (values) are

either unknown or impossible to rank on a consistent fashion.

The problem has not been one of naming, however. As Dunn himself explains, “whereas

well-structured and moderately structured problems reflect consensus, the main characteristic

of ill-structured problems is conflict among competing goals. Policy alternatives and their

outcomes may also be unknown, such that estimates of risk and uncertainty are not

possible. The problem of choice is not to uncover known deterministic relations, or to

calculate the risk or uncertainty attached to policy alternatives, but rather to define the nature

of the problem. The prototype of the ill-structured problem is the completely intransitive

decision problem, that is, one where it is impossible to select a single policy alternative that is

preferred to all others”.

The situation of ill-structured problems can be improved, however, according to Dunn, using

what he calls methods of the second type (Dunn, 1988). Criticizing Zeckhauser and Schaefer

(1968) for their picture of problems that “come in relatively well-bounded packages

involving an explicit statement of a decision maker’s preferences, a careful exposition of the

alternative actions available, and a model that relates these alternatives to the stated

preferences in a manner that permits an efficient choice among the alternatives” (Dunn, 1998,

p. 721) –that is, blaming Zeckhauser and Schaefer for the original idea of policy analysis-the

author stresses the existence of what others had previously called ill-structured problems

(Simon, 1973), squishy problems (Strauch, 1976) or messy problems (Ackoff, 1974).

What is important here is that, to Dunn, ill-structured problems can be structured. For them,

a) decision makers b) preferences or utilities c) alternatives and d) outcomes and e) states of

nature are unknown or equivocal. The reason for these difficulties is the multitude of

stakeholders involved, since “conflicting representations of problems are continuously

created, maintained, and changed by stakeholder who affect, and are affected, by the policy-

making processes of modern governments” (Dunn, 1988, p. 722).

39

There is a way out, however: the principle of methodological congruence by which “the

appropriateness of a particular type of method is a function of its congruence with the type of

method under investigation”. What is needed, then, is a new set of methods “of the second

type”.

Cost-benefit analysis, for example, is an appropriate and useful first-order method that enables

calculations of net benefits or benefit-cost ratios for a set of well-bounded alternatives. But

cost-benefit analysis does not and cannot generate the policy goals, phases and instruments

that should be included in the set (Dunn, 1988, p. 724)

In this and other publications (Dunn, 2008) the author has identified brainstorming, hierarchy

analysis, synetics, multiple perspectives analysis and policy-grid analysis as methods of the

second type.

Dunn notices the weaknesses of several of these methods for addressing the difficulties in

finding the limits of the problem. According to him, that is precisely the key strength of the

policy-grid analysis, which allows for an estimation of the boundaries of the problem –i.es,

an estimation of the complete universe of relevant representations of it in any given context,

temporal and locational.

More recently, it has been Robert Hoppe (Hoppe, 2010) who establishes a relationship

between a given kind of problem (structured, moderately structured-structured in means,

moderately structured-structured in ends, non-structured) and specific types of analysis.

The question is: what is the implication of a mismatch between type of problem and type of

analysis (used in policy design). The likely answer is: bad implementation, because the

consensus that implementation researchers have looked at (Van Meter and Van Horn, 1975;

Matland, 1995, but also Weible, Sabatier and McQueen, 2009 speaking specifically about

wicked problems and implementation) will not be there.

This is how we can say that a) Implementation research has not paid attention to the

problems of problem structuredness and b) it could understand much better consensus,

conflict, ambiguity, support, commitment and several other variables that it has found of

relevance over time. Finally, it is important to dig deeper on the usefulness of a multi-level

analysis of implementation, and a first objective for further research can be finding the

relationship between problem structure and each of those levels.

40

References

Aguilar, Luis F (ed.) (1993). La implementación de las políticas. México, D.F: Miguel

Angel Porrúa

Bardach, Eugene (1977). The implementation game:what happens after a bill becomes a

law. Cambridge, Massachusetts: The MIT Press.

Barrett, S. and C. Fudge (1981). Policy in action: essays on the implementation of public

policy. London: Methuen.

Berman, Paul (1978). The study of macro- and micro- implementation. PublicPolicy, Vol. 2

6 (n.2), pp. 157-184.

Calderón, Victoria Alejandra (2011). La perspectiva de H.D Lasswell (1902-1978) para el

análisis de las políticas públicas, Enfoques para el análisis de las políticas públicas, André

Noel Roth (ed). Bogotá: Universidad Nacional de Colombia.

deLeon, Peter (2006). The historical roots of the field. Oxford Handbook of Public Policy.

New York: Oxford University Press.

deLeon, Peter (1999). The stages approach to the policy process. What has it done? Where

is it going?. Paul A. Sabatier (ed.) Theories of the policy process. Boulder, Colorado:

Westview Press.

Derthick, Margaret (1972). New towns in-town. Washington, D.C: Urban Institute Press.

Dunn, William N. (2008 [1981). Policy analysis: an introduction. New Jersey: Pearson

Prentice Hall.

Dunn, William N. (1988). Methods of the second type: Coping with the wilderness of

conventional policy analysis. Policy Studies Review, vol. 7 (n.4), pp. 720-737.

Elmore, Richard F (1985). Forward and backward mapping: reversible logic in the analysis

of public policy. K. Hanf et al. (eds). Policy implementation in federal and unitary systems.

Dordrecht: MartinusNidjhoff

41

Elmore, Richard F (1979). Backward mapping: implementation research and policy

decisions. Political Science Quarterly, Vol. 94 (n.4), pp. 601-616

Goggin, Malcolm L (1986). The too few cases-too many variables problem in

implementation research. The Western Political Quarterly, Vol. 39 (n.2), pp. 328-347.

Goggin, Malcolm L. (1984) Review of “Implementation and public policy”. Publius, Vol.

14 (n.4), pp. 159.160

Hill, Michael and Peter Hupe (2007). Implementing public policy. London: Sage.

Hjern, Benny (1982). Organization structures: a new unit of administrative analysis.

Organization Studies, 2/3, pp. 211-227.

Hjern, Benny (1981). Implementation research: the link gone missing. (Review). Journal of

Public Policy, Vol. 2 (n.3), pp. 301-308.

Hoppe, Robert (2010). The Governance of Problems: puzzling, powering and participation.

Bristol, Gran Bretaña: The Policy Press.

Howlett, Michael (2011). Designing public policies: principles and instruments. New

York: Routledge.

Howlett, Michael y M. Ramesh (2003) .Studying public policy: policy cycles and policy

subsystems. Don Mills (Ontario), Canada: Oxford University Press.

Ingram, Helen and Anne Schneider (1990). Improving implementation through smarter

statutes. Journal of Public policy, Vol. 10 (n.1), pp. 67-88.

Kickert, W.J.M, Klijn, E.H and J. F.M Koopenjan (eds) (1997) Managing complex

networks: strategies for the public sector. London: Sage.

Linder, S.H and B. Guy Peters (1989). Instruments of Government: Perception and

contexts. Journal of Public Policy, Vol. 9 (n.1), pp. 35-58

Lipsky, Michael (1980). Street-level bureaucracy: dilemmas of the individual in public

service. New York: Russell Sage Foundation.

Lipsky, Michael (1971). Street-level bureaucracy and the analysis of urban reform. Urban

Affairs Quarterly (n.6), pp. 391-409.

42

Matland, Richard E. (1995). Synthesizing the implementation literature: the ambiguity-

conflict model of policy implementation. Journal of Public Administration Research and

Theory, Vol. 5 (n.2), pp. 145-174

May, Peter (1993). Mandate design and implementation. enhancing implementation efforts

and shaping regulatory styles. Journal of Policy Analysis and Implementation, Vol. 12

(n.4), pp. 634-663.

May, Peter (1992). Policy learning and failure. Journal of Public Policy, Vol. 12 (n.4), pp.

331-354

Mazmanian, David A. and Paul Sabatier (1989, orig. 1980).Implementation and public poli

cy. Washington, D.C: University Press of America.

Nilsen, Per, Christian Stahl, Kerstin Roback y Paul Cairney. Never the twain shall meet? A

comparison of implementation science and policy implementation research. Implementation

science, Vol. 8 (n.63), pp. 1-13

O’Toole, Laurence (2000). Research on public implementation: assessment and prospects.

Journal of Public Administration Research and Theory, Vol. 10 (n.2), pp. 263-288.

O’Toole, Laurence (1986). Policy recommendations for multi-actor implementation: an ass

essment of the field. Journal of Public Policy, Vol. 6 (n.2), pp. 181-210

O'Toole, Laurence Jr., & Montjoy, R. S. (1984). Interorganizational policy implementation:

A theoretical perspective. Public Administration Review, Vol. 44 (n.6), pp. 491-503.

Palumbo, D.J and D.J Calista (eds.) (1990). Implementation and the policy process:

Opening up the black box. New York: Greenwood Press.

Peter, B. Guy (2000). Policy instruments and public management: bridging the gaps.

Journal of Public Administration Research and Theory, Vo. 10 (n.1), pp. 35-48.

Pressman, Jeffrey and Aaron Wildavsky (1973). Implementacion. Mexico, D.F: Fondo de

Cultura Economica.

Raiffa, Howard (1968). Decision analysis. Reading, Massachusetts: Addison-Wesley.

Rein, Martin and Francine Rabinovitz (1993, orig. 1977). Implementation: a

theoreticalperspective. Aguilar, Luis F (ed.) (1993). La implementación de las políticas.

México, D.F: Miguel Angel Porrúa

43

Rittel, Horst and Melvin Webber (1973). Dilemmas in a General Theory of Planning.

Policy Sciences 4, pp. 155-169.

Robichau, Robbie and Lawrence E. Lynn (2009). The implementation of public policy: still

the missing link. The Policy Studies Journal, Vol. 37 (n.1), pp. 21-36.

Rothstein, Bo (1998). Just institutions matter.The moral and political logic of the Universal

Welfare State. Cambridge: Cambridge University Press.

Sabatier, Paul A. and Hank Jenkins-Smith (1993). Policy change and learning: an advocac

y coalition approach.

Sabatier, Paul A. and Hank Jenkins-Smith, eds. (1988). Policy change and policy-oriented n

etwork: exploring an advocacy coalition framework (symposium issue). Policy Sciences (n.

21), pp. 123-277

Saetren, Harald (2005). Facts and myths about research on public policy implementation Po

licy Studies Journal, Vol. 33 (n.4), pp. 559-582.

Sandfort, Jodi, Stephanie Moulton and Stephen Roll.

Integrating and advancing policy and

program implementation analysis. Paper presented at the Annual Research Conference of th

e Association for Public Policy Analysis and Management. Washington, D.C: November 2

013.

Selznick, Philip (1949). TVA and the Grass Roots: A study in the sociology of formal orga

nization. Berkeley, California: University of California Press.

Simon, Herbert. The structure of ill-structured problems. Artificial intelligence (4), pp. 181-

201.

Stoker, Robert P. (1991).Reluctant partners: Implementing federal policy. Pittsburgh:

University of Pittsburgh Press.

Van Meter, Donald and Carl E. Van Horn (1975). The policy implementation process: a

conceptual framework. Aguilar, Luis F (ed.) (1993). La implementación de las políticas.

México, D.F: Miguel Angel Porrúa

Weible, Christopher M., Paul A. Sabatier y Kelly McQueen (2009). Themes and variations:

taking stock of the advocacy coalition network. Policy studies journal, Vol. 37 (n.1), pp. 12

1-140.

Weible, Christopher M, Tanya Heikkila, Peter deLeon y Paul A. Sabatier (2012). Understan

ding and influencing the policy process. Policy Sci (n.45), pp. 1-21.

Wilson, James Q. (1989). Bureaucracy. New York: Basic Books.

44

Winter, Soren (1990). Integrating implementation research.Palumbo, D.J and D.J Calista

(eds.)(1990). Implementation and the policy process: Opening up the black box. New York:

Greenwood Press.

Yates, Douglas (1982). Review of “Steet-level bureaucracy: dilemmas of the individual in

public services” by Michael Lipsky. The American Political Science Review, Vol. 76 (n.1),

pp. 145-146.