Research on Policy Implementation: Assessment and ProspectsPublic+Adm+R… · Research on Policy Implementation: Assessment and Prospects Laurence J. O'Toole Jr. University of Georgia

Research on Policy Implementation:Assessment and Prospects

Laurence J. O'Toole Jr.University of Georgia

A version of this article was presentedat the fifth national Conference on PublicManagement, Texas A&M University,College Station, Texas, December 2-4,1999. Thanks are due to Peter deLeon,Kenneth J. Meier, Beryl Radin, and espe-cially Ann Chih Lin for helpful commentson the earlier draft. Responsibility, ofcourse, remains with the author.

J-PART 10(2000):2:263-288

ABSTRACT

While policy implementation no longer frames the corequestion of public management and public policy, some scholarshave debated appropriate steps for revitalization. And the practicalworld stands just as much in need now of valid knowledge aboutpolicy implementation as ever. Where has all the policy imple-mentation gone? Or at least all the scholarly signs of it? Andwhy? What has the field accomplished? Should a resurgence ofattention to the subject be exhorted? And if so, in what direc-tions?

This article considers these questions as foci of anassessment of the state of the field, and the argument reachessomewhat unconventional conclusions: There is more here thanmeets the eye. While modest to moderate progress can be notedoh a number of fronts, an initial assessment is likely to understatethe extent of work underway on matters quite close to the imple-mentation theme. Research on policy implementation-like ques-tions has partially transmogrified. One has to look, sometimes, inunusual places and be informed by a broader logic of intellectualdevelopment to make sense of the relevant scholarship. Policyimplementation work, in short, continues to bear relevance forimportant themes of policy and management. But some of the dis-course has shifted, the questions have broadened, and the agendahas become complicated. Research on implementation, underwhatever currently fashionable labels, is alive and lively.

The explicit study of policy implementation has swung inand out of fashion during the past quarter century. Beginningwith Pressman and Wildavsky's classic investigation (1984[1973]) of an Economic Development Administration projectgone sour in Oakland, California, the subject quickly gainedcachet during the Great Society postmortem and the subsequentyears of welfare state and budgetary crises.

263/Journal of Public Administration Research and Theory

at Universiteit T

wente on January 5, 2017

http://jpart.oxfordjournals.org/D

ownloaded from

http://jpart.oxfordjournals.org/

Research on Policy Implementation

Pressman and Wildavsky exaggerated mightily in assertingthat, until their efforts, virtually no one had addressed the subject(1984 [1973], xxi). But the claim was based in a real, and conse-quential, point: For all the attention to administration over theyears, the nitty-gritty of implementation had been largely a back-ground issue. Their work and the work of hundreds of othersover the next several years (see O'Toole 1986) ensured that thetheme would be important for the foreseeable future. From no-where, policy implementation moved to a position of prominence,perhaps even overemphasis (see Under and Peters 1987). Theproliferation of studies brought, in turn, an explosion in types ofresearch designs, varieties of models, and—especially—proposalsfor adding a bewildering array of variables as part of the expla-nation for the implementation process and its products. Thecornucopia of investigations catalyzed, in turn, a set of sectariandisputes: qualitative and small-n versus quantitative, large-ninvestigations; top-down versus bottom-up frameworks; policy-design versus policy-implementation emphasis, and so forth.Implementation was even seen by some worried students of tradi-tional public administration as a theme posing a hegemonic threatto the field (Kettl 1990 and 1993).

It was no accident that the interest in policy implementationsurged following the rise of and disappointment in the initiativesof the 1960s and 1970s. But many years of lowered expectations,incremental program adjustments, budget crunches, and, in thelast few years, Republican congressional control have had damp-ening effects. Implementation issues tended to recede from theheadlines, as the policy agenda became preoccupied with cut-backs, devolution, and holding the line.

Since 1990, the spate of scholarly research aimed explicidyat the implementation theme has abated. (There are exceptions,of course, including many studies referenced below.) Lynn hascommented, in particular, on the ascendency and diminishment ofimplementation research conducted in the nation's premier policyschools (Lynn 1996, 47). Others have argued that research inter-est in the subject has declined overall (deLeon 1999a and 1999b).

It is hardly as though the scholarly community has declaredvictory and moved on to other challenges, however. In fact,assessments at earlier stages generally emphasized the slow paceof progress, the seemingly intractable or at least interminablescholarly disputes remaining, and the need for bigger and betterempirical investigations. (Samples include O'Toole 1986; Gogginetal. 1990.)

26VJ-PART, April 2000

at Universiteit T



ownloaded from



'A well-known example is Sabatier'ssuggestion that investigators abandon theso-called stages heuristic that placesimplementation per se into prominence asa research subject, in favor of approachesthat emphasize policy change, policy-oriented learning, and the influence ofpolicy coalitions over time. (See Sabatier1991; Sabatier and Jenkins-Smith 1993.)

inevitably, some related topics must beneglected. One important example is thescholarship on policy design, which bearson issues of implementation—and viceversa. The integration of design andimplementation considerations is a themeimplicitly suggested by some of the recentwork on governance, a topic treated laterin this article. But research on policydesign itself cannot be addressed withinthe constraints of the present analysis.

In recent times, a number of prominent scholars of publicpolicy and administration have commented hopefully, pessimis-tically, caustically, and pragmatically regarding the likelihoodand utility of a large-scale resurgence of implementation research(see Lester and Goggin 1998; in response see deLeon 1999a;Meier 1999; Schneider 1999; Winter 1999). Some attribute theshift of interest to other policy questions to better frameworksand a sharpened sense of appropriately productive research ques-tions.1 But few aside from Lester and Goggin have called for awholesale renaissance of the subject—in the sense, at least, of aninfusion of research effort to make this topic the high-visibilitycore of public management and public policy scholarship that itwas a few years ago (Lynn 1996, 57).

Similarly, even as the researchers seem not to have solvedthe implementation puzzle, practitioners continue to find them-selves enmeshed in the vexing challenges of converting policyintent into efficacious action. Policy failures continue to beprominent, and evidence of implementers' desires to be informedin appropriate ways by the research community suggests thatmany implementation conundrums remain salient in the world ofaction. Lester and Goggin (1998, 3-4) have sketched a set ofadditional reasons the study of implementation today should becompelling; these have to do with the emerging experience inmany policy sectors during the past twenty years, the develop-ment of several shifts in intergovernmental relations, and themovement of much policy responsibility to the states in recentyears. It seems clear, at the least, that the practical world isnow just as much in need of valid knowledge about policyimplementation as it ever has been.

If scholarship has not simply solved the problem, and ifpractice continues to require attention to the issue, what hashappened? Where has all the policy implementation gone? Or atleast all the scholarly signs of it? And why? What has the fieldaccomplished? Should a resurgence of attention to the subject beexhorted? And if so, in what directions?

This article considers these questions as foci of an assess-ment of the state of the field,2 and the argument reaches some-what unconventional conclusions: There is more here than meetsthe eye. While modest to moderate progress can be noted on anumber of fronts, an initial assessment is likely to understate theextent of work underway on matters quite close to the imple-mentation theme. A considerable quantity of provocative, well-conceived, and well-executed recent scholarship bears quitedirectly on salient issues of policy implementation, even if notexplicitly and obviously framed in such terms. The second major

265/J-PART, April 2000

at Universiteit T



ownloaded from



section of the article sketches selected lines of research that haveadvanced some of the themes of implementation research in non-obvious but notable ways. The article concludes with a briefreview of prospects.

WHAT HAS HAPPENED TOIMPLEMENTATION RESEARCH?

Policy implementation is what develops between the estab-lishment of an apparent intention on the part of government to dosomething, or to stop doing something, and the ultimate impactin the world of action. Some scholars include here both theassembly of policy actors and action, on the one hand, and thecause-effect relationship between their efforts and ultimateoutcomes, on the other (for instance, Mazmanian and Sabatier1989). Others, including myself, have emphasized the importanceof making a conceptual distinction between implementation(action on behalf of the policy) and ultimate impact on the policyproblem (for initial statements, see Montjoy and O'Toole 1979;O'Toole and Montjoy 1984). Implementation research concernsthe development of systematic knowledge regarding whatemerges, or is induced, as actors deal with a policy problem.

Aside from relatively trivial circumstances (one-shot imple-mentation efforts, self-implementing policies, and small-scaleimplementation), an understanding of implementation requiresrecognition of the multiactor character of policy action (O'Toole1996b). While intraorganizational cases can sometimes be treatedin terms of the tools and theories that have been developed tounderstand the management of public organizations (see, for in-stance, Montjoy and O'Toole 1979), even these instances exhibitmultiactor features—particularly when clients, political support,and other external influences are taken into account. Clients (ormore generally, targets) of policy, for instance, must be morethan passive recipients of publicly initiated effort; they are amongthe parties who have to be active toward implementation, throughcoproduction or in some other less direct fashion. Even moreobviously, interorganizational implementation requires additionalapproaches. Here analysts and practitioners must confront aworld of multiple institutional actors—more than one govern-ment, agency, or sector—whose cooperation and perhaps coordi-nation are needed for implementation success.

Clearly, a multiactor approach to understanding implementa-tion offers a shift from the standard world presented in conven-tional organization theory. The former presents the pressing issueof how to concert action in the absence of operational authorityand across institutional lines. The theoretical and practical issues


at Universiteit T



ownloaded from



involved go considerably beyond the standard single-agency per-spective.

Even if one were to ignore for purposes of analysis theinvolvement of clients and other interested parties, the world ofmultiactor implementation is far from a trivial set. In larger-nresearch, it has been demonstrated that a substantial segment ofthe large set of U.S. national cases is multiactor (O'Toole andMontjoy 1984). Numerous researchers have argued that thiscluster of instances is important, even dominant (for instance, seeHjern 1982; Hjern and Porter 1981). A recent content analysis offederal statutes shows that multiactor cases constitute the greatmajority of new or substantively revised federal programs, andthis phenomenon can also be documented in the work of a Con-gress from a generation ago (Hall and O'Toole 2000). Othersmight argue that the number and strength of causal forces press-ing governments to organize their implementation efforts in morenetworked forms have been growing in recent years (see thecoverage in O'Toole 1997b). But the main point is that this set ofcases is a large and significant segment, and it has been the focusof the bulk of implementation research over the years (for a cri-tique of this emphasis, see Kettl 1993). The key challenges toexplanation, prediction, and performance arise here.

In 1986, I reviewed virtually the entire scope of multiactorpolicy implementation research and concluded:

The field is complex, without much cumulation or convergence. Few well-developed recommendations have been put forward by researchers, and anumber of proposals are contradictory. Almost no evidence or analysis ofutilization in this field has been produced. Two reasons for the lack ofdevelopment are analyzed: normative disagreements and the state of thefield's empirical theory. Yet there remain numerous possibilities for increas-ing the quality of the latter. Efforts in this direction are a necessarycondition of further practical advance (1986, 181).

What, then, can be said by way of progress in the succeed-ing years? The explicit evidence is mixed. Virtually all analystshave moved past the rather sterile top-down/bottom-up dispute,and some helpful proposals for synthetic or contingent perspec-tives have been offered. But consensus is not close at hand, andthere has been relatively little emphasis on parsimonious expla-nation. The dominance of the case-study approach has receded,and a number of thoughtful larger-n empirical studies have beenconducted—a point often missed by critics. But significantly moreare needed; more importantly, the recent empirical work raises aquestion about appropriate modeling strategies and specifications.The context-dependent (and primarily American) feature of muchearlier work has been exposed and theoretical efforts have

261IJ-PART, April 2000

at Universiteit T



ownloaded from



become more self-consciously general, but solid cross-nationalinvestigations are still rare. A so-called third-generation approachto implementation research has been suggested, but relativelylittle such research has been stimulated by this call. As I willshow, there are some sensible reasons for this state of affairs.Finally, and perhaps most importantly, the implementation prob-lem has been reconceptualized in somewhat different fashions,and work has proceeded along a number of parallel, overlapping,and highly relevant lines of research. These promise to expandknowledge about converting policy into action, even as theydiffuse attention away from implementation in the narrow. Farfrom signaling a failure of the research enterprise, this lastdevelopment provides evidence of impact and advance.

Efforts at Synthesis

Attempts to stimulate synthesis in theory have appearedduring the past decade. An important dispute between advocatesof the so-called top-down perspective and those identified asbottom-uppers raged for a number of years, to some limitedoverall impact. Most scholars would now agree on a few impor-tant points in this regard. First, normative top-down and bottom-up differences regarding where leverage is most appropriatelyplaced in the implementation system continue to be important inpractical terms, but heated empirical arguments have quieted.Second, variables located at the top or center can be important,as can contextual or field variables. Sufficient evidence hasaccumulated to validate partially both top-down and bottom-uparguments (Bressers and Ringeling 1989; Mazmanian and Saba-tier 1989, 302-04; Goggin et al. 1990; Stoker 1991; Matland1995; Ryan 1996).

What has not happened, however, is a careful winnowing ofthe mass of potential explanatory variables toward parsimoniousexplanation. Indeed, and this is a third point of general agree-ment, a multitude of candidate variables continue to float throughthe research literature. In my earlier analysis, I found plenty anddocumented their published sources (1986). Most continue toremain active candidates. Meier has commented recently on thispoint with a caustic suggestion: "I often characterize the theoryas 'forty-seven variables that completely explain five casestudies'. . . . I propose . . . [a]ny policy implementation scholarwho adds a new variable or a new interaction should be requiredto eliminate two existing variables" (1999, 5-6). While thischaracterization does too little justice to some extant efforts,particularly some larger-n inquiries, the main point remainsvalid. One attraction of some early efforts at formal theorybuilding in this field (see below) is that it avoids this problem.


at Universiteit T



ownloaded from



Methodological Developments

The field of implementation research was taken to task sometime ago for its overreliance on case-study research (Goggin1986). What has happened in the interim? First, it should benoted, some implementation analysts have developed smaller-nstudies of an exceedingly high standard and demonstrated thatthere are multiple routes to understanding In particular, it seemsclear that research performed in ignorance of the understandingthat implementation actors themselves have about their circum-stances is likely to miss important parts of the explanation forwhat happens (see especially Lin 2000; also Brodkin 1997;Glaser and MacDonald 1998; Lin 1998; Sandfort 1997; seedeLeon 1999b for a general brief for qualitative approaches).Second, some empirical studies have taken advantage of carefuldesign to allow for statistical inference despite relatively limitednumbers of cases (OToole 1989).

Third, a number of larger-n multivariate investigations havebeen completed in recent years, and these in general havedemonstrated the importance of both centrally controlled andcontextual variables in explaining implementation results, how-ever operationalized. (A sample includes Lester and Bowman1989; McFarlane 1989; Scheirer and Griffith 1990; Meier andMcFarlane 1995; Meier and Keiser 1996; Gerstein et al. 1997;Berry, Berry, and Foster 1998; Brown, O'Toole, and Brudney1998; Jennings and Ewalt 1998 and 2000; Heinrich and Lynn1999). The claim that empirical implementation research isnothing but case studies is no longer valid.

Yet—fourth—the move to multivariate explanation and largenumbers of cases exposes the specialty to new or renewed chal-lenges, which have yet to be addressed fully. These are partiallymethodological and partially theoretical, and they are notrestricted narrowly to questions of implementation alone. Threeissues in particular are worthy of note.

The first is not strictly new, but it is a more prominent issuenow: larger-n investigations remain almost exclusively cross sec-tional. A thorough understanding of implementation, indeed ofpolicy action more generally, requires longitudinal designs,preferably over extended periods. Executing such research istypically difficult, for all the usual reasons, but this remains asignificant challenge for the field.

The second has to do with the whole point behind imple-mentation analysis and the link between theory and practice.


at Universiteit T



ownloaded from



Meier, with coauthors, has suggested a set of refinements in theuse of statistical inference to take into account the interest,among specialists in policy and public management, in improvingperformance rather than merely documenting and explaining it.The general approach is dubbed substantively weighted analyticaltechniques, or SWAT (see Meier and Gill forthcoming). Theinitial publication in the project applied a version of SWAT to across-state analysis of policy implementation for one kind ofprogram (Meier and Keiser 1996). The main contribution was toargue for the desirability, under some circumstances, of examin-ing outlier cases distinctively in multivariate modeling in order tosee what they might reveal about unusual combinations of pro-duction factors in high- and low-performing instances. It is alsonoteworthy that this piece is framed as an implementation study.The implications of such innovations for larger-/! implementationresearch portend increased possibilities to offer guidance both inthe selection of cases for intensive examination and as a tool toguide policy designers and implementation practitioners in im-proving performance. SWAT may help to make sense out of puz-zling multivariate findings (for an inviting candidate, see Scheirerand Griffith 1990). And the possibilities of employing SWAT forimplementation research and practice have only recently beenseen. For instance, this approach can be read as implying thatserious consideration ought to be accorded interpretivist treat-ments of implementation, wherein the context-specific meaningsof implementers' words are important to an understanding of whyaction transpires as it does. And SWAT could be combined withinterpretive research as one tool in case selection. The links andimplications across both epistemological and methodological per-spectives have scarcely been considered.

The third point of interest has been, if anything, less visibleto most implementation researchers: opportunities for innovationin modeling. Multilevel program arrays, especially those thatexhibit site-specific variations that seem to matter, can beapproached by altering conventional regression analysis andframing the model in hierarchical terms, to allow for interactionacross the levels of the hierarchical model. In this fashion,administrative and policy influences can be separated from clientcharacteristics, state-level determinants can be distinguished fromsite-specific features, and models can offer both greater explana-tory power and more sensible specifications for interpretingpolicy action.

Hierarchical linear modeling offers promise in explainingaspects of performance (see, for instance, Heinrich and Lynn1999 and 2000; Roderick 2000). The technique has not beenmuch used thus far to advance work on policy implementation.


at Universiteit T



ownloaded from



Two caveats might be entered. First, its success is highly depen-dent on the availability of very large data sets, ideally withindividual-level client or target information included; thus empir-ical studies will be restricted. Second, hierarchical models areobviously meant to fit hierarchically structured contexts. To theextent that program implementation arrangements are more fluidor multicentered, this form of modeling would be inappropriate.

Research beyond the United States

Most implementation research has been conducted in andregarding the United States, albeit with the goal of truly generalunderstanding. Even in earlier years, however, contributions inWestern Europe were an important part of theory building andtesting (for instance, Hull with Hjern 1987). Solid cross-nationalcomparative work has been especially lacking (for a significantexception, see Knoepfel and Weidner 1982). This problem per-sists, although it is surely not unique to implementation research.Public management more generally is sorely in need of suchinvestigations.

Some empirical studies have been conducted outside theUnited States and Western Europe (see Grindle 1980; Ross 1984;Chan et al. 1995; Brinkerhoff 1999). Investigation suggests thatthe approaches developed in Western liberal contexts may havelimited utility in other settings (see O'Toole 1994 and 1997a), butthis subject remains primarily a frontier for further investigation.

Interestingly, one of the topics on which cross-nationalcomparative implementation research can bear fruit, and has doneso to some extent already, is on the execution of internationalagreements, especially multilateral ones. The typical circumstanceis a common policy and a number of signatory countries. Hun-dreds of such agreements now present important empirical cir-cumstances for systematic study. Early scholarship offers somecross-national comparisons (see Weiss and Jacobson 1998; Vic-tor, Raustiala, and Skolnikoff 1998), but additional investigationsare clearly needed.

Third-Generation Research: Progeny?

Goggin et al. (1990, 17-18) proposed that implementationresearch take seriously the need for advancing die field scientif-ically:

The principal aim of third-generation research is to shed new light onimplementation behavior by explaining why that behavior varies across time,policies, and units of government. . . . Third generation research is

21MJ-PART, April 2000

at Universiteit T



ownloaded from



designed to overcome the conceptual and methodological problems thatmany scholars agree have impeded progress in this field. In a word, the aimof third-generation research is simply to be more scientific than the previoustwo in its approach to the study of implementation.

To that end, Goggin et al. offer details of a set of hypothesesderived from a candidate framework and ideas about the essentialissues of measurement. The third-generation argument alsoendorses the use of multiple measures and multiple methods.

As a coauthor of that study, I am hardly distanced from itsargument and I continue to endorse the general theme. But onepoint of reflection can be noted. In advocating for a third-generation perspective, we sought to catalyze sustained empiricalinvestigation of this type. Some such studies have been attempted(see, for instance, Orth 1997; Berry, Berry, and Foster 1998;Jennings and Ewalt 2000), but there has been no sustainedinterest.

Why not? Other frameworks, of course, may be moreappropriate (in this regard, see Cline forthcoming). Beyond thisobvious point, it is useful to recognize that a potentially intimi-dating standard was designed into this vision of third-generationresearch.

Goggin et al. (1990) indicate that the best kind of implemen-tation study, at least for intergovernmental programs, consists ofinvestigations that involve numerous variables and variableclusters (and multiple measures for these) across policy types(three types are included in the volume), across the fifty states,and over at least ten years (with annual observations). The argu-ment implies a design ideal of large-n studies involving 3 (poli-cies) x 50 (state) x 10 (years), or 1500 observations for everyvariable. And the measurements are best taken via a combinationof content analyses, expert panels, elite survey responses, andexpert reassessment of the data from questionnaires and inter-views. A reasonable response to this proposal would be that itoutlines a career's worth of work, perhaps more.

Absent feasibility constraints, such a project would be ofgreat help in advancing the understanding of policy implemen-tation, particularly in the United States and for intergovernmentalprograms. But proceeding directly in this fashion could imposean unrealistic set of requirements for the advancement of empir-ical research. Needless to say, executing the agenda in partialfashions, for instance by testing parts of the framework in indi-vidual efforts, is a more feasible objective.

212IJ-PART, April 2000

at Universiteit T



ownloaded from



Viewing the Half-Full Glass

The sketch offered thus far indicates a number of develop-ments that carry significance for the systematic study of policyimplementation, but it also indicates that much remains to beaccomplished. And for every important study that has beencompleted in recent years, many studies yet remain to be done.Still, this picture alone would suggest too limited a view of thestate of the field. Scholarship that bears directly on the corequestion of implementation—what happens between the establish-ment of policy and its impact in the world of action?—has arisenin many ways that would not be immediately noticeable if onewere to observe merely those investigations with a self-proclaimed focus on implementation. In fact, much has beendeveloping that should be of interest to those studying theimplementation issue, particularly with regard to theoretical fociand themes. Most of these lines of effort contribute overtly toother forms of scholarship, but they carry import more broadlyand should be considered in a complete assessment.

Analysts have noted that a considerable amount of effort hasbeen devoted recently to implementation-like questions, underother rubrics. As Winter has said, a[i]f defined in terms ofthe problems analyzed, the 'field' of implementation researchwould be considered much more robust than simply by countingresearch under the label of implementation" (1999, 2). AndMeier, in particular, has been pointed in this regard:

My biased survey of literature suggests] that a wide range of journalspublish articles that inform the study of policy implementation—the main-stream sociology journals, most of the public administration journals, theprofessions journals (public health, social work, sometimes law or medi-cine), many of the economics journals, and on rare occasion a politicalscience journal. Much of this literature is not intended to directly answerquestions of policy implementation, but it addresses concerns that are centralto policy implementation (1999, 6-7).

Where has useful work on the subject—or, more accurately,work that is of potential benefit in elucidating the subject—beentaking place? What are the promising lines of development? Whatdo these suggest about research for understanding policy imple-mentation?

INDIRECT CONTRIBUTIONSTO IMPLEMENTATION RESEARCH

Numerous kinds of scholarly inquiry that have been devel-oped in recent years offer promise for those interested in expli-cating policy implementation, particularly the multiactor variant.


at Universiteit T



ownloaded from



This section offers a sampling, with some links to the maintheme. It should be recognized, nonetheless, that these kinds ofcontributions reach considerably beyond the implementation prob-lem; indeed, Frederickson (1999) has argued that several of thesekinds of scholarship (he emphasizes the first three covered in thissection) have infused the subject of public administration moregenerally and indicate a revitalization of that field. The treatmenthere focuses on implementation per se, with the recognition thatthis limitation is somewhat arbitrary.

Institutional Analysis

Implementation research is, in important respects, heavilyreliant on institutional scholarship. Virtually all policies andprograms depend on institutional action, and the institutionalforms now increasingly common for policy implementationembrace characteristics that extend far beyond the traditionalfocus on bureaucracy or market. A few examples illustrate theimportance of this line of research for policy implementation.

Institutional Analysis and Development. The pathbreakingscholarship of Elinor Ostrom and colleagues (Ostrom, Gardner,and Walker 1994; Ostrom 1999) has reshaped a great deal ofpolicy research. Ostrom and others have sketched an approach toinstitutional analysis that is largely based in a rational-choiceperspective and a goal of understanding the emergence andimpact of variegated institutional forms.

This rule-based perspective makes it possible to sketchinstitutional details with precision and clarity. Some might arguethat the multidimensional distinctions allowed in this approachcreate overwhelming complexity that ultimately hinders parsi-monious explanation. But the approach—institutional analysis anddevelopment (IAD)—provides not only conceptual detail but ananalytical approach to develop and extend empirical theory. Anumber of promising investigations have been completed; thesefocus particularly on how stable, long-term solutions to problemsof common-pool resource management can emerge and be sus-tained (Blomquist 1992; Ostrom, Gardner, and Walker 1994;Schlager, Blomquist, and Tang 1994). There are indications thatthe approach can be extended to other settings (Cowie 1999).

IAD is by no means a relabeled version of implementationresearch. But the route that Ostrom and colleagues have takenpromises to offer benefit for the ultimate elucidation of imple-mentation action. In particular, the perspective can comprehendand analyze institutional forms of all sorts, including multiactorarrays that vary in several ways from each other, and public-


at Universiteit T



ownloaded from



private-nonprofit combinations with subtle but potentially impor-tant institutional features. Of special advantage is the IAD'sapproach of explicitly incorporating a logic of multileveled action(Riser and Ostrom 1982).

IAD takes analysts well beyond limiting top-down/bottom-updebates, at least in principle. And it does not succumb to theatheoretical tendencies observable in some bottom-up approachesthat begin with the functioning pattern of implementation andtend to regard virtually any set of operating relations as neces-sary, perhaps even optimal (as in Hjern and Hull 1983). Addinga theoretical engine creates opportunity, as well, for generatingtestable propositions about implementation behavior and results ininstitutions of widely varying forms (for an illustration seeO'Toole 1996a).

Still, there are drawbacks. Most important is the almostexclusive attention thus far to self-organizing systems. Theexclusion of official governmental programs, especially those thatincorporate regulatory or other formally authoritative approaches,constitutes an important lacuna. In particular, studies of thecomparative performance of different institutional arrangementsfor dealing with policy issues are matters of high priority (seeTang 1991). The study of multiactor policy implementation needsa theoretical approach that combines the self-organizing potentialof combinations of actors (including corporate actors) with themandated character of certain interunit links, the latter quitetypical of at least some portions of government programs(O'Toole 1993). An adjustment of IAD to include both self-organizing and mandated elements could provide considerableenlightenment for analysts of implementation.

Actor-centered Institutionalism. With Renate Mayntz, FritzScharpf has initiated a promising line of institution-based theoriz-ing about policy (Mayntz and Scharpf 1995; Scharpf 1997). Via"actor-centered institutionalism," Scharpf has combined insightsfrom game theory, welfare economics, and institutional analysisto suggest an approach to modeling policy-relevant settings ofmany types. While mostly ignoring implementation per se in thisrecent scholarship, Scharpf offers a logic and set of conceptualtools that may be able to elucidate a number of implementationcircumstances. This is particularly the case since, like Ostrom, heexplicitly models multiactor settings in which institutional formsare complex and do not readily fit the simple market-or-hierarchydesignation.

A depiction of Scharpf s full approach would take thisexposition afield. But it is worth noting that he explores the


at Universiteit T



ownloaded from



potential for game-theoretic logic, along with the linked-gameelaboration of the basic approach, to make sense of institutionalsettings in which hierarchical authority lines do not array fullythe relevant actors. Consider as an example moves by Europeanactors enmeshed in complex settings involving both country-level(domestic political) and European-level (for instance, EuropeanUnion) patterns of interdependence. Scharpf s focus on policysettings involving governmental actions differentiates thisapproach from the main emphasis thus far in IAD. He sketchespossibilities that allow for numerous modeling opportunities. Itremains to be seen to what extent this basic perspective willcatalyze significant work.

These two approaches are a mere sampling of the relevantresearch, but they suggest some of the achievement and potentialfrom institutional analysis. Theory building and testing alongthese lines is certain to inform the understanding of policyimplementation.

The Study of Governance

Rather than concentrate narrowly on implementation to theexclusion of other forms of action and other levels of influenceon the ultimate performance of public programs, some analystshave sought to consider the more comprehensive subject of howsystems of governance deliver policy-relevant impacts. Thisbroader conceptualization is not antithetical to implementationresearch; it is designed to incorporate a more complete under-standing of the multiple levels of action and kinds of variablesthat can be expected to influence performance.

The theme and perspective of governance are difficult todenote with precision. The topic is meant to refer, nonetheless, toseveral related dimensions of the contemporary policy world.One is the multivariate character of policy action: Not only domany factors influence results, but these factors are of verydifferent types and are too often examined in isolated fashion tothe relative neglect of other parts of the explanation. Another isthe commitment to treat governance as something considerablybroader than government itself: to take account, and consider thedesign and operation, of structures and processes of policyaction, wherever they might be. In this sense, the governancetheme is quite complementary to the emphasis on institutions.Third, a governance approach emphasizes the multi-layered struc-tural context of rule-governed understandings, along with the roleof multiple social actors in arrays of negotiation, implementation,and service delivery. Addressing governance requires attending tosocial partners and ideas about how to concert action among


at Universiteit T



ownloaded from



3In his published work, Weber has notfully explicated the framework on whichhis investigations rely. But in his book-length study of innovative approaches toregulatory challenges in environmentalpolicy, the instances he explored andthe model he used reference Prisoners'Dilemma circumstances. The approach—although not the specific injunctions—canprobably be generalized to a more com-prehensive set of multiactor circum-stances.

'Corporatist nations have had substantialexperience with the kinds of gamesWeber examines, but these forms ofaction—"pluralism by me rules," as hedubs them—are relative rareties thus farin the United States.

them. It should be clear, therefore, that while governance is notan old-wine-in-new-bottles version of implementation, interest inthe governance theme indicates the continued vitality of theconcerns that have flourished under the implementation rubricsince the first critiques of simplistic top-down approaches.

Some versions of policy research aimed at the theme ofgovernance are explicitly connected with the study of policyimplementation. An example is the insightful work of Stoker(1991), who has adapted the regimes framework of Stone (1989)to intergovernmental implementation and has applied a nontech-nical form of game theory to distinguish different implementationcircumstances—especially those that involve more or less intrac-table forms of conflict among interdependent actors. Stoker'swork suggests not only that it may be possible to combine top-down and bottom-up insights in a coherent regime-basedapproach, but also that practical—and in some cases counter-intuitive—injunctions can follow from such an analysis. Impor-tantly, Stoker's implementation research is explicitly integratedinto a perspective that sees implementation questions as an aspectof the broader governance theme.

Another instance, less exclusively focused on implementa-tion per se, is the recent scholarship of Edward Weber (1998).Weber too uses an implicitly game-theoretic analytical frame-work3 and explores the requisites for cooperative approachesto environmental problem solving. While he is appropriatelyrestrained about the prospects for sustained collaboration, Weberexplicates the institutional and other preconditions for thisapproach to policy action—in effect, thereby, suggesting thedimensions of a relatively untested but potentially promisingregime form for decision making and implementation in theUnited States.4 Weber's work is an example of current, theo-retically informed, policy-focused work, which has roots in thetheme of governance and direct implications for the study andpractice of policy implementation. It also links directly to othersignificant topics that are prominent in the best current work,including a combination of top-down and bottom-up insights, theuse of analytical tools with the potential to explicate multiactorsettings, and an effort to include relevant variables that reachbeyond the scope of earlier policy investigations.

Others also are pursuing this theme in ways that are relevantto questions of implementation, even when they do not explicitlytrumpet the latter in fashions that may have caught widespreadattention. An important instance of this form of governanceresearch is the work of Lynn and colleagues (Lynn, Heinrich,and Hill 1999 and 2000). In broad outline, their project

m/J-PART, April 2000

at Universiteit T



ownloaded from



constitutes an effort to synthesize influences on policy perform-ance of several sorts, and from several levels, by taking accountof the standard concerns of implementation researchers andintegrating these with other kinds of related analyses.

Their conceptualization treats programs as units of analysisand suggests that outputs be considered a function of an array offactors (elements of the environment, treatments or programtechnologies, client or target characteristics, structure, andmanagement), some of which have received treatment in imple-mentation research over the years. Lynn, Heinrich, and Hillexpand upon standard implementation research, furthermore, byincorporating additional considerations (for instance, frominstitutional analysis and from public management) in their con-ceptualization of what drives program performance. They offer arather comprehensive assessment of many relevant literatures-including an assessment of which studies explore which sets ofthese variables, how they are measured, and what the findingshave been. Of particular note, in connection with the governancetheme and the study of policy implementation, is that theseresearchers focus especially on studies that offer investigations oftwo or more of these types of variables and thus suggest a morecomplex array of models for explaining public program perform-ance. They also identify some particularly interesting lines ofanalysis for further development.

Lynn and colleagues have not organized an agenda forimplementation research in the narrow. Instead, they havesuggested ways to link such work with additional traditions ofresearch to expand the overall understanding of what makesprograms work. Some of the current work that is being devel-oped on governance, therefore, offers the prospect of building onearlier implementation work in especially promising ways.

Other analyses could equally well be included—either hereor as part of the elucidation of closely related topics, such as thetreatment of networks (below). One instance is the recent contri-bution of Bardach (1998). Bardach, one of the influential earlyimplementation analysts, continues to eschew general theory infavor of practice-oriented, metaphorical treatments of importantissues (see Bardach 1977). In his recent work, he deals withcollaboratives of two or more units and seeks to understand whataspects of the development of inter-institutional linkages make forsuccessful joint effort. Once again, implementation per se hasmoved to the background, in favor of attention to concertedaction across institutional boundaries on behalf of public purpose.But the practical management issues overlap with the manage-ment challenges that are evident in many implementation studies.


at Universiteit T



ownloaded from



And some of the themes—like the importance of building trustfor collaborative success—are also analyzed in some of theimplementation research of recent years (for instance, O'Toole1996b).

Additional recent research has emphasized governance ratherthan management in ways that are largely consistent with theforegoing exposition. Consider some of the contributions ofMilward and Provan (especially 1999). As they put the issue,"governance . . . is concerned with creating the conditions forordered rules and collective action, often including agents in theprivate and nonprofit sectors, as well as within the public sector.The essence of governance is its focus on governing mechan-isms—grants, contracts, agreements—that do not rest solely onthe authority and sanctions of government" (p. 3).

Milward and Provan bring into clear relief an uncomfortablepoint that remains a challenge for researchers on such questions,regardless of the explicit theme: There is a "paucity of empiricalliterature on the relationship between governance structure andoutcomes" (p. 24). While they have been in the forefront ofrelating certain characteristics of governance forms to overalloutcomes, however measured (1999, 24-25; Provan and Milward1995), the variety of arrangements embraced by the governancenotion defies parsimonious theory building. Thus this problem,which has bedeviled implementation research for much of the lastfifteen years or so, is likely to reemerge under the governancerubric. (The same point can be made regarding research on net-works, an overlapping topic to which Milward and Provan alsocontribute in important ways; see below.)

Networks and Network Management

The debate between top-downers and bottom-uppers centeredin part on the question of the appropriate unit of analysis (forinstance, implementing agency vs. program and interagencyarray). Behind these arguments is an important question aboutinstitutional arrangements: What are the forms through whichmost implementation action develops? Great variety is evident.But the emergence of increasingly complex structural forms,including multiactor networked patterns, has made especiallysalient the questions of research design, conceptualization, andtheory building for networked policy action.

Partially as an independent intellectual effort, but partiallyas well in response to this debate within implementation research,the study of networks and network management has acquired con-siderable significance. Indeed, this subject has attracted growing

279/ J-PART, April 2000

at Universiteit T



ownloaded from



attention in a number of other countries as well (Hufen andRingeling 1990; Marin and Mayntz 1991; Jordan and Schubert1992; Marsh and Rhodes 1992; Scharpf 1993; Bressers, O'Toole,and Richardson 199S; Klijn 1996; Kickert, Klijn, and Koppenjan1997; Bogason and Toonen 1998). Even in theoretical develop-ments in the United States that overtly eschew an implementationfocus, versions of network analysis have occupied an importantrole (see Sabatier and Jenkins-Smith 1993).

There are compelling reasons, both empirical and theoret-ical, for scholars to treat seriously the network theme (O'Toole1997b). Some have argued that network approaches should beamong the most important analytical heuristics to be applied inunderstanding and improving public management (Lynn 1996).Others have claimed that in the current era, shifts toward massivequantities of contracting have triggered the formation of a hollowstate, with the management of public programs largely consistingof the monitoring and crafting of network forms (Milward 19%).Scholars in other countries place heavy emphasis on the net-worked character of policy implementation itself (Kickert, Klijn,and Koppenjan 1997), and cross-national comparative policystudies offer suggestive prospects (Bressers, O'Toole, andRichardson 1995).

Much of my recent work has been targeted at this set ofthemes—and this focus is a direct outgrowth of tackling imple-mentation questions (O'Toole 1993; 1995; 1996a; 1997a). Thus,for instance, I have sought to provide some systematic informa-tion about the networked character of public programs enacted atthe national level (Hall and O'Toole 2000) and in other countries(for instance, O'Toole 1998). Meier and I are at work modelingand testing for the impact of both management and structuralcontext on the operations of public programs; we explicitly treatthe extent of hierarchy or network as structural features amenableto analysis (for the initial effort see O'Toole and Meier 1999).Others have been busy along related lines. Agranoff andMcGuire, for instance, have provided convincing evidence on thedescriptive portion of the question, particularly as regards inter-governmental management (1998). And the emphasis by Provanand Milward on the link between network features and perform-ance, as well as on a range of effectiveness criteria, speaks to themore complex agenda entailed in networked forms of policyaction (1995; Milward and Provan 1999). Additional instancescould be documented at length (see LaPorte 1996 for furtherexposition).

A great deal of this research is closely related to work onimplementation. In important respects, the upsurge in network


at Universiteit T



ownloaded from



studies represents a continuation of some of the same lines ofresearch that became visible in some of the best implementationresearch a few years earlier. This is not to say that a theoreticalconsensus has emerged on how to conceptualize and model net-worked phenomena surrounding policy action. Indeed, differentapproaches compete for attention (see Milward and Provan1999). And there is some disagreement about the potentialapplicability of formal approaches (O'Toole 1993), or even thegeneral theory-building project itself (Bardach 1998).

The study of networks and public management, however,draws from promising theoretical streams with questions ofimplementation—performance via governance in the delivery ofpolicy results—as significant as ever. Further, the networkscholarship in public management also overlaps that on govern-ance and on institutional analysis. The several emergingapproaches promise considerable advance in the years ahead,even if not a clear theoretical synthesis or convergence.

Formal and Deductive Approaches

The implementation theory that developed in an explosion ofresearch effort from the 1970s through the 1980s was almostexclusively inductive, and it was characterized by a profusion ofvariables. But some efforts have been initiated to frame a moreclearly deductive and parsimonious approach.

Formal, rational-choice approaches like game theory cannotbe reviewed here, even in a cursory fashion, but implementationresearchers have seen both promising and limiting prospects.Heuristic applications have been offered by a number of scholars(Stoker 1991; Koremenos and Lynn 1996; Weber 1998). Such aline of inquiry can be useful, despite limitations, in exploring aset of persistently important implementation questions (O'Toole1993). The limitations can, somewhat paradoxically, suggestpractical options for public managers who are enmeshed in themidst of networks for implementation: Identifying limitations tomodeling can expose points of leverage for implementation man-agers (O'Toole 1995).

Additional relevant theoretical developments, thus farinsufficiently noted among implementation scholars in the UnitedStates, also have appeared. Some formal approaches haveresulted in modeling that appears to be both plausible and amen-able to testing (see especially Torenvlied 1996a and 1996b). Arelatively parsimonious, deductive theory has been developed inthe Netherlands, primarily by Hans Bressers and Pieter-Jan Klok.Dubbed—perhaps misleadingly—instrument theory, the approach


at Universiteit T



ownloaded from



takes inspiration from top-down/ bottom-up synthesis and offersthe prospect of a predictive model focused on the interactionsamong policy actors. Instrument theory builds on a core of asmall number of variables to sketch an impressively varied rangeof possible implementation results. The theory, particularly in itslater elaborations, has been extended to multiactor, networkedsettings with a range of target group possibilities (see Bressersand Klok 1988; Klok 1995; Bressers and O'Toole 1998). Thisapproach is synthetic and suggestive in another fashion, since itaims to link the theory of policy design with that of implementa-tion, thus addressing a key criticism directed at the field for anumber of years (for instance, Under and Peters 1987). Unfor-tunately, the most complete expositions of the approach remainavailable in the Dutch language only (Klok 1991), although somekey parts are also published in English (Bressers and Klok 1988).The full utility of the approach remains to be tested widely.

PROSPECTS

There are those who would argue that for all the conceptual,theoretical, and methodological effort, the implementationresearch enterprise remains stuck in neutral, or is running in acircle. DeLeon's challenge (1999b) is perhaps the most reason-ably framed in this regard (more so, for instance, than Fox1990). And his critique is direct:

First, implementation as currently structured is winning no conceptual con-verts. . . . And, second, its focus on the exceptional failure to the preclusionof the workaday successes have [sic] largely been ingrained, thus biasing theimplementation research product. . . .

The combination of a greater emphasis on a democratic orientation toimplementation, buttressed by more of a post-positivist orientation andmethodology and a realistic assessment of what implementation can deliver(as opposed to promise) will . . . give the policy community a much betterhandle on the linchpin stage that delivers the policy goods.

As should be clear, the present assessment is considerablymore optimistic about the current state of the research enterprise.On the issue of converts, the situation is better than portrayed bydeLeon: a number of promising scholars have contributed inrecent years; and when one frames implementation questionsbroadly, to include research on complex institutional arrange-ments, networks, governance, and the like, some of the supposeddefectors, like Bardach, can be seen as contributors. Second, thesupposed bias of implementation research toward the cases offailure is an outdated generalization. Even decades ago scholarstook pains to avoid such a bias toward failure in the selection ofcases for investigation (Montjoy and O'Toole 1979; O'Toole and

2S2/J-PART, April 2000

at Universiteit T



ownloaded from



Montjoy 1984). Cross-state larger-n studies have substantiallyreinforced this effort (for instance, Meier and McFariane 1995),and the third-generation work of Goggin et al. (1990) was explic-itly designed to avoid any trace of preoccupation with the"exceptional failure." Rather, the idea was to select multiplepolicies in multiple sites over extended time periods, to maximizewithin reasonable limits the amount of variance subject to expla-nation. Executing this full agenda has its difficulties, as has beenexplained, but such a perspective is clearly a far cry from theexcerpted indictment.

Has a democratic orientation been absent from implementa-tion studies until now? A reasonable case can be made that thisis not so at all. Top-down and bottom-up investigations areanimated, to be sure, by quite different notions of democracy—top-downers offer justification in conventional overhead-demo-cratic tenets, bottom-uppers focus on more interactive processes,client involvement, and coproduction. Searching for an imple-mentation approach built around a normative core of discursivedemocracy would generate a fascinating scholarly agenda, but itis not the case that an interest in democratic theory has simplybeen ignored until now.

More basically, the study of policy implementation is inmany respects in a relatively mature stage of development.Weaknesses are apparent, issues remain, and some of the mostinteresting relevant work is taking place on the edges of thespecialty or in related research fields. But these developmentspoint to an appropriate broadening rather than a shrinking of therelevant research enterprise. It behooves scholars not to drawarbitrarily narrow jurisdictional lines, nor to expend energy onsectarian causes. Explaining—and ultimately improving—the waypolicy intention influences policy action is the research agenda,by whatever name. Prospects for grappling with this importantsubject are both multiple and engaging.

As for implementation research: the top-down/bottom-updebates are ended, superceded by general recognition of thestrengths of each. Synthetic theoretical efforts have beennumerous. Inductive approaches have been supplemented by thebeginnings of deductive and formal analysis. Larger-n studies,while not plentiful, are considerably less rare than is typicallyportrayed. Certain tendentious measurement and methodologicalissues have been joined, indeed in some cases addressed. And,most significantly, a range of complementary research initiativesis now underway, and these initiatives offer prospects for shed-ding new light on the implementation question as they addressrelated challenges.

2&3/J-PART, April 2000

at Universiteit T



ownloaded from



Plenty of work remains. But the prospects for the scholarlystudy of—perhaps even the improvement of—policy performanceare reasonably bright. The study of implementation cannot pos-sibly provide the full set of relevant answers. But research onimplementation, under whatever currently fashionable labels, isalive and lively.

REFERENCES

Agranoff, Robert, and McGuire,Michael.1998 "Multinetwork Management:

Collaboration and the HollowState in Local EconomicPolicy." Journal of PublicAdministration Research andTheory 8:1:67-91.

Bardach, Eugene.1977 The Implementation Game. Cam-

bridge, Mass.: MIT Press.1998 Getting Agencies to Work

Together. Washington, D.C.:Broo Icings.

Berry, Frances Stokes; Berry,William D.; and Foster, Stephen K.1998 "The Determinants of Success in

Implementing an Expert Systemin State Government." PublicAdministration Review 58:4:293-305.

Blomquist, William.1992 Dividing the Waters: Governing

Groundwater in Southern Cali-fornia. San Francisco: Institutefor Contemporary Studies Press.

Bogason, Peter, and Toonen,Theo A.J., eds.1998 "Comparing Networks." Sym-

posium in Public Administration76:2:205-407.

Bressers, Hans, and Klok, Pieter-Jan.1988 "Fundamentals for a Theory of

Policy Instruments." Inter-national Journal of SocialEconomics 15:3/4:22-41.

Bressers, Hans Th.A., and O'Toole,Laurence J. Jr.1998 "The Selection of Policy

Instruments: A Network-Based

Perspective." Journal of PublicPolicy 18:3:213-39.

Bressers, Hans; O'Toole, Laurence J.Jr.; and Richardson, Jeremy, eds.1995 Networks for Water Policy: A

Comparative Perspective. Lon-don: Frank Cass.

Bressers, J.Th.A., and Ringeling,Arthur B.1989 "Beleidsinstrumenten in drie

arena's." Beleidswetenschap3:1:3-24.

Brinkerhoff. Derick W.1999 "State-CivD Society Networks

for Policy Implementation inDeveloping Countries." PolicyStudies Review 16:1:123-47.

Brodkin, Evelyn.1997 "Inside the Welfare Contract:

Discretion and Accountability inState Welfare Administration.*Social Service Review 71(March): 1-33.

Brown, Mary Maureen; O'Toole,Laurence J. Jr.; and Brudney,Jeffrey L.1998 "Implementing Information

Technology in Government: AnEmpirical Assessment of theRole of Local Partnerships."Journal of Public AdministrationResearch and Theory 8:4:499-525.

Chan, Hon S.; Wong, Koon-kwai;Cheung, K.C.; and Lo, Jack Man-keung.1995 "The Implementation Gap in

Environmental Management inChina: The Case of Guangzhou,Zhungshou, and Nanjing."

Public Administration Review55:4:333-40.

Cline, Kurt D.forthcoming "Defining the Implemen-

tation Problem: OrganizationalManagement versus Coopera-tion." Journal of Public Admin-istration Research and Theory.

Cowie, Gail M.1999 "Stakeholder Involvement and

Institutional Change: Cases ofInterstate River Basin Manage-ment." DPA diss. The Univer-sity of Georgia, Athens.

deLeon, Peter.1999a "Cold Comfort Indeed: A Re-

joinder to Lester and Goggin."Policy Currents 8:4:6-8.

1999b "The Missing Link Revisited:Contemporary ImplementationResearch." Policy StudiesReview 16:3/4:311-38.

Fox, Charles J.1990 "Implementation Research: Why

and How to Transcend PositivistMethodologies." In Dennis J.Palumbo and Donald J. Calista,eds. Implementation and thePolicy Process: Opening up theBlack Box, 199-212. Westport,Conn.: Greenwood.

Frederickson, H .-George.1999 "The Repositioning of American

Public Administration." GausLecture, annual meeting of theAmerican Political Science Asso-ciation, Atlanta, September 3.

Gerstein, Dean R., and others.1997 Final Report: National Treat-

ment Improvement EvaluationStudy. Washington, D.C.: U.S.

2M/J-PART, April 2000

at Universiteit T



ownloaded from



Department of Health andHuman Services, March.

Glaser, Bonnie, and MacDonaM, Karin.1998 "On die Front Lines of Welfare

Reform: Are Workers Imple-menting Policy Reform?" Jour-nal of Policy Analysis andManagement 17:1:1-22.

Goggin, Malcolm L.1986 "The 'Too Few Cases/Too

Many Variables' Problem inImplementation Research."Western Political Quarterly38:328-47.

Goggin, Malcolm L.; Bowman, AnnO'M.; Lester, James P.; and O'Toole,Laurence J. Jr.1990 Implementation Theory and

Practice: Toward a Third Gen-eration. Glenwood, 111.: ScottForesman/Little, Brown.

Grindle. Merilee S., ed.1980 Politics and Policy Implemen-

tation in the Third World.Princeton, N.J.: PrincetonUniversity Press.

Hall, Thad E., and O'Toole,Laurence J. Jr.2000 "Structures for Policy Imple-

mentation: An Analysis ofNational Legislation, 1965-66and 1993-94." Administrationand Society 31:6:667-86.

Heinrich, Carolyn J., and Lynn,Laurence E. Jr.1999 "Means and Ends: A Compara-

tive Study of Empirical Methodsfor Investigating Governance andPerformance." Paper preparedfor the Fifth National PublicManagement Research Confer-ence, Texas A&M University,College Station, December 3-4.

2000 "Governance and Performance:The Influence of Program Struc-ture and Management on JobTraining Partnership Act (JTPA)Program Outcomes." In Lynnand Heinrich, eds. Models andMethods in the Empirical Studyof Governance. Washington,D.C.: Georgetown UniversityPress.

Hjem, Benny.1982 "Implementation Research—The

Link Gone Missing.* Journal ofPublic Policy 2:301-08.

Hjem, Benny, and Hull, Chris.1983 "Implementation Research as

Empirical Constitutionalism.*European Journal of PoliticalResearch 10:2:105-15.

Hjem, Benny, and Porter, David O.1981 "Implementation Structures: A

New Unit of AdministrativeAnalysis." Organization Studies2:3:211-27.

Hufen, Hans, and Ringeling,Arthur, eds.1990 Beleidsnetwerken: Overheads-.

Semi-overheids-en Particuliere. Organisaties in Wisselwerking.• The Hague: VUGA.

Hull, Chris, with Hjem, Benny.1987 Helping Small Firms Grow.

London: Croom Helm.

Jennings, Edward T. Jr., and Ewalt,Jo Ann G.1998 "Interorganizational Coordina-

tion, Administrative Consolida-tion, and Policy Performance."Public Administration Review58:5:417-26.

2000 "Driving Caseloads Down: Wel-fare Policy Choices and Admin-istrative Action in-the States." InLaurence E. Lynn, Jr., andCarolyn J. Heinrich, eds.Models and Methods for theEmpirical Study of Governance.Washington, D.C.: GeorgetownUniversity Press.

Jordan, Grant, and Schubert, K.1992 "A Preliminary Ordering of

Policy Network Labels." Euro-pean Journal of PoliticalResearch 21:1-2:7-27.

Red, Donald F.1990 "The Perils—and Prospects—of

Public Administration." PublicAdministration Review 50:4:411-19.

1993 "Searching for Clues aboutPublic Management: Slicing dieOnion Different Ways." InBarry Bozeman, ed. Public

Management: The State of theArt, 55-68. San Francisco:Jossey-Bass.

Kkkert, Walter J.M.; Hijn, Erik-Hans;and Koppenjan, Joop P.M., eds.1997 Managing Complex Networks:

Strategies for the Public Sector.London: SAGE.

Kiser, Larry L., and Ostrom, Elinor.1982 "The Three Worlds of Action: A

Metatheoretical Synthesis ofInstitutional Approaches." InElinor Ostrom, ed. Strategies ofPolitical Inquiry, 179-222.Beverly Hills, Calif.: SAGE.

Khjn, Erik-Hans.1996 "Analyzing and Managing

Policy Processes in ComplexNetworks: A Theoretical Exam-ination of the Concept PolicyNetwork and Its Problems.Administration and Society28:1:90-119.

Hok, Pieter-Jan.1991 Een instrumententheorie voor

milieubeleid. Enschede, TheNetherlands: University ofTwente Press.

1995 "A Classification of Instrumentsfor Environmental Policy." InBruno Dentc, ed. EnvironmentalPolicy in Search of New Instru-ments, 21-36. Dordrecht, TheNetherlands: Kluwer.

Knoepfel, Peter, and Weidner, Helmut.1982 "Implementing Air Quality Con-

trol Programs in Europe."Policy Studies Journal 11(Sept.): 104-15.

Koremenos, Barbara, and Lynn,Laurence E. Jr.19% "Leadership of a State Agency."

In Donald F. Keal and H. Brin-ton Milward, eds. The State ofPublic Management, 213-40.Baltimore: Johns Hopkins Uni-versity Press.

LaPorte, Todd R.1996 "Shifting Vantage and Concep-

tual Puzzles in UnderstandingPublic Organizational Net-works." Journal of Public

285/ J-PART, April 2000

at Universiteit T



ownloaded from



Administration Research andTheory 6:1:49-74.

Lester, James P., and Bowman,AnnO'M.1989 "Implementing Intergovern-

mental Policy: A Test of theSabatier-Mazmanian Model."Polity 21:731-53.

Lester, James P., and Goggin,Malcolm L.1998 "Back to the Future: The Redis-

covery of Implementation Stud-ies." Policy Currents 8:3:1-9.

Lin, Ann Chih.1998 "Building Positivist and Inter-

pretivist Approaches to Qualita-tive Methods." Policy StudiesJournal 25:1:162-80.

2000 Reform in the Making: TheImplementation of Social Policyin Prison. Princeton, N.J.:Princeton University Press.

Linder, Stephen H., and Peters,B. Guy.1987 "A Design Perspective on Policy

Implementation: The Fallacies ofMisplaced Prescription." PolicyStudies Review 6 (Feb.):459-75.

Lynn, Laurence E. Jr.1996 Public Management as Art, Sci-

ence, and Profession. Chatham,N.J.: Chatham House.

Lynn, Laurence E. Jr.; Heinrich,Carolyn J.; and Hill, Carolyn J.1999 "The Empirical Study of Gov-

ernance: Theories, Models,Methods." Presented at theWorkshop on Models and Meth-ods for the Empirical Study ofGovernance, University of Ari-zona, Tucson, April 29-May 1.

2000 The Empirical Study of Govern-ance: Theories, Models, andMethods. Washington, D.C.:Georgetown University Press.

Marin, Bemd, and Mayntz,Renate, eds.1991 Policy Networks: Empirical Evi-

dence and Theoretical Consid-erations. Boulder, Colo.: West-view.

Marsh, D., and Rhodes, R.A.W., eds.1992 Policy Networks in British

Government. Oxford: ClarendonPress.

Madand, Richard E.1995 "Synthesizing the Implementa-

tion Literature: The Ambiguity-Conflict Model of Policy Imple-mentation." Journal of PublicAdministration Research andTheory 5:2:145-74.

Mayntz, Renate, and Scharpf, Fritz W.1995 "Der Ansatz des akteurzen-

trierten Institutionalismus."In Mayntz and Scharpf, eds.Steuenmg und Selbstorganisationin staatsnahen Sektoren, 39-72.Frankfurt am Main: Campus.

Mazmanian, Daniel A., and Sabatier,Paul A.1989 Implementation and Public

Policy, rev. ed., 1-anham Md.:University Press of America.

McFarlane, Deborah R.1989 "Testing the Statutory Coherence

Hypothesis: Implementation ofFamily Planning Policy in theStates." Administration andSociety 20:4:395^22.

Meier, Kenneth J.1999 "Are We Sure LassweU Did It

This Way? Lester, Goggin andImplementation Research."Policy Currents 9:1:5-8.

Meier, Kenneth J., and Gill, Jeff.forthcoming Substantively Weighted

Analytical Techniques: A NewApproach to Program and PolicyAnalysis. Boulder, Colo.:Westview.

Meier, Kenneth J., and Keiser, Lael R.1996 "Public Administration as a

Science of the Artificial: AMethodology for Prescription."Public Administration Review56:459-66.

Meier, Kenneth J., and McFarlane,Deborah R.1995 "Statutory Coherence and Policy

Implementation: The Case ofFamily Planning." Journal ofPublic Policy 15:3:281-301.

Meyers, Marcia; Glaser, Bonnie; andMac Donald, Karin.1998 "On the Front Lines of Welfare

Reform: Are Workers Imple-menting Policy Reforms?" Jour-nal of Policy Analysis andManagement 17:1:1-22.

Milward, H. Brinton, ed.1996 Symposium on "The Hollow

State: Capacity, Control, andPerformance in Interorganiza-tional Settings." Journal ofPublic Administration Researchand Theory 6:2:193-313.

Milward, H. Brinton, and Provan,Keith G.1999 "How Networks Are Gov-

erned." Unpublished paper.

Montjoy, Robert S., and O'Toole,Laurence J. Jr.1979 "Toward a Theory of Policy

Implementation: AnOrganizational Perspective."Public Administration Review39:5:465-76.

Onh, Caroline M.1997 "State Implementation of Air

and Water Pollution ControlPolicy: An Analysis of StateEcological Capacity." Paper pre-sented at the annual meeting ofthe Midwest Political ScienceAssociation, Chicago, April 10-12.

Ostrom, Elinor.1999 "Institutional Rational Choice:

An Assessment of the Institu-tional Analysis and DevelopmentFramework." In Paul A. Saba-tier, ed. Theories of the PolicyProcess, 35-71. Boulder, Colo.:Westview.

Ostrom, Elinor; Gardner, Roy; andWalker, James.1994 Rules, Games, and Common-

Pool Resources. Ann ArborUniversity of Michigan Press.

O'Toole, Laurence J. Jr.1986 "Policy Recommendations for

Multi-Actor Implementation: AnAssessment of the Field." Jour-nal of Public Policy 6:2:181-210.


at Universiteit T



ownloaded from



1989 "Alternative Mechanisms forMultiorganizational Implemen-tation: The Case of WastewaterManagement." Administrationand Society 21:3:313-39.

1993 "Multiorganizational PolicyImplementation: Some Limita-tions and Possibilities forRational-Choice Contributions."In Scharpf, ed., 27-64.

1994 "Economic Transition, Constitu-tional Choice, and PublicAdministration: ImplementingPrivatization in Hungary." Jour-nal of Public AdministrationResearch and Theory 4:4:493-519.

1995 "Rational Choice and PolicyImplementation." AmericanReview of Public Administration25:1:43-57.

1996a "Hollowing the Infrastructure:Revolving Loan Programs andNetwork Dynamics in the Amer-ican States." Journal of PublicAdministration Research andTheory 6:2:225-42.

1996b "Implementing PublicPrograms." In James L. Perry,ed. Handbook of Public Adminis-tration, 2d ed., 150-62. SanFrancisco: Jossey-Bass.

1997a "Networking Requirements,Institutional Capacity, andImplementation Gaps in Transi-tional Regimes: The Case ofAcidification Policy in Hun-gary." Journal of EuropeanPublic Policy 4:1:1-17.

1997b Treating Networks Seriously:Practical and Research-BasedAgendas in Public Administra-tion." Public AdministrationReview 57:1:45-52.

1998 Institutions, Policy, and Outputsfor Acidification: The Case ofHungary. Aldershot, UK: Ash-gate.

O'Toole, Laurence J. Jr., and Meier,Kenneth J.1999 "Modeling the Impact of Public

Management: Implications ofStructural Context," Journal ofPublic Administration Researchand Theory 9:4:505-26.

O'Toole, Laurence J. Jr., and Monnoy,Robert S.1984 "Interorganizational Policy

Implementation: A TheoreticalPerspective." Public Adminis-tration Review 44:6:491-503.

Pressman, Jeffrey L., and Wildavsky,Aaron.1984 [1973] Implementation.

Berkeley: University ofCalifornia Press.

Provan, Keith G., and Milward,H. Brinton.1995 "A Preliminary Theory of Net-

work Effectiveness: A Compara-tive Study of Four MentalHealth Systems." AdministrativeScience Quarterty 40:1:1-33.

Roderick, Melissa.2000 "Evaluating Chicago's Efforts

to End Social Promotion." InLaurence E. Lynn Jr. and Caro-lyn J. Heinrich, eds. Models andMethods for the Empirical Studyof Governance. Washington,D.C.: Georgetown UniversityPress.

Ross, Lester.1984 "The Implementation of

Environmental Policy in China:A Comparative Perspective."Administration and Society15:4:489-516.

Ryan, N.1996 "Some Advantages of an Inte-

grated Approach to Implemen-tation Analysis." PublicAdministration 74:737-53.

Sabatier, Paul A.1991 "Toward Better Theories of the

Policy Process." PS 24:2:147-56.

Sabatier, Paul A., and Jenkins-Smith,Hank C , eds.1993 Policy Change and Learning: An

Advocacy Coalition Approach.Boulder, Colo.: Westview.

Sandfort, Jodi.1997 "Peering into the Black Box: A

Study of the Front-Line Organi-zations Implementing Welfare

Policy in Michigan." Ph.D. diss.University of Michigan.

Scharpf, Fritz W., ed.1993 Games in Hierarchies and Net-

works: Analytical and EmpiricalApproaches to the Study ofGovernmental Institutions.Boulder, Colo.: Westview.

1997 Games Real Actors Play: Actor-Centered Institutionalism inPolicy Research. Boulder, Colo.:Westview.

Scheirer, Mary Ann, and Griffith,James.1990 "Studying the Dynamics of

Public Policy Implementation:A Third-Generation Approach."In Dennis J. Palumbo andDonald J. Calisti, eds. Imple-mentation and the Policy Pro-cess: Opening up the Black Box,163-179. New York: Green-wood.

Schlager, Edella; Blomquist, William;and Tang, Shui Yan.1994 "Mobile Flows, Storage, and

Self-Organized Institutionsfor Governing Common-PoolResources." Land Economic!70:3:294-317.

Schneider, Anne L.1999 "Terminator! Who, Me? Some

Thoughts about the Study ofPolicy Implementation." PolicyCurrents 9:1:1-5.

Stoker, Robert P.1991 Reluctant Partners: Implement-

ing Federal Policy. Pittsburgh:University of Pittsburgh Press.

Stone, Clarence.1989 Regime Politics: Governing

Atlanta 1946-1988. Lawrence:University Press of Kansas.

Tang, Shui Yan.1991 "Institutional Arrangements and

the Management of Common-Pool Resources." Public Admin-istraton Review 51:1:42-51.

Torenvlied, Rene\1996a Besluiten in uitvoering. Gronin-

gen. The Netherlands: Univer-sity of Groningen Press.


at Universiteit T



ownloaded from



1996b "Political Control of Implemen-tation Agencies: Effects of Polit-ical Consensus on Agency Com-pliance." Rationality and Society8:1:25-56.

Victor, David; Raustiala, Kal; andSkolnikoff, Eugene B., eds.1998. The Implementation and Effec-

tiveness of InternationalEnvironmental Commitments.Cambridge, Mass.: MIT Press.

Weber, Edward P.1998 Pluralism by the Rules: Conflict

and Cooperation in Environ-mental Regulation. Washington,D.C.: Georgetown UniversityPress.

Weiss, Edith Brown, and Jacobson,Harold K., eds.1998 Engaging Countries: Strengthen-

ing Compliance with Inter-national Environmental Accords.Cambridge, Mass.: MIT Press.

Winter, Seren.1999 "New Directions for Implemen-

tation Research." Policy Cur-rents 8:4:1-5.

2&S/J-PART, April 2000

at Universiteit T



ownloaded from


Documents

Research on Policy Implementation: Assessment and ProspectsPublic+Adm+R… · Research on Policy Implementation: Assessment and Prospects Laurence J. O'Toole Jr. University of Georgia