Performance Measurement in Public Agencies: The Law Enforcement Evolution

Performance Measurement in Public Agencies: The Law Enforcement EvolutionAuthor(s): Edwin W. ZedlewskiSource: Public Administration Review, Vol. 39, No. 5 (Sep. - Oct., 1979), pp. 488-493Published by: Wiley on behalf of the American Society for Public AdministrationStable URL: http://www.jstor.org/stable/3109924 .

Accessed: 15/06/2014 04:43

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

.

Wiley and American Society for Public Administration are collaborating with JSTOR to digitize, preserve andextend access to Public Administration Review.

http://www.jstor.org

This content downloaded from 185.44.77.38 on Sun, 15 Jun 2014 04:43:24 AMAll use subject to JSTOR Terms and Conditions

488 PUBLIC ADMINISTRATION REVIEW

extinction of certain paper records, the user should learn this before its installation. Living with this knowledge a few months in advance of its becoming reality might soften the blow and facilitate a less painful user adjustment. On the other hand, an accommodation to the user might be the result, so that both needs (automation and paper files) might be supported in the MIS design. Failure to try to support the user will likely result in failure of the user to support the project.

Project Management

The final role is that of management, a most common role for the generalist. Despite our conditioned associa- tion of the term "project management" with technology and technical experts, the activity need not vary significantly from routine management. Two major differences between functional and project management are the life expectancy of the management organization and the scope of management responsibilities. Project management generally forms a management organization whose lifetime is limited to the life of the project. Functional management teams, on the other hand, expect to survive as long as the organization they serve. Project management involves a systems-wide approach that considers all functions of the project, including external environmental impact, while functional manage- ment considers only a limited functional scope.

Project management involves all the traditional func- tions of management: planning, organizing, staffing, directing, coordinating, and evaluating. The fact that the project may be of a technical nature is not sufficient reason for it to require technical management in all phases. Is it assumed that every chief executive officer must know the details of every aspect of his organiza-

tion? If it were, we would have few CEO's in the world today. Similarly, the major functions of project manage- ment can be directed by a non-technical generalist.

Summary

Organizations contemplating the installation of an MIS should carefully examine the assumptions behind their allocation of human resources to the project. In particular, the assumption that the MIS should be assigned to the computer center should be replaced with a rational analysis of the objectives of the MIS and its implementers. Frequently, the association of MIS with computer technology leads one to the questionable conclusion that technical experts are the logical people to carry out the project. The nature of the MIS is sufficiently different from more conventional computer applications that technical expertise, that is, cognitive patterns developed through repeated installations of single applications, may actually form a barrier to its successful development. More careful consideration of the perspectives, talents, and loyalties of potential MIS implementers might result in more effective placement of individuals in the MIS organization.

Notes

1. Walter J. Kennevan, "MIS Universe," Data Management, September, 1970.

2. Wendell L. French and Cecil H. Bell, Jr., Organization Development (Prentice-Hall, Inc., 1973), p. 3.

3. James Martin, Principles of Data-Base Management (Prentice- Hall, Inc., 1976), p. 36.

P ERFORMANCE MEASUREMENT IN PUBLIC AGENCIES: THE LAW ENFORCEMENT EVOLUTION*

Edwin W. Zedlewski, National Institute of Law Enforcement and Criminal Justice

Public sector agencies have adopted or adapted many planning and analysis techniques from industry in order to enhance their ability to manage operations. PERT, MBO, and PPBS are just a few of the acronyms that have become part of governmental jargon. Recent additions like sunset legislation, zero-base budgeting, and tax and

*The author wishes to thank anonymous referees for their helpful suggestions in revising an earlier version of this article. Special thanks are owed to James M. H. Gregg for his comments and encouragement.

expenditure limits indicate a genuine concern in the public sector with performance. All of these techniques have steered agencies toward quantitative management.

Edwin W. Zedlewski is an operations research analyst and staff manager of the performance measurement program at the National Institute of Law Enforcement and Criminal Justice, LEAA. Previously he served in a number of management and research positions at the Center for Naval Analyses, primarily in resource analysis and strategic planning. He is also conducting research at the Institute on the interactions between public and private efforts in crime prevention.

SEPTEMBER/OCTOBER 1979

This content downloaded from 185.44.77.38 on Sun, 15 Jun 2014 04:43:24 AMAll use subject to JSTOR Terms and Conditions

PUBLIC MANAGEMENT FORUM 489

But none of them have helped to define what constitutes performance in the public sector.

Certainly public agencies, like organizations in the private sector, are expected to strive for effectiveness and efficiency. Unlike their corporate counterparts, however, they have no tests to guide them. If a corporation operating in a competitive environment produces a product or service for which there is no demand, it is ineffective and will fail. If it is unable to produce that commodity at a cost comparable to its competitor's, its market share will be devoured. When a government agency is ordered to produce a cure for cancer, however, there are no market mechanisms that will test its effectiveness and efficiency and there is no way for it to withdraw from its enterprise. Moreover, government agencies must fulfill other mandates-equi- table provision of services, documentation and deter- mination of service eligibility, and responsiveness to special interest groups and sectional priorities. Perfor- mance assessment of these agencies therefore embodies either more dimensions or semantically different inter- pretations of traditional dimensions than those required in a corporate environment.

Whatever their value to management science and public administration, the LEAA experience with goals-derived measures suggested that they neither rely upon nor stimulate the development of a rigorous foundation for performance mea- surement.

Perhaps nowhere in the public sector are these nuances of performance more subtle and intertwined than in the criminal justice system. Criminal justice agencies play a particularly complex and sensitive role in our society, and efforts to gauge their performance have not, as yet, overcome the considerable obstacles to isolating measurable elements of performance. While some progress has been made in recent years, perfor- mance measurement in criminal justice is still largely piecemeal.

To help remedy this situation, the Law Enforcement Assistance Administration (LEAA) in 1978 began a long-range program to improve the quality of perfor- mance measurement information and expand its utiliza- tion throughout the criminal justice system. The pro- gram goes far beyond an effort to simply define performance elements. It will be a rethinking of the theoretical constructs of performance. Once the concep- tual framework is complete, an extensive applied re- search effort is planned to determine such things as: Which goals and performance elements compete for an organization's resources? Which aspects of performance respond to changes in agency policy? What kinds of performance measures are most useful to agency mana- gers? A major objective is to give public administrators in criminal justice empirically-grounded information about which outcomes they can or cannot control and what mixes of policies are likely to operate in harmony.

SEPTEMBER/OCTOBER 1979

The structure of the new performance measurement program grows out of a series of lessons learned by LEAA. During the past decade, the agency supported attempts: (1) to develop performance measurement systems for the police, prosecution and public defense, courts, and corrections functions; (2) to promote perfor- mance and professional standards; and (3) to launch an extensive program of evaluation research. This article traces the contributions of these experiences to the current effort. It then discusses some of the conceptual advances anticipated from the program and how these concepts might proceed from research to action. Some concluding observations are offered to other mission-ori- ented funding agencies.

Performance Measurement Research

The need for performance statistics in criminal justice was acknowledged even before LEAA was created. The President's Commission on Law Enforcement and Ad- ministration of Justice recommended in 1967 that a National Criminal Justice Statistics Center be established within the Department of Justice to collect and analyze two basic kinds of data: characteristics of criminal careers, and crime and the system's response to it as reported by agencies at all levels.' Such data are now widely available through LEAA statistical programs, and the latter type have formed the basis for rudimentary performance measurements.

LEAA also funded more systematic studies of perfor- mance measurement in criminal justice through its research arm, the National Institute of Law Enforcement and Criminal Justice. The primary objectives of these studies have been to suggest ways that agency manage- ments can improve their functioning and to stimulate interest in quantitative management analysis.2 In gen- eral, the unit of study has been the operational agency-a police department, a district attorney or public defender's office, a court, or a prison system. The research style typically has involved a combination of expert panels and field surveys. Grantees, working with panels of experts, have identified and structured the goals and objectives. Measures of goal-achievement were then specified, some relating to operational activities and others relating only to goals or policies.

Such projects were helpful in focusing political and management attention on the need for reform and improvement in the field. Goals statements and norma- tive statements of system operations provided a broad standard against which a particular system or program could be measured. Policy makers or managers could then choose the goals or standards they considered most important and concentrate upon improving those facets of their operations. But performance information sys- tems developed by this kind of approach had severe limitations. The measures tended to have little or no diagnostic value. If an objective of a court system is to reduce the time between arrest and trial, for example, measuring the amount of time that elapses between

This content downloaded from 185.44.77.38 on Sun, 15 Jun 2014 04:43:24 AMAll use subject to JSTOR Terms and Conditions

490 PUBLIC ADMINISTRATION REVIEW

arrest and trial may offer no clues as to the cause of the delay or how to reduce it. Also, the measures are not amenable to empirical validation. One report may argue that average time to trial is a good way to measure trial delay. A second may assert that median time to trial is a better measure. There is no way to test their relative superiority. One may be more sensitive to policy changes than the other, but that does not answer the question: which is the better measure of the concept of delay. Whatever their value to management science and public administration, the LEAA experience with goals-derived measures suggested that they neither rely upon nor stimulate the development of a rigorous foundation for performance measurement.

Standards and Goals

There is a considerable disagreement among criminal justice professionals as to what "performance" means let alone how it should be measured. Much of the disagree- ment stems from the variety of values brought into the debate. There is a general consensus on goals: police should maintain law and order; prosecutors should present the public's case against the defendant; courts should ensure due process before and during trial; and the correctional system should hold offenders in humane facilities. But descend just one step below this plane and the arguments begin. Should police respond to noncrime calls? Should prosecutors press for conviction on weak cases or concentrate on improving good ones? Should every offender convicted of a given type of crime receive a similar sentence?

Standards in criminal justice have been an attempt to merge what is "right" with what is possible. The finest in research and practice converge to produce a combination of operational knowledge and ethics, and with it, implicit performance norms.

LEAA's Standards and Goals Program exemplified one standards-setting approach. Established in 1971, the National Advisory Commission on Criminal Justice Standards and Goals sought to provide federal leadership in system reform. Comprehensive sets of standards were published in 1973 for the traditional sectors of criminal justice (police, courts, and corrections) as well as for nontraditional aspects-community crime prevention, drug abuse, juvenile delinquency, organized crime, and as well as research and development.

The standards were of two kinds: (1) operational performance targets for criminal justice agencies; and (2) structural and policy reforms. Illustrative of the perfor- mance targets is Standard 4.1 for the courts: "The period from arrest to the beginning of trial of a felony prosecution should not be longer than 60 days." Courts Standard 3.1 is illustrative of the reform approach: "As soon as possible, but in no event later than 1978, negotiations between prosecutors and defendants-either personally or through their attorneys-concerning con- cessions to be made in return for guilty pleas should be prohibited."

LEAA published the Standards and Goals volumes and initiated a national program to encourage their consideration by the states. States received grants to establish the machinery to develop, adopt, and imple- ment their own standards and goals. In all, 48 states and territories participated in this effort.

The final report of the national-level evaluation of the Standards and Goals Program summarizes the results:'

The impact of the Standards and Goals Program was insignifi- cant. Nowhere did the program achieve the ultimate objectives intended for it.

As the reasons for the failure, the report goes on.:

The domain of concrete, objectively valid standards is narrow, far more so than S&G took into account." "Even when the virtues of the standards were clear, the costs of implementation could lead reasonable people to reject it, depending on local and often idiosyncratic conditions." "Many of the issues for which LEAA sought standards have no 'right' answer, even in constitu- tional law."

The state-wide standards developed as a result of the program provided a useful statement of current profes- sional consensus on broad policy and operational issues. The communication process involved in developing the standards was probably of some value to the partici- pants. But there was little evidence that the standards influenced practice in substantial and direct ways. The process envisioned for implementation was through incorporation of the standards into state comprehensive law enforcement plans prepared annually under the LEAA program. This did not work for a variety of reasons. A number of states had neither the resources nor the inclination to utilize the LEAA program as the mechanism for large scale criminal justice system reform. The standards themselves were usually a complicated mixture of policy and operational objectives. Hence, implementation would have required a highly complex set of relationships between state planning agencies, the political leadership within the state, and the heads of the operational agencies. Most state planning agencies were not in a position to effectively orchestrate or lead an implementation effort of such complexity. Finally, the standards were not empirically grounded or refined sufficiently so that they could be easily tailored to variations required by differing needs of local systems. Extensive additional work would have been required to adjust the standards to the realities of particular opera- tional situations.

Current Performance Measurement Research

It is evident that there is no such thing as a criminal justice system in the sense that police, courts, and correctional agencies operate in concert to produce an agreed-upon social outcome. If anything, objectives of individual agencies create tensions among these bodies. It is equally evident that, even if there were coordinated criminal justice systems, they would differ significantly

SEPTEMBER/OCTOBER 1979

This content downloaded from 185.44.77.38 on Sun, 15 Jun 2014 04:43:24 AMAll use subject to JSTOR Terms and Conditions

PUBLIC MANAGEMENT FORUM 491

across states as to purposes and priorities. However, there is a growing conviction that one can determine within reasonable limits the relationships of agency policies to performance outcomes.

The efforts that have been discussed-performance measurement studies and the Standards and Goals Program-have influenced LEAA's current performance measurement research to some extent. The major need- and an objective of the program-is to develop an understanding of the technology of public service delivery. It is not sufficient to assign measurable objectives to agencies. The process by which the agency will achieve the objective must also be understood and incorporated into a performance framework.

Standards in criminal justice have been an attempt to merge what is right' with what is possibk.

The goals of a police department-crime prevention, criminal apprehensions, order maintenance, and public safety-can be listed. Activities such as patrol, receipt and dispatch of calls for assistance, direction of traffic, and crime investigations can be observed. Outputs from these processes-arrests, patients delivered to hospitals, and traffic citations-can also be observed. But can a performance measurement system be constructed given these pieces of information?

Historically, the answer to that question is "yes." Performance measures abound. The literature on police productivity includes such yardsticks are: ratio of arrests to number of crimes reported, rates of convictions to numbers of arrests, and the number of traffic citations per officers But these measures are oversimplifications. For example, the ratio of arrests to crimes cannot be constructed as a useful measure of police performance unless we also know the level of police resources expended and something about the quality of the arrests made. And convictions per arrest reflect as much upon prosecutorial performance as they do upon police.

Earlier performance measurement research also failed to recognize the interdependencies among policies. This has been a significant limitation. Prosecution provides perhaps the simplest illustration. A district attorney can adopt a number of philosophical stances with respect to accepting cases. One is legal sufficiency-any case is taken where there is "probable cause" to believe that the defendant has committed the crime for which he has been charged. A more stringent policy is trial suffi- ciency-only those cases are accepted in which the prosecutor believes that "proof beyond reasonable doubt" can be established. The policy on plea negotia- tions can also be chosen. Under a liberal policy many cases will be settled privately with defense counsel, agreeing to accept a guilty plea to a lesser offense instead of pressing a charge on a more serious court. Under a no-plea policy there will be prosecution of the original charge.

SEPTEMBER/OCTOBER 1979

Given today's heavy caseloads and limited prosecu- torial resources, a district attorney who chooses a liberal intake policy (probable cause) and a strict no-plea policy may overwhelm his office with cases. The combination may create an inappropriate policy mix for the resources available. The no-plea policy may result in more cases going to trial. More trials require additional preparation on each case. More work per case means fewer cases can be accepted-a more stringent intake policy later may be necessary notwithstanding the prosecutor's original poli- cy stance. Given this simple framework, a district attorney must choose his policies in light of their interactions.

To structure these and similar policy interactions in a way that will permit better articulation and more in depth study of performance measurement issues, LEAA awarded $1.1 million in 1978 to a group of researchers.6 Four separate grants were awarded to study policing, prosecution and public defense, courts, and adult correc- tions. A fifth grant was awarded to study the system- level perspective, addressing measurement issues com- mon to each functional area and examining performance issues that arise from the daily interaction between these agencies.

The grantees operate as a consortium. Each institu- tion works to sort out the key functions and factors within their area of expertise. By reviewing the range and variations of activities currently practiced, they are attempting to define performance in a practical but meaningful way. Through regular interaction, they dis- cuss each other's progress and share perspectives on how to approach problems of mutual concern.

The consortium approach will produce five studies addressing common themes but from perspectives that reflect the sometimes conflicting objectives of the agencies. The intention is not to force a consensus that there are right and wrong ways to conduct any facet of criminal justice. The consortium acknowledges that there are divergent views among agencies derived from a variety of historical roots, legal constraints, and func- tional practicalities. What is hopes to accomplish is an exposition of these differences and their implications for performance measurement research.

Translating Theory Into Practice

The program will assess the importance of various operational and policy factors to performance measure- ment in criminal justice and synthesize the issues into new frameworks of performance. But even if successful new frameworks are developed, the program risks a limitation common to previously-funded performance research efforts: a lack of objective validity. Unless the frameworks are subjected to some form of empirical testing, they are at best sensibly constructed stories.

The performance measurement program is in the first phase of LEAA's action program development process- problem definition. Assuming that those efforts produce a satisfactory advance in performance concepts, empiri- cal testing of the concepts and measures will be

This content downloaded from 185.44.77.38 on Sun, 15 Jun 2014 04:43:24 AMAll use subject to JSTOR Terms and Conditions

492 PUBLIC ADMINISTRATION REVIEW

supported in 1980. This phase would involve the collection and analysis of pertinent data to determine which factors are critical to explaining performance variations among similar agencies and which appear most responsive to management actions.

If empirical tests are successful, the program will move to a field experimentation phase. LEAA would test the systems based upon the research to determine their utility in managing police departments, prosecu- tor's and public defender's offices, courts, and correc- tional agencies. The tests should resolve such issues as user acceptance, maintenance costs, and contributions to agency management capabilities.

The final step in the performance measurement program (assuming that earlier phases are successful) is demonstration and marketing. Program responsibility moves to the action offices of LEAA. At this point one should be able to say with confidence that the research base is solid and the concepts have proven their value in operational settings and warrant implementation.

The entire developmental process from research to action is lengthy but practitioners will derive benefits from the program at every stage, not just at its completion. Some of the mystique of performance measurement will be removed early one. It is already obvious that there are no exemplars of criminal justice agencies, nor are there singular definitions of perfor- mance. There are instead spectrums of operational styles that can be drawn out along various dimensions of performance. It is therefore important for managers to recognize that measures of "performance" are not report cards but rather diagnostic devices. Only the manager can decide where he would like to be along each spectrum and what steps should be taken given his operating environment to move in the appropriate directions.

Many management information systems developed and sponsored by LEAA can produce some statistics relevant to assessing an agency's progress, but it is neither essential nor necessarily desirable to own an information system in order to measure performance.7 It is essential that managers be made aware of the range of operational styles that are available to them and the kinds of information they need to collect in order to choose among their options. It is also essential that managers make a conscious choice. Only after they define performance can they intelligently interpret statistics relevant to it. These are the critical issues of the early phase of the program-what the options are, how to choose among them, where to measure, and when the program objectives chosen are being achieved.

Applications in Other Agencies

The research strategy of the performance measure- ment program within LEAA has applicability to pro- grams in other mission agencies such as health, educa- tion, and transportation. Although mission agency mana- gers are constrained in their funding activities by a need to produce policy-relevant outputs, policy relevance

does not preclude the sponsorship of long-range research programs. A structured, orderly process of knowledge production, verification, and application is not only sound management practice but also holds promise of creating a climate more conducive to innovative research in mission agencies committed to solving specific policy problems. Rather than relying solely on traditional approaches that have produced respectable, if not dramatic, results, agencies have greater freedom to experiment. By limiting initial investments in innova- tions until their merits have been systematically assessed and objectively demonstrated, the costs of experimenta- tion are kept within acceptable bounds. At the same time, the benefits of creative and innovative approaches can be more carefully explored and, ultimately, realized in shaping public policy.

In order to realize the benefits inherent in this approach to mission-sponsored research, an agency must create and maintain some necessary internal conditions. Both management and staff must recognize and defend the existence of research risk. Some fraction of the research projects funded or otherwise conducted will provide little, if any, opportunities for creating action programs, either because of the project's execution or because of the nature of the findings. Unless the agency is prepared to write off some of its enterprise as unlucky investments, it will confine itself to safe, narrowly-de- fined problems or worse, commit itself to throwing good money after bad.

There is a growing conviction that one can determine within reasonable limits the relation- ships of agency policies to performance out- comes.

The agency must also institute a programmatic review process and incentive structure consistent with objec- tives of selecting from among its research portfolio the most promising opportunities for field implementation and moving these opportunities into the hands of the practitioner in a form that is useful. Once a research program is begun it is difficult to stop or to channel toward action implementation. Staff are reluctant to kill their brainchildren; they are equally reluctant to move away from further research and toward the next stage in the development process. By monitoring the program turnover in its research portfolio, however, the agency can insure a continuity between its research and action activities and the progression of its research findings to the field.

Notes

1. The Challenge of Crime in a Free Society, A Report by the President's Commission on Law Enforcement and Admini- stration of Justice (February 1967).

SEPTEMBER/OCTOBER 1979

This content downloaded from 185.44.77.38 on Sun, 15 Jun 2014 04:43:24 AMAll use subject to JSTOR Terms and Conditions

PUBLIC MANAGEMENT FORUM 493

2. See for example, State Judicial Information Systems, Final Report (Phase II), Search Group, Inc., 1976 or Indicators of Justice: Measuring the Performance of Prosecution, Defense and Court Agencies Involved in Felony Proceedings, National Institute of Law Enforcement and Criminal Justice (May 1977).

3. The National Evaluation of the Standards and Goals Project, Volume 1: An Assessment of the Program, American Insti- tutes for Research, Washington, D.C. (December 1978), p. 165.

4. Ibid., pp. 174-76. 5. See Opportunities for Improving Productivity in Police

Services, National Commission on Productivity (1973) for other examples of traditional indicators of police perfor- mance.

6. The grantees are the University of North Carolina (Police); Bureau of Social Science Research, Inc. (Prosecution and Public Defense); Research Triangle Institute (Courts); The Osprey Company (Adult Corrections); Georgia Institute of Technology (Systems Issues).

7. See "Court Information Systems, Phase I Report." National Evaluation Program, National Institute of Law Enforcement and Criminal Justice, Washington, D.C., 1977, for an assess- ment of the utility of information systems in one component of criminal justice decision making.

SEPTEMBER/OCTOBER 1979

Competent, creative administration begins with -

PUBLIC ADMINISTRATION Organizations, People and Public Policy Samuel J. Bernstein & Patrick G. O'Hara This provocative, lucid text offers a comprehensive, concept-oriented introduction to public administra- tion. Organized around three focal points-organizations, people, and public policy-PUBLIC ADMINIS- TRATION synthesizes theory and analytical tools from the emerging policy sciences and from various other disciplines as well, to give your students a broad perspective.

Student ability to handle admin- istrative tasks is rapidly developed by the problem-solving approach and highly relevant examples that include a midbook case study of the circumstances surrounding the design and manufacture of the McDonnell-Douglas DC 10.

Written by two notable adminis- tration experts, this text has been acclaimed by instructors from coast to coast. Instructor's Manual also available. 448 pages. 0-06-040649-6. March 1979.

To request examination copies, write to Suite 5D (800), Harper & Row, 10 East 53d St., New York, N.Y. 10022. Please include course title, enrollment, and present text.

New York * San Francisco

This content downloaded from 185.44.77.38 on Sun, 15 Jun 2014 04:43:24 AMAll use subject to JSTOR Terms and Conditions