CAUSAL ANALYSIS: ITS PROMISE FOR POLICY STUDIESDonald S. VanMeter and Herbert B. Asher

Ohio State University

IntroductionSome of the central issues raised in the field of policy analysis can

accurately be described as questions of cause and effect. For example,in the comparative state policy literature, the investigation of thedeterminants of public policy is basically a concern with the socio-economic and political causal antecedents of policy decisions. Similar-ly, impact and evaluation studies raise causal questions about theeiiects of policy decisions. While causal themes are implicit in theseresearch areas, often the theorizing process is not explicitly causalnor do the methodologies employed facilitate causal analysis. Hence,we will argue in this paper for a more causal approach to theorizingand to data analysis.

Causal TheorizingIt may be the case that the use of causal data analysis techniques

will be difficult or even impossible; certain assumptions may not bemet, data may be unavailable, and equation systems may be unidenti-fied. Even in such situations, a causal approach to theorizing isvaluable for its heuristic value. Thinking causally about a problemand constructing an arrow diagram that reflects causal processes mayoften facilitate the clearer statement of hypotheses and the generationof additional insights into the topic at hand. The heuristic value ofcausal thinking and path diagrams is demonstrated by the followingexample.

Consider the case of a decision-maker whose goal is to improvestudent performance on various standardized tests. (The reader shouldbear in mind that the construction of reliable and valid indicators isnot our prime concern here, although this is certainly a must if we areto have any confidence in our analysis.) Assume that the decision-maker has one basic variable over which he has control: the amountof money to be spent on education. Such a variable might be labeled amanipulable since its level can be varied by the conscious decision ofactors, within certain limits imposed by various external constraints.But rather than simply hypothesize that increased expenditures foreducation will improve student performance, an assertion contradictedby several studies, one might ask the more causally relevant question:how is it that increased expenditures for education might translate intobetter student performance? That is, what are the ways in which in-creased expenditures actually produce improved student performance?

The decision-maker might well recognize that options are availableas to how additional moneys might best be allocated to improve studentperformance. For example, should money be channeled into hiring moreteachers so as to lower the pupil/teacher ratio, into attracting betterteachers, or into improving facilities and developing (or expanding)innovative programs? What might be the optimal mix of funding forthese three options? We might represent the decision-maker's situationat this stage by the following diagram:

- 103-

Education _Expenditures

Lower Pupil/Teacher Ratio

Higher Teacher Quality

Better Facilities & Programs

ImprovedStudent

Performance

But there are additional variables that impinge upon student per-formance that are not as directly under the control of the decision-maker. These might include the supportiveness of the family environ-ment toward educational achievement, the student's abilities, and thestudent's affect toward the educational system. In addition, theremight be interrelationships among the manipulable and non-manipulablevariables. For example, perhaps more and better teachers, facilities,and programs will influence the student's affect toward education.Hence, we might come up with the following more complete model:

TEACHER/PUPILRATIO

EDUCATIONEXPENDITURES

FAMILYENVIRONMENT

TEACHERQUALITY

INNOVATIVEPROGRAMS

\V STUDENT

-7 PERFORMANCE/N

STUDENTAFFECT

STUDENTABILITY

Note how the model not only specifies the relationships between theindependent variables and the ultimate dependent variable of interest(student performance), but also makes explicit the relationships amongthe prior variables. Each linkage included implicitly represents anhypothesis which would be tested by estimating the magnitude of therelationship. While actual estimation of the linkages may or may notbe possible, depending upon whether satisfactory indicators can beconstructed, appropriate data collected, and the like, the point to bemade here is that the kind of causal thinking illustrated by the examplehas greater promise for better elucidating the processes whereby policydecisions are made—and consequences are determined—than simplycorrelating independent and dependent variables in a relatively un-thinking fashion.

- 104 -

The discussion of manipulable variables has implications for thegoals of research, the type of statements that one makes, and the kindsof variables that one employs in the actual data analysis. For example,in the comparative smte policy literature, the use of factor or compos-ite variables is not uncommon(Grumm, 1971; Sharkansky and Hofferbert,1969; Van Meter, 1972). Such variables are constructed by factoranalyzing a set of indicators of some underlying construct (e.g., socio-economic development), thus creating a factor-variable composed of aweighted combination of the input indicators. Such a variable mightthen be used in correlation and regression analysis. This proceduremight be satisfactory if one is mainly interested in prediction (perhapsas measured by the size of R^s) and is less concerned about deline-ating underlying causal processes. But if one wants to make causalstatements such as a unit of change in X produces a certain change inY, what does it mean to talk about a unit change in a socio-economicfactor variable? What components of the factor variables are actuallyproducing the change in the dependent variable? Single item indicators,while perhaps having lower predictive power, are conceptually clearerand lend themselves to more meaningful causal interpretations.

Single item indicators are also superior if one's research is to serveas a productive guide to action for policy-makers. Factor variables areessentially non-manipulable, although their individual components mayvery well be. For instance, whereas conscious decisions can lead tochanges in the salaries of legislators, the availability of staff ser-vices, and the number of days the legislature is in session each year,it is difficult to conceive of actors manipulating a factor labeled"legislative professionalism" (Grumm, 1971). Hence, our researchwould have greater relevance for decision-makers if we choose for ouranalysis the best single manipulable variables.

Causal Data Analysisa. InfrodlucfionGreater attention should also be given to causal data analysis

(i.e., causal modeling). A number of techniques can be classified underthe heading of causal modeling: these include Simon-Blalock arrowtesting, recursive path estimation, and nonrecursive path estimationwhich often involves fairly complex econometric estimation techniques(Simon, 1957; Blalock, 1964; Land, 1969; Duncan, 1966; Duncan,Haller, and Portes, 1968; Stokes, 1971; and Alker, 1969). The Simon-Blalock procedure is highly unsatisfactory for it requires a tremendousinvestment in often unrealistic and restrictive assumptions and yieldsonly weak results-whether or not a linkage belongs in a model withoutany information as to the magnitude of that linkage. Recursive pathestimation, which requires a similar set of assumptions to the Simon-Blalock technique, does yield estimates of the magnitude of the link-ages between variables. This allows one to make statements about howa change in one variable affects another variable and enables one totalk about direct and indirect effects of one variable upon another.Nonrecursive path estimation allows one to handle reciprocal relation-ships, thereby yielding a better representation of real world processes.

- 105 -

b. Recursive Path EstimationRecursive path estimation is probably the causal technique dbat is

most compatible with Ae existing statistical skills of policy analystsand that has widespread applicability. In order to perform recursivepath estimation, certain assumptions must be met; these include thestandard ones associated with the use of multiple regression (Draperand Smith, 1966; Wonnacott and Wonnacott, 1970). Hence, if theseassumptions are met, then the estimation does not require the acquisi-tion of new, sophisticated analysis techniques. The standard list ofassumptions in the recursive case includes: (I) linear and additiverelationships; (2) interval level data; (3) disturbance terms uncorre-lated with the explanatory (predictor or independent) variables in theequations in which they appear; and (4) no confounding unmeasuredvariables.

To clarify these last assumptions, consider the following simple,three variable model:

'V

= socio-economic development

= inter-party competition

X2 = public welfare expenditures

i , X2, and X3 are all measured variables, while Ry and Rv are dis-turbance terms representing those variables not formally included inthe model that influence the endogenous variables. P21, P31J and P32are path coefficients representing the causal impact of one variableupon another. The diagram can be represented by the following twoequations; no equation is written for Xl since it is considered anexogenous variable uninfluenced by any of the other measured vari-ables:

(V) X2= P21X1

(2) X3= P31X1

Assumption 3 above says that Ru and Xl, Ry and Xl, and Ry and X2must be uncorrelated. Assumption 4 says there should not be any un-measured variable (say X4) that directly influences any two of ourmeasured variables; otherwise, X4 would have to be formally incorpo-rated (measured) in the model to avoid possible misleading inferences.

Techniques for relaxing the first two assumptions are becomingmore prominent so one should not despair of using causal techniqueswhen there is interaction (i.e., non-additive relationships) or whenthere is less than interval level data (Wilson, 1971; Boyle, 1970).Also, non-linear relationships can sometimes be made linear by an

- 106 -

appropriate transformation (Strouse and Williams, 1972). The third andfourth assumptions are best justified by serious substantive reasoningby the investigator.

Once the path coefficients are estimated by ordinary regressiontechniques, the residual path coefficients representing the relationshipbetween the unmeasured disturbance terms and their associated en-dogenous variables can be determined. This gives one a direct indi-cator of the explanatory power of the model. In addition to estimation,path analysis allows one to talk about the direct and indirect effectsthat one variable has upon another. For example, socio-economicdevelopment (Xi) has a direct impact on public welf̂ are expenditures(X3) and an indirect impact on expenditure levels via its impact oninter-party competition {X2). One can compute the magnitude of thedirect and indirect effects which helps delineate the operative causalmechanisms.

In addition, with path analysis we can decompose the correlationbetween any two variables into a sum of simple and compound pathswith some of the compound paths being substantively meaningfulindirect effects and others perhaps not (Stokes, 1971). In our simpleexample, the only decomposition that can be performed that involves asubstantively meaningful indirect effect is that between Xj and X3:

rj^ = P31 + P21P32- Rank ordering the components of the decompo-sition would again give us insight into the causal processes.

Finally, the decomposition of a correlation, in addition to yieldinginformation about the causal processes, also provides a way in whichto test the adequacy of the model if some linkages have initially beenomitted. If the model were specified correctly, then (except for meas-urement error and sampling error when relevant) the empirical corre-lation between any two variables should be numerically equal to thesum of the simple and compound paths linking the two variables. If theequality does not hold, this suggests that the model may be improperlyspecified and in need of revision. The most common type of revisionis to include a linkage that was previously omitted. The major short-coming of this procedure is that, unguided by theoretical insight, onewill likely end up with a model that includes all possible (recursive)linkages. If so, no model testing is possible as the model is exactlydetermined. The major shortcoming of a recursive model is its unreal-istic omission of feedback processes. But techniques are available forhandling nonrecursive situations and as computer software is develop-ed, such analyses will become much more common. The basic problemin this area concerns the identification of the equation system.

ConclusionCausal thinking, as reflected in the construction of arrow diagrams,

is a significant task. However, if we cannot handle certain problemsand meet certain assumptions, then analysis will be stymied, estima-tion impossible, and important questions unanswered. Indeed, we willoften find ourselves in such a situation as causal modeling techniquescannot automatically be applied without some prior work on the part ofthe investigator. Among the problems that he must first surmount arethe proper specification of models, often in areas where there is little

- 107 -

solid theory upon which to build, and the satisfactory operationaliza-tion of key concepts. It was not our purpose here to go into strategiesof model construction and operationalization except to say that theresults of one's causal analysis are only as valid as the initial de-cisions made in building the model and operationalizing the variables.

We have argued in this paper for the use of causal thinking andcausal techniques in policy studies. Even where the actual estimationcannot be performed, a causal style of thinking is still beneficial fortheory building and hypothesis generation. Path analysis proceduresare important for they enable one to move beyond mindless correlationanalysis and the simple estimation of direct effects, the basic outputof ordinary regression. Rather, path analysis allows the investigator toexamine the causal processes underlying his observed relationshipsand to get a handle on the relative importance of alternative paths ofinfluence. The model testing permitted by path analysis proceduresalso encourages a more explicitly causal approach in the search forexplanations of the phenomena under consideration. For all of thesereasons, it appears that causal analysis holds substantial promise forpolicy studies.

REFERENCES

Alker, H. R., Jr. (1969) "Statistics and Politics: The Need for Causal Data Analysis,"pp. 244-313 in S. M. Lipset (ed.) POLITICO AND THE SOCIAL SCIENCES. NewYork: Oxford University Press.

Blaiock, H. (1964) CAUSAL INFERENCES IN NONEXPERIMENTAL RESEARCH.Chapel Hill: University of North Carolina Press.

Boyle, R. P. (1970) "Path Analysis and Ordinal Data." AMERICAN JOURNAL OFSOCIOLOGY 7 5 (September): 461-80.

Draper, N. and H. Smith (1966) APPLIED REGRESSION ANALYSIS. New York: JohnWiley and Sons.

Duncan, O. D. (1966) "Path Analysis: Sociological Examples." AMERICAN JOURNALOF SOCIOLOGY 72 (July): 1-16.

Duncan, O. D., A. O. Haller, and A. Portes (1968) "Peer Influences on Aspirations: AReinterpretation." AMERICAN JOURNAL OF SOCIOLOGY 74: 119-37.

Grumm, J. (I97I) "The Effects of Legislative Structure on Legislative Performance,"pp. 298-322 m R. Hofferbert and I. Sharkansky (eds.) STATE AND URBAN POLI-TICS. Boston: Little, Brown.

Land, K. C. (1969) "Principles of Path Analysis," pp. 3-37 in E. Borgatta (ed.)SOCIOLOGICAL METHODOLOGY 1969. San Francisco: Jossey-Bass.

Sharkansky, 1. and R. Hofferbert (1969) "Dimensions of State Politics, Economics, andPublic Policy." THE AMERICAN POLITICAL SCIENCE REVIEW 63 (September):867-79.

Simon, H. (1957) "Spurious Correlation: A Causal Interpretation," pp. 37-49 in MODELSOF MAN. New York: John Wiley and Sons.

Stokes, D. E. (1971) "Compound Paths in Political Analysis," pp. 70-92 in J. F.Herndon and J. L. Bernd (eds.) MATHEMATICAL APPLICATIONS IN POLITICALSCIENCE V. Charlottesville: The University Press of Virginia.

Strouse, J. and J. O. Williams (1972) "A Non-additive Model for State Policy Research."JOURNAL OF POLITICS 34 (May): 648-57.

- 108-

VanMeter, D. S. (1972) "The Policy Implications of State Legislative Reapportioomeat:A Longitudinal Analysis. Ph.D. dissertation. Madison: The University of Wis-consin.

Wilson, T. P. (1971) "Critique of Ordinal Variables," pp. 415-31 in H. M Blalock (t^A ^CAUSAL MODELS IN THE SOCIAL SCIENCES. Chicago: Aldine-Atherton. ^

Wonnacott, R. J. and T. H. Wonnacott (1970) ECONOMETRICS. New York: John Wileyand Sons.

ASSESSING THE IMPACT OF INCREMENTAL POLICIESE. Terrence Jones

University of Missouri — St. Louis

During the past decade social scientists have become increasinglyinterested in using social science methodology to assess the extent towhich public policies achieve their intended objectives. From a meth-odological viewpoint, this movement has been one of adapting researchtools developed in other contexts to a new set of circumstances.

Up to now, most policy evaluation studies have dealt with majorchanges in policy. Such a concern is quite understandable since, ifany policies are making a difference which can be detected by evalu-ation studies, it should be policies which have undergone a substantialtransformation. The most frequent policies made by governments arenot, however, major changes; instead, the most common governmentalactions are regular (usually annual) shifts in expenditure and manpowerlevels. Many of these adjustments are intended to achieve changes insocietal conditions; examples include police protection (minimizecrime), fire protection (minimize fire damage), and health facilitiesand manpower (minimize mortality and morbidity). Thus we can viewsuch budgetary and personnel decisions as goal-oriented policies andproceed to evaluate their effectiveness. For instance, what impact doyear-to-year changes in fire protection expenditures have on year-to-year changes in fire damage? This article discusses some of the re-search design, measurement, and data analysis problems involved inevaluating the impact of these everyday incremental policies.

Research DesignIn the recent past, most policy impact studies have employed one of

three designs: cross-sectional (e.g., the Coleman Report); a trueexperimental design involving randomized assignment of the inde-pendent variable (e.g.. The New Jersey Graduated Work IncentiveExperiment); and the quasi-experimental time-series design (e.g.,Campbell and Ross's evaluation of the Connecticut crackdown onspeeding). None of these designs, however, is well-suited for assess-ing incremental policy impact. The cross-sectional design, by defi-nition, does not include time as a variable and hence makes it moredifficult to infer causation. The randomized experiment must be plan-ned in advance, frequently presents ethical and political problems,and consequently can seldom be employed. Finally, the time-seriesdesigns are by far best suited for major policy changes. Such designsneed a minimum of three time-series observations of the dependent

- 109-