Policy and Evidence: An Uneasy but Essential Partnership

Mark E. CourtneyFred H. WulczynChapin Hall at the University of Chicago

The policy realm is where conflicting values play out most directly

Practice concerns individual-level effects on outcomes that are generally agreed upon

Administration is organized around achieving the goals of policy

Social welfare policy is often the result of highly contested and shifting values

Examples: Willingness to pay Child protection versus family preservation Benefit-cost analysis of extending foster care past

18

Types of Evidence Needed for Policy Making

Characteristics/needs of the population Characteristics and behaviors of systems

Capacities Political environment Organizational cultures

System dynamics

System Dynamics: Patterns Identify Targets of Policy

System Dynamics: Patterns Identify Targets of Policy

Limits of randomized experiments for informing policy

Some policy changes cannot be assessed at all using experiments Mandatory child maltreatment reporting Extending foster care past 18

Constellations of policies are difficult to assess due to design complexities Extending care, health insurance extension, and education

vouchers Time required leads to changes in context that can

compromise experiments, but actually benefit non-experimental methods

…but, experiments are VERY valuable!

Principles and Lessons

Sound information to guide policy will come as much or more from sound data on populations and services as from evaluation research per se

Experiments are best implemented in the context of sophisticated knowledge of populations, service contexts, and system dynamics

At best, research can often only help illustrate the tradeoffs of policy choices Do not oversell the implications of findings for policy, lest you

get what you asked for Illustrating tradeoffs helps policymakers understand when and

how ideology guides their decision making