Embed Size (px)
Citation preview
Integrating complexity into policy evaluation
and enabling better policy making:
Introducing the Centre for the Evaluation of
Complexity Across the Nexus (CECAN)
Dr Candice Howarth
Senior Research Fellow & Knowledge Integrator, CECAN
University of Surrey
5th European Environmental Evaluators Network Forum, Copenhagen, 15-16 Sept 2016
Policies across the environment, food, energy, water nexus are entangled in complex ways
∙ Span multiple sectors
∙ Cross stakeholder impact and influence
∙ Conflicting timescales and cultures
Influenced by pressing issues such as climate change, loss of
biodiversity, poverty, food scarcity etc.
Innovative methods are needed to evaluate complex policy interventions
∙ Bring evidence & expertise from range of domains & stakeholders
∙ Cross sector and integrative
Exploring the nexus
CECAN aims to transform the practice of
evaluation in the Nexus to make it fit for a
complex world
Evaluation questions
Impact evaluation
What difference did the policy make?
Process evaluation
How was the policy delivered?
Economic evaluation
Did the benefits justify the costs?
WHY IS
EVALUATION
DIFFICULT?
Society is inter-connected…
…multi-level
What is a complex system?
Many independent, heterogeneous units,
…that interact,
… give rise to system behaviour as a whole
…which is more than the sum of its parts
Micro to macro systems – neutrons to nations
Multi-level, nested interactions
Emergence – behaviour of a system emerges
Feedback is important
WHAT IS
CECAN DOING
CECAN Vision
To integrate complexity into policy evaluation
and to enable more effective policy-making
To catalyse a step-change policy evaluation for
complexity and Nexus issues
Developing methods for policy evaluation in
complex domains
Testing these methods with a range of case
studies
Supporting knowledge exchange and capacity
building
CECAN is..
Co-Funders
Partners
ACTIVITIES
CECAN acknowledges from the outset, the issue of
equifinality: there are many pathways to the same outcome
Developing:
• Timely methodological solutions that actively respond to
the fact that context matters and different approaches
may be needed (even for the same problem)
• Methodologies that are also suitable for a non-nexus
topics and can be re-purposed in other complex social
settings (e.g. health care, urban planning)
Methodological development
Addressing methodological challenges
Policy design and evaluation
Scaling Causality
Context TimeInter-
disciplinarity
Framing Big data Cost
Potential case studies
Regulating our Future' evaluating new approaches to food safety
regulation - Food Standards Agency
Lessons from evaluating complexity in the Transitional Arrangements
of Electricity Market Reform - BEIS
Procuring complexity-ready evaluation - lessons from Water
Abstraction Reform Agent Based Modelling evaluation - Defra
Evaluation strategies for Marine Plans in England - Defra
Understanding the effectiveness of enforcement interventions on
waste crime - Environment Agency
Programme impact and scheme interactions in the Rural
Development Programme for England - Defra
A full system analysis of the Renewable Heat Incentive - BEIS
Example of methodologies to explore
Agent based modelling
Type of simulation where 'agents' (e.g. individuals, communities,
neighbourhoods, etc.) assigned rules about how they interact over time & space
Strengths
• Observing at a macro-level sum of micro-level level behaviours & interactions
• Handling non-homogeneous &inter-dependent behaviour
• allowing the effect of a policy intervention to be simulated &explored
Usage
• Modelling ex ante ‘what if?’ questions about policy options, and exploring ex post models as a way of understanding the causes of observed changes due to policy and to construct credible counterfactuals;
• Developing a user-friendly toolkit to help policymakers determine when and how an evaluation would benefit from agent based modelling
Example of methodologies to explore
Involving citizens
The growth of crowd-funding, crowdsourcing and citizen science has shown the
opportunities for recruiting the public to assist with evaluation
Strengths
• Relatively low cost
• Large numbers of people with diverse expertise and interests can be involved
• Provides much greater quantity of data than would otherwise be available
• Increasing the transparency of the evaluation and associated policy intervention
Challenges
• Demands effective leadership
• Appropriate motivation to sustain contributions
• Support of technology to gather and organise the data
Evaluation Policy and Practice Notes
Bayesian Updating for Contribution Claims
Qualitative Comparative Analysis (QCA)
Dependency Modelling
Agent Based Modelling
and more…
Seminars, workshops and Fellowships
Monthly seminars in central London
Residential Workshops
One day courses
Fellowships
• Fellows – policy professionals,
academics, practitioners –work within the
Centre. Applications now open!
