Andy Stirling - Addressing Uncertainty, Ambiguity and Ignorance in Sustainability Appraisal

Addressing Uncertainty, Ambiguity and

Ignorance in Sustainability Appraisal

Andy StirlingSPRU – science and technology policy research

presentation to interdisciplinary workshop on 'Cost‐Benefit Analysis: Uncertainty, Discounting and the Sustainable Future’

Technical University Eindhoven, 12‐13th April, 2010

‘Sound Science’ in Policy Under Uncertainty

on chemicals:

“ …sound science will be the basis of the Commission's legislative proposal…”

- EC RTD Commissioner, Philippe Busquin

on genetic modification:

“… this government's approach is to make decisions … on the basis of sound science”

former UK Prime Minister, Tony Blair

on energy:

“[n]ow is the right time for a cool-headed, evidence based assessment of the options open to us … I want to sweep away historic prejudice and put in its place evidence and science”

UK Energy Minister Malcolm Wicks

Converts messy political problems into neat technical puzzles

Powerful pressures for ‘decision justification’

0.001 0.1 10 1000externality’: cUS/kWh (after Sundqvist et al, 2005)high SUSTAINABILITY low

CBA delivers precise orderings of options

coal

oil

gas

nuclear

hydro

wind

solar

biomass

‘Sound Science’ in CBA? – the energy sector example

0.001 0.1 10 1000

coal

oil

gas

nuclear

hydro

36

wind

solar

biomass

n =

‘externality’: cUS/kWh (after Sundqvist et al, 2005)

minimum maximum

25% 75%

high SUSTAINABILITY low

CBA delivers precise orderings of options, but is sensitive to ‘framing’


0.001 0.1 10 1000

coal

oil

gas

nuclear

hydro

36

20

wind 18

solar 11

biomass 22

31

21

16

n =

‘externality’: cUS/kWh (after Sundqvist et al, 2005)high SUSTAINABILITY low

CBA delivers precise orderings of options, but is sensitive to ‘framing’


challenges centrality of determinate probability & magnitude

setting agendas defining problems characterising options

posing questions prioritising issues formulating criteria

deciding context establishing baselines drawing boundaries

discounting time adopting methods engaging disciplines

choosing cases recruiting expertise commissioning research

constituting ‘proof’ exploring sensitivities interpreting results

allocating resources policing remits sourcing probabilities

including scenarios considering uncertainties reviewing findings

Some dimensions of ‘framing’ in sustainability appraisal

not just operational matter of consistency, completeness, commensuration

but fundamental interpretive flexibility – as true of principles as methods

All analysis requires framing … all framing involves values

‘Framing’ in CBA

Each aspect of framing mediates distinctive uncertainties…

abatement cost

mitigationcost

hypotheticalmarkets

damage cost

Hohmeyer, 1988

Ottinger et al, 1990

Externe, 1995

principal method

Tellus, 1991

Framing: divergent methods in CBA

Tension between standardisation and triangulation

Hohmeyer, 1988


FUEL CYCLE STAGE

LIFE CYCLE PHASE

extraction

processing

transport

storage

conversion

residues

materials

energy

construction

operation

capacity

decommissioning

DTI, 1992

effect addressed

effect partly addressed

Framing: ‘system boundaries’ in CBA

Tension between narrow but precise and broad but aggregated

global warming

nuclear proliferation

ecosystem damage

aesthetic impacts

Schuman et al, 1982

Hohmeyer, 1988


Externe, 1995

effect addressed

Framing: ‘sustainability’ issues in CBA

Not just about consistency: to what extent and how to be complete?

acoustic noise marine debrisaerial visibility effects marine hydrocarbon releasesagricultural intensification marine life morbidityair quality mechanical hazardsaltered flow rates and patterns micro-climate effectsambient temperature change mutagenicity or teratogenicityaqueous radioactive pollution navigational hazards behavioural interference nitrogen oxide emissionscarbon monoxide emissions occupational healthcatastrophic dam burst occupational safetycatastrophic pollution potential particulate emissions or community disruption prompt public health impactsdrainage disruption psychological trauma ecological or habitat disturbance radioactive emissions electromagnetic interference resource depletionelevated water levels road traffic endangered species impacts rural population impactsenhanced coastal erosion salinity changeenshading or reflection sedimentation changes entrainment of aquatic biota soil acidification eutrophication soil erosionexplosive or incendiary effects soil sterilisationfisheries interference soil structure loss chain residues soil toxification or contaminationgaseous waste volume management solid waste volume management greenhouse gas emissions stratospheric ozone depletion hazard to bird flight sulphur oxide emissions metal releases thermal radiation effects hydrogen sulphide emissions toxic aerial emissions seismicity tritiated water emissions induced subsidence tropospheric ozone enhancement interference with migration turbidity changes land use change visual aesthetic offence latent human health effects volatile organic emissionsliquid waste volume management water abstraction demand local ambient CO2/O2 balance water quality effects

Categories of

‘environmental impact’

variously addressed

in CBAs in OECD

countries of electricity

supply technologies

(studies reviewed

earlier)…


… even greater numbers of non-

environmental aspects of

sustainability (‘externalities’)

THE FORM OF EFFECTS eg: death / injury / diseaseroutine marginal / novel catastrophicreversible / irreversible

DISTRIBUTIONAL ISSUES eg: concentrated / dispersedpublic / workers benefits / burdensfuture / presenthuman / non-human

AUTONOMY OF AFFECTED eg: voluntary / controllable / familiarinstitutional trust


Each presents a source of uncertainty in final orderings …

Different issues raise contrasting dimensions of value

Incommensurability: ‘apples & oranges’

• completeness we should be able to compare all alternatives

‘Impossibility Theorem’ (51; 63): principles not all reconcilable

• plural society: no guaranteed single ‘objective’ or ‘definitive’ preference ordering

• NB: undemanding criteria – eg: does not impose equality of influence

• substantive rationality problematic – look to procedural rationality

• unanimity if everyone prefers A to B, then society should too

• non-dictatorship no-one should always get their way, no matter what

• transitivity if A preferred to B and B to C, then A preferred to C

• independence… ordering of A and B independent of other alternatives

• universality all possible preference orderings are admissible

Arrow: axiomatic rationality to explore ordinal social choice

Much discussed, but not refuted within rational choice paradigmeconomics is curiously reticent to highlight this Nobel-winning work

Subjectivity of utility means no cardinal preference orderings

SALIENT POSSIBILITIES DEFINED POSSIBILITIES UNDERDEFINED

Aspects of Incertitude – a heuristic framework

- engineering failure

- transport safety

- familiar chemicals

- routine epidemics

- well tried software

- unfamiliar agents / vectors

- excluded conditions

- human actions / intentions

- complex systems

- open dynamic contexts

SALIENT POSSIBILITIES DEFINED

RISK

all relevant

probabilities

are known

UNCERTAINTY

not all relevant

probabilities

are known

Aspects of Incertitude after: Collingridge, Faber, Funtowicz

Keynes, Knight, O’Neill, Proops,

Ravetz, Wynne…

POSSIBILITIES UNDERDEFINED

… many analytic schemes for problems deriving probabilities – in terms of:

- source: context; data; model; expertise;

- levels: statistical; scenarios; scientific;

- nature: epistemic; ontological, normative;

- technical: inputs; structure; parameters;

- practical: variability; sensitivity; precision

- locus: institutional, moral, legal, situation;

… RIVM, van Asselt, de Marchi


RISK

EMPIRICAL UNCERTAINTY

intrinsic to data

THEORETICAL UNCERTANTY

inherent in science or

models

after: Collingridge, Faber, Funtowicz


Ravetz, Wynne…

UNCERTAINTY

not all relevant

probabilities

are known

Aspects of Incertitude


RISK UNCERTAINTY AMBIGUITY

constituting, bounding,

partitioning or ordering of

salient possibilities is unclear or contested


empirical uncertainty

theoretical uncertainty




Ravetz, Wynne…


IGNORANCE

at least some salient

possibilities are

indeterminate or

indeterminable

…divergent framings of ‘possibilities’ in appraisal, eg:

- basis: knowledge; experience; sensibility; expectation

- normativity: rights; interests; priority; values; virtues

- focus: trust; legitimacy; authenticity; accountability; blame

- contexts: marginal-routine; transformative-catastrophic

- ordering: benefit; harm; scenarios; principles; norms

-distribution: fairness; equity; equality; tolerance; plurality

- ethics: consequences; utility; instrumental; deontic; lexical

RISK UNCERTAINTY IGNORANCE






Ravetz, Wynne…

AMBIGUITY

salient possibilities are

unclear or contested



RISK UNCERTAINTY IGNORANCEAMBIGUITY


theoretical uncertainty UNCHARACTERISABILITY

problematic definition, bounding or partitioning

NONCOMPARABILITY

problematic basis for comparison

INCOMMENSURABILITY

problematic aggregate relative orderings

UNQUANTIFIABILITY

problematic aggregate cardinal ratios &increments




Ravetz, Wynne…



ignorance is rarely explicit in CBA, thus effectively denying even possibility of:

- unknowns - surprise - novelty - new alternatives or unexpected conditions.

But these are central to sustainability:

- CFCs and ozone depletion; - robustness of cross-species TSEs; - endocrine disruption as new mechanism

RISK UNCERTAINTY

OPEN IGNORANCE

acknowledged

IGNORANCE

CLOSED IGNORANCE

unacknowledged

AMBIGUITY

uncharact-erisability

noncom-parability

ordinal inco-mensurability

cardinal un-quantifiability






Ravetz, Wynne…



only rarely discussed… but then

tends to be in terms of ‘reducibility’.

This assumes that knowledge is

always additive and increased by

research. But this not always true

(eg: CFCs, TSEs, EDCs)



noncom-parability








Ravetz, Wynne…

OPEN closed

ACCESSIBLE INACCESSIBLE



eg:

CFCs

eg:

TSEs; EDCs



noncom-parability





INSTITUTIONAL IGNORANCE

salient knowledges available, but not accounted for in policy

SOCIETAL IGNORANCE

knowledges accessible, but

not yet available




Ravetz, Wynne…

OPEN closed

ACCESSIBLE INACCESSIBLE



RISK UNCERTAINTY

institutional ignorance

societal ignorance

AMBIGUITY


noncom-parability





PHENOMENOLOGICAL IGNORANCE

intrinsic to nature of world

EPISTEMIC IGNORANCE

intrinsic to nature of understanding




Ravetz, Wynne…

IGNORANCE

INACCESSIBLE

OPEN closed

ACCESSIBLE





societal ignorance

AMBIGUITY

EPISTEMICPHENOMENOLOGICAL


noncom-parability





INDETERMINACY eg: genotypic change



Ravetz, Wynne…

INACCESSIBLE

OPEN closed

ACCESSIBLE


DETERMISTIC CHAOS

eg: butterfly effect




societal ignorance

indeter-minacy

deterministic chaos

AMBIGUITY



noncom-parability





HERMENEUTIC IGNORANCE intrinsic to language

AXIOMATIC IGNORANCE

inherent in assumptions

LOGICAL IGNORANCE

incomplete logic system



Ravetz, Wynne…

OPEN closed

INACCESSIBLEACCESSIBLE





Ravetz, Wynne…

UNCERTAINTY IGNORANCE


societal ignorance

indeter-minacy

deterministic chaos

hermeneutic ignorance

axiomatic ignorance

logical ignorance

AMBIGUITY




noncom-parability






CBA TENDS TO TREAT ALL INCERTITUDE (IF AT ALL) AS RISK

RISK

INACCESSIBLEACCESSIBLE

OPEN closed


POSSIBILITIES UNDERDEFINED SALIENT POSSIBILITIES DEFINED

IGNORANCE



Ravetz, Wynne…

BUT NEGLECTED ASPECTS CAN ALSO BE EFFECTIVELY ADDRESSED

inaccessibleACCESSIBLE

OPEN

RISK UNCERTAINTY AMBIGUITY

humility and transparency

closed


societal ignorance

research and monitoring

transdisciplinary learning

precautionary appraisal

participatory deliberation


reductive aggregation

unproblematic

problematic

unproblematic problematic

knowledge about likelihoods

knowledge about possibilities

some specific, concrete practical lessons for CBA

Responses to Incertitude

unproblematic

problematic




RISK

UNCERTAINTY

AMBIGUITY

IGNORANCE

risk assessment , cost-benefit analysis decision theory optimising models



unproblematic

problematic



AMBIGUITY

IGNORANCE

RISK

UNCERTAINTY


uncertainty heuristics

interval analysis

sensitivity testing




scenarios / backcasting

interactive modeling

mapping / Q-methods

participatory deliberation

unproblematic

problematic



AMBIGUITY

IGNORANCE

RISK

UNCERTAINTY



interval analysis

sensitivity testing




unproblematic

problematic



AMBIGUITY

IGNORANCE

RISK

UNCERTAINTY



interval analysis

sensitivity testing

scenarios / backcasting

interactive modeling

mapping / Q-methods

inclusive engagement

monitor, surveil, research

diversity, flexibility, learning

resilience, adaptability




unproblematic

problematic




RISK

UNCERTAINTY

AMBIGUITY decision rules aggregative analysis deliberative process political closure

reductive modelingstochastic reasoningrules of thumbinsurance

` evidence-basing agenda-setting horizon scanning transdisciplinarity

liability lawharm definitions indicators / metrics institutional remits

‘Reductive aggregation’ presents powerful means to justify decisions This ‘closing down’ around risk is Beck’s “organised irresponsibility”

Institutional Pressures for Closure

IGNORANCE

unproblematic

problematic



Precaution and Participation as Analytical Rigour

PRECAUTIONARY APPRAISAL

PARTICIPATORY DELIBERATION

REDUCTIVE AGGREGATION

RISK

UNCERTAINTY

AMBIGUITY

IGNORANCE


HUMILITY, LEARNING,

ENQUIRY

TRANSDISCIPLINARY REFLEXIVITY AND RIGOUR

extend scope additive, cumulative, synergistic effects; life cycles, compliance real world effects: CFCs, DES; ‘closed systems’: MTBE, PCBs

humility on science sensitivities & proxies: mobility, persistence, bioaccumulation omission of persistence in organochlorines,

MTBE, CFCspro-active research prioritise open monitoring & surveillance & targeted experiment neglected: TBT, BSE; no monitoring: asbestos, benzene,

PCBsdeliberate argument levels of proof, burden of evidence, onus of persuasion Swann committee on antimicrobials, 1967 later ignored

alternative options pros, cons, justifications for range of options & substitutes ALARA, BAT, BPM – ionising radiation, fisheries,

acid rain

engage public independence through pluralism and robustness on values benzene, DES, asbestos, acid rain, fisheries

X-discipline learning collect all relevant knowledge, beyond ‘usual suspects’ MTBE / engineers; BSE / vets (clinical / toxicology /

epidem.)

Precaution as ‘Broadening Out’ Appraisal (cf: EEA, 2001)

Deliberative process (not just ‘decision rule’) transcends CBA

explicit incertitude explicitly engage with uncertainty, ambiguity and ignorance

organic

environmental

intensive

GM + labelling

GM + monitoring

GM + voluntary controls

low sustainability high

‘Mapping’ Key Uncertainties in Sustainability AppraisalDivergent expert views of risks and benefits of different agricultural strategieselicited using multicriteria mapping method in Unillever-sponsored research (2003)

A Practical Example

organic

environmental

intensive

GM + labelling

GM + monitoring


organic

environmental

intensive

GM + labelling

GM + monitoring




A Practical Example

GOVERNMENT

organic

environmental

intensive

GM + labelling

GM + monitoring


organic

environmental

intensive

GM + labelling

GM + monitoring



A Practical Example


GOVERNMENT INDUSTRY

organic

environmental

intensive

GM + labelling

GM + monitoring


GOVERNMENT INDUSTRY

organic

environmental

intensive

GM + labelling

GM + monitoring


PUBLIC INTEREST


A Practical Example


- Move beyond ‘sound science’ aspiration and rhetorics

explicitly acknowledge importance of framing in analysis real world effects: CFCs, DES; ‘closed systems’: MTBE, PCBs

Conclusions

Address uncertainty, ambiguity & ignorance in appraisal

- Acknowledge that CBA is highly circumscribed as appraisal method

‘horses for courses’ – contrasting tools for different contexts

- Be reflexive about dynamics of closure in appraisal

pressures for decision justification

- Recognise intrinsic rigour of precaution and participation

for ‘broadening out’ and ‘opening up’ appraisal of sustainability

Seize a rare opportunity

shared imperatives of analytical rigour and democratic accountability

Education

Andy Stirling - Addressing Uncertainty, Ambiguity and Ignorance in Sustainability Appraisal