Andy Stirling - nexus methods

‘Nexus Methods’

opening up democracy as rigour

Andy StirlingSPRU & STEPS Centre

University of Sussex

presentation to conference session on Nexus Methods ESRC Methods Festival

University of Bath5th July 2016

www.steps-centre.org/www.sussex.ac.uk/spru/www.multicriteriamapping.com www.sussex.ac.uk/spru/people/peoplelists/person/7513

http://www.steps-centre.org/

http://www.sussex.ac.uk/spru/

http://www.multicriteria-mapping.org/

http://www.sussex.ac.uk/spru/people/peoplelists/person/7513

How Shall We Play This?

1000

1130

1100

1130

1200

1230

STEPSMethods talk then discussion Andy Stirling

Open Spacevolunteersconvene groups

Ses

sion

1br

eak

Ses

sion

2

Nexus Methodology talk then discussionAndy Stirling

Open Spacevolunteersconvene groups

STEPSMethods Andy Stirling

OPTION 3OPTION 1 OPTION 2

What is ‘the Nexus’?

food

water

energy

What is ‘the Nexus’?

food

water

energy

Substantive imperatives for ‘joined up policy…?

… or instrumental pressure for policy justification?

What is ‘the Nexus’?climate

developmentfood

environment

water

security

globalisationenergy

population

migrationurbanisation

Phenonema Under Scrutiny

• social and material world

•“systems” and “contexts”

• “scales” and “levels”

• “actors” and “networks”

• “values” and “interests”

• “frames” and “narratives”

• “causes” and “effects”

• processes and relations

• “knowledges”,“incertitudes”

• “positives” and “negatives”

• “structures” and “agents”

• “actions” and “reactions”

• “imaginations” and “visions”

• “metrics” and ”indices”

Complexity is not made by pathways methods…

…but addressed by them – brings a need for “reflexivity”

cost-benefit analysisrisk assessmentdecision analysismultiattribute utility theory technology assessmentlife cycle analysisoptimisation modellingBayesian networksextended accountingdata miningdelphi methods

Quantitative Culture Qualitative Culture

A Juicy Opportunity for Appraisal Disciplines!

No shortage of candidate ‘Nexus methods’

scenario workshopsfocus groups

participatory appraisalstakeholder deliberation

ethnomethodologycollaborative design

capabilities assessment strategic appraisal

action researchcooperative research

study groups

Energy regulation: most mature, sophisticated comparative analysis…

A Key Common Problem: Concealed Ambiguity

0.001 0.1 10 1000externality’: cUS/kWh (after Sundqvist et al, 2005)low RISK high

coal

oil

gas

nuclear

hydro

wind

solar

biomass



0.001 0.1 10 1000

coal

oil

gas

nuclear

hydro21

wind

solar

biomass

n =

‘externality’: cUS/kWh (after Sundqvist et al, 2005)

minimum maximum25% 75%

low RISK high



coal

oil

gas

nuclear

hydro

36

20

wind 18

solar 11

biomass 22

31

21

16

n =

…‘sound science’, ‘evidence based’ Nexus analyses justify many policies



All Quantification is Qualitatively FramedEqually true of qualitative research, but this is better appreciated

under-determined realities

Framing ‘the Nexus’

‘system’‘focus’

under-determined realities picture of problem/solution


‘scope’




‘scope’


‘cause’

‘effect’



environment‘scope’


subjective framing

‘effect’


‘cause’view 1

picture of problem/solution


environment

‘system’

under-determined realities diverse picturesplural frames

‘effect’

‘cause’

‘scope’

‘focus’

view 2

view 1

conte

nding

know

ledge

s

and v

alues


plural frames

‘system’

‘cause’

‘effect’

under-determined realities diverse pictures

‘scope’

‘focus’

view 3

view 1

view 2

conte

nding

know

ledge

s

and v

alues


view 4

plural frames

‘system’

‘cause’

‘effect’

under-determined realities diverse pictures

‘scope’

‘focus’

conte

nding

know

ledge

s

and v

alues

view 1

view 2

view 3


incumben

ts

power in knowledge

‘system’

‘cause’

‘effect’


academic

government

local people

diverse pictures

‘scope’

‘focus’

conte

nding

know

ledge

s

and v

alues


“THE NEXUS”

fundamental nature

biophysical environments

societies & economies

cultures &

institutions

Science (of all kinds!) is a key means by

which to remind “the real world”

of politics about the

“real real world” of everything else

The Conditionality of Assessment


knowledge practices

fundamental nature



cultures & instituions

But knowledge is produced by people,

with cultures, pursuing practices

in institutions

So it is jointly shaped to reflect

both social and natural orders

KNOWLEDGE PRACTICES

knowledge practices

fundamental nature



INSTITUTIONS


This leads to a paradox

What is inside…… is also outside

knowledge practices



INSTITUTIONS

The Knowing-Known Dualism in Assessment

This leads to a paradox

What is inside…… is also outside

… as in a Klein Bottle, knowledge is on the

inside and the outside of human action

KNOWLEDGE PRACTICES

‘Reflexivity’

A capacity for self reflection…

… understanding how answers depend on questions

reflexivity over subjectivity, contingency, contextuality

refle

ctio

n ov

er ro

bust

ness

, qua

lity,

err

or‘Reflexivity’ is not about ‘Anything Goes’

normatively and/or epistemically wrong

normatively and/or epistemically right

CONCEPTUAL SPACE OF ALL CANDIDATE POSSIBILITIES

FOR WHAT IS NORMATIVELY OR EPISTEMICALLY ‘RIGHT’

KEY

normatively and/or epistemically wrong

normatively and/or epistemically right

naïve realism

“one thing is precisely right”

mainstream fallibilism

“one thing is approximately right”

caricature relativism

“all things are equally right”

conditional plurality

“contrasting things are equally right… xxxx…but much else is just plain wrong!”

reflexivity over subjectivity, contingency, contextuality

refle

ctio

n ov

er ro

bust

ness

, qua

lity,

err

or‘Reflexivity’ is not about ‘Anything Goes’

reflexivity: ‘right’ answers & solutions vary by framings of questions & problems

molecular biologists

ecologists economists

sociologists

Review of >100 different kinds of ‘Nexus-related method’

Background Approach

- epistemic cultures: quantitative / hybrid / qualitative

- styles of reasoning: deductive / inductive / abductive

- relations & practices: analytic / interactive

- procedural functions: frameworks / techniques / tools

RESEARCH METHODSprimarily qualitative mostly quantitative

EPISTEMIC CULTUREexplicitly mixed

MO

DE

OF

ENQ

UIR

Yla

rgel

y de

duct

ive

chie

fly a

bduc

tive

mai

nly

indu

ctiv

eRESEARCH METHODS

QUALITATIVE APPREHENSION HYBRID APPRAISAL EMPIRICAL QUANTIFICATION

GROUNDED METHODS

INTERPRETIVE ANALYSIS QUANTITATIVE ASSESSMENT

APPRECIATIVE APPROACHES

MIXED THEORY

EVALUATIVE TECHNIQUES

integrated assessment

largely qualitative mostly quantitative EPISTEMIC CULTURE

explicitly mixed

STYL

E O

F EN

QU

IRY

dedu

ctiv

e st

yle

abdu

ctiv

e st

yle

indu

ctiv

e st

yle QUALITATIVE APPREHENSION HYBRID APPRAISAL EMPIRICAL QUANTIFICATION

GROUNDED METHODS


social assessment

longitudinal methods

cladistic taxonomy

natural experiment

value chain analysis

monitoring

multi-sited ethnography

scientometrics

surveillance data mining

econometrics

case studies

network analysis

narrative analysis

historiography

comprehensive archive research

historical method

discourse analysis

process tracing

situational analysis

studying up

systems theory`

soft systems theory`

information theory

meta analysis

life cycle approaches

optimisation modelling

Bayesian models

probabilistic risk analysis

uncertainty analysis


MIXED THEORY

EVALUATIVE TECHNIQUESenvironmental

assessmentevaluation methods

statistical methods

spatial analysis

critical pedagogy

technology assessment

panel studies

social indicators

technometricscontent analysis

social experiment

randomised control trials

remote sensing

agent-based modelling

needs analysis

road-mapping

ethno-methodology

action research

counter-factuals

co-word analysis

RESEARCH METHODS

innovation histories

evidentiary presumptions

systems histories`

precautionary appraisal

critical accounting

influence mapping

post-normal science

resilience analysis

alternatives assessment

critical triangulation

cross-frame analysis

imaginaries analysis

arts catalyst

capabilities assessment

sensitivity analysis

key: analytic / interactive method

costbenefit analysis

ecological footprint

integrated assessment

largely qualitative mostly quantitative EPISTEMIC CULTURE

explicitly mixed

STYL

E O

F EN

QU

IRY

dedu

ctiv

e st

yle

abdu

ctiv

e st

yle

indu

ctiv

e st

yle QUALITATIVE APPREHENSION HYBRID APPRAISAL EMPIRICAL QUANTIFICATION

GROUNDED METHODS


social assessment

extended foresight longitudinal

methods

cladistic taxonomy

natural experiment

value chain analysis

monitoring

multi-sited ethnography

scientometrics

surveillance data mining

econometrics

delphi analysis

photo-elicitation

case studies

network analysis

narrative analysis

historiography

comprehensive archive research

historical method

discourse analysis

process tracing

situational analysis

studying up

systems theory`

soft systems theory`

information theory

meta analysis

life cycle approaches

optimisation modelling

Bayesian models

probabilistic risk analysis

uncertainty analysis


MIXED THEORY

EVALUATIVE TECHNIQUESenvironmental

assessmentevaluation methods

statistical methods

spatial analysis

opinion surveys

attitudinal scaling

structured deliberation

dissensus groups

citizen science

crowd sourcing

scenario workshops

futures visioning

performative approaches

focus groups

semi-structured interviews

critical pedagogy

decision analysis

contingent valuation

multicriteria mapping

Q methodgaming

techniques

open space

technology assessment

constructive technology assessment

interactive diagrams

panel studies

social indicators

technometricscontent analysis

social experiment

experimental economics

randomised control trials

remote sensing

interactive visualisation

interactive modelling

agent-based modelling

paricipatory backcasting

needs analysis

road-mapping

ethno-methodology

situational activism

action research

biography

counter-factuals

co-word analysis

participatory theatre

RESEARCH METHODS

social multicriteria analysis

stakeholder negotiation


concurrent evidence

evidentiary presumptions

systems histories`

precautionary appraisal

critical accounting

do-it-yourself juries

influence mapping

post-normal science

resilience analysis

alternatives assessment

critical triangulation

cross-frame analysis

power tools

imaginaries analysis

deliberative polling

diversity mapping

co-operative research

collaborative design

arts catalyst

community art

participatory rural appraisal

study groups

capabilities assessment

planning cells


key: analytic / interactive method

costbenefit analysis

ecological footprint

INPUTS

(aspects taken into account within practice of research or appraisal)

problems, options, pros / cons, issues, uncertainties, perspectives

INPUTS

(things that are taken into account)

Pro

blems, options, pros / cons, issues, uncertainties, perspectives

narrow

broad

closing down opening up

expert / analytic

participatory / deliberative

OUTPUTS

(aspects that are conveyed outwards into wider discourse)

’Plural conditional’ conclusions…

… if X then A … if Y then B …

Rigour in Nexus-Focused Methodologies

Combining reflexivity about framing and reflection about error in informing policy and politics

narrow

broad


expert / analytic


decision analysis

INPU

TS

OUTPUTS

sustainability

safety

impacts



narrow

broad


expert / analytic


citizen’s juries

INPU

TS

OUTPUTS



narrow

broad


expert / analytic


citizen’s juries

participatory appraisal

q-method

scientometric mapping

open space


extended foresightcitizen’s juries

decision analysis


participatory sensitivity analysis

cost-benefit analysis

risk assessment


structured interviews

participant observation

multi-site ethnographic-

methods

citizen’s juries

consensus conference

open hearings

concurrent evidence

critical narratives

intervention futures

PIPA

plural photovoice

system histories


INPU

TS

OUTPUTS



spot-the-narrative

narrow

broad


expert / analytic


citizen’s juries

decision analysis

participatory rural appraisal


q-method


deliberative mapping

scientometric mapping

open space


risk assessment


structured interviews

narrative-based participant observation

multi-site ethnographic-

methods

citizen’s juries

consensus conference

open hearings

dissenting opinions

multi-criteria mapping

extended foresight



risk assessment

INPU

TS

OUTPUTS

Power Closes Down Nexus Research and AppraisalPressure to command authority, foster trust, secure acceptance, manage blame

expert / analytic


Challenges for Research and Appraisal

Reflexively means actively balancing bias in conventional appraisal

narrow

broad


participatory appraisal

q-method open space


participatory sensitivity

analysis

intervention futures PIPA

plural photovoice

system histories


INPU

TS

OUTPUTS

narrow

broad


expert / analytic


Power Closes Down Research and Appraisal

A responsibility for neutrality means independent innovation research and policy appraisal should deliberately counter pressures for closure

MONODISCIPLINARITY

eg: Q-method comes out of social psychology

INPU

TS

OUTPUTS

narrow

broad


expert / analytic




MONODISCIPLINARITY - enquiry is structured by community practices - institutionalised as “normal science” within paradigm - authority through self-confident coherence

eg: in Q-method analysis is subject to particular assumptions and conventions

INPU

TS

OUTPUTS

MONODISCIPLINARITY - enquiry is structured by community practices - institutionalised as “normal science” within paradigm - authority through self-confident coherence

Participatory methods can also be monodisciplinary

eg: participatory panel follows particular rules

narrow

broad


expert / analytic




INPU

TS

OUTPUTS

narrow

broad


expert / analytic




CROSS-DISCIPLINARITY

- task- and context-specific - allows cross-critique and triangulation - authority through juxtaposition

eg:participatory panel providescomplement to discourse analysis

INPU

TS

OUTPUTS

MULTIDISCIPLINARITY

- enquiry structured hierarchically - impressive scope, reassuring applicability - authority through integration

eg:integrated assessmentglobal panels

narrow

broad


expert / analytic




INPU

TS

OUTPUTS

INTERDISCIPLINARITY

- enquiry structured by collaboration - attention targeted on focal problem - authority thro’ fidelity in addressing complexity

eg: collaborative networksmultiple partnerships

narrow

broad


expert / analytic




INPU

TS

OUTPUTS

TRANSDISCIPLINARITY

- enquiry reflects wider extant framings - flexible to divergent contexts - authority thro’ salience and legitimacy

eg:co-designed with stakeholders

narrow

broad


expert / analytic




INPU

TS

OUTPUTS

Mapping Perspectives for Deliberation Multicriteria Mapping ‘opens up’ politics and power in expertise

Analysis of 12 UK government GM advisors (2001)

Mapping Perspectives for Deliberation Multicriteria Mapping ‘opens up’ politics and power in expertise

Analysis of 12 UK government GM advisors (2001)

organicslow input intensive

GM 1GM 2GM 3


GM 1GM 2GM 3

UK Governmentecology chair


GM 1GM 2GM 3


GM 1GM 2GM 3

UK Governmentsafety chair

GM industry research executive

Green NGO scientist

Acknowledging assumptions, values, uncertainties ‘plural & conditional’ approach is rigorous & democratic

… if A, then x; … if B, then y

Multicriteria Mapping ‘opens up’ politics and power in expertise

Mapping Perspectives for Deliberation