View
44
Download
0
Category
Tags:
Preview:
DESCRIPTION
Defining and Evaluating ‘Science for Sustainability’. AIRP-SD Project Paul Weaver & Leo Jansen pweaver@noos.fr jansenleo@hetnet.nl. AIRP-SD Goals. Develop an approach for ex ante , concomitant and ex post evaluation of sustainability-oriented RTD programmes - PowerPoint PPT Presentation
Citation preview
NZSSES/ICSER Conference
Defining and Evaluating Defining and Evaluating ‘Science for Sustainability’‘Science for Sustainability’
AIRP-SD Project
Paul Weaver & Leo Jansenpweaver@noos.fr
jansenleo@hetnet.nl
NZSSES/ICSER Conference
AIRP-SD GoalsAIRP-SD Goals
Develop an approach for ex ante, concomitant and ex post evaluation of sustainability-oriented RTD programmes
Describe innovative RTD approaches Identify good and bad practices Assess transferability of practice Establish programme design guidelines
NZSSES/ICSER Conference
Toward an evaluation architectureToward an evaluation architecture
Lack of theory and practice suggests need to develop both through an iterative, heuristic procedure – begin with tentative hypotheses about the nature
of the research processes that might support SD; test/refine these through the evaluation process itself
Need for externally-specified evaluation criteria Depends on establishing a conceptual model of
sustainability-oriented RTD
NZSSES/ICSER Conference
AIRP-SD Procedure
Establish conceptual model of sustainability research
Select case study research programmes/projects
Describe research outcomes, designs, contexts
Evaluate outcomes, design, context separately
Relate research outcomes to design
Relate design to context
Relate outcomes to context Revise conceptual m
odel
Recommendations for research management and design
Identify good practice
NZSSES/ICSER Conference
Derive conceptual model from:Derive conceptual model from:
the nature and concept of sustainable development
the nature of the socio-ecological systems of interest
the nature of SD challenges to science and society
NZSSES/ICSER Conference
SD challengesSD challenges
Orientation and ambition - paradigm change - transition - prescriptive - normative - vision-led
Complexity of socio-ecological systems of interest, indeterminacy, irreversibility, risk, precaution
Multiple stakeholders, high stakes, uncertainty, non-market values
Need to communicate with non-scientists
NZSSES/ICSER Conference
Evaluation objects (outcomes)Evaluation objects (outcomes)
Outcomes
Contribution to strategic orientation
Contribution to solution set (quantity/quality)
Contribution to strategic
management
Contribution to social capital
Contribution to scientific capital and capacities
Contribution to influencing society
NZSSES/ICSER Conference
Evaluation objects (D&P)Evaluation objects (D&P)
Design and Process
Management and finance model
Procedure for initiating/defining programme
Research model (methods, tools,
procedures)
Procedure for handling risk and uncertainty
Procedure and basis for decision making
Procedure for communication & dissemination
NZSSES/ICSER Conference
Evaluation objects (context)Evaluation objects (context)
Context
Established social capital
Established RTD management capital
Established scientific/research
capital
Extent & urgency of unsustainability or vulnerability
Understanding & awareness of (un) sustainability across society
NZSSES/ICSER Conference
Sustainability principlesSustainability principles
Global and systemic approaches Knowledge sharing and mutual learning Broad participation Environmental heritage Precaution / Action to prevent risk Resilience
NZSSES/ICSER Conference
Sustainability principles (cont.)Sustainability principles (cont.)
Transparency and justification of decision making
Appropriate scale-issue relation Inter-generational equity Intra-generational equity
NZSSES/ICSER Conference
AIRP-SD working hypothesesAIRP-SD working hypotheses
Hypothesis I
A wide set of generic research outcomes can be identified and evaluated on a set of externally specified sustainability goals and criteria
Hypothesis II
Research designs and processes that conform strongly with the principles of sustainable development are likely to contribute to strong research outcomes
Hypothesis III
Contexts where sustainability problems or awareness is high are likely to facilitate SD research. Strong scientific/social capital in relation to SD challenges will facilitate SD research and strong outcomes
NZSSES/ICSER Conference
Hypothesis IVHypothesis IV
CONTEXT
Research design and process
Research outcomes
Research context, design/process and outcomes are connected in a loop
NZSSES/ICSER Conference
Nine case studiesNine case studies
ALR ATSD CUTS ECO-CYCLE HABIFORUM MAHRE SEP STD ZERI
NZSSES/ICSER Conference
Fact finding versus evaluationFact finding versus evaluation
Separate tasks Data collection – factual information – What?
How? Who? Evaluation – analytical, judgmental – Whether?
How well? Evaluative questions shape the mindset and
background for fact-finding
NZSSES/ICSER Conference
Methodological issuesMethodological issues
Restricted time-frame for analysis; some outcomes yet to emerge: interim outcomes, process-orientation
Ambiguities; e.g., financing model: detailed specification and differentiation
Subjectivity: consistent definitions, inter-subjectivity and cross-checking
Depth vs breadth: encourage evaluation to enable validation in due course
NZSSES/ICSER Conference
Multi-level approach (outcomes)Multi-level approach (outcomes)
N ove ltyG en era lisab ility
A ffo rd ab ilitye tc .
N ew p rod u c tion sys tem sN ew su ffic ien cy so lu tion s
e tc .
R an g e o f so lu tion s
C om p atib ilityIn teg ra tion
B a lan ce , e tc .
G lob a l issu esE coeffic ien cyE q u ity, e tc .
Q u a lity o f so lu tion s
C om p atib ilityD irec tion a lly re liab le
A ccep tedA d ap tab le , e tc
F u tu re vis ion sM etap h ors o f life q u a lity
In d ica to rs
S tra teg ic reo rien ta tionan d p a th fin d in g
S ys tem s-leve lP rocess o rien ted
C on tin g en c iesB arrie rs /b o tt len ecks
S cen ariosTech n o log y assessm en t
A c tion p lan s , e tc .
S tra teg ic m an ag em en t
LEVEL III
LEVEL II
LEVEL I
OUTCOM ES
NZSSES/ICSER Conference
Evaluation of each objectEvaluation of each object
Evaluate each outcome using SD goals Evaluate each D&P feature using SD principles Evaluate each context feature using SD
challenges Strength (strong, weak, neutral) Direction (positive, negative)
NZSSES/ICSER Conference
Evaluation of interrelationsEvaluation of interrelations
Evaluate influence of each D&P feature on outcomes
Evaluate influence of each D&P feature on each other (synergies)
Evaluate influence of context (context dependency, transferability) of each D&P feature
Evaluate any RTD induced change in context
NZSSES/ICSER Conference
Data analysisData analysis
NZSSES/ICSER Conference
Results: good practicesResults: good practices
Initiation– stakeholder inventory/analysis– broad participation– re-conceptualisation, scope/goals
Finance– co-funding arrangements– dedicated funds for re-conceptualisation,
stakeholder participation, communication– phased funding arrangements
NZSSES/ICSER Conference
Results: good practices (cont.)Results: good practices (cont.)
Management– internal learning (iterative, adaptive)– openness, transparency, accountability– continuous monitoring and quality assurance– re-defined set of success criteria
Research– vision-led, system-based, long-term oriented,
inter/trans-disciplinary, iterative– multi-functional approaches/tools
NZSSES/ICSER Conference
ConclusionsConclusions
Importance of context Importance of structure and process Importance of goal setting Importance of SD principles Synergy among design features - need for
thorough integration Feedback - practice makes perfect
NZSSES/ICSER Conference
Defining and Evaluating Defining and Evaluating ‘Science for Sustainability’‘Science for Sustainability’
AIRP-SD Project
Paul Weaver & Leo Jansenpweaver@noos.fr
jansenleo@hetnet.nl
Recommended