Premises and practices in combining quantitative and qualitative FTA methods Anchor paper Theme 3

Premises and practices in combining quantitative and qualitative FTA methods - Anchor paper Theme 3

Premises and practices in combining quantitative and qualitative FTA methods

Anchor paper Theme 3

Fabiana Scapolo & Andrea Ricci

[European Commission Joint Research Centre & ISIS Institute of Studies for the

Integration of Systems]

The 4th International Seville Conference onFuture-Oriented Technology Analysis (FTA)

12 & 13 May 2011

Premises and practices in combining quantitative and qualitative FTA methods - Anchor paper Theme 3Premises and practices in combining quantitative and qualitative FTA methods - Anchor paper Theme 3

OutlineOutline

1. The methodological debate in FTA2. Current practises in combining quantitative and

qualitative methods in FTA3. Barriers to integration of methods and how to

overcome them4. Recent developments in FTA methods

5. Research questions


The methodological debate in FTAThe methodological debate in FTA• A relevant element of the series of the FTA conferences since its first

edition in 2004

• The increased policy demand for robust evidence for decision making, indicate that there is scope for methodological integration in order to increase relevance of FTA outcomes for policy, business and society

• FTA differs from other policy-support techniques as it applies six distinctive principles:

– Future-oriented; participation, evidence based, multidisciplinarity; coordinated mobilisation of people and resources; action oriented

• Key aspect of the debate is the combination of qualitative and quantitative methods to produce rounded results by bringing together disciplines within FTA

– Type of data: numeric information vs. collective knowledge based on expert judgement

– Analytical objectives: numerical measurement and combination of empirical, deductive and experimental studies to obtain objective results vs. exploration of meanings and how experts understand present and future issues on the basis of their knowledge and experiences.


Current practices in combining quantitative Current practices in combining quantitative and qualitative methods in FTAand qualitative methods in FTA

• Lack of systematic stock taking of FTA practices mixing Q&Q approaches, and…

• Lack of evaluation guidance on FTA “mixed” studies• Growing interest of FTA community translates into varied

attempts, e.g.– Bibliometrics and patent–based assessments to detect emerging

technologies/breakthroughs– Visual inspection to identify hidden trends and patterns– Feeding expert judgment into quantitative models (narrative

visions to get the ball rolling)– Relying on the power of number crunching to detect consistency

and other phenomena “invisible to the naked eye/brain”


Barriers to integration of methods (1) Barriers to integration of methods (1)

• An epistemological divide– Are Q&Q compatible? Can and should they mix?– Forecast/predict Vs shape/forge– Disagreements reveal profound, cultural barriers

• A variety of futures communities– Foresight/forecast/technology assessment/forward looking– Competition Vs collaboration

• Adaptation of methods and tools– Building bridges (e.g. indicators) to increase compatibility– But may reinforce and consolidate the isolation of Q&Q

communities…



• Misconception 1: models may be Q or Q, but their predictive value always decreases as the time horizon stretches – the value of the process prevails over that of the ouptut

• Misconception 2: subjectivity is a resource for FTA, not a disturbance– Value judgments count (and are inevitable anyway, even in

numeric models through assumptions)– Future will be (partly) shaped precisely by the subjectivity of

decision makers



• Misconception 3: Participatory ≠ Qualitative– Quantitative models increasingly recur to consultation of

stakeholders groups– Inspiring storylines often written by visionary solitaires…

• The vicious triangle

Epistemological & cultural barriers

Epistemological & cultural barriers

Lack of trustLack of trust Lack of skillsLack of skills


Addressing the barriersAddressing the barriers

– Identifying priority areas where Q&Q can be integrated more smoothly and readily

• Functional (scenario development, ex ante impact assessment)• Sectoral (energy, transport, climate change)

– Clarify/disambiguate the scope and time limits of Q&Q (short Vs long term)

– Assess and represent existing interfaces

– Enforce systematic and extensive participation of stakeholders in all steps of FTA

– Promote the creation and use of a stock taking repository of FTA methods and practices

– Increase trust through enhanced credibility• sell the process, not the outcome• Seek consensus on the need for trade offs

– Turn the cultural clash into an asset


DevelopmentsDevelopments of FTA methodsof FTA methods

• New tools and methods, novel applications of existing ones and (new) disciplines entering FTA

• New technologies e.g. web 2.0 can be used to increase the possibility of interactive participation of stakeholders and speed up the provision of information and feedbacks

• Use of collaborative tools e.g. social web platform are still limited not only in FTA but in general to link policy and research


Combining qualitative and quantitative Combining qualitative and quantitative methodsmethods

• Millennium Ecosystem Assessment Scenarios

• Objective: provide decision makers and stakeholders with scientific information on the links between ecosystem change and human well-being

• Methods: combination of storylines developments and quantitative modelling

– Scenarios captured aspects of ecosystem that are not possible to quantify or difficult to be expressed in quantitative terms i.e. complex aspects of society and ecosystem

– Models helped ensuring consistency of storylines and provided numerical information where quantification was possible

• Lessons learned: – scenarios helped to identify areas where analytical tools are weak and can be

improved– Advance model developments on ecosystem service to further disaggregate

services to local scales, address cultural and supporting ecosystem services and consider feedbacks between ecosystem change and human development


Combining Foresight and participatory Combining Foresight and participatory Technology Assessment Technology Assessment

• Citizens visions and experts and stakeholders • Objective: develop a list of new and emerging issues for European

S&T, policy options for future European research programme• Focus: demand side (i.e. needs and trends of society and societal

developments)• Methodological approach in three steps:

– citizens panels of (seven Member States) developed long-term view on needs, wishes, concerns and challenges on the future (69 visions)

– Experts and stakeholders analysed visions and transformed them into future research agendas and provided a list of recommendations that was derived with a stream model widely applied in policy analysis

– List returned to citizens to be validated and prioritised

• Advantages: opening up possibilities for a more structured involvement of citizens in the definition of future European policies


FTA and strategic designFTA and strategic design• Strategic design (or design thinking) address societal challenges that are falling

at the intersection of current knowledge– It applies future-oriented design principles to increase organisation innovative and

competitive qualities, suitable for FTA at corporate and business but applied by Finnish Innovation Fund to develop holistic understanding on issues related to ageing, education and sustainability

– Proposes itself as a capability to enhance innovation in contexts where typical decision-making practices have been ineffective at meeting the demands of the challenge ahead

– Principle: issues co-evolve with the understanding of the intrinsic elements that characterise these issues. The space where is co-evolution happens is the overlap between analysis and execution

– It offers a way of understanding problems and solutions in a feedback loop rather that linear relationship (i.e. the traditional 1:1 relationship).

• A fragment increase of our understanding of a challenge enables better ideas towards its resolution

• A partial solution can inspire new questions about the problem space (e.g. the challenge) and that is when the cycle starts again

– Similarities with FTA: • open-mindedness looking for insights in atypical places without favouring hard facts over soft

evidence; • a non-linear way of working which involves cycling between refining the understanding of the

problems and searching for new solutions;


Foresight and social network analysisForesight and social network analysis• Social network analysis reveals relationships and links that make up

various social processes. In S&T to understand clusters of collaboration and diffusion of networks by focusing on social relations amongst a set of actors

• Network analysis can be used as a useful tool in Foresight to:– Analyse data (helps codification and enable robust analysis)

– Reveal structural features of the data, inform the process on emerging links or relationships, groups or clusters

– Introduce a ‘systemic’ perspective by emphasising relationships between actors, key issues and trends

– Provides benefits in various phases of the process (i.e. scoping, participation, generation, action and evaluation)


FTA & social scanning and prediction marketsFTA & social scanning and prediction markets

• Improve professional forecasting and foresight in an era of complex phenomena and disruptive events with high level of uncertainties

– Prediction markets: could encourage to think more clearly about impacts of discrete events, what discrete events are needed to reach a particular future, how they believe different trends will interact (avoid biases)

– Scanning: should be made public as it would allow people concerned with complex phenomena or issues at the interplay between science and society to keep track of the content of each other intellectual portfolio of knowledge


Questions for the debateQuestions for the debateUnderstanding the problems• Do researchers – and all the communities under the FTA umbrella (and beyond) - have an

interest in integrating methods and practices?

• Do misconceptions between communities increase the epistemological barriers?

• What is the impact of the educational systems on the qualitative and quantitative divide?

Moving forward• Which areas and sectors have high potential for increasing transparency and convergence

of quantitative and qualitative methods?

• How can participatory approaches be designed to balance quantitative and qualitative dimensions?

• Which evaluation approaches are needed to identify good practices in the integration of methods, and to select the best combination of methods?

• How can transparency and trust be increased between the FTA communities and users?

Documents

Premises and practices in combining quantitative and qualitative FTA methods Anchor paper Theme 3