Upload
frank-catanzaro
View
2.912
Download
2
Tags:
Embed Size (px)
DESCRIPTION
A Briefing on the Real Time Delphi software tool developed by Ted Gordon of the WFUNA Millennium Project
Citation preview
Ted GordonSenior Research FellowThe Millennium ProjectWorld Federation of United Nations Associations
First developed at RAND in the 1960’s A means of eliciting and combining expert
judgments while avoiding the pitfalls of conference room confrontations.
Used a sequential questionnaires, each building on the previous round.
Required features: Participants guaranteed anonymity Feedback of reason for extreme opinions
Has been used in thousands of studies
Economically useful desalination of sea water 1970 1964-1980
Effective fertility control by oral contraceptive 1970 1970-1983
Development of new synthetic ultra light materials 1971 1970-1978
Automated language translators 1972 1968-1976
New organs through transplanting or prosthesis 1972 1968-1982
Reliable weather forecasts 1975 1972-1988
Operation of a central data storage facility 1980 1971-1991
Reformation of quantum relativity 1980 1975-1993
Implanted artificial organs made of plastic and electronic components 1981 1975-1988
Widespread and socially acceptable use of psychotropic drugs 1983 1980-2000
Creation of a form of artificial life (at least in the form of self replicating molecules) 1989 1979-2000
Economically useful exploitation of the ocean bottom 1980 1980-2000
Feasibility of limited weather control 1990 1987-2000
Commercial generation of protein for food 1990 1985-2003
Drug treatment for psychotic cases 1992 1983-2017
Biochemical general immunization 1994 1983-2000 From Gordon and Helmer, A Long Range Forecasting Study, RAND, 1964
Sociologist Economist Technologist
Environment
Person 1 XXX XXX
Person 2 XXX
Person 3 XXX
Person 4 XXX
Person 5 XXX XXX
Person 6 XXX
Person 7 XXX XXX
Person 8 XXX
TOTAL 3 2 2 3
First developed in a Defense Advanced Research Projects Agency (DARPA) study by Gordon and Pease (2004).
The object was to invent a technique by which a commander could rapidly collect and synthesize opinions of field personnel, using Delphi, to help him reach a decision.
Work was published in Technological Forecasting and Social Change.
Since then, many improvements and adaptations.
An on-screen form presents a series of questions to each participant, and for each: The average (or median) response of the group
so far (and possibly the distribution of responses) The number of responses made so far A button that opens a window showing reasons
that others have given for their responses. A button that opens a window that allows the
respondent to type in justifications for their own answer.
A space for the new numerical estimate, answering the question.
There is only one round; no explicit second round is necessary as in a conventional Delphi.
When a respondent revisits the study in a minute or a day, the original input form is presented with their prior answers in place and the new average or median of the group, and if desired, the distribution of responses.
By then others may have contributed judgments, so the averages or medians may have changed since the original answers were provided.
If an answer is beyond a pre-specified distance from the average an indicator “flag” can be used to show the respondents the questions in which their answers differ significantly from the group.
When the flag is “up” reasons become particularly important
The respondents are not shown the group average until after they have inserted an answer and a selected number of responses have been received or a given time has elapse since the study has started.
The answers can be submitted cell by cell or as an entire document.
To avoid crowding, the reasons can be submitted and appear on a separate page.
A notation appears after the respondent has submitted an answer indicating that the response has been received and entered into the database.
Notations also appear if individual responses are outside of predetermined plausible ranges.
Extensive hyperlink reference material can be included to permit the respondents to retrieve data on line while completing the questionnaire.
Reference material is included directly on the questionnaire
Split samples can be used to reduce the effect of bias introduced by question placement on the questionnaire.
A permanent record is maintained of responses including both quantitative and narrative responses.
Collection of judgments about when certain developments might occur, their importance, and impacts and reasons for extreme opinions
Choosing from among alternative decisions based on clear cut criteria
Providing information for and sharpening of scenarios
Input/output and cross impact matrixes Environmental monitoring
Decision models: Real Time Decision Making. (DARPA) Choosing a Configuration. (auto mfg)
Priority studies: Assessing Priorities of Proposed Millennium Project Study
Topic, 2006. Assessing Priorities of Proposed Millennium Project Study
Topic. 2007 Assessing Priorities of Proposed Millennium Project Study
Topic. 2008 SOFI studies:
Variables for the Global State of the Future Index Variables for the Republic of Korea State of the Future Index Variables for the South Africa Global State of the Future
Index
Substantive studies: Future education and learning possibilities Future Energy Developments
Studies in support of UNESCO:
Judgments on acceptability of a Delphi process. Scenario Drivers. Report construction
Resource allocation (WB subcontractor) Performance Biodiversity Climate Change
Possible Development Likelihood by 2030 (%)
1. National programs for improving collective intelligence
Some richer as well as lower income countries have (by this year of 2030) made improving collective intelligence a national goal; this includes improving individual as well as intelligence for their nations-as-whole. Click here to see references
Likelihood by 2030 The average group answer: 61.7Respondents: 195
Comments on this possibility (page 2) click here
2. Just in time knowledge and learning
Rote learning has diminished in importance. With ubiquitous computing and education for life-long learning, 'just in time knowledge' has become the norm. Reasoning, problem solving, and learning strategies form the core focus of public educational systems. Click here to see references
Likelihood by 2030 The average group answer: 71.1 Respondents: 185
Comments on this possibility (page 2) click here
1. What might encourage this possibility? What positive consequences might follow? Please enter positive developments and consequences. If you wish to change your prior input, type it here:
There is a great need for this I believe
***1. In ouropinion, knowledge is the result of the epistemological process of human brain triggered by information from the surroundings. What ipossible is that a group of persons, preferably comprehensivelydeveloped, in a community with shared goals work together, a synegenerated yielding results greater than the sum of its parts, oftenawesome. We firmly believe that this cooperation will be widely spraround the world by the year 2030.***I believe that public and private investments in social aspects anin education of the members of the society they can contribute.***complex and urgent national and international problems and disa
Average, median, standard deviation
Distribution of opinions
Summaries by region, occupation, gender
Correlation of qualitative responses with quantitative responses (e.g. “those who answered with a low estimate, had these reasons…”)