Scenario based analysis for decision making: A condition for … · 2018. 3. 28. · In this presentation we formulate conditions for successful application of scenario-based analysis,

May 2007Scenario-Based Analysis for Decision-Making 1

Scenario based analysis for decision making:

A condition for success in crisis management

Willem van Santen (Peak & Valley, DECIS Lab); Eddy van der Heijden (DECIS Lab, CICON); Adri Janssen (Peak & Valley); Niek Wijngaards (DECIS lab, TRT-NL)

Humans primarily assess situations and plan for actions by (implicit) scenario-based analysis. Decision making in crisis management situations in The Netherlands unfortunately does not explicitly feature scenario-based analysis; not within teams nor between teams. In this presentation we formulate conditions for successful application of scenario-based analysis, based on our experiences in crisis management and crisis management exercises. The conditions are formulated and briefly assessed in a number of cases. An important implication for information systems support is identified and future research is announced.

Practitioner Proceedings

ISBN 978-90-9021871-7

Scenario-Based Analysis for Decision-Making 2

Scenario Based Analysis for Decision Making:Scenario Based Analysis for Decision Making:A condition for success in crisis managementA condition for success in crisis management

Willem van SantenWillem van Santen(Peak & Valley, DECIS Lab)(Peak & Valley, DECIS Lab)

www.peakandvalley.nlwww.peakandvalley.nlw.van.santen@[email protected]

Eddy van der HeijdenEddy van der Heijden(DECIS Lab, CICON)(DECIS Lab, CICON)

www.decis.nlwww.decis.nlEddy.vanderheijden@[email protected]

Adri JanssenAdri Janssen(Peak & Valley)(Peak & Valley)

www.peakandvalley.nlwww.peakandvalley.nla.janssen@[email protected]

Niek WijngaardsNiek Wijngaards(DECIS Lab, TRT(DECIS Lab, TRT--NL)NL)

www.decis.nlwww.decis.nlNiek.wijngaards@[email protected]


Contents of the presentationContents of the presentation

•• Problem StatementProblem Statement

•• Scenario Scenario –– Based Analysis (SBA)Based Analysis (SBA)

•• Preconditions for SBAPreconditions for SBA

•• Feasibility and Practice of SBAFeasibility and Practice of SBA

•• ConclusionsConclusions


Problem StatementProblem Statement

•• Crises are complex, dynamic and uniqueCrises are complex, dynamic and uniquePeople try to comprehend them by forming hypotheses based on forPeople try to comprehend them by forming hypotheses based on former mer experiences (experiences (““replaying old movies)replaying old movies)

•• In crises many different actors are involvedIn crises many different actors are involvedTo manage the crisis, their hypotheses should be shared and inteTo manage the crisis, their hypotheses should be shared and integrated.grated.

•• Are shared scenarios used in Are shared scenarios used in Dutch crisis management practice?Dutch crisis management practice?


ScenarioScenario--Based Analysis (1)Based Analysis (1)

FactsDoctrines

HistoryStory-lines

PlanActionEvent

Scenario-based analysis

Plan-based action


ScenarioScenario--Based Analysis (2)Based Analysis (2)

Creating a Story line involves:Creating a Story line involves:–– Fact findingFact finding

(environment, organization, situation)(environment, organization, situation)

–– AnalysisAnalysis(higher objective, history, doctrines, models, semantics)(higher objective, history, doctrines, models, semantics)

–– Expectation explicationExpectation explication(worst case, best case, other alternative)(worst case, best case, other alternative)


Preconditions for SBAPreconditions for SBA

•• Shared higher objectivesShared higher objectives

•• Notion of own role (Notion of own role (individual, organizationalindividual, organizational))

•• Notion of otherNotion of other’’s roles (s roles (individual, organizationalindividual, organizational))

•• Entrance to the physical networkEntrance to the physical network

•• Unambiguous termsUnambiguous terms

•• Correct logging and archivingCorrect logging and archiving


Feasibility and practice of SBAFeasibility and practice of SBAHAGARHAGAR HELGAHELGA BorseleBorsele

Shared higher Shared higher objectivesobjectives

Notion of own roleNotion of own role ☺☺Notion of otherNotion of other’’s roless roles ☺☺Entrance to physical Entrance to physical networknetwork

Unambiguous termsUnambiguous terms

Correct logging and Correct logging and archivingarchiving

Dutch Crisis Management Exercises, see References


ConclusionsConclusions

•• In the exercises we reviewed, the preconditions for In the exercises we reviewed, the preconditions for SBA were not metSBA were not met

•• The exercises show that crisis management teams feel The exercises show that crisis management teams feel the need to the need to ““come togethercome together””

•• In policy documents, we find a converging set of In policy documents, we find a converging set of measures to share doctrines, terminology and measures to share doctrines, terminology and informationinformation

•• Based on these observations, we expect that SBA will Based on these observations, we expect that SBA will play a more explicit role in future crisis management.play a more explicit role in future crisis management.


Research RecommendationsResearch Recommendations

•• Start experimenting with explicit ScenarioStart experimenting with explicit Scenario--Based Based Analysis in crisis management exercises;Analysis in crisis management exercises;

•• Investigate organizational change in relation to Investigate organizational change in relation to ScenarioScenario--Based Analysis in Decision Making Based Analysis in Decision Making ProcessesProcesses

•• Investigate intelligent agentsInvestigate intelligent agents’’ support for support for ScenarioScenario--Based Analysis in Decision Making Based Analysis in Decision Making ProcessesProcesses

Van Santen et al. Scenario-Based Analysis for Decision Making

Scenario-Based Analysis for Decision Making: A Condition for Success in Crisis Management

Willem van Santen Peak & Valley / DECIS Lab

www.peakandvalley.nl [email protected]

Adri Janssen Peak & Valley

www.peakandvalley.nl [email protected]

Eddy van der Heijden DECIS Lab / CICON

www.decis.nl [email protected]

Niek Wijngaards

DECIS Lab / Thales Research & Technology Netherlands

www.decis.nl [email protected]

ABSTRACT

Humans primarily assess situations and plan for actions by (implicit) scenario-based analysis. Decision making in crisis management situations in The Netherlands unfortunately does not explicitly feature scenario-based analysis; not within teams nor between teams. In this article we formulate conditions for successful application of scenario-based analysis, based on our experiences in crisis management and crisis management exercises. The conditions are formulated and briefly assessed in a number of cases. An important implication for information systems support is identified and future research is announced.

Keywords

Scenario-based analysis, decision making, story-lines, facts, analysis expectations, crisis management.

INTRODUCTION

Crises are renowned for their dynamic character and uniqueness; and are as a result unforeseeable. In crisis response, many different actors are involved, distributed over time, space and activities. The un-predictability, dynamics and large number of people involved make crisis response very complex. People often cope with complexity by formulating hypotheses about complex situations in their mind based on old (and not necessarily comparable) experiences. By replaying an ‘old movie’ and trying to fit plausible events and actions, people try to define possible problems and find mitigating measures.

Decision making during crises not only takes place in the mind of one single person, but also in a team or even in several teams involved in crisis response. The same ‘movie’ must therefore be accessible to multiple persons in one team and/or different teams. Not only must the ‘movie’ be shared, but also the identified problems and possible measures, with the joint intent to form shared crisis response awareness and, based on this awareness, optimal, interactive decisionmaking. The key topic of this position paper discusses whether this is the case: are shared (possibly generic or specific) scenarios used in decision making in crisis response and –management in the Netherlands?

In the following paragraphs, first scenario-based analysis for decision making in crisis teams is described. Second, conditions for successful scenario-based analysis are identified. Third, these aspects and conditions are assessed in actual crisis oriented decision making in The Netherlands. This assessment is based on a number of evaluations of

Practitioner Proceedings ISCRAM2007 May 2007


calamities, crises and exercises, as well as the experiments with crisis management by the authors. Finally, this paper concludes with an outlook on future research1.

SCENARIO-BASED ANALYSIS

Scenario-based analysis is intended for developing and reviewing strategies for action [1, p.21]. Scenarios lie at the basis of decision making and give insight in, and rationales for, possible actions and their impact via the formation of plans and sub-plans. Scenario-based analysis is by definition aimed at the future. The past and the present do however influence scenarios: the past provides lessons learned, the present offers facts of the current situation. Below is an illustration of the use of scenario-based analysis in operational and administrative decision making processes:

Facts Doctrines

History Story-lines

Figure 1. Scenario-based analysis in decision making.

Scenario-based analysis basically consists of two steps. In the first step a story-line is developed, leading to several different expectations of events in the (nearby) future. In the second step, per storyline, a concept plan of action is formed. These concept plans will be assessed by the decision making teams, leading to a decision on the course of action (the Plan). A consequence of the use of scenario-based analysis is a clearer, and shareable, perception of risks [1, p. 110-121] and interventions and reflection on the situation – important aspects of crisis management [8].

A story-line describes a (possible) course of a crisis, an incident within a crisis or an event within an incident. The boundaries of a story-line differ by “organizational level” of the crisis response operation. Moreover, story-lines may be aimed at certain functionalities of crisis suppression (for example the commitment of police forces or availability and use of the traffic infrastructure). A story-line at the highest organizational level may lead to a number of story-lines at lower levels; the same holds for functionalities. Every crisis response organization must be explicitly aware of these interdependencies and correctly plan the alignment of story-lines.

The creation of a story-line involves:

1. Fact finding. Facts2 represent the “present” and describe the current situation of the crisis. It is very important to verify the

1 The (intermediate) results of scenario-based analysis are described without making any assumptions on the processes to arrive at these results. Scenario-based analysis for decision making hinges on sharing results, not on homogenizing processes.

PlanActionEvent

Scenario-based analysis

Plan-based action



(assumed) facts before using them in decision-making processes at all. Facts cover three aspects:

a. Environment. These facts concern, e.g., how many inhabitants and which crops are in the affected area, locations of chemical installations, etc.

b. Organization. These facts concern, e.g., which units are already in the area, location of committed special units, processes without adequate resources, etc.

c. Situation. These facts concern, e.g., damage caused by the incident, pretexts of the incident, scale of effects, etc.

2. Analysis. In the assessment and analysis of the facts, the “past” is explicitly involved. Based on experience, the seriousness of certain facts or the opportunities they offer, is assessed. In addition, possible consequences of events or facts are assessed. The analysis always involves addressing the following aspects:

a. Higher objective. The basic intention with which every involved unit acts in the crisis, becomes focused (or aligned) with this objective. For example, is the intent to limit the spreading of the crisis, or to secure as many victims as possible? When dealing with scarcity of resources, units can base their decisions on this higher objective.

b. History. The history covers information from old plans or old crises. History offers lessons learned and “best practices” for the crisis organization. In addition, the history of a story-line covers actions already brought into execution. Whether actions are still in execution, or already carried out: their impact must still be visible, and should be taken into account during the analysis.

c. Doctrines. Doctrines describe the work method of operational and administrative units. Possible response times, displacement times, acting times, but also security instructions are inferred from doctrines. Doctrines not only help with elaborating the scenario, but also with assessing the actions and work method of colleague crisis organizations in the same area or in adjacent areas.

d. Models. Models provide a forecast of the development of events. For example, to predict overflow of a certain area, trajectory of a fire, completion of an evacuation and seriousness of impact of multiple incidents.

e. Semantics. Sharing story-lines, plans and decisions requires the use of unambiguous definitions of terms. This not only applies to the “crisis management language”, but also for the naming of locations and events. In general, terms must be specific and univocal.

3. Expectation explicitation. Expectations describe possible futures: possible courses of the crisis and its repression. Scenario-based analysis is (most) effective when per cycle of figure 1 at least three expectations are developed. These expectations provide alternative views on the development of the crisis. Which of the three expectations is first chosen depends on the analysis. In general, a possible classification of expectations is: a worst-case expectation, a best-case expectation and another alternative. An expectation has an associated probability, as well as an estimation of its impact.

On basis of the story-line a potential plan can be developed for each expectation. This is a natural moment to distribute the story-line to other organizations to (re-)align story-lines. Care should be taken to prevent overloading crisis response organizations. Criteria for dissemination of story-lines can be formulated – which is out of scope for this position paper.

2 A “fact” is not necessarily accurate, any inaccuracy should be considered explicitly part of the ”fact”.



Decision making is based on the results of scenario-based analysis. The expectations and associated potential plans lead to a decision regarding the development of a plan and the possible start of sub-plan formation. Scenario-based analysis starts again at the individual or team’s request, or when the course of the crisis differs from the adopted plan and from the alternative plans, or for other reasons which lead to adaptation of the decisions. It is important to clearly delineate the boundaries of a story-line: which incident or group of incidents relates to the story-line, which functionalities have been incorporated in the story-line, and how to determine when the course of the crisis renders the story-line obsolete and ergo a new story-line should be started.

PRECONDITIONS

Using scenario-based analysis in and between teams is not obvious. The members of crisis organizations must have experience in clarifying facts, in making shared analyses and in creating shared expectations. An important precondition is the acceptance of scenario-based analysis for decision making in the organization [2, p. 109-110; 180]. There is a number of other conditions which, when satisfied, enable scenario-based analysis for decision making within and between crisis response teams:

1. Shared higher objectives; 2. A correct and detailed notion of the individual (formal and informal) team member’s own role within a team,

but also of his or her organization (e.g. the borders of the team’s responsibility within the storyline); 3. A correct and detailed notion of the individual (formal and informal) role of the other team members, but also of

their organizations. These notions should be the same for all team members. 4. Entrance to the “physical” network;

This involves the ability to share information with team members, other teams and other relevant organizations; 5. Unambiguous terms in crisis management;

This not only concerns the definition of static terms (the same for all crises), but also of dynamic terms, which are frequently unique for a crisis and during this crisis.

6. Good logging and archiving procedures and -methods; Logging and archiving are not only important factors for good fact analysis, but also of great importance for transfer of tasks at shift changes or task changes and for after-evaluation of the crisis response.

When these preconditions are met, information systems can come into play to support scenario-based analysis in crisis management, at multiple levels in the entire crisis organization, ranging from operational levels to more managerial / staff levels. Although the functionality offered by a specific information system can differ, e.g. depending on the workflow or processes in place within a team or organization, it is of importance that these information systems facilitate:

… making the distinction between facts, analysis, expectations, plans, and actions.

… separating scenario-based analysis from actual acting: after taking actions again the facts need to be assessed to determine the consequences of the (in)actions taken.

… sharing scenario-based analysis (including sharing facts, analysis, expectations, and managing interdependencies between scenarios) across teams and organizations.

FEASIBILITY AND PRACTICE OF SCENARIO-BASED ANALYSIS

Scenario-based analysis can, and will, take place when the aforementioned preconditions are met. The signs, however, are not very promising. In general, there is a notion that teams in crisis management situations are so much involved in reacting to the incidents that they have insufficient time for pro-action: to look and plan forward. As a result, there is a focus on operational issues, even in teams with a tactical or strategic role. Furthermore, the ICT and means of communication differ between the different operational services (police, fire brigade, healthcare, local government) and each service uses its own doctrine and “language”. How does that work out in practice?

In 2005 and 2006, the authors observed several exercises that took place in the Netherlands. The following table presents a very condensed summary of the observations in three exercises. Exercises HAGAR [3] and HELGA [4] were large, regional exercises in which several Dutch and German decision-making teams took part. These two exercises were part of a large flood-management program called “program VIKING”. Exercise Borsele was a local government exercise, during which coordination between local teams was an important goal [10].



Precondition Exercise “HAGAR”

Nov 2005

Exercise “HELGA”

May 2006

Exercise “Borsele”

Nov 2006

Shared higher objectives There were no specific higher objectives, other than a common notion to prevent dike openings and flood and, if necessary, timely evacuation.

There were no specific higher objectives, other than a common notion to prevent dike openings and flood and, if necessary, timely evacuation.

Higher objectives introduced during the crisis were not adequately shared – e.g. timing and teams.

Notion of individual and organizational own role

The participants in the exercise had a clear view on their own individual role in the crisis and on their organization’s role.

Not all (some) participants in the exercise had a clear view on their own individual role in the crisis and most participants had an overall view on their organization’s role.

Most participants had an overview on their individual role; some did not have an overview of their organization’s role.

Notion of other individual and other organizational roles

Most participants had an overall explicit view on the individual and organizational other’s roles.

Most participants had a minimal explicit view on the individual and organizational other’s roles.

Most participants had a minimal explicit view on the roles of other individuals and organizations.

Entrance to the physical network

All participants had entrance to the physical network. Participants used different information systems and different means of communication (the network was not optimally used for sharing information).

All participants had entrance to the physical network. Participants were not sufficiently trained on the specific tools. Participants made almost no use of the physical network. Coordination between teams happened during physical meetings.

A few participants had entrance to the physical network; the majority did not receive information and did not partake in the shared situation awareness.

Unambiguous terms Participants used “different languages” sometimes leading to misunderstanding of messages.

Participants used “different languages” sometimes leading to misunderstanding of messages.

Participants used “different languages” and terminology leading to misunderstanding of messages.

Correct logging and archiving

There were no shared structured logging and archiving procedures, other then the “saving everything” procedure used by the evaluation organization.

There were no shared structured logging and archiving procedures, other then the “saving everything” procedure used by the evaluation organization.

The logging and archiving procedures in place are not sufficient; the “save everything” procedure by the evaluation organization provided additional information.

Table 1: Our observations during exercises in 2005 and 2006

The observations in the abovementioned exercises are not unique. They are supported by other evaluation reports (e.g. the observation report of the large “Bonfire exercise” in 2005 [5] and the report on 41 exercises by the Netherlands Healthcare Inspectorate [6]). They are also supported by several government policy analyses, such as the ACIR report [7] on the use of ICT in disaster management and the evaluation of real disasters such as the Volendam fire-incident [11] and the Enschede firework disaster [9].



From our own experiences we conclude that the preconditions for effective scenario-based analysis are still not met. Fortunately, as a result of crisis management programs such as VIKING the involved crisis management teams work hard to “come together”. We observe a converging set of measures on national, regional and local levels to share information and develop a shared doctrine and terminology. Based on these observations, we expect that scenario-based analysis will play an important role in future crisis management.

FUTURE

In this experience-based position paper, we have outlined our view on scenario-based analysis for decision making in crisis management. Our experiences in crisis management and crisis management exercises, as well as our analysis of reports of crises, support our belief that this approach yields high benefits. In the near future, we intend to further explore this topic from a number of angles: In a number of Dutch and Dutch-German crisis management programs, experiments will be conducted regarding scenario-based analysis for decision making. Additionally, we intend to explore the relationship with organizational change in crisis management. And in relation to information systems, we intend to research a multi-agent system (software agents), supporting humans involved in incident management using scenario-based analysis.

ACKNOWLEDGEMENTS

The authors are grateful to the organization of the Dutch-German VIKING crisis management training program at the Province of Gelderland, The Netherlands and Nord-Rhein Westfalen, Germany, www.progammaviking.nl, in which the potential of scenario-based analysis for decision making became tangible. The research reported here is part of the Interactive Collaborative Information Systems (ICIS) project (http://www.icis.decis.nl/), supported by the Dutch Ministry of Economic Affairs, grant nr: BSIK03024. The ICIS project is hosted by the DECIS Lab (http://www.decis.nl/), the open research partnership of Thales Nederland, the Delft University of Technology, the University of Amsterdam and the Netherlands Organization for Applied Scientific Research TNO.

REFERENCES

1. Lindgren and Bandhold, Scenario planning, the link between future and strategy, Palgrave Macmillan, 2003. 2. Ringland, Scenario planning, 2nd edition, Wiley & Sons, 2006. 3. Programma VIKING, Evaluatierapport Hagar, november 2005. 4. Programma VIKING, Evaluatierapport Helga, september 2006. 5. COT, observatieverslag Bonfire, 2005. 6. IGZ, Oefenen? Noodzaak!, toetsing oefeningen GHOR ter voorbereiding op de bestrijding van rampen en

zware ongevallen, april 2005. 7. ACIR, de vrijblijvendheid voorbij, op weg naar een effectieve multidisciplinaire informatievoorziening bij

grootschalig gezamenlijk optreden in onze gedecentraliseerde eenheidsstaat, maart 2005. 8. Ministerie van Binnenlandse Zaken en Koninkrijksrelaties, Crisisbeheersing in scenario’s, december 2005. 9. Commissie Oosting, eindrapport de vuurwerkramp, februari 2001. 10. Gouman, Kempen, de Vree, Capello, Van der Heijden and Wijngaards, The Borsele Files: The challenge of

acquiring usable data under chaotic circumstances. Submitted to ISCRAM 2007. 11. Commissie Alders, Onderzoek cafebrand nieuwjaarsnacht 2001, beschrijving uitvoering gezamenlijk onderzoek

rijksinspecties. Samenvatting, conclusies en aanbevelingen, juni 2005.


Documents

Scenario based analysis for decision making: A condition for … · 2018. 3. 28. · In this presentation we formulate conditions for successful application of scenario-based analysis,